One of the great things about juggling is, at least as a beginner, you don't need much by the way of equipment. People have learned with tennis balls, rolled up socks, and any number of other objects. Lacrosse balls, available for around two bucks apiece from any sports supply store, are pretty good for juggling.
As you start to juggle more, you may want to get some specialized equipment. There aren't really any juggling vendors local to Ann Arbor, but last time I checked, Fantasy Attic Costumes (305 S Main St, 734-665-2680) had a few juggling beanbags and some clubs - mostly jugglebugs, which I would avoid, but they also carry airflites, which are good 1-piece clubs, for $14 a piece. Also In Flight Sports (215 S State St, 734-995-3323) has some stuff. Also Jireh Distributing Company (616-754-3487, email@example.com) is a Michigan mail-order place, and they usually ply their wares at local festivals.
Mail order is the way to go for most equipment. Luckily, most of the major vendors have online catalogs. The Juggling Information Service's Juggler's Mall is probably the best place to start. Popular American makers include Brian Dube, Renegade, and Todd Smith.
Our club also has catalogs from various companies--just let us know that you want to see them.
Juggling clubs (those things that look like light plastic bowling pins) are the one big item most jugglers end up acquiring sooner or later. Bruce has written a page specifically about buying juggling clubs.