Sounds like flea beetles to me also. Many people say they won't do that much damage, but they riddle the leaves of my eggplant to make it look like lace and stunt the growth. I use Sevin spray to allow the eggplant to grow to where it is big enough and strong enough to withstand them.
A couple of possible organic controls is to use a sticky trap or vacuum. These recommendations come from Rodale's Vegetable Garden Problem Solver by Fern Marshall Bradley, to give what I hope is proper credit. They also mention using a garlic spray on the plants to deter feeding.
Take a piece of yellow cardboard or maybe paint a piece of plywood bright yellow. Many insects are attracted to yellow. For fleas beetles it probably does not need to be yellow, but you can use these for other insects as well. Coat the surface with a clear sticky substance. Or you can buy commercial sticky traps. Hold the trap close to the plant and brush the foilage. Some will jump up and be trapped. Don't leave the sticky trap in the garden permanently as it will attract beneficial insects.
If the plants are strong enough, you can use a hand held vacuum cleaner to suck the beetles off. The book mentions that you may need to do this frequently, maybe daily, to make a dent in the population. I'd suspect it may take more than on trip through the eggplant with sticky traps to make a difference too.
If your plants are growing and developing, you probably don't need to do anything. If the leaves look like lace and dry out enough to turn turn brown and crispy, some control is a pretty good idea.
