Container size can be a personal preference. Consider how much space you have available, whether you need to (or have someone else reasonably, physically able to) move them around while the damp dirt is in them. That can be a pain once you get past a single-digit # of gallons capacity.
Below a certain size per type of plant you'll be limiting the plant size, BUT at the same time you could have one plant in 20 gallons of dirt or two plants in 10 gallons of dirt, it's more about surface area to catch sunlight at some point except for the water issue. The longer your growing season the larger a pot you may need.
I usually use 5 gallon containers, a few a bit smaller, and window box sized for herbs. With those I MUST water twice a day in the heat of summer which is usually around 90F +-5 degrees in my zone. If you use a smaller container you can just grow a smaller plant and not need to water more often but be careful to limit the fertilizer you give the plant or else you will have a plant get too big for the container and lose water too quickly unless you don't mind watering several times a day or set up a drip irrigation system.
Another issue with some plants is wind. Last year I planted a grape tomato in an ~ 8 gallon pot. It grew well, I put a stake in the pot and it was about 8 foot tall before July! Great I thought, until a strong gust of wind came along one day and blew the entire thing on its side, killing the plant. The pot simply didn't weigh enough or have large enough diameter vs the leveraged force of the wind against the staked plant in the pot. Some people will put a big rock or brick in their pot to compensate a bit, or I had a few spare cinder blocks I put around some pots when I knew a lot of strong wind was coming.
With all that in mind, you might easily double your yield by using something 5 gallons or larger. I just use 5 gallon because that's the size of all those leftover plastic buckets that everything from sunflower seeds to drywall paste come in, so they were free and readily available. Some people are pickier about how their containers look though...
I should also mention that in 5 gallon buckets, most of my plants only get to about 70% the size of the same things I plant in the ground, though I suspect if my growing season were longer the difference might be greater and there's the location issue, things I have in pots are mostly shaded in the evening.
