Usually they're picked when they are full-sized and well filled out, and maybe some are starting to show the first tiny tinge of change in color (to black or red). A good failsafe with any kind of produce is, if you think maybe they're ready to pick, pick one and cut it open and eat it and see if you were right
But there is nothing actually wrong with letting them go further, especially if you are not trying to totally maximize the crop, and personally I much prefer them fully ripened. To each their own
Pat, growing no peppers at all this year, probably a good thing considering how darn COLD it's been.
I usually pick mine when they're about 3-4". Mine have gotten sunburned (purple on the side facing the sun), but they still taste fine. Actually, I hung some up to dry last year, and they turned red while drying inside my house. I ground them up & made some super-hot pepper flakes!