Here's a link to a pretty good calculator, unfortunately, the one I usually go by when double-checking myself was hosted on geocities, and is now gone. The calculator towards the bottom of the webpage goes by dimensions, so there won't be any question about how large your 20 gallon cage is. It's pretty liberal, though, you don't necessarily need this much space per mouse:
Those numbers are for females, because males (unless they're littermates, who seem to get along better than strangers, but I've heard that once they start breeding they become more aggressive to other males) are territorial, and in too small of a space, will smell the other males all over everything and become aggressive. If you have a 20 gallon long, and I'm only trying to picture it, not looking at one, so don't quote me, but I'd say you could go up to 15 or so mice before it's TOO crowded, but I wouldn't keep it that way. 5-10 would be my guess at happy mice, especially with hiding places. As for the five gallons, you could probably do 1-3 females or one male, or use them for birthing cages. Just guesses. You can probably just put something similarly sized to your mice in the tank, visualize it, and use common sense about it.
In my mousery, I have all sorts of different population of meeces in similar cages. Some ten gallon tanks have only 3 or 4 in them; some have eight or ten. I try to house meeces of similar age together wherever possible. Family groupings like mother and daughters or father and sons work well enough in many cases. Both bucks and does often become less accepting of any new meece after having been bred, so that sometimes ends up with one or two of them being moved in order to maintain peace. I have about a half a dozen bucks housed separately, and some of those were removed for health issues. They remain in bachelor quarters after completion of their treatment. Each situation is handled according to the very simple principle of what works and what doesn't. I'm a big fan of that principle in most areas of my life, actually.