...we used to have giggle fits turning off the sound whenever Hubert Humphrey came on the screen. (Must have been the '68 election.) He was as funny as any cartoon character, and we had no idea what he was talking about, so his voice would have just been distracting noise.

There's one other aspect of lower frequencies. The physiological studies show we have less directionality (sensitivity to which direction a sound comes from) as frequencies decrease in this range, so a lower voice is more enveloping.

This is simply a result of the physical size of the head, which determines the separation distance between the ears, and also the size of the voice producing components. Longer wavelengths equals lower frequencies, so larger and taller people on average will have lower voices. Perhaps when we're little, a woman's voice is as authoritative sounding in this sense as a man's voice is when we're fully grown.