Families with children are a protected class under the federal Fair Housing Act. It's unlawful to discriminate in housing terms or conditions on the basis of familial status. See 42 U.S. Code 3604 at
http://www.law.cornell.edu/uscode/42/3604.html.
HOAs are permitted to establish swimming pool rules that are necessary for health or safety. These might include requiring a parent to accompany a young child in the pool, or banning children who are not toilet trained. But rules designed for the convenience of adults have been found to be unlawful.
In HUD v. Paradise Gardens, an HOA was found liable for restricting children between 5 and 16 to pool hours between 11:00 and 2:00, and for completely banning children under 5 from the pool. You can read the case at
http://www.hud.gov/offices/oha/oalj/cases/fha/files/HUD 04-90-0321-1.pdf. Excerpt:
Although an association of residents may be empowered to establish rules to accommodate the varying interests of residents to make the community enjoyable, it cannot set rules that would, in effect, deny or unduly limit the use of facilities based on familial status.
A more recent case is Landesman v. The Keys, available at
http://www.homeownersassociationman...sman-v.-The-Keys-Condominium-Owners-Assn.html.
If the housing in question is "housing for older persons", the HOA would have more leeway in restricting the activities of children.
