Because others in his community drink and solicit whores does not mean that he has no choice in the matter; he willfully decides his course. He does not have to decide correctly and will be limited by his understanding, but he is a free agent for his personal happiness
You ignore the effects of social motivation and peer pressure on actions. Especially, ESPECIALLY in the case of children. If a kid grows up in a poor area and his high school friends are drug dealers, then it's pretty much already too late.
That doesn't even address the issue of childhood development. If a baby is neglected in the first year of life because of no father and the mother's working two jobs, it's already too late. The child will have moderate to severe social and developmental problems for the rest of its life. I'd rather have that mother on welfare so she can properly raise her child than have that child waste state resources down the line when he/she drops out of school and/or goes to jail.
I think it was Locke who spoke of the "tabula rasa." Never underestimate the effects of nurture.