You are ignoring the context of your own quote. Obviously he loves his g'ma, but is describing what he sees as the reaction of the typical white person when they pass a black stranger on the street.
He saw it first hand from both sides, as that black person and as the son of the white mother on the street.
You would be silly to deny that many, if not most white folks would be wary of encountering a strange person on the street, much less a black person. That is simply reality.
Also, what does he mean when he says typical? Politicians often try to capture in a phrase the persona of the typical American. Phrases you often hear are "Joe Six Pack", and names like that.