There is more to it than the fact that a taller person has to move the weight farther.
1. At the bottom position, the weight, which is distributed through the hips, is farther away from the fulcrum (the knees). Think of it like this: Which is more difficult, picking up a 10 pound hammer with a 2 foot shaft, or picking up a 10 pound hammer with a 5 foot shaft? The longer lever (the thighs) make getting out of the hole more difficult.
2. Taller people tend to weigh more than shorter people. When you squat, you are squatting bodyweight in addition to the weight on the bar.
That said, it is possible for taller people to make up the difference by getting bigger than the shorter guys. Ed Coad can squat over 1000 pounds about about 5'5", and so can Gary Frank at 6'3". The difference is that Ed only weighs 242 by Gary is around 350.