On the contrary, we recover faster because our muscle volume is much smaller than the males. Less muscle mass, less damage caused by training, faster recovery.
This also occurs between males. Smaller males recover faster than larger males. A 70kg weight lifter recovers faster between training sessions and between sets than a 140kg weight lifter.