if no one is having babies and the world is full of walkers.... it isn't hard to assume the world will be extremely low on survivors.
I also thought everyone was infected so whenever a 'survivor' died (unless they were properly dealt with) they would just be adding to the walker population