I've never heard of anybody being hit by a rogue cat ladder. They start close to the wall of the building, so are not easily dislodged by wind. Besides, even if the owner is not concerned about pedestrians, they tend to be concerned about the safety of their cat.
Flower pots, OTOH, are an entirely different thing…