Inferring historical population sizes is key to identifying drivers of ecological and evolutionary change and crucial to predicting the future of species on our rapidly changing planet. The pairwise sequentially Markovian coalescent (PSMC) method provided a revolutionary framework to reconstruct species' demographic histories over millions of years based on the genome sequence of a single individual. Here, we detected and solved a common artifact in PSMC and related methods: recent population peaks followed by population collapses. Combining real and simulated genomes, we show that these peaks do not represent true population dynamics. Instead, ill-set default parameters cause false peaks in our own and published data, which can be avoided by adjusting parameter settings. Furthermore, we show that certain changes in population structure can cause similar patterns. Newer methods, like Beta-PSMC, perform better but do not always avoid this artifact. Our results suggest testing multiple parameters that split the first time window before interpreting recent population peaks followed by collapses and call for the development of robust methods.