The fact that there's almost nothing that overlaps with Christmas besides the timing, and that the timing has other plausible explanations.

FWIW, I'm very open to the timing having been shifted to coincide with other festivals, but that's not what most people mean when they say Christmas has pagan origins. They're not saying that Christmas isn't actually the day Jesus was born (I'm totally on board with that idea), they're saying that X, Y, and Z aspects of the Christmas celebration were originally pagan and were adapted for Christianity. I have seen no compelling evidence in favor of that claim about any aspect of Christmas traditions, and I've seen plenty against.

> fact that there's almost nothing that overlaps with Christmas besides the timing

The drunken revelry?

Given these two explanations for the drunken revelry,

1. Christmas has its origins in Saturnalia, but the only remaining similarity is the approximate date and the drunkenness.

2. Humans will happily accept any excuse to get drunk during the longest nights of the year.

I lean towards option 2 as a much simpler and more likely explanation.