I can imagine some unintended consequences from this. If I'm a hard-going reviewer, people might choose to avoid me, and I'll be compensated less. Conversely, if I'm easy-going, people might find ways to get their papers to me.
I suppose there can be some other layer of reviewer meta-review to account for this, much like the role 'acceptance rate' has come to have for journals and conferences. But, following Goodhart's law, even that has come to be gamed now that a meaning - a proxy for journal prestige or quality - has been placed on it.
That's why I use the term "credit". It needs to have real-world value of some sort. I don't think just paying people is the solution, that would just mean the richer you are, the more reviews you get, and the better your reviews are. You can argue they should be paid via a neutral source of money, but there is no such thing.
If the problem of gaming the system could be gotten around, something very similar to internet comments could work - anyone can post a public review of anyone's work, in which case reputation would become the currency. Successfully criticizing someone with higher reputation than yourself would gain more status than praising them, but also come with more risk. It would also make it a lot less meaningful to get those 'not new ideas lol' comments that others on this thread have complained of.
I quite like the fact that whatever someone says on the internet now, the first comment is usually the best-argued counterpoint. It seems like a dynamic that could be brought to science quite well. I may not have the time to parse an entire paper in a field unfamiliar to me, but if I come across it and the top comment is 'there's a math error in table 6 that throws paragraph 4 into doubt', I can certainly verify that for myself. That sort of 'peer' review would make science much more accessible to the interested layperson.
