Peer reviews should be done in public and considered an ongoing process rather than a one-time thing. If an expert thinks some questions are worth asking, then the resulting discussion should be available for younger generations to learn from it as well. I think the open source ecosystem has shown the effectiveness of such a system. As more data is made public it becomes easier and more effective to sniff out bad actors as well.
We could also take steps towards helping establish high reputation for certain papers by introducing a mechanism other than citation count. Maybe some kind of stamp of approval.
Yes, this is more or less how the non-profit journal https://elifesciences.org (funded by HHMI, Wellcome, Max Planck, and Wallenberg Foundation) works.
A problematic aspect of non-public peer reviews is waste and political battles. I have witnessed first-hand how big names in a particular field reject articles from incumbents that are perfectly sound just to delay their publication and/or to copy them.
A public review introduces some skin in the game and avoids this kind of behavior, as well as rejections or requests to make changes because of reviewer incompetence. It also avoids the opposite thing, blind acceptance of flawed studies.
> Yes, this is more or less how the non-profit journal https://elifesciences.org (funded by HHMI, Wellcome, Max Planck, and Wallenberg Foundation) works.
Also, all 19 journals published by EGU (European Geophysical Union)
I'd take it even further: make peer review public and open to all members of that community. Imagine a forum-like discussion anybody can anonymously review any submission and all reviews are public.
Becoming a reviewer should still be invite-only and the system should keep track of the reviewer's identity behind the scenes to monitor for abuse, of course. The review can include coarse-grained reputation signals like "has reviewed 100+ papers in the last 5 years".
It might be worth embargoing reviews with a fixed time delay before making them public to prevent bandwagon effects and disincentivise review plagiarism tho. The reviewer's identity should be deanonymised too after something along the same time scale as when the paper author's identity is revealed.
We could also take steps towards helping establish high reputation for certain papers by introducing a mechanism other than citation count. Maybe some kind of stamp of approval.