Abstract, for those not interested in reading TFA:
"Large language models now write papers and referee reports in economics. The field must develop best practices for handling AI-generated text in the publication process. Otherwise, peer review is in serious danger of collapsing. We propose that AI detectors should become a standard tool of the profession. Our experiment shows that they flag referee reports written by AI, not reports that AI merely proofread or helped edit.
Aided by a simple model, we argue that a paper flagged as AI-generated should trigger a closer reading and can be rejected at the first mistake, and that an AI-generated referee report adds nothing that editors could not get by running the prompt themselves. Such reports are unacceptable: they damage authors and journals and should be treated like plagiarism. Individual restraint has already shown its limits, and the profession needs to develop norms to solve this collective action problem. We close with a practical guide for authors, referees, and editors."
The more I get serious about this, the more I want to see Machines produce Machine-native output. I don't want crossover or doubt. I want high confidence that machine output:
> Aided by a simple model, we argue that a paper flagged as AI-generated should trigger a closer reading and can be rejected at the first mistake
“We propose a witch hunt fueled by unreliable, unaccountable, proprietary detection technology that’s roughly equivalent to tea leaf reading and can be fooled by trivial rewrite templates that anyone can find with a 5-second web search.”
Conferences are now capping authors to a maximum of 8 papers which of course screws up university departments where everyone adds their name. This ETH professor says there should be a deposit paid to avoid AI spam https://www.linkedin.com/posts/omutlu_international-symposiu...
Same professor was rejected twice by peer review for what later became landmark papers one of which even got some Intel industry award. He did not change the paper and merely resubmitted that's how broken peer review already is.
Clearly it's not worth wasting human attention on AI-generated text that doesn't reflect considerable human contribution.
Is the same true for AI-generated text that helps a human contributor contribute more than they otherwise would be able to? (like ESL, or someone who's just not great at writing) As in, the thoughts are human but the text is AI.
To be fair the peer review model has been falling apart long before AI generated text was a thing, mostly due to the exploitative nature of academic journals leading to incentive structures that don't align with producing good science.
Reviewers don't get paid and have zero incentive to do a good job. You can in many cases just submit a paper over and over to different journals until it encounters a reviewer that can't be bothered reading it, upon which it gets rubber stamped through. The result is that time and time again, incredibly shoddy and too often downright fraudulent science has gotten through peer review.
Reviewers don't get paid but that doesn't mean they have zero incentive to do a good job.
First, reviewers do get compensated, just not with money but with reviews from other researchers when they submit their papers. Not every review is good, but I've gotten enough helpful reviews from highly competent researchers whose time is very valuable which I didn't have to pay a cent for, that I'm incentivized to do the same for others. Also I want the system to work, so that's also an incentive.
> You can in many cases just submit a paper over and over to different journals until it encounters a reviewer that can't be bothered reading it, upon which it gets rubber stamped through.
This is not how to build a career in research, only high impact journals and conferences matter.
> The result is that time and time again, incredibly shoddy and too often downright fraudulent science has gotten through peer review.
Peer review isn't supposed to detect fraud, it's not designed to do that, and the people reviewing aren't trained in fraud detection, nor should they be.
The system is supposed to detect fraud through reproducible studies, but if you hop over to this thread (https://news.ycombinator.com/item?id=50538303) you'll see people complaining all over that the results are obvious so what was the point of the research. Then you might appreciate how hard it is to make a career off reproducing results. If we think fraud in research is such a problem, we have to actually allocate funds toward reproducing results.
But to your point, peer review does stop plenty of shoddy research, just not all of it, nor is it reasonable to expect it to, nor is all shoddy research worthless and worth driving to 0. Also we might be working on different and conflicting definitions of shoddy.
Well in the scenario you're a fraud or a hack, you're likely aware that you're a fraud or a hack, so you're unlikely to pursue a career in the traditional high impact journal sense. Though you may still wish to build credibility via volume of work, or to perpetuate some agenda, perhaps manufacturing a scientific basis for a business you're running.
We know replication doesn't prevent fraud either. Like in many fields most studies simply don't replicate. It's a whole thing.
I’m going to say a class of opinion that has historically been overwhelmingly rejected by this crowd:
Peer review will no longer matter in about 2 years and it will feel silly that we were ever asking ourselves these questions.
Despite being proven time and time again that performance trends are incredibly stable and if anything accelerating slightly, that pretraining scaling laws and their analogs for post training are reliable enough for this to be mathematically uncontroversial, it is just as reliable that these clearly accurate projections are so incredible that people still don’t seem to accept it. This sentence has been true and could have been said at any point in time over the last 4-5 years (and it has, many times).
The AI 2027 scenario while hilarious when posted is alarmingly not that far off from where we are today. As Dwarkesh reports from his conversations with Anthropic researchers and those at other labs, we are on track to solve RSI sometime in 2027 (again a relatively easy thing to project if you know a little bit about performance trends and how model performance is measured).
The world will not implode like a dying star, and I expect the experience to be more or less what we’re going through today, though some capabilities will experience a much higher rate of improvement, and since there are unavoidable bottlenecks that AI cannot get around ever and some that will take years to figure out how to break down. But “oh no we don’t have humans to peer review” is incredible to me: peer review is a quality filter, and humans are notoriously bad at it. Models are already better in many ways at peer review and will become so obviously better that having a human in the loop at all will be like hailing a horse and buggy ride from the airport.
It's amazingly offensive to LLM-review a paper that someone has put months or years of their life into writing, for report that will partly determine whether it can be published in a high quality journal. And the career consequences that come for it. For those unaware, tenure decisions in economics can rest on 3-5 top-level publications. The number of pubs is much smaller than in adjacent fields, like OR.
There should be strong, public, and embarrassing consequences for reviewers who do this. And the same applies to authors who use AI-generated text.
I just can't believe that people feel that they can inflict AI text on other humans as though it were their own. We can't normalize this, it's incredibly offensive. I recently received an AI communication from a doctor. Did the doctor even think about it, or did they just press enter? How much credence do I give their decision on the question I asked? Certainly much less than if they had written it themselves.
That said, the marketplace for ideas is going to look a lot different in the future, and that's probably ok. There needs to be a role for agentic or hybrid communication and discovery, and we need to find new ways to ferret out the truth in the midst of all of that. The old gatekeepers will still play some role, but it will be different. But feeding colleagues AI writing as your own needs to be a hard no.
That's fine, as long as it's labeled accordingly. I opt for organic, sustainable options in everything where possible. Maybe journals just need a "Certified Free-Range Human" seal so readers know what they're getting.
It almost definitely cannot, because it takes 1 minute to generate a research paper and it takes much more time to review it and make sure it's trash AI slop.
"Large language models now write papers and referee reports in economics. The field must develop best practices for handling AI-generated text in the publication process. Otherwise, peer review is in serious danger of collapsing. We propose that AI detectors should become a standard tool of the profession. Our experiment shows that they flag referee reports written by AI, not reports that AI merely proofread or helped edit.
Aided by a simple model, we argue that a paper flagged as AI-generated should trigger a closer reading and can be rejected at the first mistake, and that an AI-generated referee report adds nothing that editors could not get by running the prompt themselves. Such reports are unacceptable: they damage authors and journals and should be treated like plagiarism. Individual restraint has already shown its limits, and the profession needs to develop norms to solve this collective action problem. We close with a practical guide for authors, referees, and editors."
- is optimized for machine input
- is convertible to human-native symbols, context
- includes metadata and proof-of-custody/origin
- leans into its role as an evolving language
- is necessarily terse and token-efficient
“We propose a witch hunt fueled by unreliable, unaccountable, proprietary detection technology that’s roughly equivalent to tea leaf reading and can be fooled by trivial rewrite templates that anyone can find with a 5-second web search.”
Same professor was rejected twice by peer review for what later became landmark papers one of which even got some Intel industry award. He did not change the paper and merely resubmitted that's how broken peer review already is.
Is the same true for AI-generated text that helps a human contributor contribute more than they otherwise would be able to? (like ESL, or someone who's just not great at writing) As in, the thoughts are human but the text is AI.
How are we supposed to gauge human contribution?
Reviewers don't get paid and have zero incentive to do a good job. You can in many cases just submit a paper over and over to different journals until it encounters a reviewer that can't be bothered reading it, upon which it gets rubber stamped through. The result is that time and time again, incredibly shoddy and too often downright fraudulent science has gotten through peer review.
First, reviewers do get compensated, just not with money but with reviews from other researchers when they submit their papers. Not every review is good, but I've gotten enough helpful reviews from highly competent researchers whose time is very valuable which I didn't have to pay a cent for, that I'm incentivized to do the same for others. Also I want the system to work, so that's also an incentive.
> You can in many cases just submit a paper over and over to different journals until it encounters a reviewer that can't be bothered reading it, upon which it gets rubber stamped through.
This is not how to build a career in research, only high impact journals and conferences matter.
> The result is that time and time again, incredibly shoddy and too often downright fraudulent science has gotten through peer review.
Peer review isn't supposed to detect fraud, it's not designed to do that, and the people reviewing aren't trained in fraud detection, nor should they be.
The system is supposed to detect fraud through reproducible studies, but if you hop over to this thread (https://news.ycombinator.com/item?id=50538303) you'll see people complaining all over that the results are obvious so what was the point of the research. Then you might appreciate how hard it is to make a career off reproducing results. If we think fraud in research is such a problem, we have to actually allocate funds toward reproducing results.
But to your point, peer review does stop plenty of shoddy research, just not all of it, nor is it reasonable to expect it to, nor is all shoddy research worthless and worth driving to 0. Also we might be working on different and conflicting definitions of shoddy.
We know replication doesn't prevent fraud either. Like in many fields most studies simply don't replicate. It's a whole thing.
Peer review will no longer matter in about 2 years and it will feel silly that we were ever asking ourselves these questions.
Despite being proven time and time again that performance trends are incredibly stable and if anything accelerating slightly, that pretraining scaling laws and their analogs for post training are reliable enough for this to be mathematically uncontroversial, it is just as reliable that these clearly accurate projections are so incredible that people still don’t seem to accept it. This sentence has been true and could have been said at any point in time over the last 4-5 years (and it has, many times).
The AI 2027 scenario while hilarious when posted is alarmingly not that far off from where we are today. As Dwarkesh reports from his conversations with Anthropic researchers and those at other labs, we are on track to solve RSI sometime in 2027 (again a relatively easy thing to project if you know a little bit about performance trends and how model performance is measured).
The world will not implode like a dying star, and I expect the experience to be more or less what we’re going through today, though some capabilities will experience a much higher rate of improvement, and since there are unavoidable bottlenecks that AI cannot get around ever and some that will take years to figure out how to break down. But “oh no we don’t have humans to peer review” is incredible to me: peer review is a quality filter, and humans are notoriously bad at it. Models are already better in many ways at peer review and will become so obviously better that having a human in the loop at all will be like hailing a horse and buggy ride from the airport.
There should be strong, public, and embarrassing consequences for reviewers who do this. And the same applies to authors who use AI-generated text.
I just can't believe that people feel that they can inflict AI text on other humans as though it were their own. We can't normalize this, it's incredibly offensive. I recently received an AI communication from a doctor. Did the doctor even think about it, or did they just press enter? How much credence do I give their decision on the question I asked? Certainly much less than if they had written it themselves.
That said, the marketplace for ideas is going to look a lot different in the future, and that's probably ok. There needs to be a role for agentic or hybrid communication and discovery, and we need to find new ways to ferret out the truth in the midst of all of that. The old gatekeepers will still play some role, but it will be different. But feeding colleagues AI writing as your own needs to be a hard no.
Framed another way: AIs are arguing with AIs about the merits of AIs, while the Board of Governors of the Federal Reserve System is nodding.
So far we’ve been using volume and time as constraints to justify why reviewing so much AI-generated text and code is humanly impossible.
But if we solve time and cognitive constraints for subject matter experts, Can AI-generated text and code survive peer reviews?