Studies link AI to more academic papers and lower review quality

In short: Researchers and journal editors are reporting a rise in academic paper submissions that seems linked to AI, along with signs that quality is slipping.

What's going on

Editors at the academic journal Organization Science say they noticed something change in 2023. They were seeing more papers submitted, and many seemed weaker than before. A new study by Claudine Gartenberg at the University of Pennsylvania and colleagues argues that AI is a key reason.

The study reports a 42% increase in submissions to the journal since 2022. It also reports a “significant drop” in quality, based on AI detection and other checks. The authors say peer reviews also declined, with reviewers focusing more on abstract ideas and less on what the paper actually found in the data.

Other examples point in the same direction. A paper in the Journal of Economic Literature found that AI could generate nearly 400 finance papers that looked ready to submit in 12 hours. Separately, a journal editor in Italy described receiving a 2,127-word review reply in 14 minutes, which raised concerns it may have been written with AI.

Some academics also worry about confidentiality. Reviewers may upload unpublished drafts into “large language models” (AI systems that predict the next word, like very advanced autocomplete). That could risk leaking ideas, since some tools may store or learn from what users paste in.

What to watch

Journals and universities may tighten rules for using AI in writing and peer review, or add stronger checks for rushed or low-quality work. In fields like medicine, researchers warn that flawed AI-assisted studies can create misleading results, like finding patterns that do not reflect real cause and effect.

Source: Financial Times