“Vague, confusing and nothing to enhance my work”: How Ki can undermine the peer review

At the start of this yr I received comments on an educational manuscript from me as a part of the same old peer review process and noticed something strange.

My research focuses on ensuring that trustworthy evidence is used for the data of politics, practice and decision -making. I often work with groups corresponding to the World Health Organization to perform systematic reviews to tell the rules or guidelines for clinical and public health. The paper I had submitted for the review of Peer was about systematic checking behavior.

What I noticed has played my concerns in regards to the growing role of artificial intelligence (AI) within the scientific process.

A service on the community

Peer Review is of fundamental importance for educational publishing system to make sure that research is strictly criticized before publication and distribution. In this process, researchers send their work right into a diary wherein editors invite expert colleagues to provide feedback. This advantages all the pieces involved.

For Peer experts, it’s favored when applying for funds or promotion, because it is taken into account a service for the community. For researchers, it asks them to refine their methods to make clear their arguments and to repair weaknesses with the intention to prove that their work is definitely worth the publication. Peer Review ensures that research is trustworthy for the general public.

Even at first glance, the comments that I received in my manuscript in January this yr seemed strange.

First, the sound was far too uniform and general. There was also an unexpected lack of nuance, depth or personality. And the reviewer had no page or line numbers and no specific examples provided what needed to be improved to steer my revisions.

For example, they suggested that I “remove redundant explanations”. However, they didn’t state which explanations were superfluous or where they appeared within the manuscript.

They also suggested ordering my reference list bizarre that ignored the journal requirements and doesn’t follow any format that I replicated in a scientific journal. They provided comments on underpants that didn’t exist.

And although the journal didn’t require a piece “Discussion”, the peer reviewer had given the next proposal to enhance my non-existent discussion: “Future instructions for the further refinement of (the content of the paper) would improve the future-oriented perspective of the paper”.

Test my suspicion

In order to check my suspicion, the evaluation was at the very least partially written by Ai, I had my very own manuscript on three AI models uploaded-chatt-4o, Gemini 1.5pro and deepseek-V3. I then compared comments from the Peer Review with the edition of the models.

For example, in keeping with Peer Reviewer's comment on the summary:

Briefly discuss the broader effects of (predominant edition of paper) on systematic review results to emphasise their importance.

The edition of Chatgpt-4o regarding the abstract reading:

Use a sentence that summarizes the broader effects or possible effects (predominant edition of paper) on systematic checks or evidence -based practice.

Peer Reviewer's comment on the methods was:

The methodological transparency is commendable, with an in depth documentation of the (process carried out by us) and the reasoning. The orientation with (gold standard) reporting requirements is a specialty that ensures compatibility with current best practices.

The output of Chatgpt-4o in relation to the methods was:

Clearly describes the technique of (process that we carried out) to make sure transparency within the methodology. Emphasizes the alignment of the tool with (gold standard) guidelines that reinforce the methodological strict.

The biggest red flag was, nonetheless, the difference between the feedback from the peer reviewer and the feedback from the Associate Editor of the Journal, which I had submitted my manuscript. Where the feedback from the Associate Editor was clearly, instructive and helpful, the feedback from the Peer Reviewer was vaguely, confusing and did nothing to enhance my work.

I expressed my concerns on to the editor -in -chief. To your honor, I used to be immediately thanked that I had reported the issues and documented my investigation – which, as they said, was “worrying and unveiling”.

A careful supervision is required

I even have no final proof that the peer review of my manuscript was generated. However, the similarities between the comments of the peer review and the output of the AI ​​models were noticeable.

KI models make research faster, easier and more accessible. However, your implementation as an instrument for supporting the peer requires careful surveillance, whereby the present instructions for AI use within the check of the peer mixedand its effectiveness unclear.

If AI models are to be utilized in the peer review, the authors have the precise to be told and the chance to provide themselves to separate. Reviewers must also disclose the usage of AI of their review. However, the enforcement of this problem stays an issue and must fall to the magazines and editors to make sure that peer reviewers who use AI models inappropriately are marked.

I submitted my research to “experts” by my colleagues in front of the realm and received the feedback with AI-generated feedback, which ultimately didn’t improve my work. If I had accepted these comments and not using a query – and if the Associate Editor had not given such an exemplary feedback – there may be every probability that this might be unnoticed.

My work can have been assumed for publication without being properly checked and was spread to the general public as a “fact”, which my colleagues confirmed, although my colleagues didn’t check this work themselves.