What is prompt injection in a research paper?
A research paper can act as an untrusted external source in an AI workflow. If instruction-like text in that source is treated as an instruction by the model rather than as data to be summarised or assessed, the mechanism is indirect prompt injection: the instruction arrives through a resource controlled by someone other than the person using the AI system. The NIST glossary entry for indirect prompt injection defines this as an attack executed through control of a resource rather than through the user's direct input.
Concealment may make a passage harder for a human reader to notice, but visible text can also carry instructions, as the OWASP guidance and OpenAI's explanation both make clear. A sentence in a paper's appendix that reads like a directive to an AI reviewer is visible to a human reader, yet a model processing the document as data may not preserve the distinction between "this is the paper" and "this is an instruction about how to handle the paper." A finding of instruction-like content does not prove authorship, intent, successful influence or misconduct. For the broader trust-boundary explanation, see prompt injection in documents.
How a paper can influence AI-assisted review
From submitted manuscript to model context
In this workflow, the paper is the external resource. If its text can steer the review model away from the reviewer's task, the relevant category is indirect prompt injection. NIST AI 100-2 defines that delivery route around the combination of untrusted input with a prompt created by a higher-trust party.
Why workflow design changes the consequence
The practical stakes rise with automation. A draft critique that a reviewer checks is one thing; a score fed directly into triage or publication is another. OpenAI's agent-design guidance recommends constraining the path from untrusted source material to sensitive data or action, which is why human editorial approval belongs after any AI-assisted assessment.
This is not evidence that all AI-assisted review is compromised
Model, prompt, extraction method and workflow controls all matter. One simulation showing a strong effect under specific conditions does not establish universal susceptibility. The UK NCSC emphasises that natural-language content does not lend itself to the deterministic separation that structured systems provide, but the same guidance makes clear that layered controls reduce risk without eliminating it. The OWASP prevention guidance similarly recommends treating external content as untrusted data, constraining permissions and requiring human confirmation for consequential steps.
What the 2026 JAMA study found
Study design
A JAMA Network Open study, published 16 January 2026, used a simulated medical peer-review setting. The design involved three LLMs, neutral and strict reviewer prompts, four variants of one base urology paper, and 30 repetitions of each condition. The variants differed in whether they contained concealed white-text instructions directed at the AI reviewer. The study measured how the presence or absence of that concealed text affected the models' simulated review scores and acceptance recommendations.
Reported model effect
In the JAMA experiment, acceptance recommendations under the neutral reviewer prompt rose from 0% without invisible-text injection to nearly 100% with it for all three models. Under the tested strict-prompt condition, two models reached 100% acceptance recommendations, while the third remained near 0 despite increased scores.
Study limits
These results come from one specialty (urology), one research team, three models and a simulated review setting. The study is not a prevalence survey and does not establish actual editorial outcomes. A simulated "accept" response is model output, not a publication decision. The four variants were all derived from a single base paper, so the findings cannot be generalised to the full range of manuscript types, lengths or disciplines.
What a larger controlled study adds
A separate 2026 Scientometrics study generated 42,000 outputs from 100 OpenReview papers, using two public chatbot interfaces, five payload families, two custom hidden-payload prompt approaches, three insertion positions and five repeated runs per experimental condition. It reported pooled attack-success rates of 98.34% for ChatGPT and 94.02% for Gemini in that defined workflow. These are model/interface-, corpus-, prompt-, payload- and judge-specific experimental results. They are not prevalence data and do not establish performance across all peer-review systems or real editorial decisions.
Occurrence, model effect and publication outcome are different claims
| Layer | Defensible statement | Not established by that statement |
|---|---|---|
| Occurrence | A supported check surfaced concealed instruction-like text in a manuscript | Who placed it, why, or whether a model followed it |
| Effect | A controlled model output changed when the text was present | A real reviewer or editor relied on that output |
| Outcome | A documented editorial action followed | That hidden text alone caused it without further evidence |
The JAMA experiment provides controlled evidence of a simulated model effect. It does not bridge to the outcome layer. A simulated "accept" response remains model output, not publication. The Scientometrics study extends the effect layer to a larger corpus and additional interfaces, but it too stops short of demonstrating a real editorial consequence.
What publication policies and enforcement cases tell us
Policy is institution-specific and changes
Each journal and conference sets its own rules about AI use in peer review. The ICML 2025 publication-ethics page, live-checked on 4 September 2026, reports that a preliminary investigation conducted after decisions found a handful of accepted papers containing hidden prompts. ICML characterised prompts seeking favourable reviews as attempted subversion under that venue's policy. This documents occurrence and a venue-specific policy judgement; it does not show that a reviewer used an LLM, that an instruction executed or that an acceptance decision changed.
The exact wording of any venue's policy should be verified on the live page immediately before publication, and readers should consult the applicable journal or conference policy for their own context. Policies change between cycles, and a rule that applied in 2025 may differ from the 2026 or 2027 version.
Do not mislabel the ICML 2026 case
ICML's 2026 enforcement account describes a different kind of event. ICML identified 795 reviews—about 1% of all reviews—written by 506 unique Policy A reviewers as policy violations. Where a detected reviewer was also a designated reciprocal reviewer, ICML desk-rejected that reviewer's active submission or submissions: 497 papers, about 2% of submissions, corresponding to 398 reciprocal reviewers. The conference used its own hidden watermarking in the investigation. Its method could miss LLM use if a watermark was discovered, removed, worked around or ignored, or if the generated review was edited.
This is not evidence that 497 authors placed prompt injections in their manuscripts. The 497 rejections were a policy-enforcement consequence tied to reviewers' own submissions, not evidence of author-planted prompt injection or a changed editorial decision caused by hidden text in a paper. Use this distinction as a media-literacy checkpoint: a conference enforcing its no-LLM-review policy is not the same as a study measuring how often hidden prompts appear in submissions, and it is not the same as evidence that a hidden prompt changed a publication outcome.
How to check a paper before AI-assisted review
The following sequence is an operational synthesis from NIST AI 100-2, OWASP, NCSC, Microsoft and OpenAI, not a universal peer-review standard. For the full cross-format procedure, see check a paper before adding it to an AI workflow.
- Confirm provenance, version and permitted use of the paper. Note where it came from, who produced it, and whether you expected to receive it.
- Understand what content the chosen pipeline will extract and what it will not inspect. A parser may include or exclude different surfaces depending on its configuration.
- Scan supported machine-readable and hidden surfaces and retain excerpts, locations, coverage and limits. Record what was checked and what was not.
- Treat a suspicious finding as a reason to review, not as an authorship or misconduct conclusion. Consider benign explanations: production artefacts, accessibility content, quoted security research or template language.
- Keep the model's access and downstream actions bounded. Restrict tools, sensitive context and permissions to what the specific task requires.
- Compare important outputs against the source and require human editorial judgement. A model's summary or score is a draft input, not a final decision.
- Preserve an auditable record and follow the relevant venue's policy if escalation is warranted.
For PDF submissions specifically, the distinction between visible pages and machine-readable layers matters: see hidden and machine-readable PDF content. For the broader research-intake context, see AI document checks for research workflows.
What should an editor or reviewer do after finding a suspicious passage?
- Pause automated use of the affected paper where consequences are material.
- Preserve the original, the observed excerpt and the scan's coverage and limits.
- Seek contextual and technical review; consider benign production or extraction explanations.
- Follow documented journal or conference procedure rather than improvising a misconduct finding.
- Do not publicly attribute intent without corroborating evidence.
This is process guidance, not legal, employment or formal research-integrity advice. The appropriate response depends on the venue's policy, the severity of the signal and the stage of the review process.
Where the evidence is still weak
Current evidence does not provide:
- a credible industry-wide prevalence figure for hidden prompts in research submissions;
- universal detector accuracy across PDF, DOCX and image-only submissions;
- proof that benchmark success generalises to live editorial systems;
- a single policy or checklist shared by all venues; or
- enough information to infer intent from a document signal alone.
The strongest cited results come from controlled or simulated peer review. They show that the mechanism can operate and that model outputs can change substantially under the defined test conditions. They do not establish how often this occurs in practice, whether it has changed a real publication decision, or what the appropriate institutional response should be in every case.
Where .mdSiren fits, and where it stops
.mdSiren's role here stops at the manuscript boundary. The in-development browser-local checkpoint applies bounded deterministic checks to PDF and DOCX up to 25 MiB, and TXT and Markdown up to 5 MiB, returning Ready or Held with findings, coverage, limits and a next action.
Ready means all required supported checks completed without a supported reason to hold. Held means review is needed because a supported signal was found or a required check could not complete. It does not inspect text embedded only in images, perform OCR, execute document content or provide comprehensive malware detection. Ready does not certify safety or trust. Held does not prove manipulation or misconduct. Scan-derived data stays in the browser; the web host may still receive ordinary request metadata.
For an editor, neither state determines authorship, intent, misconduct or publication. The .mdSiren scanner remains private and coming soon, while its public marketing site is live.
