The FDA AI tool Elsa was supposed to streamline drug approvals. Instead, insiders say it invents non-existent studies and misrepresents research. Staff have sidelined the tool for core review work due to frequent hallucinations and unreliable outputs.
What Elsa Was Meant to Do
The FDA launched Elsa in early June to boost efficiency across the agency. Leaders framed it as a way to speed drug approvals and reduce healthcare costs. Drug affordability remains a key political issue for the current administration.
What Insiders Report
FDA employees told CNN that Elsa confidently fabricates medical studies and miscites research. One employee said, “Anything that you don’t have time to double-check is unreliable.” Staff also said the tool often answers incorrectly on FDA research areas and drug labels. It cannot link to third-party citations from medical journals.
Why Staff Sidelined the Tool
Employees now avoid using Elsa for regulatory reviews. They say the FDA AI tool wastes time because every answer needs verification. It still helps with low-risk tasks, like meeting summaries and email templates. Even then, staff remain cautious.
The FDA’s Official Line
Jeremy Walsh, the FDA’s head of AI, admitted Elsa hallucinates like other generative AI models. Commissioner Marty Makary stressed Elsa is not integrated into clinical review protocols. He said employees only use it for organizational duties, and only if they choose to.
HHS Pushes Back
The Department of Health and Human Services disputed CNN’s report. Officials said the network relied on “disgruntled former employees” who never used the current version of Elsa. HHS argued, “The only thing ‘hallucinating’ in this story is CNN’s failed reporting.”
Why Hallucinations Are So Risky
Regulatory work demands precision, provenance, and verifiable sources. Hallucinated data can mislead reviewers, distort risk assessments, and undermine public trust. If regulators lean on fabricated outputs, the errors can cascade into policy, clinical, and legal decisions.
What the FDA Should Do Next
The FDA should harden controls around the FDA AI tool:
- Enforce human-in-the-loop review for any decision-impacting output.
- Log prompts, versions, and sources for full traceability.
- Pair generation with retrieval so every fact cites a source.
- Publish red-team findings and model cards for transparency.
- Restrict Elsa to low-risk tasks until validation improves.
What Healthcare and Industry Should Learn
Hospitals, payers, and vendors must not trust black-box claims. They should benchmark hallucination rates, log every output, and tag synthetic content. Staff need training to detect and escalate AI errors fast. Independent audits should monitor models in production, not only pre-deployment.
Conclusion
The FDA AI tool Elsa shows how fast hype can outrun assurance. Hallucinated data breaks trust, slows teams, and raises compliance risk. The FDA, HHS, and industry must demand auditability, verifiable sourcing, and strict human oversight. Responsible AI is mandatory when public health is on the line.
0 responses to “FDA AI Tool Elsa Invents Fake Data, Insiders Warn”