It has been discovered that 60% of approved medical AI systems that automatically summarize medical consultation details have misidentified drug names and recorded non-existent symptoms.
The Auditor General's office in Ontario, Canada, investigated AI systems used by the provincial government and discovered a series of serious errors in AI-powered medical record-keeping systems for healthcare institutions.
Office of the Auditor General of Ontario
Ontario auditors find doctors' AI note takers routine blow basic facts
https://www.theregister.com/ai-ml/2026/05/14/ontario-auditors-find-doctors-ai-note-takers-routinely-blow-basic-facts/5240771
The system in question is called 'AI Scribe,' which automatically creates medical notes by recording conversations during consultations. During the audit, 20 approved systems were evaluated using simulated medical recordings, and it was found that 12 systems recorded medications other than those actually prescribed in the medical notes.
AI-powered medical record systems are tools designed to reduce the manual input required by doctors and nurses to record patient conversations. However, if the AI misremembers drug names, symptoms, or treatment plans, it could not only be a simple note-taking error but could also impact the patient's overall medical care.
The audit revealed not only errors in drug names, but also instances where AI arbitrarily added information that was not spoken during the recording. Nine out of 20 systems generated treatment plan suggestions for patients, and in some cases, recorded information that was not present in the recording, such as 'no lumps were found' or 'the patient has anxiety symptoms.' In addition, 17 systems overlooked important information regarding the patient's mental health.
The audit report also pointed out problems with the evaluation methods used during the review. In the review of the AI medical record creation system, 'provincial presence,' such as whether the company was based in Ontario, accounted for 30% of the overall evaluation. On the other hand, the accuracy of medical notes accounted for only 4%, bias prevention for 2%, and threat, risk, and privacy assessments for 2%, indicating that accuracy and safety, which are crucial in the medical field, were given very little weight.
Furthermore, OntarioMD, a medical IT support organization for physicians in Ontario, recommended that physicians manually review AI-generated notes, but the approved systems did not have a function to make physician review mandatory. This meant that there was still room for AI-generated, incorrect medical notes to be used without modification.
It should be noted that this audit was 'verification of approved AI systems using simulated medical recordings,' and does not constitute a finding that medication errors occurred daily in all actual clinical settings. Nevertheless, the fact that medical AI can potentially misidentify basic information such as drug names, symptoms, and treatment plans indicates significant challenges in pre-implementation review and post-implementation verification systems.
According to the news outlet The Register, a spokesperson for the Ontario Department of Health stated that 'more than 5,000 doctors in Ontario are participating in the AI Scribe program, and there have been no known reports of patient harm related to the technology.' However, the audit report points out the risk that 'improper weighting of evaluations could lead to the selection of AI tools that create inaccurate medical records or fail to adequately protect sensitive patient health information.'
Related Posts:
in AI, Posted by log1d_ts