Healthcare AI Scribe Vendor Assessments: Use case and AI Confidentiality
Before a practice extends an AI scribe beyond its trial, someone has to write down what the product does with patient information. The practice manager collects the vendor’s answers, a doctor’s error audit and what staff have said, then pastes the lot into ChatGPT or Claude to turn it into a recommendation. The assessment names the vendor, says where the audio is processed, records one patient’s request that the scribe has not honoured, counts the errors a doctor found and quotes a nurse who uses a free chatbot. This page marks each item in a routine assessment that the TGA, Ahpra, the Privacy Act and NSW health records law bear on, and shows how the same document is worked on Ermos inside the practice.
The AI scribe vendor assessment
Written by a practice manager or clinical lead before the practice decides whether to adopt, extend or stop using an AI scribe. Ahpra says all tools should be tested by the user or organisation for fitness for purpose before clinical use, and the RACGP says practices should seek independent legal advice or consult their medical defence organisation on security, privacy and contract terms.
AI scribe vendor assessment: Nightjar Notes
Kestrel Lane Medical Practice · Monday 16 November 2026 · Prepared by Helen Tran, Practice Manager, with Dr Chidi Eze · For decision at the practice meeting on Thursday 26 November
1. The product and the trial. Nightjar Notes, supplied by Nightjar Clinical Pty Ltd (account manager Tom Mannering). Trial since Monday 5 October 2026 with Dr Voss and Dr Eze. The scribe was offered in 188 consultations and 14 patients declined (7.4%).1
2. Is it a medical device? The vendor’s sales deck describes a Clinical insights tab that suggests likely diagnoses and tests. It is switched on by default and Dr Eze has opened it twice. Mr Mannering could not tell us whether any part of Nightjar Notes is on the ARTG and said he would find out.2
3. Where the data goes. Audio is stored in an Australian region and deleted after 30 days. Transcripts and draft notes stay in the vendor account until we delete them. The language model that drafts the note is run by a sub-processor in the United States. Mr Mannering says nothing is used to train models by default, but de-identified text may be used to improve the service unless we opt out in writing. No sub-processor list has been supplied.3
4. A patient’s request. In Mrs Margaret Ellison’s consultation on 12 October she asked that her drinking be left out of the notes. Dr Voss removed the line from the clinical note. The transcript in the vendor account still contains it. The audio was due to be deleted on 11 November; the transcript has no deletion date.4
5. Accuracy. Dr Eze checked 40 draft notes against the audio. Seven (17.5%) needed a clinical correction. One recorded no drug allergies when the patient had not been asked, one gave the wrong side for a knee injection and one said a medicine was stopped when the patient had stopped it herself.5
6. Consent. Dr Voss and Dr Eze ask for verbal consent at the start of each consultation and record scribe consent in the note. There is no patient information sheet and no sign at reception. Nothing was recorded for the 14 patients who declined.6
7. Other AI use found. On 10 November Sam Kovacs RN told Helen Tran that Sam pastes immunisation recall letters into a free chatbot on a personal phone to make the wording plainer. The letters carry the patient’s name and date of birth. This is outside the trial.7
8. Contract and insurance. The draft agreement limits Nightjar’s liability to the fees paid in the previous 12 months, requires breach notice “without undue delay” and gives us no right to audit sub-processors. On 12 November Helen Tran asked Darren Bastow at Moreton Gate Health Indemnity whether the practice’s policy responds to AI tools. No reply yet.8
9. Recommendation. Do not extend beyond the two trial doctors. Before any extension we need in writing: the Clinical insights tab switched off, the ARTG status, the sub-processor list and countries, an opt-out from any use of our data to improve the service, deletion of transcripts on request, multifactor sign-in, a patient information sheet and a written practice policy. Decision at the practice meeting on Thursday 26 November.9
Helen Tran, Practice Manager, with Dr Chidi Eze. Not yet reviewed by Dr Helena Voss.10
An invented assessment. The vendor, its product and its staff are fictional; the practice and patients are the same invented cast as the other healthcare documents. Hover or tap any highlighted item for the analysis; the full list is below.
AI scribe vendor assessment compiler
An Agent+ that compiles the vendor assessment from the vendor's own documents, your trial notes and the checklist your practice uses. It sets each answer beside its source, records where data is held and processed as the vendor states it, and lists the questions still open.
What it does
- Reads the vendor's documents and your trial notes
- Compares them against your assessment checklist
- Lists unanswered questions and gaps for follow-up
What stays with your people. The practice manager and clinical lead decide whether to adopt, extend or stop. Legal and privacy questions go to your own adviser or medical defence organisation.
An Agent+ is built once and anyone in the practice can run it. It works from your own documents and runs on Ermos Dominion, the AI unit in your office, or on Ermos Edge, your own instance in an Australian data centre.
What leaves the practice
- 1The vendor, the contact and the trial numbers
Names a vendor and its account manager and gives the practice’s own trial figures. The figures are aggregate, but they show how the trial is going. Ahpra says practitioners should be aware of the governance arrangements their practice sets up to oversee the use and monitoring of AI, and that all tools should be tested for fitness for purpose before clinical use. Patient data guide.
Frontier Models (ChatGPT, Claude) A named vendor contact and the trial figures transmitted Ermos Edge Stays on your instance; a permitted excerpt may go to the named Australian provider, which is contractually barred from keeping it Ermos Dominion Stays on the unit; low risk alone Use case AI scribe vendor assessment compiler. Compares a scribe vendor's documents with your own assessment checklist and lists the gaps and unanswered questions for the practice manager to follow up. See the use case - 2Whether the scribe is a medical device
The TGA says a digital scribe that only records, transcribes and drafts documentation for the clinician to review is not a medical device, and that one that identifies, recommends or suggests diagnoses, treatments or investigations is likely to be, and must be on the ARTG unless exempt. A suggestion tab is the kind of function the TGA points to, and the TGA says a manufacturer’s claims, including in sales material, can show a therapeutic purpose. The TGA asks health professionals to check regularly whether the intended purpose meets the definition and whether updates have added functions. Ahpra adds that TGA approval does not change the practitioner’s duty of human oversight. A draft written by an AI tool can state a product’s regulatory status wrongly, so the answer has to come from the vendor in writing and the ARTG. Patient data guide.
Frontier Models (ChatGPT, Claude) The TGA’s published tests are public; the practice’s unanswered question is a small leak Ermos Edge Stays on your instance, but a person must still ask the vendor and check the ARTG Ermos Dominion Stays on the unit, but a person must still ask the vendor and check the ARTG Use case AI scribe vendor assessment compiler. Compares a scribe vendor's documents with your own assessment checklist and lists the gaps and unanswered questions for the practice manager to follow up. See the use case - 3Where the audio, the transcript and the model are
Describes storage in Australia, a model run by a sub-processor in the United States and a possible use of de-identified text. Under APP 8.1 an entity must take reasonable steps before disclosing personal information to an overseas recipient to ensure it does not breach the APPs (other than APP 1), subject to the exceptions in APP 8.2. HPP 14 in NSW says an organisation must not transfer health information to a body outside NSW unless, for example, it reasonably believes the recipient is subject to a law, scheme or contract that upholds similar principles, or the individual consents. HPP 5(1)(d) asks that everything reasonably within the organisation’s power is done to prevent unauthorised use or disclosure when information is given to a person providing a service to it. The RACGP says vendors may use data to train models or on-sell it and ideally data is stored in Australia. Whether this flow is a disclosure or a transfer is a question for the practice’s adviser. Privacy Act guide.
Frontier Models (ChatGPT, Claude) The practice’s data-flow gaps and the vendor’s sub-processor arrangement transmitted before they are resolved Ermos Edge Stays on your instance, but where the scribe vendor sends the audio is for the practice to settle with the vendor Ermos Dominion Stays on the unit, but where the scribe vendor sends the audio is for the practice to settle with the vendor Use case AI scribe vendor assessment compiler. Compares a scribe vendor's documents with your own assessment checklist and lists the gaps and unanswered questions for the practice manager to follow up. See the use case - 4A patient’s request and the copy that remains
Names a patient and her alcohol use, which is health information. A patient asked for the line to be left out of her notes and the scribe kept it. Whether alcohol use that bears on her care can be left out of the clinical note is for the doctor to decide: Good medical practice 10.5.1 asks for records that report relevant details of clinical history, and this page does not say how a patient’s request is to be weighed against that. Whatever the doctor decides, a copy that stays in a vendor account with no deletion date raises HPP 5(1)(a) (keep health information no longer than necessary) and APP 11.2 (destroy or de-identify what is no longer needed, unless a law requires it to be kept). Patient data guide.
Frontier Models (ChatGPT, Claude) A named patient and her drinking transmitted, the very information she asked to keep out of the record Ermos Edge Stays on your instance, but the doctor still decides what belongs in the clinical note Ermos Dominion Stays on the unit, but the doctor still decides what belongs in the clinical note Use case AI scribe vendor assessment compiler. Compares a scribe vendor's documents with your own assessment checklist and lists the gaps and unanswered questions for the practice manager to follow up. See the use case - 5The error audit
Counts the errors in a sample of notes and describes three of them. The patient is not named, but the errors are clinical: an allergy status nobody asked about, a wrong side and a medicine stopped by the patient. Ahpra says the practitioner is responsible for checking the accuracy and relevance of records created with generative AI, the TGA says the health professional is responsible for verifying the accuracy of information entered into a patient’s record, and Good medical practice 10.5.1 asks for accurate records. Patient data guide.
Frontier Models (ChatGPT, Claude) Clinical errors in a sample of notes transmitted; no patient named Ermos Edge Stays on your instance, but the AI can still draft a wrong note and a person must check it Ermos Dominion Stays on the unit, but the AI can still draft a wrong note and a person must check it Use case AI scribe vendor assessment compiler. Compares a scribe vendor's documents with your own assessment checklist and lists the gaps and unanswered questions for the practice manager to follow up. See the use case - 6The consent process and the decline figure
Describes how consent is obtained and what is missing. Ahpra says to obtain informed consent and ideally note the response in the health record, and that scribing software should include an explicit consent step. The TGA says patients can withdraw consent and ask for another method of recording. Good medical practice 4.5 describes informed consent as a voluntary decision made with knowledge and understanding of the benefits and risks. Ahpra warns of possible criminal implications if consent is not obtained before a private conversation is recorded, and the TGA and the RACGP point to state and territory surveillance devices laws; this page does not interpret them. The text names no patient. Patient data guide.
Frontier Models (ChatGPT, Claude) A consent routine and its gaps described, with no patient named; low risk on its own Ermos Edge Not about where data goes Ermos Dominion Not about where data goes Use case AI scribe vendor assessment compiler. Compares a scribe vendor's documents with your own assessment checklist and lists the gaps and unanswered questions for the practice manager to follow up. See the use case - 7A named nurse’s use of a free chatbot
Names an employee and a practice that sits outside the trial, with no written practice policy yet to measure it against. Ahpra says that when a publicly available generative AI tool is used, personal information can enter the public domain. A patient’s name and date of birth in a recall letter is health information once it shows the person is due for a service (Privacy Act s 6FA(a)(iii) and (b)), and APP 1.2 asks an entity to take reasonable steps to implement practices that ensure it complies with the APPs. Privacy Act guide.
Frontier Models (ChatGPT, Claude) A named employee and a use of an outside chatbot with patient details transmitted Ermos Edge Stays on your instance, but staff still need a policy and training to stop them using a personal chatbot Ermos Dominion Stays on the unit, but staff still need a policy and training to stop them using a personal chatbot Use case AI scribe vendor assessment compiler. Compares a scribe vendor's documents with your own assessment checklist and lists the gaps and unanswered questions for the practice manager to follow up. See the use case - 8The contract terms and the insurer
Sets out liability, breach notice and audit terms from a draft agreement and names an insurer and its adviser. Contract terms from an unsigned draft are commercially sensitive. Ahpra asks practitioners to consult their indemnity provider if they are unsure whether AI tools used in practice are covered, and the RACGP asks practices to get independent legal advice or consult their medical defence organisation on security, privacy and contract aspects. A draft can summarise a clause wrongly. HPP 5(1)(d) is the NSW rule on information given to a service provider.
Frontier Models (ChatGPT, Claude) Draft contract terms and a named insurer adviser transmitted Ermos Edge Stays on your instance, but a person must still check any clause summary against the contract Ermos Dominion Stays on the unit, but a person must still check any clause summary against the contract Use case AI scribe vendor assessment compiler. Compares a scribe vendor's documents with your own assessment checklist and lists the gaps and unanswered questions for the practice manager to follow up. See the use case - 9The recommendation and its conditions
The practice’s own plan. Several conditions track the RACGP fact sheet (assurances on how data is encrypted, stored and destroyed, secondary use for training, multifactor authentication) and the TGA list of things to check. The conditions name no patient. The plan and the decision date are internal until the practice acts on them.
Frontier Models (ChatGPT, Claude) The practice’s own plan and dates; low risk on its own Ermos Edge Not about where data goes Ermos Dominion Not about where data goes Use case AI scribe vendor assessment compiler. Compares a scribe vendor's documents with your own assessment checklist and lists the gaps and unanswered questions for the practice manager to follow up. See the use case - 10The authors and the review status
Names the practice manager and the doctor who prepared the assessment and records that the principal has not yet reviewed it. It shows who is accountable for what the document says and what stage it has reached.
Frontier Models (ChatGPT, Claude) Names two staff and the review status Ermos Edge Not about where data goes Ermos Dominion Not about where data goes Use case AI scribe vendor assessment compiler. Compares a scribe vendor's documents with your own assessment checklist and lists the gaps and unanswered questions for the practice manager to follow up. See the use case
What happens in Frontier Models (ChatGPT, Claude)
- The whole document is transmitted to the provider’s servers. On personal plans the user cannot choose where it is stored or processed, and for the major providers that has generally meant outside Australia, under that country’s legal process. Some business plans offer storage in Australia, but the provider still receives the content.
- On consumer tiers, prompts can be retained and used to train future models.
- The provider’s staff and systems can reach the content under the provider’s own policies.
- Once sent, the practice no longer controls it. Retention and deletion happen on the provider’s terms.
The same document on Ermos
Ermos Dominion
The document is drafted on a unit in your office, from your own templates and the patient record. Every marked item stays on the unit, which keeps working with the internet switched off. Nothing in it is transmitted to an AI provider, so there is no disclosure to analyse: the confidentiality work is the same as for your file server.
Ermos Edge
Your documents, the index and your access boundaries stay on your own instance in an Australian data centre. When a staff member asks for this draft, only their question and the excerpts they are permitted to see go to a named Australian provider, who is contractually barred from keeping the content or training any model on it. Nothing is processed outside Australia.
| Frontier Models (ChatGPT, Claude) | Ermos Edge | Ermos Dominion | |
|---|---|---|---|
| Where the document is processed | Provider servers, generally offshore on personal plans | Your instance in an Australian data centre; answers generated in Australia | The unit in your office |
| Who can read it | Provider staff and systems, under policy | Your staff, within the boundaries you set. The named Australian provider processes only permitted excerpts, under contract, and keeps nothing | Your staff only |
| Is it kept | On the provider’s terms | Kept on your instance; the provider is barred from retaining it | Stays on the unit |
| Can it train a model | On consumer tiers, yes | No, by contract | Only your own unit learns from it, on your hardware |
Part of Use cases in healthcare practices. See how this works in practice: Practice policies and forms on Ermos.
Common questions
Do ChatGPT and Claude use the information in an AI scribe vendor assessment?
Is it safe to paste an AI scribe vendor assessment into ChatGPT or Claude?
What if staff use their own ChatGPT or Claude account for an AI scribe vendor assessment?
How can a medical practice use AI for an AI scribe vendor assessment without sending it to ChatGPT or Claude?
When is an AI scribe regulated as a medical device in Australia?
What does Ahpra expect of a doctor who uses an AI scribe?
What should a practice ask an AI scribe vendor?
See Dominion answering from your own documents.
A demonstration takes 30 minutes, uses no client data, and comes with no obligation.
Book a demo