Updated October 14, 2026
AI should be evaluated against a specific barrier to care and the people affected by it. Faster administration or a stronger model score may be useful, but neither establishes that a service has become more equitable.
The Robert Wood Johnson Foundation’s explanation of health equity connects fair opportunities for health with removing social and institutional barriers. That framing places technology within a wider effort involving services, resources and policy. It also gives leaders a practical starting point: identify the barrier before selecting a tool.
Define a bounded use case
Consider administrative assistance, such as preparing appointment reminders from approved templates or flagging incomplete referral paperwork for staff review. Define the permitted inputs, output and next action. Check whether a simpler change to the process would address the problem.
Administrative work can still affect access. A scheduling system that repeatedly fails to reach someone may delay their care. A referral tool can misroute a request. Include these consequences in the assessment, even when the system does not provide medical advice.
Clinical uses, including diagnosis, triage and treatment recommendations, require clinical expertise, evidence for the intended population and setting, and the relevant regulatory and institutional review. An administrative pilot cannot establish a tool’s clinical safety or effectiveness.
Examine what the model is actually predicting
A convenient measure may represent access to services more closely than need. In a 2019 study, Obermeyer and colleagues examined an algorithm that used healthcare spending as a proxy for health need. Unequal access to care contributed to racial bias in its allocation of additional support.
The practical lesson is to inspect the target and its assumptions. Ask what an apparently successful prediction means for patients. Review who is missing from the data and whether historical decisions reflect barriers the new service is intended to reduce.
Validate across the people who will use the service
Before deployment, compare the proposed workflow with current practice using relevant local data. Examine performance across patient groups, languages, sites and access channels where appropriate and permitted. Investigate errors that could delay care or prevent people receiving support.
Report sample sizes and uncertainty. A small subgroup cannot support a confident claim of equal performance. When evidence is insufficient, narrow the intended use or collect additional evidence before extending it.
Include patients, community representatives and frontline staff in defining success. Alongside technical measures, consider completed appointments, time to assistance, unresolved requests and the burden placed on patients. These are proposed evaluation measures, not claims of outcomes already achieved.
Make oversight and access practical
The WHO’s guidance on AI for health places ethics, human rights and accountability at the centre of design and use. Its six guiding principles emphasise human control, safety, transparency and equitable access.
Put those principles into the workflow. Name the accountable service owner. Give trained reviewers time, context and authority to intervene. Explain the tool’s role in accessible language and provide a route to correction or human assistance.
Retain workable alternatives for people who cannot use a digital channel. Confirm appropriate data access and privacy protections. Monitor real use, complaints and differences between groups, with a clear process for pausing or changing the service.
Claims about better access or health outcomes should follow evidence from the deployed setting. Until then, describe the intended benefit and the evaluation still needed.
