The 3-question test African health AI fails quite often

A foreign AI scribe sat on my computer for months. It was meant to transcribe my consultations, save me time at the end of long clinic days, and produce cleaner notes than I could type by hand. It could not understand my patients. When a mother in Kibera switched between Swahili and Sheng to describe her child’s pain, the scribe produced a transcript that looked confident and was almost entirely wrong. When an older patient from Kisii broke into vernacular mid-sentence, the scribe simply stopped listening. The software ran. The consultation failed.

This is not a vendor failure. It is a design failure. And it is the rule, not the exception.

By most credible estimates from health system reviews and donor evaluations, around 80 per cent of AI health pilots in Africa do not move past pilot. The technology is rarely the reason. The reasons are the ones we refuse to name. No clinician on the design team. No clause on who owns the data after the pilot ends. No local accountability when the algorithm gets it wrong. No training budget that survives the launch event.

We do not have an AI problem in African health. We have a co-design problem dressed up as an innovation problem.

This argument is uncomfortable to make in a year when nearly every health summit on the continent is framed around the promise of AI. Africa CDC, WHO Africa and Kenya’s draft AI Bill of 2026 all speak about AI as the lever that will close our diagnostic gap, fix our workforce shortage and modernise our health systems. The framing is intoxicating. It is also incomplete.

What the framing leaves out is who the AI is being built for, and on whose terms.

Consider what is happening only a few kilometres apart in Nairobi. Aga Khan University, the University of Michigan Center for Global Health Equity and Tenwek Mission Hospital in Bomet are running an NIH-funded AI project for colorectal cancer diagnosis. Kenyan tissue samples. Kenyan principal investigators, including Professors Mansoor Saleh and Shahin Sayed at Aga Khan. A rural mission hospital treated as a principal site, not a pilot site. A multi-year partnership, not a one-time landing.

This is what right looks like. We should name it more often, and fund more of it.

The argument is not that AI in African health is broken. The argument is that we want more projects that pass a basic test, and fewer that fail it. The test has three questions.

The first is who designed it. Not who funded the design. Who sat in the room when the problem was defined? If no African clinician, patient advocate or frontline health worker was in that room from week one, the tool was deployed in Africa. It was not built for it.

The second is who owns the data. If training, validation and inference data leave the continent, and the contract does not give African institutions equal rights to the resulting model, we are paying foreign vendors to train their own products on our patients. This is not capacity building. It is a procurement strategy.

The third is who is accountable when it fails. If the tool produces a missed sarcoma, a delayed referral, an incorrect triage, who answers? If the only answer is the local clinician using the tool, the governance is a fiction.

I have lived this argument from inside an African consortium. Over the past two years I served as co-principal investigator on an AI and machine learning prosthetics project under the African Engineering and Technology Network, a multi-country consortium hosted by Carnegie Mellon University Africa.

The clinical questions were defined in Nairobi, by clinicians who fit and adjust prostheses in our own outpatient clinics. The engineering was led from Dedan Kimathi University of Technology in Nyeri, working alongside teams at partner universities elsewhere on the continent. Grant resources for the Kenyan arm came through to a Kenyan institution. The AI components were not imported. They were co-developed.

The project taught me what the three-question test looks like in practice, and it is the standard against which I now read every AI proposal that crosses my desk.

Most tools sold to African health systems as AI cannot pass even one of these three questions. They carry an African name, an African pilot site, an African testimonial, and still fail the test. The label is not the test. The questions are the test.

We can do better. We are already doing better, in places. The scribe that failed me has an answer, and it is African. Dr Tobi Olatunji, a Nigerian physician turned machine learning scientist, founded Intron Health to build clinical voice AI for African accents and code-switching.

Its Sahara model is trained on what is now the largest clinical speech database on the continent, drawn from contributors across 29 African countries, and is in use in dozens of hospitals across Nigeria, Kenya and South Africa. The mother in Kibera and the patient from Kisii would not have broken it.

Olatunji is not alone. Dr Rose Nakasi’s malaria microscopy work at Makerere. Dr William Wasswa’s cervical cancer screening tool at Mbarara. The AfriMed-QA medical reasoning benchmark built from over 60 African medical schools. Digital pathology systems validated on Kenyan and Ugandan patient data. These are not the names that headline summits. They should be.

Kenya is now drafting its AI Bill. The Senate has invited submissions. The three questions above belong in those submissions, and in the procurement guidance, research ethics review and public tender documents that will follow. Co-design from week one. African data ownership. African principals, not African participants.

The next time a foreign delegation lands in Nairobi to talk about AI for Africa, we should be ready with three questions before they finish the welcome address. If the answers are not clean, we should be ready to say no. Politely. Firmly. And with our own builders in the room.

Surgeon, writer and advocate of healthcare reform and leadership in Africa