Solutions · Healthcare

Risk scores a clinician can question.

For the teams that build and govern clinical and operational models. See why a patient was flagged, check that an imaging model looked at the anatomy and not the artefact, and notice when a model meets a population it was not validated on.

Data
99,343inpatient encounters from 130 US hospitals; 69,990 patients
Discrimination
0.665 AUCon 19,907 encounters from patients the model never saw
Who gets a call
26%of flagged discharges return within 30 days, against 11% of all discharges
A new unit
PSI 6.73on age, flagged in the first batch from a geriatric ward

Where it fits

For clinical and operational models, and the people who govern them.

Care transitions

Readmission and deterioration risk

Why this patient was flagged, from the model's own inputs, in terms a care team can check against the chart.

Imaging

Image classifiers

Grad-CAM, occlusion and LIME on a classifier's input, with a check of what the model says when the region it relied on is hidden.

Operations

Length of stay, no-shows and demand

Numeric and yes/no predictions explained in their own units, and watched for drift as case mix changes.

Payers

Risk and utilisation models

A stored explanation for each score, retrievable by your own case id.

Go-live

Validation at a new site

Compare the patients a model meets in production with the population it was validated on, input by input.

Research

Cohort studies and model papers

Exact, seeded and exportable results that another group can reproduce.

A worked example

A readmission model, opened up.

Public dataDiabetes 130-US Hospitals, 1999–2008, UCI Machine Learning Repository (Strack et al., 2014).readmission-30dscikit-learn · gradient boosting

A gradient-boosted model estimates the risk that a patient with diabetes is readmitted within 30 days of discharge, from ten inputs known at discharge. Race and gender are left out, and patients are kept apart between training and test. A discharge is flagged for a follow-up call at 20%. The data is de-identified and public; every figure below is what the Kaimei API returned.

readmission-30d · Explain
scikit-learn · gradient boosting
raiseslowers
Diabetes 130-US Hospitals, 1999–2008 POST https://api.kaimei.ai/v1/explanations

One patient

A flag that can be checked against the chart.

Discharge A is scored at 59.7%, against an average of 12.2%. Almost all of it is history: 8 inpatient stays in the past year add +32.8 points, and 5 emergency visits add +6.4.

Discharge C sits just over the line at 21.3%. Its largest push is 2 prior stays (+5.7), then an 8-day stay (+1.8). A nurse can see at a glance what the model saw, and what it did not.

At this threshold the model flags 7.6% of discharges and finds 18% of the readmissions. That trade-off is the hospital's to set; Kaimei reports it.

Global importance · readmission-30dpermutation · 2,000 rows
Inpatient stays, past year38.0%
Discharged to19.8%
Age band7.1%
Length of stay6.9%
Lab procedures6.5%
Diagnoses recorded6.1%
Medications5.8%
Emergency visits, past year4.9%
prior_inpatient is the most influential input, followed by discharge and age. Together they carry 65% of the measured importance (permutation importance over 2000 rows).

What it learned

Plausible where it is strong, honest where it is not.

prior_inpatient is the most influential input, followed by discharge and age. Together they carry 65% of the measured importance (permutation importance over 2000 rows).

The weaker effects are reported as they are. Length of stay and age do not move risk in one direction, and Kaimei says so rather than smoothing them over.

  • Higher prior_inpatient increases the probability of readmitted, most sharply between 2 and 4 (range 9% → 23%).
  • discharge = care facility pushes the probability of readmitted highest and discharge = home lowest (range 10% → 16%).
  • days_in_hospital has a non-monotonic effect on the probability of readmitted: it peaks around 8 days (range 10% → 12%).
Robustness · readmission-30d300 rows · 5%-σ noise

0.6% of decisions flip under 5%-σ input noise. Most fragile inputs: lab_procedures, medications.

On 2,000 reference rows the model predicts readmitted 7% of the time; 46% of cases sit within ±0.10 of the decision threshold.

With a 20% threshold and an average risk of 12%, many patients sit near the line. The behaviour insight counts them.

A new population

The model meets patients it was not validated on.

Five hundred held-out encounters are logged as normal traffic, and every input is stable. Then a geriatric unit joins the deployment: five hundred encounters of patients aged 85 and over, drawn from the same data. Age leaves the reference range at once, and discharge destination follows. The model flags 7.6% of them against 5.8% before. Whether its validation still holds for this ward is a clinical governance question. Kaimei makes sure it is asked on the first day.

Normal trafficstable
Length of stay0.03
Diagnoses recorded0.02
Medications0.02
Lab procedures0.01
Age band0.01
Medication changed0.01
Recent inputs match the reference population (max PSI 0.03). 500 logged predictions against 5,000 reference rows.
The geriatric unitsignificant
Age band6.73
Discharged to0.36
Diagnoses recorded0.09
Other procedures0.09
Length of stay0.03
Medications0.03
Recent traffic has significant drift (PSI 6.73). Shifted inputs: age, discharge. 500 logged predictions against 5,000 reference rows.

Population stability index per input. Bars are cut off at 0.5; the line marks 0.25, where Kaimei calls drift significant.

Imaging

Did it look at the anatomy, or at the label in the corner?

For an image classifier Kaimei draws where the evidence for a class sits, outlines the smallest region holding half of it, and then asks the model twice more: with that region hidden, and with only that region shown.

If hiding the focus barely changes the answer, the model is relying on something else in the picture. One plain sentence states where the model looked and whether that region alone decides.

  • Grad-CAM for uploaded Keras and PyTorch convolutional networks
  • Occlusion and LIME for any image classifier, including one behind your own endpoint
  • Classification only: not segmentation or detection
Follow one image through a network
An image explanation, as the API writes itfrom the docs

The model looks mostly at the centre of the photo: half of its evidence for dog sits in 12% of the picture. That region alone is enough to say dog (P(dog) = 99.7%); with it hidden, P(dog) falls from 90.4% to 5.0%.

The example in the guide to reading explanations, from the built-in demonstration model. The same check runs on any image classifier you connect.

Privacy

Patient data stays where you decide.

Run Kaimei yourself

The API runs on your own infrastructure, against your own database. Patient-level inputs never leave your environment.

Self-hosting

Keep the model at home

Point Kaimei at a scoring endpoint you host. It sends inputs and reads scores; the model itself is never uploaded.

inference="remote"

Share shapes, not weights

Upload an architecture with shapes and statistics only. It can be drawn and reviewed, and the weights stay with you.

weights="summary"

Who it is for

Builders, governance and research.

01

Clinical informatics and data science

Upload a fitted model in one call. See what it relies on, where its effects are weak, and how a retrain differs from the version in use.

02

Clinical governance and quality

Read why a patient was flagged in plain terms, and be told when the population a model serves is no longer the one it was validated on.

03

Research and regulatory

Exact, seeded results for a paper or a submission file, stored with the model version and exportable as JSON.

Rules and guidance

Context for regulated and reimbursed settings.

Rule or guidanceWhat it concernsWhere Kaimei helps
FDA guiding principles for machine learning in medical devicesGood Machine Learning Practice (2021) and Transparency for Machine Learning-Enabled Medical Devices (2024), with Health Canada and the MHRADevelopment practice, monitoring of deployed models, and clear information for the people who use them.Documented model behaviour, per-prediction explanations and monitoring of deployed models for the developer's files.
Predetermined change control plansFDA final guidance for AI-enabled device software functions, December 2024 (current version August 2025); the guidance for all devices is still a draftHow planned changes to an AI-enabled device are described and verified.Version-by-version comparison of what a model learned, stored with each release.
EU AI ActAI that is a safety component of a medical device is high-risk under Annex I; those obligations apply from 2 August 2028Risk management, logging, transparency, human oversight, accuracy and robustness.Logged predictions and explanations, robustness and drift analyses.
Hospital Readmissions Reduction ProgramCMS; payment reductions capped at 3%Why hospitals model 30-day readmission in the first place.A readmission model whose flags can be explained and whose population is watched.

Coming soon

Next on the roadmap.

What we are building now, and what to use until each one ships.

  • Coming soonSubgroup performance and fairness

    Accuracy, calibration and attributions compared across the patient groups you define. Until then, explanations help investigate a concern; they are not a test.

  • Coming soonClinical notes and text

    Word- and passage-level attributions for models that read free text, such as discharge summaries and referral letters.

  • Coming soonDetection and segmentation

    Explanations for the boxes and masks a model draws on a scan, not only for its label. Image classification works today.

  • Coming soonData quality checks

    Missing values, out-of-range readings and unit changes caught as predictions are logged. Until then, Kaimei compares live inputs with the reference population.

See it on your own model.

Bring a clinical or operational model. Start with it running entirely inside your environment.