Chat with FHIR data: how an LLM reads a patient chart

An LLM has no native understanding of FHIR. When you ask an agent to look up a patient, what happens is:

1. The agent calls a FHIR API (Medplum, in Last EHR's case) and fetches structured resources: Patient, Condition, AllergyIntolerance, Observation, Communication, MedicationRequest, Immunization.
2. Those resources are projected into a compact summary: a list of conditions, allergies, medications with dosages, recent vitals and labs, and prior notes.
3. That summary, along with your question, goes to the language model as a prompt.
4. The model reads the context and responds.

The model itself never touches your FHIR backend. The agent sits in the middle and does the translation. Your backend does not need to know about the model provider, and the model never makes a FHIR API call.

Raw FHIR is powerful but verbose. One patient's chart can be dozens of resources, each with codes, references, and metadata a chat does not need. The agent's job is to extract the clinically relevant parts and present them cleanly.

Last EHR's show_patient_info tool does exactly this: it fetches demographics, conditions, allergies, observations (sorted by date), notes, medications, and immunizations, then projects each resource down to the fields that matter (a label, a value, a date). The UI renders those as structured cards; the same projection becomes the model's context.

This grounding step is where ambiguity starts. If an observation is missing a date, or a dosage lives in free text instead of structured fields, the model's context gets weaker. Consistent coding and complete fields in the backend are what make the difference between a model that reflects the chart and one that guesses.

The read path is a privacy question. Whenever the agent opens a chart, that context goes to the model provider with no approval step. This is by design (the agent needs the picture to be useful), but it means your provider choice and agreements govern reads, not any UI control.

The write path is a safety question. When the agent proposes a write, it does not execute. Last EHR shows an approval card with exactly what will be saved, and only your click runs the write. The backend's AccessPolicy then bounds what the signed-in user can actually create. The full mechanics are on the approval-gated writes page, including what the gate does not protect against (approval fatigue, hallucinated content, and the ungated read path).

The fastest way to understand the flow is the live demo. No sign-up; everything runs on seeded synthetic patients. Try:

• "Find patients named Smith": the agent searches and shows results with a View record button.
• "Show me Jane Smith's chart": the full chart renders as cards: demographics, conditions, medications, observations, immunizations, notes.
• "Record a heart rate of 72 bpm for Maria Garcia": the agent proposes an Observation and stops at the approval card. Nothing saves until you click Approve.

On the public demo, writes are tagged per visitor session, so you only ever see the seed data plus your own edits. That keeps a shared demo from becoming a mess of strangers' writes; it is a demo mechanism, not a product feature.

Last EHR is open-source alpha software, and it is not a HIPAA-covered service. If you point it at real patient data, compliance is on you. The essentials:

• Reads require a provider agreement. Chart context goes to your model provider under your key. Some providers offer BAAs covering API traffic on qualifying plans (OpenAI and Anthropic among them); consumer plans do not qualify.
• The backend must be HIPAA-eligible. Medplum offers HIPAA-eligible hosted plans with BAAs; check their current terms. If you self-host a FHIR server, securing it is your job.
• The layer stores nothing. Last EHR keeps no database and no copy of chart data. Data lives in your backend and, during a request, at your model provider.
• Synthetic data is the right default. The seed script loads four synthetic patients with realistic but fake charts. The live demo uses exactly that data.

Last EHR is not the only project connecting language models and FHIR, and it is worth knowing the categories.

Research projects such as LLMonFHIR (an academic open-source effort) study how to ground LLMs on FHIR data: terminology, normalization, evaluation. Useful reading if you want the theory behind context assembly.

MCP servers, including Medplum's own, expose FHIR data to MCP-compatible clients over the Model Context Protocol. That is programmatic plumbing: the right layer if you are building your own agent and want standardized access to a backend. It does not come with a clinician-facing UI or an approval gate.

Last EHR is an application layer: a ready-to-run chat UI with four FHIR tools and the approval card built in. It runs on Medplum today; other FHIR backends are a goal, not yet supported. It is not a replacement for MCP servers or the research; it is what you run when you want a working, permissioned chat over a chart without building one.

You need a Medplum project (their free tier works for evaluation), a model API key (OpenAI or Anthropic), and the code:

git clone https://github.com/cbetz/last-ehr.git
cd last-ehr
npm install
cp .env.example .env.local   # add your Medplum + model keys
npm run seed                 # load synthetic patients
npm run dev                  # http://localhost:3000/demo

The FHIR tools live in lib/ai/tools.ts and the agent route in app/api/chat/route.ts. Multi-tenancy, users, and permissions come from your Medplum project (Project, ProjectMembership, AccessPolicy); the layer does not reimplement them. Porting to Aidbox, HAPI, or Firely means adapting the tool implementations; that is the intended seam, and contributions are welcome. Setup details are on the Medplum AI agent page.