At Junction 2026 I called the real Gemini API on fake clinic messages and fine-tuned Gemma into a local decision model, reranker and transcriber, so real patient messages never leave the clinic. It won one of the three Best Use of Gemma prizes.
Published
AI
Last weekend at Junction 2026 in Espoo, my project won Google DeepMind’s Best Use of Gemma challenge, as one of three winning teams. The prize has no ranking between the three, and each team gets €1,000.
The project answers one question: how do you use a strong AI model on data that must never go to a public API? My example was a clinic inbox full of patient messages. Gemini sorts them well, but you can’t send it real patient messages. So I called the real Gemini API on fake messages, and used its answers to fine-tune small Gemma models that run on a laptop.
I turned Gemma into three things it isn’t out of the box:

Rahmy was a team of one: me.
A clinic inbox fills up with refill requests, billing questions, and now and then a voicemail where someone’s dad can’t speak properly. Someone has to read every one and decide how urgent it is and who should handle it.
A model like Gemini does this well. But patient messages, health records and legal or HR cases are exactly the data you’re not allowed to send to someone else’s server. The models you can run locally, like a 1B Gemma, are small and don’t know the task.
The fix is distillation: a big “teacher” model labels examples, and a small “student” model is fine-tuned to give the same answers. The trick is what goes into the teacher. It’s the real, production Gemini API, but it only ever sees fake data. That makes it safe to call as often as you like, and the labels it gives are the real thing.
I used this loop three times. For the decision model and the reranker, Gemini labels fake messages. For the transcriber, Gemini writes fake voicemails and speaks them, so the audio and the true text both come from the API.
Fake data is only useful if it covers the real cases. So each message starts as a tuple of six traits: topic, urgency, who is writing, how clear they are, their style, and the channel. Here is one tuple:
prescription refill · routine · a parent of a young child · very worried about a minor problem · formal · voicemail transcript
Gemini turns each tuple into a message, and a realism filter drops the ones that read wrong. About 85% pass. I split the test set in two: coverage messages that look like normal traffic, and challenge messages built to be tricky, like that worried parent asking for a routine refill.
The Gemini Decisions API answers typed questions and gives a probability for every option, not only the top answer. I used one question of each type:
| Question | Type | Asks |
|---|---|---|
triage |
choice | emergency, same day, routine or admin? |
route |
choice | nurse, doctor, pharmacy, front desk or billing? |
callback_today |
yes/no | must a clinician phone back today? |
severity |
score | none to critical, five levels |
The probabilities matter. A message that Gemini rates 55% routine and 30% same day teaches the student something that a plain “routine” label doesn’t.
I trained three students on 1,125 messages and tested them on 296 they never saw:
| Student | Agrees with Gemini | Train time | Time per message |
|---|---|---|---|
| Always give Gemini’s average answer | 56.8% | – | – |
| Plain Gemma 3 1B, no training | 40.3% | – | – |
| Gemma 3 1B + LoRA | 91.7% | 16 min | 92 ms |
| EmbeddingGemma + LoRA | 91.0% | 2.5 min | 5.6 ms |
| EmbeddingGemma, frozen | 86.7% | 10 s | 4.4 ms |
“Agrees” means the student’s top answer is Gemini’s top answer, averaged over the four questions. Times are on an NVIDIA A10.
Plain Gemma is worse than a constant guess. Untrained, Gemma 3 1B scores 40.3%, below the 56.8% you’d get by always giving Gemini’s average answer. A small model doesn’t know the task. After training it does.
The small embedding model is almost as good and 16 times faster. The EmbeddingGemma student is 0.7 points behind the Gemma 3 1B student, trains in a sixth of the time and answers in 5.6 ms instead of 92 ms. For a busy inbox, that’s the one I shipped in my demo app.
More fake data kept helping. On the same 137 test messages, the Gemma student went from 88.9% to 90.9% with a bigger adapter, and to 92.3% with twice the data. More fake messages only cost Gemini calls, so this is the cheapest lever.
Newer wasn’t better. EmbeddingGemma 2 scored below the first EmbeddingGemma, at 88.8% against 91.0%. It lost on search as well.
A clinic inbox needs more than triage, so I turned Gemma into two more tools.
The reranker. Gemini wrote 200 fake staff searches and graded every message against every search. On 36 test searches, a Gemma 3 1B student that reads the search and the message together reaches an NDCG@10 of 0.800 (1.0 means it ranks exactly like Gemini). Plain EmbeddingGemma gets 0.652 and plain Gemma 0.451. An EmbeddingGemma student is a bit worse at 0.776, but its message vectors can be stored, so a search takes 0.17 ms per message instead of 20 ms. The test set is small, so treat those numbers as rough.
The transcriber. EmbeddingGemma 2 has an audio encoder, so it can hear, but it can’t write text. I trained a small reader (20M numbers) that maps each 40 ms of sound into EmbeddingGemma 2’s text space and picks the closest of about 200,000 English words. Training data was 460 hours of LibriSpeech plus fake voicemails: Gemini wrote the scripts, spoke them with 30 voices, and I added a phone-line effect.
| Transcriber | Word error rate on phone voicemails |
|---|---|
| Gemini’s own transcriber (the bar) | 2.8% |
| 100 h of audio, small reader | 11.6% |
| 460 h, bigger reader | 9.3% |
| + overlapping 30 s windows | 8.1% |
| + a word language model | 6.9% |
It’s still more than twice Gemini’s error rate, and the errors are the kind that matter. In one test voicemail, “his speech sounds kind of slurred” came out as “kind of blurred”. Slurred speech is a stroke sign.
The local server takes the same request as the Gemini Decisions API and gives
the same answer shape. An app switches from Gemini to the local copy by
changing one URL. It listens on 127.0.0.1 only and keeps nothing it reads:
curl -s 127.0.0.1:8250/v1/decisions -H 'content-type: application/json' -d '{
"input": "my dad is very confused since this morning and his face looks droopy",
"questions": {"triage": {"type": "choice", "instructions": "How urgent?",
"criteria": {"emergency": "…", "same_day": "…", "routine": "…", "admin": "…"}}}}'
It also has /v1/rerank for search and /v1/transcribe for audio. The demo
page puts Gemini’s triage slip on top of the local one, so you can lift it and
compare the two on each test message.
Benchmarks don’t show what a model is for, so I also built Kotona, a healthcare app that uses all three tools. It’s for people who are back home after TMS treatment for depression. Each day they say or type a few sentences about how they slept, what they did and how they feel.
A fixed safety rule reads every check-in before the model does. Words about self-harm put the person at the top of the care team’s list, and the model can add a flag but never clear one. All three models run on the clinic’s own server. Kotona uses the EmbeddingGemma decision model, because Gemma 3 1B is more accurate but too large and slow for a small clinic server.
The challenge had one criterion: the best use of Gemma, judged on a creative use, strong technical work and a real-world problem. I think four things got it there.
The code, the fake data and every benchmark report are in Rahmakyan/gemma-lab.