Every code excerpt below is a verbatim quote from the JemmaPass repository. No pseudo-code, no simplification — exactly what runs on a Pixel 9 when you press "Scan QR".
A patient summary entering the app travels top-down through perception, reasoning, shielding, forging, and transfer. Each layer has clear contracts and runs entirely on-device.
┌─────────────────────────────────────────────────────────────────┐
│ ① PERCEPTION 📷 Camera · 🎙️ Mic · ⌨️ Text · 📡 NFC/Nearby/QR │
│ └─ ML Kit (Barcode, Document Scanner, Text Recognition) │
│ └─ AudioRecord 16kHz mono PCM → 44-byte WAV header │
├─────────────────────────────────────────────────────────────────┤
│ ② REASONING 🧠 Gemma 4 E4B (3.4 GB) via LiteRT-LM 0.11.0 │
│ └─ Native @Tool function calling (JSON schema) │
│ └─ 68 tools across 6 surfaces (Allergies, Patient, Jemma, │
│ Identify, MedScan, Explain) │
├─────────────────────────────────────────────────────────────────┤
│ ③ SHIELDING 🛡️ KnowledgeBaseManager — SQLite FTS5 │
│ └─ knowledge_full.db (3.36 GB) · 58 tables · 88 indexes │
│ └─ DDInter 2.0 + SNOMED IPS + ATC + RxNorm + UMLS + LOINC │
│ └─ Sub-50 ms query latency on Pixel 9 (cold), <5 ms (warm) │
├─────────────────────────────────────────────────────────────────┤
│ ④ FORGING 🏛️ JemmaProfileJ → JemmaPayloadPruner │
│ └─ Schema _j 1.2 — 18 IPS pillars (3 active, 15 stubs) │
│ └─ Pruner: 1009 b raw → 777 b lean → 500 b wire (~50%) │
│ └─ Hash canonicalisation (deterministic SHA-256 fingerprint) │
├─────────────────────────────────────────────────────────────────┤
│ ⑤ TRANSFER 📡 Triple-Layer QR + Nearby Mesh + NFC HCE │
│ └─ Layer A: _j2: binary compressed (deflate-raw RFC 1951) │
│ └─ Layer B: multilingual text fallback (any phone reads it) │
│ └─ Layer C: Triple QR Frame Splitter for large profiles │
│ └─ Mesh: Strategy.P2P_CLUSTER, 6 chunk types (V/VR/S/SR/E/F) │
│ └─ Lock-screen AppWidget · double-tap → 0-unlock SOS │
└─────────────────────────────────────────────────────────────────┘
│
▼
┌──────────────────────────┐
│ Any rescuer's phone. │
│ Any language. │
│ Zero internet. │
└──────────────────────────┘
📷 Multimodal input · camera · mic · NFC · QR
The user never has to type a clinical code. JemmaPass reads your environment — pill boxes via camera, symptoms via voice, NFC tags via touch, QR badges via scan. The Gallery audio pipeline (inspired by Google AI Edge Gallery) wraps raw PCM into a 44-byte WAV header before passing to Gemma 4 — necessary because LiteRT-LM's miniaudio decoder rejects raw PCM with error MA_INVALID_FILE.
// Audio capture pipeline — verified compatible with Gemma 4 multimodal
private fun captureAudioForGemma(): ByteArray {
val sampleRate = 16000 // 16 kHz mono (Gemma 4 USM encoder spec)
val bitsPerSample = 16
val channels = 1
// PCM → WAV (Gemma 4 multimodal audio input requires a container)
// Reference: Google AI Edge Gallery ChatMessage.kt:187 genByteArrayForWav
val byteRate = sampleRate * channels * bitsPerSample / 8 // = 32000
val wavHeader = buildRiffHeader(pcm.size, sampleRate, channels, bitsPerSample)
return wavHeader + pcm
}
// Send to Gemma 4 with audio Content
val engineConfig = EngineConfig.Builder()
.setModelPath(modelPath)
.setAudioBackend(Backend.CPU()) // required for audio decoding
.build()
val content = Content.audioBytes(audioBytes) // not raw PCM!
_j2: direct (no manifest fetch), QR multi-frame, Aztec, Code 128. Zero camera permissions thanks to GmsBarcodeScannerOptions.
_j2: wire format. No app on rescuer side required if they use a TWA browser.
🧠 Gemma 4 + 49 @Tool functions via LiteRT-LM
Gemma 4's killer feature for medical AI is native structured tool calling. Instead of asking the LLM to hallucinate clinical knowledge, we expose 49 typed Kotlin functions (the @Tool surface) that Gemma can invoke synchronously from its native thread. Each tool emits a JemmaToolEvent for UI streaming. The LLM never hallucinates a SNOMED code — it asks our database for one.
ui/profile/allergies/chat/ai/JemmaTools.ktai/patient/ai/livescan/ai/gemma/ai/livescan/checkOneDrugAgainstFocusProfileThis is the function called in the demo when Kamekichi scans Augmentin. It resolves the drug to ATC, walks the ATC hierarchy upward (because penicillins share J01C*), and matches it against every allergy in the focus profile. Less than 50 ms on a Pixel 9.
@Tool(
description = "MASTER cross-check: given a candidate drug name, " +
"check it against ALL three pillars of the focus profile " +
"(allergies, current medications for DDI, and known " +
"conditions for contraindications). Returns a structured " +
"collision report. Call this whenever the rescuer scans " +
"a new medication or considers giving a drug to the patient."
)
fun checkOneDrugAgainstFocusProfile(
@ToolParam(description = "Candidate drug — common name (any language), " +
"brand, INN, or RxNorm/ATC code.")
drugName: String
): Map<String, Any> = runBlocking(Dispatchers.IO) {
val p = focusRef.get() ?: return@runBlocking profileMissingMap()
val lang = currentLang()
val tStart = System.currentTimeMillis()
val result = xcheck.checkOneDrugAgainstProfile(
drugName, p.al, p.md, p.cn, lang
)
Log.i(TAG, "[t=${System.currentTimeMillis()}] " +
"🚨 checkOneDrugAgainstFocusProfile('$drugName') → " +
"🩹${result.allergyHits.size} 💊${result.ddiHits.size} " +
"🦠${result.drugDiseaseHits.size} · " +
"${System.currentTimeMillis() - tStart}ms")
crossCheckResultToMap(result, drugName, lang)
}
resolveDrugBefore any cross-check can happen, Gemma needs to resolve a free-text drug name (in any language) to a canonical KB entry with its ATC code and 25+ language displays. This is the most-called @Tool in the surface — almost every other tool depends on its output.
@Tool(
description = "Resolve a drug name (any language, brand, INN, " +
"RxNorm or ATC code) to its KB entry. Returns ATC, " +
"RxNorm, canonical and localized display. Always call " +
"this first when only a free-text drug name is available " +
"before invoking checkDdi or any cross-check tool."
)
fun resolveDrug(
@ToolParam(description = "Drug name in any supported language, " +
"or a code (RxNorm CUI, ATC).")
name: String
): String {
emitStep(JemmaToolEvent.ToolStart("resolveDrug", mapOf("name" to name)))
val candidate = kb.resolveDrugByAnyName(name)
?: return jsonError("No KB entry for '$name'.")
// Walk ATC hierarchy upward (for class-level allergy matches)
val ancestors = kb.getAtcAncestors(candidate.atc)
val result = JemmaDrugResult(
rxnormCui = candidate.rxnorm,
atc = candidate.atc,
atcAncestors = ancestors,
canonicalName = candidate.canonical,
localizedDisplays = candidate.displays
)
emitStep(JemmaToolEvent.ToolEnd("resolveDrug", result))
return moshi.toJson(result)
}
resolveDrug returns the full ATC ancestor chain, the allergy cross-check upstream doesn't need to know that "amoxicillin" → "penicillin family" — Jemma's data shape encodes the medical taxonomy. The LLM only orchestrates; the safety logic lives in deterministic SQL.
miniaudio as the decoder, which requires a recognized file container (WAV/MP3/FLAC). Sending raw 16 kHz PCM directly throws Failed to initialize miniaudio decoder, error code: -10 (MA_INVALID_FILE). JemmaPass wraps PCM in a 44-byte RIFF/WAVE header — see MedScanController.kt snippet above and Gallery's Consts.kt:64 for the canonical byteRate=32000.
🛡️ The 3.36 GB clinical knowledge base · 37.9M rows curated
The Clinical Shield is a single SQLite file shipped with the app. 3.36 GB · 58 tables · 88 indexes · 4 FTS5 virtual tables · 3 clinical views · 37.9M curated rows. Every column was auto-validated by jemma_kb_audit.py against the live binary. Gemma 4 is the conductor, the KB is the orchestra.
ddi_facts · 260,100 curated DDI records (3NF)ddi_atc_pairs · 876,277 indexed ATC × ATC pairsv_ddi_emergency view · 802,352 Major+Moderateddi_alternatives · 3,299,967 safe substitutionsatc_hierarchy · 6,934 codes · 7 levelsatc_alternatives · 643,763 ATC sibling/parent linksterminology_codes · 1,452,451 cross-system bridgeconcept_relations · 2,688,211 UMLS relationssemantic_types · 1,737,853 UMLS STY assignmentsrxnorm_concepts + rxnorm_relations · ~1.8M rowsword_index · 15,500,225 multilingual word indexips_valuesets_translations · 85,736 rows · 20 langsterminology_cjk FTS5 · 160,078 rows · ja + koterminology_latin_content FTS5 · 3,474,727 rowsdrug_names_multilingual + drug_names_cjk FTS5interactions_food · 857 DFI · DDInter2drug_disease_interactions · 8,121 DDSIallergy_cross_reactivity · 52 rules (WHO/UpToDate/ANSM/BNF)who_aware_2024 · 64 antibiotics · Access/Watch/Reservewho_eml_2023 · 151 essential medicines (WHO 23rd EML)therapeutic_duplications · 741 records-- The "emergency view" — 802,352 rows · queried in <50ms via indexed ATC lookup
CREATE VIEW v_ddi_emergency AS
SELECT p.drug_a_atc, p.drug_b_atc, f.severity, f.mechanism_category,
f.description_en, f.management_en, f.alternative_atc
FROM ddi_atc_pairs p
JOIN ddi_facts f USING (fact_id)
WHERE f.severity IN ('Major', 'Moderate');
-- Used in JemmaTools.checkDdi() — single statement, indexed on (drug_a_atc, drug_b_atc)
SELECT severity, mechanism_category, description_en, management_en, alternative_atc
FROM v_ddi_emergency
WHERE (drug_a_atc = ? AND drug_b_atc = ?)
OR (drug_a_atc = ? AND drug_b_atc = ?)
LIMIT 1;
Android 15 enforces 16 KB native page alignment for all .so files shipped via Play Store (deadline November 1, 2025). The SQLite library we use (requery:sqlite-android) was upgraded from 3.45.0 to 3.49.0 to gain compliance.
// 16 KB page-size aligned (verified with llvm-objdump on Pixel 9 Pro XL)
// align 2**14 ← required for Android 15+ (Google Play Nov 2025)
implementation("com.github.requery:sqlite-android:3.49.0")
🏛️ FHIR IPS R4 → deflate-raw → 500-byte QR
The IPS profile leaves Kotlin as a JemmaProfileJ object (Moshi-serialised), passes through the Pruner (clinical-aware field stripping), then the Codec (deflate-raw RFC 1951). The result is base64-URL-encoded and prefixed with _j2:. Haru's full profile: 1009 bytes raw → 500 bytes wire (~50% reduction).
_j2:)
The clever bit: the Android java.util.zip.Inflater(nowrap=true) is exactly bit-for-bit compatible with the WebView's DecompressionStream('deflate-raw'). So the JS-side QR generator and the Kotlin-side QR reader interop with zero conversion overhead.
/**
* Encode a JemmaProfileJ to wire format _j2:<base64-url(deflate-raw(JSON))>.
*
* Pipeline:
* 1. Moshi serializes the profile to JSON
* 2. Pruner strips empty arrays / default values
* 3. Deflater(nowrap=true) → deflate-raw RFC 1951 bytes
* 4. base64-url encoding (no padding, URL-safe alphabet)
* 5. prefix _j2: → final string fits in a single QR
*/
fun encode(profile: JemmaProfileJ): EncodeResult {
val rawJson = moshi.adapter(JemmaProfileJ::class.java).toJson(profile)
val prunedJson = JemmaPayloadPruner.smartPrune(rawJson)
val deflater = Deflater(Deflater.BEST_COMPRESSION, true) // nowrap=true
deflater.setInput(prunedJson.toByteArray(Charsets.UTF_8))
deflater.finish()
val out = ByteArrayOutputStream()
val buffer = ByteArray(1024)
while (!deflater.finished()) {
out.write(buffer, 0, deflater.deflate(buffer))
}
deflater.end()
val b64url = Base64.getUrlEncoder().withoutPadding().encodeToString(out.toByteArray())
return EncodeResult.Success(
wire = "_j2:$b64url",
rawSize = rawJson.length,
prunedSize = prunedJson.length,
compressedSize = out.size()
)
}
nowrap=true).
📡 Nearby mesh + Triple-Layer QR + NFC HCE + lock-screen widget
JemmaPass offers three independent transfer paths, all entirely offline:
A single QR can hold ~4 KB at low error correction. For most profiles that's enough, but the Triple-Layer QR design adds redundancy and interop:
Prefix _j2: · deflate-raw + base64-url. ~500 bytes for typical IPS. Only JemmaPass-aware readers decode this — but it's the densest payload.
Plain-text Unicode summary in the patient's language(s). Any phone camera (iOS native, Google Lens, anywhere) can read it. Trades density for universality.
For profiles too large for a single QR, JemmaQrFrameSplitter.kt emits JF:n/N|-prefixed frames the reader reassembles. Each frame is independently valid.
The mesh wire format uses 6 discriminated chunk types for forward-compatible upgrades. Old phones see new chunk types and silently drop them (per RFC 9171 BPv7 design principle).
enum class ChunkType(val prefix: String, val isRelayed: Boolean) {
/** Direct victim profile chunk (5-chunk rotation, no relay).
* Format: V|<sid4>|<chunkIdx>|<chunkTotal>|<payload> */
VICTIM_DIRECT ("V", isRelayed = false),
/** Multi-hop relayed victim chunk (carries TTL).
* Format: VR|<sid4>|<chunkIdx>|<chunkTotal>|<ttl>|<payload> */
VICTIM_RELAYED ("VR", isRelayed = true),
/** Direct rescuer beacon (single chunk).
* Format: S|<sid4>|<name_ascii>|<lang>|<lat>|<lon> */
RESCUER_DIRECT ("S", isRelayed = false),
/** Multi-hop relayed rescuer beacon.
* Format: SR|<sid4>|<name_ascii>|<lang>|<lat>|<lon>|<ttl> */
RESCUER_RELAYED("SR", isRelayed = true),
/** Triage event (status change, HELP, STAB, EVAC, DCD…).
* Format: E|<source>|<victim>|<status>|<rescuer>|<ts>|<ttl>|<seq> */
EVENT ("E", isRelayed = false),
/** Profile fingerprint advertisement (cache-skip optimization).
* Format: F|<sid>|<sha256_hex_8>|<lastModifiedTs>|<chunkVersion>
* Sent every 10s vs 1.5s for V| → ~75 broadcasts saved per cache hit. */
FINGERPRINT ("F", isRelayed = false);
}
companion object {
// P2P_CLUSTER: M-to-N topology, mesh-friendly, small payloads.
// Google Nearby docs: "ideal for mesh-like experiences"
const val SERVICE_ID = "be.heyman.android.jemmapassdemo.sos.SOS"
val STRATEGY: Strategy = Strategy.P2P_CLUSTER
// 131-byte endpointName cap (BLE advertising packet constraint).
// We rotate through 5 chunks of a victim profile, every 1.5s.
// Full cycle = 5 × 1500ms = 7.5s, well under the 9s discovery timeout.
const val CHUNK_ROTATION_MS = 1500L
}
In a real emergency, the victim cannot afford the friction of unlock → find app → tap SOS. JemmaPass ships a true Android AppWidget with widgetCategory="home_screen|keyguard" — meaning the same widget can be placed on the home screen and on the lock screen itself. A double-tap on the widget starts the foreground SOS broadcast service, no PIN / biometric / app launch required.
<appwidget-provider xmlns:android="http://schemas.android.com/apk/res/android"
android:minWidth="250dp"
android:minHeight="110dp"
android:targetCellWidth="4"
android:targetCellHeight="2"
android:resizeMode="horizontal|vertical"
android:widgetCategory="home_screen|keyguard"
android:updatePeriodMillis="1800000"
android:initialLayout="@layout/jemma_emergency_widget"
android:initialKeyguardLayout="@layout/jemma_emergency_widget"
android:previewLayout="@layout/jemma_emergency_widget" />
The widget guards against accidental activation with a 2-second double-tap window — a single tap shows the toggle ready state; a second tap inside 2000 ms commits the broadcast.
private const val DOUBLE_TAP_TIMEOUT = 2000L
private var lastTapTime = 0L
override fun onReceive(context: Context, intent: Intent) {
super.onReceive(context, intent)
if (intent.action == ACTION_SOS_TAP) {
val now = System.currentTimeMillis()
if (now - lastTapTime < DOUBLE_TAP_TIMEOUT) {
// Double tap confirmed — start the emergency service.
Log.i(TAG, "Double tap confirmed, starting emergency service")
lastTapTime = 0L
startEmergencyService(context, appWidgetId)
} else {
// First tap — arm the widget, await second tap.
lastTapTime = now
}
}
}
Once activated, a sibling JemmaWidgetEmergencyService (foreground service, foregroundServiceType="connectedDevice") starts the Nearby broadcast described above and keeps it alive even if the user's main app is killed. The widget itself uses a ViewFlipper to cycle through the victim's IPS chunks in the device language plus an English fallback — so a rescuer who picks up the phone sees the patient's allergies and medications scrolling on the lock screen without unlocking anything.
Every box below is a real function in the codebase. Total wall-clock latency measured on Pixel 9 Pro XL.
ML Kit Barcode Scanner decodes the GS1 DataMatrix from the Augmentin box → 06291213049487 (GTIN-14).
SQLite query against drug_gtin_lookup returns amoxicillin/clavulanate 875mg/125mg.
resolveDrug @Tool
LiteRT-LM synchronously calls Kotlin function. KB returns ATC = J01CR02, ancestors = [J01C, J01, J].
checkOneDrugAgainstFocusProfile
Focus profile is Kurodo's. KB walks his allergies, finds SNOMED 91936005 (Allergy to penicillin).
SNOMED 91936005 maps to ATC family J01C* in allergy_cross_reactivity. Augmentin's ATC J01CR02 belongs to J01C*. HIT.
Query ips_valuesets_translations for the rescuer's locale (ja-JP). Returns 「ペニシリンアレルギー」「アモキシシリン」.
Compose recomposition with the alert panel. Haptic feedback fired. Background turns crimson via state change.
Streaming token-by-token: "Augmentin contains amoxicillin, a penicillin. Anaphylaxis risk extreme. Do NOT administer."
Uses the rescuer's locale TTS engine. Speech rate adjustable. Vibration during speech.
Event emitted with E| chunk to nearby mesh (other rescuers see the alert too). Stored in local-only event log for post-incident debrief.
~200 ms total from camera click to spoken red alert. Zero network packets. Zero cloud servers. Zero possibility of failing because Vodafone is down.
Three companion deep-dives complete the technical picture.