An Offline First-Aid AI Runs on a Phone—but Its Advice Falls Short

When an emergency happens somewhere without reliable phone service, online assistants may be unavailable. A small language model running directly on a phone could still respond to a question. But first aid is unforgiving: a fluent answer that omits a crucial step or gives unsafe advice may be worse than no answer at all.

That tension is the focus of HandsOnAid, a research prototype described in an arXiv preprint by Rezwan Islam Salvi, Saiyma Sittul Muna and Mushfiqur Rahman Mushfique. The researchers built a compact model and tested both its responses and its ability to run offline on a smartphone. Their main message is mixed: fine-tuning improved performance, but the answers were not reliable enough for emergency use.

Training a small model for first aid

The team started with Gemma 2B, a language model with about two billion parameters, and adapted it using QLoRA, a method that fine-tunes a model with relatively modest computing resources. The training material, FirstAidQA, contains 5,550 first-aid question-and-answer pairs. The researchers tested seven configurations, including the original model, fine-tuned versions, and additions such as retrieval, a safety gate and a prompt to encourage longer answers.

To assess responses, they created 41 questions with reference answers based on Australian and New Zealand Committee on Resuscitation (ANZCOR) guidelines. A medical professional reviewed the questions and references. Four language models then independently scored answers without being told which configuration had produced them. The scoring scale ran from zero to five: a top score required an answer that was correct, clear and complete enough for a layperson to follow. Specific dangerous instructions could trigger safety overrides and sharply limit a score.

The researchers also tested the judges with 45 hidden control answers, including deliberately unsafe examples. All four judges placed the controls in the expected score ranges and identified the planted violations. A medical professional separately assessed 40 answers from selected configurations; the scores and overall ordering were similar to the language-model judges’ results.

Better scores, but not complete answers

Fine-tuning raised the average score from 1.26 for the base model to 2.34 out of five. The tuned model did better on 34 of the 41 questions and worse on five. The best average, 2.51, came from the fine-tuned model with retrieval, which supplied a relevant example from the same training set. However, its improvement over fine-tuning alone was small and was not statistically confirmed in the study’s tests. The other add-ons likewise did not show a confirmed improvement.

A higher average did not mean the system was ready to guide someone through first aid. For the retrieval configuration, no answer received a score of five from three of the four judges. The judges flagged at least one safety concern in eight of its 41 answers. The study describes examples where the model failed to recommend a tourniquet after pressure had not stopped bleeding, or gave inappropriate advice for a conscious choking adult.

The authors looked for possible reasons by auditing the training answers for selected details required by their rubric. They found little coverage of some guideline-specific steps. For example, the training set rarely contained details such as a tourniquet’s placement, and mentioned chest thrusts for choking only once. This offers a plausible explanation for some failures: retrieval cannot supply a correct example if the relevant information is missing or misleading in the source corpus. But the audit does not prove that training-data gaps are the only cause. The team tested one model, one English-language corpus and one guideline-based evaluation.

Offline, but not yet dependable

On a Google Pixel 7 Pro, using six CPU threads with the network disabled, the fine-tuned model began displaying text after an average of 6.86 seconds and completed answers in 14.88 seconds across ten questions. This demonstrates that the tested configuration can run locally on that phone. It does not show that the system works on every device, in every emergency, or quickly enough for every situation.

The limits of the evaluation matter too. The answer bank had only 41 questions, and the human assessment involved one medical professional and a subset of 40 answers. The study used single-turn questions, and its safety-critical labels were assigned by a language model. Keyword audits can also miss equivalent wording. The authors identify these as limitations rather than evidence that the model has been comprehensively tested.

HandsOnAid’s contribution is therefore not a validated emergency aid, but a demonstration and an evaluation of where a small offline model struggles. Its authors explicitly say it is not a medical device or a substitute for professional care, and that the observed scores could put users at risk. The results support further work on aligning first-aid training data with established protocols and testing systems more broadly—not relying on this prototype for real emergencies. This article is based on an arXiv preprint; its findings should be treated as provisional, not as clinical validation.