Pairing predictive AI with a medical reference chatbot to explain diabetes classifications

The study, by Lorenzo Mannocci, Francesca Naretto, Lucia Passaro and Anna Monreale, addresses a problem distinct from prediction accuracy: how to communicate what a model is doing. A list of influential variables can help explain a classification, yet still require interpretation. Adding a language model could make that information more accessible, but also introduce plausible-sounding statements unsupported by the model or medical evidence.

The researchers therefore combine three components: a predictive model, a technical explanation method and a retrieval-augmented generation system, or RAG. Retrieval supplies relevant passages from an external collection before a language model generates its response. The aim is to provide medical context and identifiable sources rather than rely entirely on information encoded during the language model’s training. That design offers a route to checking claims; it does not guarantee their correctness.

The work is available as an arXiv preprint posted on November 6, 2026. Both the manuscript and its landing record identify it as the preprint version of a conference paper published by Springer in 2026 in Communications in Computer and Information Science, associated with HC_AIxIA_HYDRA 2025. It should therefore not be described simply as unpublished research.

For their preliminary experiment, the team used the public Pima Indians Diabetes Database, which contains clinical measurements and diabetes labels. They standardized the measurements and used random oversampling, which repeats minority-class records, to balance the classes. A random forest classifier was trained using an 80/20 training–test split, with a grid search and three-fold cross-validation to select its settings.

The technical explanations came from LIME. This method estimates which input features influenced an individual prediction by examining model behavior around that record. Such an explanation describes the classifier’s behavior locally; it does not establish that a particular measurement causes diabetes.

The conversational component used the MedRAG framework, the StatPearls medical reference corpus, the biomedical MedCPT retriever and the Mixtral-8x7B-Instruct language model. The proposed system supports three tasks. It can rewrite and contextualize the classifier’s prediction and LIME explanation, generate its own classification with an accompanying explanation, or answer follow-up questions using the record, prediction and feature information.

A separate selector governs whether the independently generated RAG explanation is displayed. In the tested implementation, that explanation is released only when the RAG system and random forest predict the same class. This is an agreement check, not an independent verification of either the prediction or its justification. It also does not mean that every other explanation is withheld when the models disagree.

The conventional classifier performed better on the reported test metrics. Its overall accuracy was 75%, compared with 71% for MedRAG. For records labeled as diabetes, the random forest’s recall was 69%, versus 56% for MedRAG: it identified a larger share of the positive cases. Its positive-class F1 score, which combines precision and recall, was 0.66, compared with 0.58.

The two systems agreed on 82% of predictions. The authors report that agreement was most often correct, but that the next most frequent outcome was agreement on an incorrect answer. When predictions differed, the random forest was more often correct. These findings support retaining the conventional classifier in this particular architecture rather than replacing it with the retrieval-based language model. They do not establish that agreement makes an explanation trustworthy.

Evidence about the explanations themselves is much thinner. The paper presents an example with references to retrieved documents, but reports no clinician user study, systematic assessment of explanation faithfulness or measured reduction in hallucinations. A readable explanation backed by document references may still misrepresent what a classifier learned or draw conclusions its sources do not justify. The reported classification scores cannot resolve those questions.

The authors also acknowledge the limited experimental scope and computational cost of RAG. Testing on one dataset does not establish performance across clinical populations, diseases or workflows. Privacy is another unresolved concern because generating explanations can require sensitive patient information; privacy-preserving approaches are discussed as future work rather than demonstrated safeguards.

The contribution is consequently a preliminary architecture and prediction comparison, not evidence of medical benefit. It shows how retrieval, feature explanations and conversation might be connected, while leaving open whether that combination helps clinicians detect errors, understand uncertainty or make better decisions.