A Smaller AI Model for Assamese Shows Promise—and the Limits of Fluent Answers

An AI model can appear to understand Assamese yet struggle to produce a usable answer. Pragyan Phukan, of Moran College in Assam, investigates that gap in the preprint AssameseLM-E2B: An Open-Weight Assamese Language Model for Efficient Consumer-Grade Deployment, submitted to arXiv on November 4, 2026. Version 1 describes a roughly two-billion-parameter base model and an instruction-tuned assistant, AssameseLM-E2B-it. The supplied record does not establish peer review.

The project targets two problems: inefficient text encoding and interference between Assamese and Bengali, which share much of their script. Phukan reports that tested multilingual models sometimes understood Assamese questions but answered with poor grammar, inappropriate language mixing or Bengali forms. These observations concern the evaluated models and prompts, not every multilingual system.

Making Assamese take fewer tokens

Language models process tokens—pieces of text that may be words, word fragments or smaller units. If Assamese words fragment excessively, the same passage occupies more of a model’s limited input space.

Phukan expanded the tokenizer of what the paper identifies as Gemma-4-E2B with Assamese subwords. The method trained a vocabulary on Assamese text and filtered candidate tokens, including fragments beginning with dependent vowel signs or other modifiers. It then enlarged the model’s input and output vocabulary layers.

On the reported FLORES-200 Assamese test text, AssameseLM-E2B required 35,107 tokens, compared with 55,383 for its Gemma baseline and 166,218 for Llama-3.2-3B. That is approximately 37% fewer tokens than Gemma and 79% fewer than Llama. These counts are clearer than the abstract’s mathematically misleading claim of reducing overhead by “nearly 500%.” Shorter sequences can accommodate more text within a fixed token limit, but these measurements do not establish a particular real-world speedup.

Adapting the model without retraining everything

The training corpus combined web collections, Wikipedia, news and approximately 100 OCR-digitized books, including literature and textbooks. The paper describes error correction, deduplication and filtering intended to reduce unwanted language contamination. Because Assamese and Bengali share a Unicode block, however, Unicode filtering alone cannot establish complete linguistic separation.

Continued pretraining covered 0.5 billion Assamese tokens. A “Hybrid LoRA” approach trained compact adapters inside the model while also updating its vocabulary embedding and output layers, allowing newly added tokens to acquire useful representations. Instruction tuning then used 29,319 examples covering direct answers, context-based questions, English instructions and search-related tasks.

For language prediction, the paper reports 0.7488 bits per byte on FLORES-200. This metric normalizes prediction loss by text bytes rather than tokenizer-specific units; lower is better. The reported result improves on Gemma’s 0.9570, but remains slightly worse than Llama’s 0.7416. It supports improved predictive compression relative to the starting model, not a general claim of superior reasoning or factual knowledge.

A strong result on a very small test

The assistant’s headline result is a 96.7% overall score on ten retrieval-augmented generation, or RAG, prompts. Each supplied numbered passages and asked a question. This tests answering from provided evidence; it does not demonstrate the performance of a complete document-retrieval system.

The score averaged six metrics. The paper reports perfect scores for correctness, faithfulness, citation attribution, refusal and output cleanliness, with 80% for matching the requested script. Two answers used Assamese script when Romanized Assamese was expected.

Crucially, the reported 100% refusal accuracy means successful abstention on just two questions whose answers were absent from the passages. It is not evidence of universal hallucination prevention. Grading relied on an AI judge, identified as Claude Opus 4.6. The paper also inconsistently identifies some comparison models, including alternating between one-billion- and three-billion-parameter Llama variants, complicating precise baseline comparisons.

Fluency is not factual reliability

A separate 150-prompt chat evaluation, including 55 Romanized prompts, presents a less uniform picture. AssameseLM-E2B-it averaged 7.30 out of 10 for linguistic accuracy and 8.06 for query understanding, but only 4.89 for factuality and 4.62 for instruction following.

Phukan attributes part of this weakness to an overlearned refusal habit: the model sometimes declined ordinary questions because no external context was supplied. The appendix also documents factual hallucinations, including incorrect explanations about protons and neutrons.

Evaluation itself proved fragile. An initial Gemini-based judge awarded perfect factuality and understanding scores to 43 Qwen outputs the paper describes as incoherent. Native Assamese evaluators corrected those cases. That intervention addresses identified errors, but does not establish that every remaining judgment is unbiased.

The paper reports releasing both checkpoints. Its local-deployment ambition is plausible for a compact model, but training used GPUs with 48 GB of memory, and no consumer-device latency or memory benchmark is reported. The contribution is therefore a promising adaptation recipe—not proof of a deployment-ready, hallucination-free assistant.