A Stock-News RAG Prototype Reports Fast Answers, but Accuracy Remains an Open Question

A system that answers financial questions needs more than fluent language: it needs relevant information to work from. In a preprint titled “Enhancing Stock Market Information Retrieval with Retrieval-Augmented Generation,” researchers at FPT University in Can Tho, Viet Nam, describe a prototype that searches Vietnamese stock-market news before generating an answer.

The authors report strong similarity between generated responses and reference texts, alongside an average response time of roughly 1.16 seconds. Those results describe performance on a small evaluation. They do not, by themselves, demonstrate reliable investment advice or accurate answers across changing market conditions. The manuscript is a preprint marked as submitted to Expert Systems with Applications, not evidence of an accepted journal publication.

A small news collection supplies the context

The system draws on 30 stock-market articles collected from VnExpress, a Vietnamese news outlet. The collection includes eight articles about market trends and index performance, nine about corporate stocks and sectors, seven about foreign investment and liquidity, and six about regulatory and corporate actions.

This is a document-question-answering experiment, not a test of stock-price forecasting or trading returns. Its central task is to retrieve relevant reporting and use that material to formulate a response.

The paper describes cleaning the articles by removing URLs and special characters, converting text to lowercase, segmenting words and removing common Vietnamese stop words. There is an implementation ambiguity: the preprocessing section names Underthesea for Vietnamese segmentation, while the embedding section describes using Jieba. The manuscript does not clearly explain how these two descriptions fit together, limiting reproducibility.

Searching by meaning, then by words

The retrieval pipeline combines two ways of matching a question to documents.

First, a multilingual embedding model called LaBSE converts documents and queries into numerical vectors. These representations allow the system to compare texts in a shared mathematical space. A FAISS index retrieves candidate documents using Euclidean distance, with smaller distances indicating closer vector matches.

Second, the system reranks those candidates using BM25, a keyword-based scoring method that accounts for term frequency, document length and how common a term is across the collection. The intended combination is straightforward: vector search identifies related material, while keyword ranking gives weight to specific terms in the question.

The selected documents then enter a structured prompt alongside the user’s query and instructions. Gemini-1.5-Flash generates the response from this supplied context. This architecture gives the generator access to external news material, but supplying documents is not itself proof that every resulting statement faithfully reflects them.

What the reported scores show

The evaluation covers 100 stock-market queries and their generated responses. The authors report average ROUGE F-measures of 0.9555 for ROUGE-1, 0.9314 for ROUGE-2 and 0.9436 for ROUGE-L.

These metrics assess overlap with reference texts: individual words, consecutive word pairs and the longest shared subsequence, respectively. The high scores indicate close wording alignment in the reported evaluation. They should not be read as percentages of factually correct answers.

The latency measurements span the interval from submitting a query to generating a response. Across 100 queries, the reported average is 1.1626 seconds, with a minimum of 0.917 seconds and a maximum of 2.455 seconds. These figures characterize the tested setup; they do not establish performance under heavier demand or with a much larger document collection.

Important questions remain unanswered

The paper does not report a comparison against Gemini without retrieval, a vector-only retriever or another alternative. Consequently, the results do not isolate how much benefit comes from retrieval, BM25 reranking or the particular generator.

The evaluation also lacks a reported human assessment of factual correctness or a separate measurement of unsupported claims. ROUGE cannot resolve those questions simply by detecting shared wording. The manuscript provides limited detail about how queries and reference answers were constructed, making the unusually high overlap difficult to interpret beyond this experiment.

Finally, fast response generation is different from fresh information. The experiment uses a collected set of news articles and does not demonstrate continuous updating against live market developments.

The prototype therefore offers a concrete example of combining semantic search, keyword ranking and generation for Vietnamese financial news. Its reported speed and reference-text overlap are encouraging within that narrow setting. Broader claims about trustworthy financial insights remain unproven by the evaluation presented.