TactiNavi Gives an AI Game Coach Rules for When Advice Applies

A strong battle lineup is useless if a player does not own the required characters. TactiNavi tackles this problem by checking a strategy’s prerequisites before an AI coach writes its answer—and preserving those checks so engineers can investigate mistakes later.

Knowing a strategy is not enough

An AI game coach needs more than accurate information about characters and combat mechanics. Its recommendations must fit the player’s roster, resources, progress and game version. Advice that works for one player can be impossible for another.

In the TactiNavi preprint, Qi Liu and colleagues describe the system behind a coaching assistant deployed in a commercial open-world game with strategic battles. Their central idea is to separate what a strategy recommends from the conditions under which it applies. Those conditions become explicit checks rather than qualifications buried in prose.

The researchers retrospectively evaluated 1,122 logged coaching turns from three consecutive two-day deployment periods, separated by two production revision rounds. Their findings concern both the quality of advice and the difficulty of proving that subsequent changes actually improve it.

Three answers to a prerequisite

TactiNavi prepares versioned “strategy packages” offline using game evidence and expert guidance. Each package contains supporting evidence, prerequisites, counterexamples and instructions explaining how to use the strategy. Calculation tools check relevant mechanics, while a critic searches for unsupported assumptions.

When a player asks a question, the serving system retrieves relevant packages and evaluates their requirements against structured player state. Each condition has three possible outcomes: true, false or unknown.

Consider a lineup requiring a support-role character. If the player owns an eligible character, the strategy is permitted. If none is available, the coach offers an alternative and explains the missing requirement. If ownership information is absent, it seeks more evidence or asks the player rather than guessing.

These checks do not require language-model calls. Their results, together with the evidence and instructions, guide the model that writes the reply.

Better content, more careful application

On 197 strategic turns from the final deployment period, strategy packages achieved 82.0% correctness, compared with 57.9% for isolated character and mechanic facts. The 24.1-percentage-point improvement had a reported 95% confidence interval of 16.4 to 31.7 points. Correctness required no contradicted claims and successful checks covering matters such as character ownership, skills and resources.

However, an unstructured prose guide containing comparable strategic information scored 82.5%. Explicit conditions therefore did not improve recorded-turn correctness over that guide. The distinction emerged when the researchers changed player state.

Using 36 base turns, they constructed 108 variants in which a strategic requirement was satisfied, violated or unknown. On false and unknown variants, applicability violations occurred in 18.1% of guide responses and 10.4% of package responses. Unknown information produced the largest gap: 26.4% versus 15.3%.

This is promising rather than conclusive evidence. The guide–package comparison’s exact paired test gave p = 0.062, short of conventional statistical significance. The study separates the benefit of providing synthesized strategic content from the more tentative evidence that explicit conditions improve its application.

Keeping a record of why

Every response produces a decision trace connecting the advice to package versions, player-state fields, condition outcomes, evidence and tool results. That record matters when a complaint refers to an answer several turns earlier.

The diagnostic evaluator checks individual claims and follows their evidence chains to identify unsupported stages. A faulty prerequisite can implicate the strategy package; an unenforced valid prerequisite can implicate an execution rule.

On 48 expert-adjudicated player doubts, TactiNavi achieved 86.4% precision and 52.8% recall for founded complaints. It rejected nine of 12 unfounded doubts and located the relevant earlier response in 26.7% of delayed-feedback failures. These results do not establish uniformly superior diagnosis: a category-routing baseline had higher precision and recall.

Why a successful rerun is not a repair

The revision results provide a warning. Simply rerunning the unchanged system recovered 38.4% of previously failing turns. A formerly wrong answer becoming correct is therefore insufficient evidence that an edit worked.

TactiNavi’s proposed release checks compare candidates on independent held-out cases, including affected conditions, counterexamples and broader regression cases. They require a positive lower confidence bound for in-scope improvement and restrict losses outside the declared scope.

These gates were assessed retrospectively, not used to approve the two production revision rounds. Only one of 15 shipped revision units passed. Thirteen lacked sufficient gain beyond generation variation. Another improved in-scope correctness by 25.0 points but reduced out-of-scope correctness by 20.8 points.

The final deployed version also showed no measurable correctness improvement over the first on matched replay.

A bounded result

Strategy packages used about 14% more knowledge tokens than prose guides. Median reply generation took 2.07 seconds, but that measurement excludes the complete online workflow.

The evidence comes from one game and two revision rounds. Replay held planning decisions and tool outputs fixed, and the automated judge shared the coach’s underlying model. Within those boundaries, the paper offers a concrete engineering lesson: make applicability inspectable, preserve the evidence behind advice, and compare proposed fixes against what an unchanged system already does.