canalhandia/docs/plans/2026-08-05-ia-improvements-design.md

Design — grounding and feedback for the /ia module

Date: 2026-08-05 Status: approved

Why

/ia shipped earlier today and was used five times by real players. Every substantive answer contained an error:

asked	answered	reality
how to get a camel	spawns in desert villages "and badlands"	badlands is wrong; never said a saddle is involved
crossing the nether roof	"boat on soul sand, water on top, push the boat"	invented; not a technique
best mob farm spot	"Y 0–64, mobs spawnam mais concentrated there"	dubious, and English leaked into the Portuguese

The model states Minecraft mechanics confidently and incorrectly. No amount of prompt tuning fixes that, because the model does not know what it does not know. It needs sources.

Two further defects were found while measuring:

• Empty answers. M2.7 emits hidden reasoning that counts against max_tokens. At the deployed value of 300 it returned empty content twice in testing. Players see "Não consegui resposta agora" and assume the feature is broken.
• Foreign-token leakage. Replies contained 搭档 (Chinese) and contiennent (French) mid-sentence.

What was measured

Everything below is measured against the live MiniMax key, not assumed.

Retrieval source

pt.minecraft.wiki answers in Portuguese, needs no API key, and responds in ~0.3s. Its camel article contains the exact fact the model missed. It returns HTTP 403 to a default user agent — an identifying User-Agent is required.

How to choose the search term

MediaWiki cannot parse conversational Portuguese, so the raw question is useless as a query. Four methods, same questions:

method	usable terms
raw question	0 / 6
local stopword strip	1 / 6
free-text extraction call	2 / 7
tool call with forced tool_choice	5 / 5

Free-text extraction fails for the same reason answers came back empty: hidden reasoning eats a small max_tokens budget. A tool call returns a structured argument, which survives. Optional tool choice is not enough — with tool_choice: auto the model skipped the search on the hardest question, the one it had already answered wrong twice. Forcing the call fixes that.

How much of the article to send

exintro returns only the lead paragraph. Grounded on intros alone, both models honestly answered "não tenho certeza" to questions they had previously answered correctly — truthful but useless. Sending the full article text (~7k chars) produced correct answers for camels, mob farms and creeper drops.

Which model

M2.7 and M3, both grounded on full articles:

	tokens	outcome
M2.7	6647	correct on camel, mob farm, creeper
M3	7749	refused the mob farm question M2.7 answered well

M2.7 is chosen. M3 costs ~17% more and refuses more. Ungrounded, M3 also invented "poeira de guncotton" as a creeper drop.

The gap wiki text cannot close

explaintext strips tables, and brewing and crafting recipes live in tables. The fire-resistance article is only 1086 characters for this reason, and the recipe question failed even fully grounded. Recipes therefore come from Bukkit.recipeIterator() — the running server's own data, authoritative for this exact version, free and instant.

Constraints

• No restart now. The jar is built and deployed dormant; it goes live at the next restart.
• Cost is not a constraint. The MiniMax Plus plan allows 4.5–12M tokens per 5 hours. At ~2000 tokens per grounded question that is ~1800 questions per window, which a five-player server will never approach. limite-diario stays, but as chat-spam protection, not spend control.
• Latency is the real budget. Answers already take 4–6s. Every added context token makes chat feel slower.

Architecture

/ia <pergunta>
  │
  ├─ permission · module · cooldown · daily cap        (existing)
  │
  ├─ context assembly
  │    contexto.yml      always            server facts, hand written
  │    correcoes.yml     on keyword match  operator corrections
  │    recipe lookup     recipe questions  Bukkit.recipeIterator()
  │    wiki article      PRECISO only      forced tool call → pt.minecraft.wiki
  │    last 3 exchanges  same player, 10 min window
  │
  ├─ MiniMax-M2.7, max_tokens 1200
  │
  └─ sanitise → chat, with reaction buttons

Both HTTP calls run on the async thread that already exists; only delivery hops back to the main thread.

Caching. Wiki articles cached in memory by title, 200 entries, 6h TTL. Five people asking about creepers costs one fetch. Lost on restart, which is fine.

Profiles. ia.perfil: ECONOMICO | PRECISO, switched live with /ia perfil <nome>, no restart. ECONOMICO skips the wiki round trip. Its purpose is latency, not cost.

Safety

The boundary is structural, not prompt-level, and it does not weaken by adding a tool:

• Tools may only read. Never write, never execute, never touch game state. buscar_wiki performs one HTTPS GET against a hardcoded host with a URL-encoded term. The model chooses an argument, never an operation.
• The reply goes to sendMessage and nowhere else. It never reaches the command dispatcher.
• sanitise() strips § codes, markdown, emoji and leading slashes.

A player asking the model to run ls or /give gets a sentence back. There is no function that could do otherwise.

Error handling

failure	behaviour
empty content	retry once at a higher token ceiling, then apologise
reply contains CJK or other foreign script	discard, retry once
wiki 403 / timeout / no hit	answer without the article, and say the wiki was not consulted
MiniMax non-zero base_resp	log and apologise; HTTP 200 does not mean success
tool call absent despite forcing	fall back to answering ungrounded

Features

• Privacy per question. /ia public, /iap visible only to the asker. Operators can force the module private in config.
• Reactions on answers, reusing the existing Reactions class so Bedrock gets typed equivalents.
• Correction loop. The asker marks an answer wrong; an operator writes the right answer with /ia corrigir; it is stored in correcoes.yml and injected on similar future questions. No fine-tuning, no extra API cost.
• New permissions. canalhandia.ia.privado, canalhandia.ia.corrigir, canalhandia.ia.perfil.

Testing

• Unit: sanitise() against markdown, emoji, §, leading slashes, CJK.
• Unit: term extraction from a stubbed tool response; recipe lookup for a known item.
• Integration, against the live key: the five questions in this document, with the camel, mob farm and creeper answers checked for the specific facts they previously got wrong.
• Live: after the next restart, confirm /canalhandia status reports the profile and that a real question is grounded.