canalhandia/docs/plans/2026-08-05-ia-improvements-design.md

Design — grounding and feedback for the /ia module

Date: 2026-08-05 Status: approved

Why

/ia shipped earlier today and was used five times by real players. Every substantive answer contained an error:

asked	answered	reality
how to get a camel	spawns in desert villages "and badlands"	badlands is wrong; never said a saddle is involved
crossing the nether roof	"boat on soul sand, water on top, push the boat"	invented; not a technique
best mob farm spot	"Y 0–64, mobs spawnam mais concentrated there"	dubious, and English leaked into the Portuguese

The model states Minecraft mechanics confidently and incorrectly. No amount of prompt tuning fixes that, because the model does not know what it does not know. It needs sources.

Two further defects were found while measuring:

• Empty answers. M2.7 emits hidden reasoning that counts against max_tokens. At the deployed value of 300 it returned empty content twice in testing. Players see "Não consegui resposta agora" and assume the feature is broken.
• Foreign-token leakage. Replies contained 搭档 (Chinese) and contiennent (French) mid-sentence.

What was measured

Everything below is measured against the live MiniMax key, not assumed.

Retrieval source

pt.minecraft.wiki answers in Portuguese, needs no API key, and responds in ~0.3s. Its camel article contains the exact fact the model missed. It returns HTTP 403 to a default user agent — an identifying User-Agent is required.

How to choose the search term

MediaWiki cannot parse conversational Portuguese, so the raw question is useless as a query. Four methods, same questions:

method	usable terms
raw question	0 / 6
local stopword strip	1 / 6
free-text extraction call	2 / 7
tool call with forced tool_choice	5 / 5

Free-text extraction fails for the same reason answers came back empty: hidden reasoning eats a small max_tokens budget. A tool call returns a structured argument, which survives. Optional tool choice is not enough — with tool_choice: auto the model skipped the search on the hardest question, the one it had already answered wrong twice. Forcing the call fixes that.

How much of the article to send

exintro returns only the lead paragraph. Grounded on intros alone, both models honestly answered "não tenho certeza" to questions they had previously answered correctly — truthful but useless. Sending the full article text (~7k chars) produced correct answers for camels, mob farms and creeper drops.

Which model

M2.7 and M3, both grounded on full articles:

	tokens	outcome
M2.7	6647	correct on camel, mob farm, creeper
M3	7749	refused the mob farm question M2.7 answered well

M2.7 is chosen. M3 costs ~17% more and refuses more. Ungrounded, M3 also invented "poeira de guncotton" as a creeper drop.

The gap wiki text cannot close

explaintext strips tables, and brewing and crafting recipes live in tables. The fire-resistance article is only 1086 characters for this reason, and the recipe question failed even fully grounded.

Crafting and brewing close differently, and an earlier draft of this document was wrong to treat them as one problem.

Crafting comes from the running server. Bukkit.recipeIterator() is authoritative for this exact version, free and instant. It is read through getChoiceMap() / getChoiceList(); the older getIngredientMap() / getIngredientList() are deprecated and collapse a choice to one arbitrary stack, which prints "oak planks" where the recipe really accepts any plank.

Item names are translated through the wiki. Material names are English and players ask in Portuguese, and the two do not meet on their own: scanning every material name for a substring of the question was measured over twenty realistic pt-BR questions and resolved 1 of 20 — and that one, "tridente" containing "trident", by coincidence rather than translation. So the question's subject is searched on the pt wiki and the article's prop=langlinks&lllang=en gives the English title, which uppercases onto the enum: "Espada de Diamante" → "Diamond Sword" → DIAMOND_SWORD. The same twenty questions now resolve 17 of 20. Articles without an English link yield no grounding rather than a guess.

The match is exact, plus a small hand-checked alias table ("Redstone Repeater" → REPEATER, "Book and Quill" → WRITABLE_BOOK). There is deliberately no fuzzy fallback. Resolving to the longest material name contained in the title was tried and returned confidently wrong items: "Redstone Repeater" tied on length and gave REDSTONE, so a player asking about repeaters got the recipe for redstone dust; "Minecart with Chest" gave MINECART; "Rabbit's Foot" gave RABBIT, the raw meat. A suffix rule fails identically — "axe" is a suffix of "pickaxe". Colour and material families (Bed, Wool) therefore resolve to null, which is correct: a plausible recipe for the wrong item is the failure this feature exists to remove, and it is worse than no answer.

Brewing is not exposed by Bukkit at all, so potions use a hardcoded table. Checked against the 26.2 API: there is no brewing Recipe implementation; PotionBrewer has addPotionMix / removePotionMix / resetPotionMixes but no getter and no iterator; and vanilla brewing is hardcoded in PotionBrewing rather than registered as a recipe, so recipeIterator() never yields it. The server genuinely cannot supply this. RecipeBook.BREWING therefore lists the ~17 base potions in pt-BR by hand. This is the part that actually answers the fire-resistance question that motivated the feature — recipeIterator() alone would never have fixed it.

Constraints

• No restart now. The jar is built and deployed dormant; it goes live at the next restart.
• Cost is not a constraint. The MiniMax Plus plan allows 4.5–12M tokens per 5 hours. At ~2000 tokens per grounded question that is ~1800 questions per window, which a five-player server will never approach. limite-diario stays, but as chat-spam protection, not spend control.
• Latency is the real budget. Answers already take 4–6s. Every added context token makes chat feel slower.

Architecture

/ia <pergunta>
  │
  ├─ permission · module · cooldown · daily cap        (existing)
  │
  ├─ context assembly
  │    contexto.yml      always            server facts, hand written
  │    correcoes.yml     on keyword match  operator corrections
  │    recipe lookup     recipe questions  server registry + wiki langlinks;
│                                        potions from a hardcoded table
  │    wiki article      PRECISO only      forced tool call → pt.minecraft.wiki
  │    last 3 exchanges  same player, 10 min window
  │
  ├─ MiniMax-M2.7, max_tokens 1200
  │
  └─ sanitise → chat, with reaction buttons

Both HTTP calls run on the async thread that already exists; only delivery hops back to the main thread.

Caching. Wiki articles cached in memory by title, 200 entries, 6h TTL. Five people asking about creepers costs one fetch. Lost on restart, which is fine.

Profiles. ia.perfil: ECONOMICO | PRECISO, switched live with /ia perfil <nome>, no restart. ECONOMICO skips the wiki round trip. Its purpose is latency, not cost.

Safety

The boundary is structural, not prompt-level, and it does not weaken by adding a tool:

• Tools may only read. Never write, never execute, never touch game state. buscar_wiki performs one HTTPS GET against a hardcoded host with a URL-encoded term. The model chooses an argument, never an operation.
• The reply goes to sendMessage and nowhere else. It never reaches the command dispatcher.
• sanitise() strips § codes, markdown, emoji and leading slashes.

A player asking the model to run ls or /give gets a sentence back. There is no function that could do otherwise.

Error handling

failure	behaviour
empty content	retry once at a higher token ceiling, then apologise
reply contains a non-Latin script (CJK, Cyrillic, Arabic, …)	discard, retry once
wiki 403 / timeout / no hit	answer without the article, and say the wiki was not consulted
MiniMax non-zero base_resp	log and apologise; HTTP 200 does not mean success
tool call absent despite forcing	fall back to answering ungrounded

Known gap: the foreign-script check works by alphabet, so it only catches non-Latin scripts. Latin-script leakage — the observed French contiennent in an otherwise Portuguese answer — passes straight through, and this design does not close that. Catching it would need dictionary or language-identification work that is out of scope here.

Features

• Privacy per question. /ia public, /iap visible only to the asker. Operators can force the module private in config.
• Reactions on answers, reusing the existing Reactions class so Bedrock gets typed equivalents.
• Correction loop. The asker marks an answer wrong; an operator writes the right answer with /ia corrigir; it is stored in correcoes.yml and injected on similar future questions. No fine-tuning, no extra API cost.
• New permissions. canalhandia.ia.privado, canalhandia.ia.corrigir, canalhandia.ia.perfil.

Testing

• Unit: sanitise() against markdown, emoji, §, leading slashes, CJK.
• Unit: term extraction from a stubbed tool response; recipe lookup for a known item.
• Integration, against the live key: the five questions in this document, with the camel, mob farm and creeper answers checked for the specific facts they previously got wrong.
• Live: after the next restart, confirm /canalhandia status reports the profile and that a real question is grounded.