canalhandia/docs/plans/2026-08-05-ia-improvements-design.md

# Design — grounding and feedback for the `/ia` module

**Date:** 2026-08-05
**Status:** approved

## Why

`/ia` shipped earlier today and was used five times by real players. Every
substantive answer contained an error:

| asked | answered | reality |
| --- | --- | --- |
| how to get a camel | spawns in desert villages "and badlands" | badlands is wrong; never said a saddle is involved |
| crossing the nether roof | "boat on soul sand, water on top, push the boat" | invented; not a technique |
| best mob farm spot | "Y 0–64, mobs spawnam mais *concentrated there*" | dubious, and English leaked into the Portuguese |

The model states Minecraft mechanics confidently and incorrectly. No amount of
prompt tuning fixes that, because the model does not know what it does not
know. It needs sources.

Two further defects were found while measuring:

- **Empty answers.** M2.7 emits hidden reasoning that counts against
  `max_tokens`. At the deployed value of 300 it returned empty content twice in
  testing. Players see "Não consegui resposta agora" and assume the feature is
  broken.
- **Foreign-token leakage.** Replies contained `搭档` (Chinese) and
  `contiennent` (French) mid-sentence.

## What was measured

Everything below is measured against the live MiniMax key, not assumed.

### Retrieval source

`pt.minecraft.wiki` answers in Portuguese, needs no API key, and responds in
~0.3s. Its camel article contains the exact fact the model missed. It returns
**HTTP 403 to a default user agent** — an identifying `User-Agent` is required.

### How to choose the search term

MediaWiki cannot parse conversational Portuguese, so the raw question is
useless as a query. Four methods, same questions:

| method | usable terms |
| --- | --- |
| raw question | 0 / 6 |
| local stopword strip | 1 / 6 |
| free-text extraction call | 2 / 7 |
| **tool call with forced `tool_choice`** | **5 / 5** |

Free-text extraction fails for the same reason answers came back empty: hidden
reasoning eats a small `max_tokens` budget. A tool call returns a *structured*
argument, which survives. Optional tool choice is not enough — with
`tool_choice: auto` the model skipped the search on the hardest question, the
one it had already answered wrong twice. Forcing the call fixes that.

### How much of the article to send

`exintro` returns only the lead paragraph. Grounded on intros alone, both
models honestly answered "não tenho certeza" to questions they had previously
answered correctly — truthful but useless. Sending the **full article text**
(~7k chars) produced correct answers for camels, mob farms and creeper drops.

### Which model

M2.7 and M3, both grounded on full articles:

| | tokens | outcome |
| --- | --- | --- |
| **M2.7** | 6647 | correct on camel, mob farm, creeper |
| M3 | 7749 | refused the mob farm question M2.7 answered well |

M2.7 is chosen. M3 costs ~17% more and refuses more. Ungrounded, M3 also
invented "poeira de guncotton" as a creeper drop.

### The gap wiki text cannot close

`explaintext` strips tables, and brewing and crafting recipes live in tables.
The fire-resistance article is only 1086 characters for this reason, and the
recipe question failed even fully grounded. Recipes therefore come from
`Bukkit.recipeIterator()` — the running server's own data, authoritative for
this exact version, free and instant.

## Constraints

- **No restart now.** The jar is built and deployed dormant; it goes live at
  the next restart.
- **Cost is not a constraint.** The MiniMax Plus plan allows 4.5–12M tokens per
  5 hours. At ~2000 tokens per grounded question that is ~1800 questions per
  window, which a five-player server will never approach. `limite-diario`
  stays, but as chat-spam protection, not spend control.
- **Latency is the real budget.** Answers already take 4–6s. Every added
  context token makes chat feel slower.

## Architecture

```
/ia <pergunta>
  │
  ├─ permission · module · cooldown · daily cap        (existing)
  │
  ├─ context assembly
  │    contexto.yml      always            server facts, hand written
  │    correcoes.yml     on keyword match  operator corrections
  │    recipe lookup     recipe questions  Bukkit.recipeIterator()
  │    wiki article      PRECISO only      forced tool call → pt.minecraft.wiki
  │    last 3 exchanges  same player, 10 min window
  │
  ├─ MiniMax-M2.7, max_tokens 1200
  │
  └─ sanitise → chat, with reaction buttons
```

Both HTTP calls run on the async thread that already exists; only delivery hops
back to the main thread.

**Caching.** Wiki articles cached in memory by title, 200 entries, 6h TTL.
Five people asking about creepers costs one fetch. Lost on restart, which is
fine.

**Profiles.** `ia.perfil: ECONOMICO | PRECISO`, switched live with
`/ia perfil <nome>`, no restart. ECONOMICO skips the wiki round trip. Its
purpose is latency, not cost.

## Safety

The boundary is structural, not prompt-level, and it does not weaken by adding
a tool:

- **Tools may only read.** Never write, never execute, never touch game state.
  `buscar_wiki` performs one HTTPS GET against a hardcoded host with a
  URL-encoded term. The model chooses an *argument*, never an operation.
- The reply goes to `sendMessage` and nowhere else. It never reaches the
  command dispatcher.
- `sanitise()` strips `§` codes, markdown, emoji and leading slashes.

A player asking the model to run `ls` or `/give` gets a sentence back. There is
no function that could do otherwise.

## Error handling

| failure | behaviour |
| --- | --- |
| empty content | retry once at a higher token ceiling, then apologise |
| reply contains a non-Latin script (CJK, Cyrillic, Arabic, …) | discard, retry once |
| wiki 403 / timeout / no hit | answer without the article, and say the wiki was not consulted |
| MiniMax non-zero `base_resp` | log and apologise; HTTP 200 does not mean success |
| tool call absent despite forcing | fall back to answering ungrounded |

Known gap: the foreign-script check works by alphabet, so it only catches
non-Latin scripts. Latin-script leakage — the observed French `contiennent` in
an otherwise Portuguese answer — passes straight through, and this design does
not close that. Catching it would need dictionary or language-identification
work that is out of scope here.

## Features

- **Privacy per question.** `/ia` public, `/iap` visible only to the asker.
  Operators can force the module private in config.
- **Reactions** on answers, reusing the existing `Reactions` class so Bedrock
  gets typed equivalents.
- **Correction loop.** The asker marks an answer wrong; an operator writes the
  right answer with `/ia corrigir`; it is stored in `correcoes.yml` and
  injected on similar future questions. No fine-tuning, no extra API cost.
- **New permissions.** `canalhandia.ia.privado`, `canalhandia.ia.corrigir`,
  `canalhandia.ia.perfil`.

## Testing

- Unit: `sanitise()` against markdown, emoji, `§`, leading slashes, CJK.
- Unit: term extraction from a stubbed tool response; recipe lookup for a known
  item.
- Integration, against the live key: the five questions in this document, with
  the camel, mob farm and creeper answers checked for the specific facts they
  previously got wrong.
- Live: after the next restart, confirm `/canalhandia status` reports the
  profile and that a real question is grounded.