marcos
dba64df917
Crafting recipes come from Bukkit.recipeIterator(), which is authoritative
for this exact version. Player questions are Portuguese and Material names
are English, so the subject of the question is searched on the pt wiki and
prop=langlinks&lllang=en gives the English title, which uppercases onto the
enum constant.
Matching material names against the question directly does not work.
Measured over twenty realistic pt-BR questions it resolved 1 of 20, and that
one ("tridente" containing "trident") by coincidence rather than
translation. Through langlinks the same twenty resolve 17 of 20.
Brewing is not exposed by Bukkit at all: there is no brewing Recipe type,
PotionBrewer has no getter or iterator, and vanilla brewing is hardcoded in
PotionBrewing rather than registered as a recipe. Potions therefore come
from a hand-written pt-BR table. This is what actually answers the
fire-resistance question that motivated the feature; recipeIterator() alone
never could have.
Ingredients are read through getChoiceMap/getChoiceList. The deprecated
getIngredientMap/getIngredientList collapse a choice to one arbitrary stack,
printing "oak planks" where the recipe accepts any plank.
The design doc claimed recipeIterator() closed the potion case. It did not;
corrected to record what is true.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F3jSvSTrG4qsniLSr6TpC6
208 lines
9.0 KiB
Markdown
208 lines
9.0 KiB
Markdown
# Design — grounding and feedback for the `/ia` module
|
||
|
||
**Date:** 2026-08-05
|
||
**Status:** approved
|
||
|
||
## Why
|
||
|
||
`/ia` shipped earlier today and was used five times by real players. Every
|
||
substantive answer contained an error:
|
||
|
||
| asked | answered | reality |
|
||
| --- | --- | --- |
|
||
| how to get a camel | spawns in desert villages "and badlands" | badlands is wrong; never said a saddle is involved |
|
||
| crossing the nether roof | "boat on soul sand, water on top, push the boat" | invented; not a technique |
|
||
| best mob farm spot | "Y 0–64, mobs spawnam mais *concentrated there*" | dubious, and English leaked into the Portuguese |
|
||
|
||
The model states Minecraft mechanics confidently and incorrectly. No amount of
|
||
prompt tuning fixes that, because the model does not know what it does not
|
||
know. It needs sources.
|
||
|
||
Two further defects were found while measuring:
|
||
|
||
- **Empty answers.** M2.7 emits hidden reasoning that counts against
|
||
`max_tokens`. At the deployed value of 300 it returned empty content twice in
|
||
testing. Players see "Não consegui resposta agora" and assume the feature is
|
||
broken.
|
||
- **Foreign-token leakage.** Replies contained `搭档` (Chinese) and
|
||
`contiennent` (French) mid-sentence.
|
||
|
||
## What was measured
|
||
|
||
Everything below is measured against the live MiniMax key, not assumed.
|
||
|
||
### Retrieval source
|
||
|
||
`pt.minecraft.wiki` answers in Portuguese, needs no API key, and responds in
|
||
~0.3s. Its camel article contains the exact fact the model missed. It returns
|
||
**HTTP 403 to a default user agent** — an identifying `User-Agent` is required.
|
||
|
||
### How to choose the search term
|
||
|
||
MediaWiki cannot parse conversational Portuguese, so the raw question is
|
||
useless as a query. Four methods, same questions:
|
||
|
||
| method | usable terms |
|
||
| --- | --- |
|
||
| raw question | 0 / 6 |
|
||
| local stopword strip | 1 / 6 |
|
||
| free-text extraction call | 2 / 7 |
|
||
| **tool call with forced `tool_choice`** | **5 / 5** |
|
||
|
||
Free-text extraction fails for the same reason answers came back empty: hidden
|
||
reasoning eats a small `max_tokens` budget. A tool call returns a *structured*
|
||
argument, which survives. Optional tool choice is not enough — with
|
||
`tool_choice: auto` the model skipped the search on the hardest question, the
|
||
one it had already answered wrong twice. Forcing the call fixes that.
|
||
|
||
### How much of the article to send
|
||
|
||
`exintro` returns only the lead paragraph. Grounded on intros alone, both
|
||
models honestly answered "não tenho certeza" to questions they had previously
|
||
answered correctly — truthful but useless. Sending the **full article text**
|
||
(~7k chars) produced correct answers for camels, mob farms and creeper drops.
|
||
|
||
### Which model
|
||
|
||
M2.7 and M3, both grounded on full articles:
|
||
|
||
| | tokens | outcome |
|
||
| --- | --- | --- |
|
||
| **M2.7** | 6647 | correct on camel, mob farm, creeper |
|
||
| M3 | 7749 | refused the mob farm question M2.7 answered well |
|
||
|
||
M2.7 is chosen. M3 costs ~17% more and refuses more. Ungrounded, M3 also
|
||
invented "poeira de guncotton" as a creeper drop.
|
||
|
||
### The gap wiki text cannot close
|
||
|
||
`explaintext` strips tables, and brewing and crafting recipes live in tables.
|
||
The fire-resistance article is only 1086 characters for this reason, and the
|
||
recipe question failed even fully grounded.
|
||
|
||
Crafting and brewing close differently, and an earlier draft of this document
|
||
was wrong to treat them as one problem.
|
||
|
||
**Crafting comes from the running server.** `Bukkit.recipeIterator()` is
|
||
authoritative for this exact version, free and instant. It is read through
|
||
`getChoiceMap()` / `getChoiceList()`; the older `getIngredientMap()` /
|
||
`getIngredientList()` are deprecated and collapse a choice to one arbitrary
|
||
stack, which prints "oak planks" where the recipe really accepts any plank.
|
||
|
||
**Item names are translated through the wiki.** `Material` names are English
|
||
and players ask in Portuguese, and the two do not meet on their own: scanning
|
||
every material name for a substring of the question was measured over twenty
|
||
realistic pt-BR questions and resolved **1 of 20** — and that one, "tridente"
|
||
containing "trident", by coincidence rather than translation. So the question's
|
||
subject is searched on the pt wiki and the article's `prop=langlinks&lllang=en`
|
||
gives the English title, which uppercases onto the enum: "Espada de Diamante"
|
||
→ "Diamond Sword" → `DIAMOND_SWORD`. The same twenty questions now resolve
|
||
**17 of 20**. Articles without an English link (real case: "Mesa de
|
||
Encantamento") yield no grounding rather than a guess.
|
||
|
||
**Brewing is not exposed by Bukkit at all, so potions use a hardcoded table.**
|
||
Checked against the 26.2 API: there is no brewing `Recipe` implementation;
|
||
`PotionBrewer` has `addPotionMix` / `removePotionMix` / `resetPotionMixes` but
|
||
no getter and no iterator; and vanilla brewing is hardcoded in `PotionBrewing`
|
||
rather than registered as a recipe, so `recipeIterator()` never yields it. The
|
||
server genuinely cannot supply this. `RecipeBook.BREWING` therefore lists the
|
||
~17 base potions in pt-BR by hand. This is the part that actually answers the
|
||
fire-resistance question that motivated the feature — `recipeIterator()` alone
|
||
would never have fixed it.
|
||
|
||
## Constraints
|
||
|
||
- **No restart now.** The jar is built and deployed dormant; it goes live at
|
||
the next restart.
|
||
- **Cost is not a constraint.** The MiniMax Plus plan allows 4.5–12M tokens per
|
||
5 hours. At ~2000 tokens per grounded question that is ~1800 questions per
|
||
window, which a five-player server will never approach. `limite-diario`
|
||
stays, but as chat-spam protection, not spend control.
|
||
- **Latency is the real budget.** Answers already take 4–6s. Every added
|
||
context token makes chat feel slower.
|
||
|
||
## Architecture
|
||
|
||
```
|
||
/ia <pergunta>
|
||
│
|
||
├─ permission · module · cooldown · daily cap (existing)
|
||
│
|
||
├─ context assembly
|
||
│ contexto.yml always server facts, hand written
|
||
│ correcoes.yml on keyword match operator corrections
|
||
│ recipe lookup recipe questions server registry + wiki langlinks;
|
||
│ potions from a hardcoded table
|
||
│ wiki article PRECISO only forced tool call → pt.minecraft.wiki
|
||
│ last 3 exchanges same player, 10 min window
|
||
│
|
||
├─ MiniMax-M2.7, max_tokens 1200
|
||
│
|
||
└─ sanitise → chat, with reaction buttons
|
||
```
|
||
|
||
Both HTTP calls run on the async thread that already exists; only delivery hops
|
||
back to the main thread.
|
||
|
||
**Caching.** Wiki articles cached in memory by title, 200 entries, 6h TTL.
|
||
Five people asking about creepers costs one fetch. Lost on restart, which is
|
||
fine.
|
||
|
||
**Profiles.** `ia.perfil: ECONOMICO | PRECISO`, switched live with
|
||
`/ia perfil <nome>`, no restart. ECONOMICO skips the wiki round trip. Its
|
||
purpose is latency, not cost.
|
||
|
||
## Safety
|
||
|
||
The boundary is structural, not prompt-level, and it does not weaken by adding
|
||
a tool:
|
||
|
||
- **Tools may only read.** Never write, never execute, never touch game state.
|
||
`buscar_wiki` performs one HTTPS GET against a hardcoded host with a
|
||
URL-encoded term. The model chooses an *argument*, never an operation.
|
||
- The reply goes to `sendMessage` and nowhere else. It never reaches the
|
||
command dispatcher.
|
||
- `sanitise()` strips `§` codes, markdown, emoji and leading slashes.
|
||
|
||
A player asking the model to run `ls` or `/give` gets a sentence back. There is
|
||
no function that could do otherwise.
|
||
|
||
## Error handling
|
||
|
||
| failure | behaviour |
|
||
| --- | --- |
|
||
| empty content | retry once at a higher token ceiling, then apologise |
|
||
| reply contains a non-Latin script (CJK, Cyrillic, Arabic, …) | discard, retry once |
|
||
| wiki 403 / timeout / no hit | answer without the article, and say the wiki was not consulted |
|
||
| MiniMax non-zero `base_resp` | log and apologise; HTTP 200 does not mean success |
|
||
| tool call absent despite forcing | fall back to answering ungrounded |
|
||
|
||
Known gap: the foreign-script check works by alphabet, so it only catches
|
||
non-Latin scripts. Latin-script leakage — the observed French `contiennent` in
|
||
an otherwise Portuguese answer — passes straight through, and this design does
|
||
not close that. Catching it would need dictionary or language-identification
|
||
work that is out of scope here.
|
||
|
||
## Features
|
||
|
||
- **Privacy per question.** `/ia` public, `/iap` visible only to the asker.
|
||
Operators can force the module private in config.
|
||
- **Reactions** on answers, reusing the existing `Reactions` class so Bedrock
|
||
gets typed equivalents.
|
||
- **Correction loop.** The asker marks an answer wrong; an operator writes the
|
||
right answer with `/ia corrigir`; it is stored in `correcoes.yml` and
|
||
injected on similar future questions. No fine-tuning, no extra API cost.
|
||
- **New permissions.** `canalhandia.ia.privado`, `canalhandia.ia.corrigir`,
|
||
`canalhandia.ia.perfil`.
|
||
|
||
## Testing
|
||
|
||
- Unit: `sanitise()` against markdown, emoji, `§`, leading slashes, CJK.
|
||
- Unit: term extraction from a stubbed tool response; recipe lookup for a known
|
||
item.
|
||
- Integration, against the live key: the five questions in this document, with
|
||
the camel, mob farm and creeper answers checked for the specific facts they
|
||
previously got wrong.
|
||
- Live: after the next restart, confirm `/canalhandia status` reports the
|
||
profile and that a real question is grounded.
|