45f197250b
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
14 KiB
14 KiB
Chat layout: design by conversation
Why
Today an agent drives PaperCraft from outside: a terminal running Claude Code or Codex, pointed at the MCP server. The app shows a working pill, the agent runs and their review, but the conversation itself lives elsewhere. OpenDesign and Cursor's agent view put the conversation next to the result. PaperCraft already has everything behind that: 61 tools, agent runs with auto-versions and review, skills, design systems and libraries.
Decisions (user, 2026-10-21)
- Backend: both local agent CLIs (Claude Code, Codex) and a bring-your-own-key Anthropic API path.
- Layout: a separate Chat layout, chat column left, live canvas right, switched from the top bar (
Design | Chat). - Conversations live in the project (
chats/), linked to the agent runs they produce. - Mock-ups first.
What exists today
- MCP server on
127.0.0.1:<PAPERCRAFT_MCP_PORT>(default 29980), loopback only, rejectsOriginheaders (src/mcp/server.rs). Every tool runs on the UI thread throughApp::run_mcp_callsandmcp::exec::call. - Agent runs (
src/runs.rs): editing calls grouped into runs with auto-versions, review, revert, the finish toast. - Tool list and schemas:
mcp::server::tool_list(),crate::mcp::tools::edits_doc. - Skills:
skills/papercraft,skills/papercraft-design,skills/papercraft-design-system(in repo) and the installed copies under~/.agents/skills/. - No HTTP client for outbound API calls except
reqwest(already inCargo.lockthrough rmcp) andcurlused by the CLI's icon download.
Layout
- Mode switch in the top bar:
Design | Chat(segmented control), shortcut ⌘J. The mode is per window and remembered per project. - Chat mode: left column 420–640 px wide (resizable, default 480), canvas on the right with the normal top bar, zoom and a slim tool strip; the layers, inspector and side views are hidden (a toggle brings the inspector back as a narrow overlay).
- Chat column, top to bottom:
- header: chat title (rename), chat list menu (new chat, switch, delete with confirm), agent picker chip (
Claude Code · Opus 5.5,Codex · gpt-6.1-sol,API · Opus 5.5) and a settings icon; - message list: user messages; assistant text streamed as it arrives (markdown: paragraphs, lists, code, inline code, links); thinking/progress shown as a collapsed one-line "Thinking…" row when the backend provides it; tool-call cards: verb + target ("Drawing layout · Card", from
agent_verb), status (running spinner, done, failed with the error), collapsible args/result JSON, a thumbnail forget_screenshotresults; a run card when an agent run ends (Agent finished, N changeswith Review, Revert, Accept, reusing the run review); errors as a red row with Retry; - composer: multi-line input (⏎ sends, ⇧⏎ newline), attachment chips (current selection as
@Frame namechips added by a button or ⌘L, images by paste/drag, the active page), a Stop button while a turn runs, a model/effort menu for the API backend; - empty state: suggestions built from the project ("Design a sign-in screen using this project's variables", "Lint this page and fix warnings", "Make a dark mode").
- header: chat title (rename), chat list menu (new chat, switch, delete with confirm), agent picker chip (
- Patterns from shadcn AI Elements (ui.shadcn.com/docs/ai-elements; React components, so only their patterns and names are reused, built from
src/ui/kit.rs):- Chat thread with a message scroller that sticks to the bottom while streaming and keeps your place when you scroll up (a "Jump to latest" pill appears).
- Message: user and assistant turns; assistant actions on hover (copy, retry).
- Reasoning: collapsible, streams while open, collapses to "Thought for 12s" when done.
- Tool: one card per call, states pending / running / done / error, input and output sections collapsible; a screenshot output shows inline.
- Task / Chain of Thought: consecutive tool calls in one assistant turn group into a step list ("Built the card · 5 steps") that expands.
- Plan / Queue: when the agent keeps a todo list (Claude Code
TodoWrite, or the API backend's plan), show remaining and completed items pinned above the composer. - Confirmation: approve or deny before destructive tools run (
delete_pages,delete_nodeson more than 20 layers,disable_library,import_design_system); API backend pauses the loop; CLI backends rely on the CLI's own permission prompt where it exists, else the tool stays allowed and the run can be reverted. - Checkpoint: each agent run's auto-version shows as a checkpoint in the thread; "Restore to here" reverts the runs after it (reuses
runs::revert). - Context: a small meter in the header with tokens used of the model's context window (API and Claude Code report usage).
- Model Selector: backend + model menu and an effort menu.
- Prompt Input: multi-line field, attachments (selection, images, page), send / stop.
- Suggestion: one-click prompts in the empty state and after a run ("Lint this", "Make a dark mode", "Review changes").
- Subagent: a nested, collapsible block when a CLI agent spawns a sub-agent.
- Code Block: fenced code in replies with a copy button.
- Not used: Sources, Inline Citation, File Tree, Terminal, Stack Trace, Test Results (not design work).
- Canvas side: follows the agent: when a tool call targets a node, the canvas pans to it unless the user moved the camera in the last 10 s; the run overlay (green/yellow/red outlines) shows while a run is open.
- Design mode keeps working as today; the chat keeps running in the background when you switch.
Backends
All backends drive the same tools, so runs, undo, review and revert work the same.
Local agent CLIs
- Claude Code: spawn
claude -pwith streaming JSON in and out (--input-format stream-json --output-format stream-json --verbose), an MCP config pointing at the app's server (--mcp-configwith{"mcpServers":{"papercraft":{"type":"http","url":"http://127.0.0.1:<port>/mcp"}}}), the papercraft tools allowed (--allowedTools "mcp__papercraft__*"), working directory = the project folder, and the system prompt appended with the PaperCraft skill text. Resume with--resume <session_id>(stored on the chat). Exact flags are checked againstclaude --helpat implementation time and wrapped in one adapter. - Codex: spawn
codex exec --jsonwith the server added through-c mcp_servers.papercraft.url=...(or the current equivalent, checked againstcodex exec --help), resume with its session id. Same adapter shape. - Detection: Settings → Chat lists the CLIs found on
PATH(plus a path override) with their version; a missing CLI shows install hints, never an error dialog. - Events: each adapter turns the CLI's JSON lines into one internal event stream:
Text(delta),Thinking(delta),ToolStart{id,name,args},ToolEnd{id,result|error},TurnEnd{usage?},Error. Tool calls also arrive through the MCP server, so the card status is confirmed by the app itself (log_agent), not only by the CLI's report. - Stop: sends an interrupt (stdin control message where supported, else SIGINT, then kill after 3 s). The run so far stays reviewable.
- Security: the CLI runs with the user's own permissions and logins; PaperCraft passes no keys. Only papercraft tools are pre-allowed; other tools follow the CLI's own permission settings (non-interactive: denied).
Anthropic API (bring your own key)
- Raw HTTPS to
POST https://api.anthropic.com/v1/messageswithstream: true(no Rust SDK exists), usingreqwest(already a dependency). - Key: entered in Settings → Chat, stored in the macOS Keychain (Linux: Secret Service; Windows: Credential Manager) through the
keyringcrate, never in project files or logs.ANTHROPIC_API_KEYin the environment is used when no key is stored. - Model: default
claude-opus-5-5; menu also offersclaude-sonnet-5-5andclaude-haiku-5-5. Thinking adaptive (default on these models),output_config.effortset explicitly (defaultmedium, menu low/medium/high),thinking.display: "summarized"so the Thinking row has text. Server-side refusal fallback on (fallbacks: "default"with betaserver-side-fallback-2026-07-01; for Haiku, show the refusal instead). - Tools: the app's own tool list (
tool_list()converted to Messages APItools: name, description, input_schema), witheager_input_streaming: true; inputs are parsed tolerantly, validated against the schema before running, and run throughmcp::exec::callon the UI thread like any MCP call (so runs and undo behave the same). Severaltool_useblocks in one turn run in order and alltool_resultblocks go back in one user message; failures go back asis_error: true. No forcedtool_choice(not supported on these models). - Loop: stream → collect
tool_use→ run → appendresponse.contentunchanged (thinking blocks included) + tool results → next request, untilstop_reasonisend_turn;max_tokens→ continue once with a note;refusal→ show the category;pause_turn→ continue. Max 40 tool rounds per user message (then stop and say so). - Caching:
cache_controlbreakpoints on the tool list and on the system prompt (skill text + project context), stable ordering so they hit; per-turn volatile context (selection, active page) goes in the user message, not the system prompt.usage.cache_read_input_tokensis logged to verify. - Images:
get_screenshotresults go back as image blocks so the model sees what it made; user-attached images too. - Cost line: per turn input/output/cache tokens and an estimated cost from a small price table, shown under the turn in a muted row.
- Implementers read the bundled claude-api skill files
curl/examples.md,shared/tool-use-concepts.md,shared/prompt-caching.md,shared/model-migration.md(Opus 5.5 section) before writing this adapter.
Custom endpoints (generic, no provider list)
The API backend is not tied to Anthropic. Settings → Chat holds one or more endpoints, each { name, format, base_url, model, key }:
format:anthropic(Messages API,POST {base}/v1/messages,x-api-key) oropenai(Chat Completions,POST {base}/chat/completions,Authorization: Bearer, streaming SSEchoices[].delta, tools as{"type":"function","function":{name,description,parameters}},tool_callswithargumentsstreamed as string fragments, results asrole: "tool"messages).base_url: free text (e.g.https://api.anthropic.com, any Anthropic-compatible URL,http://127.0.0.1:11434/v1).model: free text.key: optional (local servers need none), stored in the OS keychain under the endpoint's base URL, never in settings files.- Claude-only request fields (
output_config.effort,thinking.display,fallbacksand its beta header,eager_input_streaming) are sent only when the base URL host isapi.anthropic.com; other Anthropic-format endpoints get a plain request (adaptive thinking omitted), and are tolerant of missingusage,thinkingblocks, orcache_*fields. - The cost line shows a price only for models in the built-in Claude price table; otherwise tokens only (or nothing when the server reports no usage).
- No provider names, presets or logos in the app. A default endpoint
Anthropic(anthropic,https://api.anthropic.com,claude-opus-5-5) exists so the first-run path stays one key away. - The agent picker lists every configured endpoint by its name.
Not in scope (v1)
- Named provider presets.
- Multiple chats running at once in one project (one active turn per project).
- Voice, file uploads other than images.
Storage
chats/<id>.jsonin the project folder:{ id, title, created, updated, backend: {kind, model?, effort?, cli_session?}, messages: [ {role, text, tools:[{id,name,args,result,error,ms}], run_id?, usage?, at} ] }. API backend also keeps the rawmessagesarray (for exact replay, including thinking blocks) inchats/<id>.api.json.- Chats sync with the project (relay). Deleting a chat deletes both files after a confirm.
- The project's newest chat opens when entering Chat mode.
MCP and CLI
- No new tools needed.
papercraft-cliis unchanged.
Acceptance
- Chat mode toggles from the top bar and ⌘J; layout matches the mock-ups at 1440p and 1280x800.
- With Claude Code installed: "Create a 390x844 artboard named Test with a centred title" streams text, shows tool cards for
create_artboardandwrite_html, the canvas follows, a run card appears at the end, Review/Revert work; a follow-up message continues the same CLI session. - Same with Codex.
- With an API key: the same prompt works through the API backend; thinking row shows a summary; cost line shows tokens; a second turn reads from cache (
cache_read_input_tokens > 0). - Stop during a long turn ends it within 3 s; tools already applied stay and are revertible.
- Missing CLI and missing key show setup states, not errors; a bad key shows the API error message.
- Chats persist in
chats/and reopen after restart; a project withoutchats/loads unchanged. - The key never appears in project files, logs or crash output (test greps the project folder and the log after a run).
- Tests: event parsing for each adapter from recorded JSON fixtures (Claude Code stream-json, Codex --json, Anthropic SSE incl. thinking, tool_use with input_json_delta, refusal, max_tokens); tool-schema conversion; chat file round trip; the API loop with a fake HTTP server (two tool rounds then end_turn).
cargo test --no-fail-fast, wasm and Windows checks pass (chat backends are native-only; the web build shows Chat mode disabled with a note).