indd-utils/ARCHITECTURE.md

Architecture

This document explains how the converter is built: its layers, how data flows from INDD bytes to an IDML package, where each kind of code lives, where the evidence for format facts is kept, and how the generated value files work. For how to change the code, see CONTRIBUTING.md.

Layers

Each layer uses only the layers above it in this table.

Layer	Module	Input → output	Knows about
Header	src/header.rs	first bytes → Header (byte order, version)	the fixed header
Container	src/container.rs	file bytes → pages, master page, contiguous objects	the page layout
Database	src/database.rs	pages → object bytes by UID, class of each UID	the object trees and data pages
Objects	src/object.rs	object bytes → chunks; Cursor decodes numbers, strings and text	the byte order and string tag (Encoding)
Model	src/model/	objects → Document	INDD classes, chunks and attribute IDs
Writer	src/idml/	Document → IDML package (ZIP)	IDML elements, attributes and naming rules
Audit	src/audit.rs	what the model read → Audit report	nothing of the format; records reads

The model holds decoded INDD data. The writer only maps model types to IDML: it does not read INDD bytes. When the writer needs a new INDD value, decode it in the model and add a field or a Value variant.

Data flow

indd::convert_into (src/lib.rs) runs these steps:

1. Container::parse checks the header and splits the file into pages.
2. Container::database reads the object trees into a Database. The database gets the file's Encoding from the header.
3. model::Reader::document reads the document object (UID 1), then every object of the class list whose class it knows, and builds a Document. Each object comes from Database::get and carries the Encoding, so every decoder reads it in the file's byte order.
4. idml::write writes the package parts (designmap.xml, spreads, stories, resources, styles) and stores them in a ZIP file.
5. The warnings of the model and of the writer are returned with the package.

Step 3 also records, for each placed graphic whose file the document does not hold, the raw data object of its preview (Graphic::preview). indd::convert returns the package as bytes, with those previews, read after step 4 from the database; indd::convert_file reads a file first. The CLI (src/main.rs) uses convert_into to write to a file, which does not read the preview data.

No global state

A conversion keeps all its state in its own values:

• The byte order and the string tag are an Encoding, held by the Database, each Object and each Cursor.
• The Reader holds its object cache, warnings and the state it collects while reading (XML nodes, frame orientations, the group path).
• indd audit gives the Database an audit::Recorder; objects and attribute lists carry it and report their reads to it. A normal conversion has no recorder.

The only statics are caches of the constant value files (OnceLock). Documents can therefore be converted on several threads at once; the fixture test conversions_on_several_threads_are_independent checks it.

Errors and warnings

Problem	Result
The file is not an INDD file, is truncated, or its database structures are damaged	Err(Error): no package
One object cannot be read (a story, spread, page item, style, colour, …)	The object is left out with a warning
An attribute list cannot be parsed	The list is left out with a warning
A value has no IDML equivalent, or one the samples do not show	The value is left out, with a warning where a reader would miss it; unknown codes are reported to indd audit

The model also guards against damaged structures that would recurse without end or allocate too much: groups that contain themselves, style group cycles, table sizes larger than their cell data.

Module map

src/model/

Module	Reads
reader.rs	Reader: object cache, chunk access, child lists, warnings
document.rs	Document; Reader::document and one function per class or area
ids.rs	class, chunk and strand IDs and enumeration codes
attrs.rs	attribute lists (Attrs, Value), including structured values: tab lists, nested styles, bullet characters, opacity stops
spread.rs	spreads, pages, guides, layers, sections
item.rs	page items: transforms, paths, frames, form fields and their states, export options, graphics and their properties and layers, links, text wrap
story.rs	stories: text runs, anchored objects, tables in text, hyperlink sources, XML markers
style.rs	paragraph, character, object and TOC styles, style groups, anchored object, text frame and story settings
table.rs	tables, rows, columns, cells, cell and table styles
color.rs, font.rs, cjk.rs	swatches and inks; font families; composite fonts and kinsoku and mojikumi tables
hyperlink.rs, xref.rs, variable.rs, xml.rs	hyperlinks and bookmarks; cross-reference formats; text variables; the XML structure
settings.rs, prefs.rs	document-level lists and preferences
xmp.rs	dates of the XMP packet (time zone offsets for link times) and its black swatch name (the saving edition)
strings.rs	searches for names at positions that are not decoded

src/idml/

Module	Writes
mod.rs	Writer and write: the package parts in order
designmap.rs	designmap.xml: document settings, languages, layers, sections, hyperlinks, TOC styles, text variables
spread.rs	spreads and master spreads: pages, guides, page items, form fields
graphic.rs	placed graphics: settings, clipping, import options, layers, links
export.rs	ObjectExportOption of page items and object styles, by version
story.rs	stories, tables in stories, the XML backing story
styles.rs	Resources/Styles.xml
resources.rs	Resources/Graphic.xml, Fonts.xml, Preferences.xml
pages.rs	page numbering and page names
attrs.rs	attribute tables: INDD attribute ID → IDML name and value kind
kind.rs	Kind: how each kind of value becomes IDML text
transparency.rs	transparency effects of page items
applied.rs	the effective values of an object style, which page item values are compared with
values.rs	lookups in the generated value files
format.rs	number, matrix, base64 and name formats
xml.rs, zip.rs	the XML and ZIP writers

Where evidence lives

What	Where
How the format was learned, and the rules for new facts	CLEANROOM.md
Every fact about the INDD format, with the files and counts that support it	docs/format/, one file per area
Values written because every InDesign export has them	docs/format/idml-values.md
How fidelity is measured (stale pairs, value coverage)	docs/measurement.md
The samples	corpus/, local and git-ignored; corpus/inventory.tsv lists version and pairing

Code comments cite the docs/format/ section a decoder relies on. Code uses only facts recorded there.

Generated value files

IDML consumers apply their own defaults to attributes that are missing. To avoid that, the writer also writes values that every InDesign export in the corpus has, even when the INDD has no field for them. These values are facts about InDesign's IDML output, so their evidence is the corpus IDML files alone (idml-values.md).

File	Generator	Holds	Read by (src/idml/values.rs)
src/idml/element_values.xml	tools/element_values.py	values of page items, spreads, pages, swatches, styles, document elements and preferences; lists such as trap presets; values keyed on another attribute	element, element_attrs, when_written, present, list, keyed
src/idml/root_values.xml	tools/root_values.py	values of the five root styles	root_style
src/idml/object_style_values.xml	tools/root_values.py	object styles other than [None]	object_style
src/idml/preference_values.xml	tools/root_values.py	the elements of Resources/Preferences.xml	preferences

How they work:

• A generator reads every corpus IDML and keeps a value when every element on the path has it, in every file of a range of DOM versions. Blocks in the file carry that range (MinimumVersion, MaximumVersion); the writer asks for the document's version.
• The files are committed. Regenerate them with --write after the corpus changes, and review the diff: a value that varies in new samples disappears.
• root_values.py collects the root styles, the other object styles and the preferences from the pairs that compare.py compares.

Precedence. A value read from the INDD always wins over an observed value. The XML writer (src/idml/xml.rs) keeps the attributes of an open start tag until it closes: Xml::attr sets a value, replacing an observed one in place, and Xml::attrs_missing adds observed values only for attributes the element does not have. For value trees (values::Node), Node::set replaces and Node::merge adds missing values.

A few observed values are still written as literals in the writer, where the generators cannot express them: AccurateLABSpots of the document (one DOM 7 file lacks it) and the default numbering list for a file whose lists cannot be read. Each cites idml-values.md.

Tests and tools

Check	What it covers
Unit tests (cargo test)	decoders on synthetic bytes (database::synthetic builds objects and databases), writer functions on small models
tests/fixtures.rs	conversion of open-licensed fixtures (tools/fetch_fixtures.py), both byte orders, thread independence, audit consistency
tests/corpus.rs	every corpus file parses and converts; skipped without corpus/
src/hostile.rs	truncated and byte-flipped synthetic documents and fixtures convert or fail without a panic or a hang (INDD_HOSTILE_SCALE, INDD_HOSTILE_SEED for larger sweeps)

| tools/compare.py | fidelity against the reference IDML files, schema validation | | tools/diff_outputs.py | byte-for-byte output of two revisions over the corpus | | tools/check_wasm.sh | the library builds for wasm32-unknown-unknown, a conversion there imports no host function, and its output is the same as native |

| indd audit, tools/audit_corpus.py | what the converter does not read yet |