IDML writes a style's text attributes in ascending attribute ID order.
The IDs between known ones, with their payload types, place 79 more
attributes: ruby, kenten, tatechuyoko and warichu settings, OpenType
features, gap tints and colours, overprint flags, paragraph border
options and others. Styles and text runs write them wherever they are
stored; the root style values come from the INDD instead of the
observed values.
- BeforeGroupingLayerReference from the u32 at 0 of chunk 0x324, a
spread layer resolved to its layer.
- TableStyle CaptionPosition from DOM 21.5: attribute 0x1047D = 1 gives
BeforeTable, absent gives AfterTable.
- AIGeneratedAltText, AltTextGenerationError and AltTextCropSyncRect of
page items from chunks 0x1E262, 0x1E260 and 0x1E261.
- The Name of a nested style group is its full path joined by ':'.
An image without a link whose chunk 0x8C23 is 0 or absent keeps its
file in its preview (0x170D -> class 0x1708 -> 0x119 -> raw data
object). Images of InDesign 2.0 to 5.0 whose chunk 0x8C1B names a class
0x8C01 object keep the preview as a preview.
`read_previews` also collects the page items of graphic table cells and
the items of form field states, which the package holds as well. Over
the open corpus this adds previews for 657 graphics in cells and 385 in
states; every cell graphic without `Contents` has one. Items anchored in
cell text were already in the story anchors.
A preview chain that can't be read gives no preview instead of failing
the graphic, and with it the conversion.
The README and ARCHITECTURE.md say that every conversion looks up each
graphic's preview object; only `convert` reads the preview data.
`indd::convert` now returns `Conversion::previews`: InDesign's screen
preview (chunk 0x170D -> class 0x1708 -> chunk 0x119 -> raw data) of each
placed graphic without `Contents`, one entry per preview file with the
`Self` of every graphic that uses it. IDML has no place for a preview, so
the package is unchanged; an application can show the preview when it
cannot read a linked file. `convert_into` and `indd audit` do not read
previews.
Output: `diff_outputs.py --exclude own/` over 23 files (the fixtures and
part of the open corpus): 0 differing outputs, 0 changed warnings or
failures.
Stories and XML stories get the Ai* attributes from DOM 21.5; links the
Ai* attributes from 21.5 and SourceObjectName from 21.4; every typed
ContourOption the four contour parameters from 21.4; object styles
EpubAriaLabel and EpubAriaLabelSourceType from 21.2; designmap.xml
Html5ExportPreference from 21.1. LockPosition and AnchorSpaceAbove go on
every AnchoredObjectSetting with all ten settings, polygons included,
in place of the per-tag WhenWritten blocks.
Chunk 0x2C1A gives the clipping type (None, DetectEdges, AlphaChannel,
PhotoshopPath) and a flags byte (high-resolution image, invert and
restrict to frame). Graphics of types 1 to 3 write their chunk 0x162B
paths as ClippingPathGeometry; PhotoshopPath names come from key 0x7D
of the image properties when ASCII, AlphaChannel names from chunk
0x177C. Placed pages get the element with the values of graphics
without the chunk.
Chunk 0x808 of the preferences is the ruler origin (0 spread, 1 page,
2 spine). With PageOrigin a page guide is measured from its own page,
with SpineOrigin a vertical guide on a two-page spread from the spine.
The difference is taken after adding the spread's translation, which
gives IDML's 0 for guides on the origin.
A story without chunk 0x2EE has OpticalMarginSize 9.2125984251969 (13 Q)
when the text size unit is Q, and 12 otherwise: 33,351 of 33,351
stories of the trustworthy pairs. The preferences' size of new frames
does not give it (3 pairs differ).
Section chunk 0x4C02 stores IncludeSectionPrefix as a u16 after a u16 0.
The section Name is its prefix; pages of a section that includes the
prefix are named with it. The first section starts at its stored start
even when it continues numbering, and the page descriptor writes a
return in the marker as U+2029.
IDML starts a ParagraphStyleRange at every paragraph-style run record,
also between neighbouring records with equal formatting. Document the
rule and the paragraph length list of the records in objects.md, and
the remaining splits inside one record in unknowns.md.
A plain rectangle has exactly four straight points along the axes. Other
frames take the one number from offset 0 of chunk 0x3723, whatever their
four insets, and before DOM 11 always have InsetSpacing.
Every ContourOption has ContourPathName: `$ID/`, or for an AlphaChannel
contour the first channel name of image chunk 0x177C. Frames take their
graphic's name; frames whose object style turns the text wrap category
on get a ContourOption with only the name. Object styles write `$ID/`.
A wrap path object (class 0x3702) flagged custom in chunk 0x3709 has its
0x162B paths written as TextWrapPreference/Properties/PathGeometry, before
TextWrapOffset.
Decode the 29-byte contour chunk (0x373D on placed graphics, 0x373F in
the preferences, 0x3777 on object styles): u32 contour type at 0 and
IncludeInsideEdges at 17. Graphics without the chunk take the
preferences' contour, else SameAsClipping; a frame with one graphic
takes its graphic's contour unless its object style turns the text wrap
category on; object styles without the chunk have SameAsClipping.
Over the trustworthy pairs every ContourType and IncludeInsideEdges the
converter writes equals the reference (graphics 9,535, frames 8,965,
object styles 5,556, preferences 1,251); 14 graphics get a ContourType
the reference leaves out.
kMojikumiDefaultName10 is LineEndAllOneEmEnum: the BasedOnMojikumiSet of
7 custom mojikumi tables in 7 trustworthy pairs (code 10).
kMojikumiDefaultName14 is LineEndPeriodOneEmEnum: the Mojikumi of one
paragraph range in one trustworthy pair. Codes 3, 5, 6, 9 and 11 have
no pair and stay unwritten (docs/format/unknowns.md).
compare.py --exclude own/: trustworthy values reproduced 28,699,017 ->
28,699,026 (+7 BasedOnMojikumiSet, +2 Mojikumi); coverage 99.484 % ->
99.485 %; extras 92,883 unchanged; no pair loses values.
The u32 read as Level is a u16 level and a u16 that is 0 in every entry
of the pairs and 1 in some entries of files without one, where the u32
gave levels such as 65537 that the schema does not allow. Outputs of the
pairs do not change.
Page items:
- drop shadow KnockedOut (0x1EB6F), true where neither item nor style
stores it
- text wrap: the u32 at 40 of 0x3703 is the master page flag and the
wrap side (two u16, as the big-endian pair shows)
- groups get the fill, stroke and corner values all their children have
- dashes (0x5A35: list and corner code) on items whose stroke type in
effect is Dashed
- BeforeGroupingLayerPosition (0x324) from 21.4; export SizeType
(0x1E22B); no generated alt text attributes on groups
- anchored settings on items inside anchored items
- IgnoreWrap false without the chunk, MinimumFirstBaselineOffset 0 by
the baseline category, InsetSpacing before DOM 11 by the general
category; EPS contents from 0x8C23
- custom striped and dashed stroke styles (classes 0xB016, 0x5A43) and
StrokeType code 0x5A42
Preferences: column direction (0x555, else the story orientation),
optical margin size by the text size unit, highlight flags false without
their chunk, grid and gutter defaults and print marks by the edition
code, watermark font by the assignment name, EPUB ViewDocumentAfterExport
for DOM 8 and 9, HTMLFXLExportPreference from DOM 10.1, MathML document
values from DOM 20, index header type by variant name, three more
built-in kinsoku and mojikumi tables.
Resources and text: the saving edition from the XMP black name (tints
of black, unknown users, endnote and index titles, endnote marker), TOC
style entries, PageNumberType (0x1B09), CustomGlyph (0x1B5E), alignment
codes of 0x4266, 0x1B3E and 0xB676, spot colour alias and DOM 8 section
pagination (observed values), text variable settings and result texts,
object styles' AppliedParagraphStyle without a chunk and flex spacing,
stories of frames anchored in the XML backing story.
Fixes: flattener resolutions of 0x10833 were swapped; colour group Self
follows the DOM version.
compare.py, trustworthy pairs: value coverage 98.96 % -> 99.48 %
(+150,288 values), documents >= 99 % 914 -> 1,021, story text 37,129 ->
37,147 of 37,150 (0 missing), extras 92,838 -> 92,883 (+788 from the
KnockedOut and group-corner exceptions, -744 from fixed extras). No pair
loses values against f21267f. diff_outputs.py: 5,743 of 5,765 outputs
differ, no failure or warning differs.
Inserted text over several text runs is written as a child of its
paragraph range, or of the story when it spans paragraph ranges, with
the ranges split at its ends, instead of one Change per character range.
A character-level hyperlink source whose whole extent is inserted text
is written inside the Change. Changes over tables, anchored items or
XML markers, and changes that do not nest with a paragraph-level
source, keep the per-range form.
compare.py, trustworthy pairs: 3,960 of the 3,962 reference Change
elements have an equal one in the output (3,969 written);
HyperlinkTextSource/Change extras fall from 220 values in 10 documents
to 0; 13 more text range starts. Only stories with tracked changes or
sources in changes differ in the output.
Chunk 0x1E234 of an object style holds four u16 flags and four f64:
X, Y, height and width of TransformAttributeOption. IDML writes each
value whose flag is 1 (36 styles; 2,390 with all flags 0 have none).
compare.py --exclude own/, trustworthy pairs: TransformAttrWidth
0 -> 30/30, TransformAttrHeight 0 -> 11/11, TransformAttrX 0 -> 3/3,
TransformAttrY 0 -> 1/1; no extra values.
- Item attribute 0x10837 is the drop shadow EffectColor, written by the
object style comparison of the other transparency values. Where the
style stores no swatch, Black is left out: one item has it, and its
IDML has no EffectColor.
- ColumnRuleOverride is the u32 at 0 of chunk 0x22646 and is written as
true or false.
compare.py --exclude own/, trustworthy pairs: DropShadowSetting
EffectColor 0 -> 11/11, TextFramePreference ColumnRuleOverride
21,165 -> 21,167/21,167, 4 more Color elements (swatches the shadows
use). No new extra values; no pair loses values.
Decoded chunks:
- EPUB (0x21A1A), HTML (0x21A19), fixed layout EPUB (0x21A25) and
Publish Online (0x21A20) export options; the EPUB chunk is read in the
layout of its two variable blocks that ends at the chunk's end. Files
without a chunk get the values every such file has.
- Adjust layout (0x7020): EnableAdjustLayout,
AllowFontSizeAndLeadingAdjustment, EnableAutoAdjustMargins.
- Language quotes (0x2D26), and $ID/ for an empty vendor of a language
whose vendor is never stored; Romanian quotes without the chunk.
Values without a field follow the InDesign that exported the IDML,
taken from the save history or the edition that made the document (the
language of its assignment name); preferences.md lists the rules and
their exceptions: DOMVersion by the last session's release, Publish
Online resolution and PublishPdf, EnableAdjustLayout after 14.0.0 and
14.0.1, the index title and separator, SnapZone, BaselineFrameGridColor
and ColumnGuideColor by the last session's language code, and style
shortcut key codes by its platform (also on character styles of DOM 7
to 12, which got none).
compare.py --exclude own/, trustworthy pairs: 98.74 % -> 98.96 %
together with the three commits before (+62,457 values, 0 lost on any
key); e.g. EPubExportPreference 0 -> 1,242 per attribute, DOMVersion
668 -> 1,164 of 1,251, IndexOptions Title 284 -> 1,162, SnapZone 68 ->
1,240, Language HyphenationVendor 1,945 -> 2,818 of 2,829,
BaselineFrameGridColor 934 -> 1,154 (object styles 4,163 -> 5,136),
ColumnGuideColor 406 -> 1,228, CharacterStyle KeyboardShortcut 2,547 ->
3,154 of 3,154. Extra values 93,502 -> 92,838: +346 on Language
elements the reference IDML leaves out (their new vendors and quotes),
one PublishPdf of an IDML exported by another release.
The eight bytes after a TOC style's name are a u16 (0 in every style),
the u16 IncludeBookDocuments and the u32 CreateBookmarks. They match all
1,520 TOC styles of the corpus whose IDML style has the same name.
compare.py --exclude own/, trustworthy pairs: TOCStyle
IncludeBookDocuments and CreateBookmarks 0 -> 1,304 of 1,304 each; no
extra value.
A colour object with model code 3 is a mixed ink: chunk 0x1F09 lists its
inks, chunk 0x1F01 (colour space 9) their fractions and chunk 0x102D the
spot colours. With chunk 0x1F24 it is a mixed ink group; colours of
space 9 whose chunk 0x117 names a group are its members and take the
group's lists. They are written as MixedInkGroup and MixedInk elements
after the inks (mixed inks sorted by name) instead of Color elements,
and fills and colour group swatches refer to them. A tint gets the
alternate colour of its base colour. Evidence in objects.md.
compare.py --exclude own/, trustworthy pairs: 81 of 81 mixed inks and
groups with every attribute (+1,140 values), Tint AlternateSpace and
AlternateColorValue 0 -> 171 of 171, ColorGroupSwatch and fill colour
references +132; 0 lost on any key. Extra values: the 1,077 values of
Color elements written for mixed inks are gone; 66 more on tints the
reference IDML leaves out (their two new attributes).
Chunk 0x2800 has one u16 per attribute: AnchoredPosition at 40,
HorizontalReferencePoint at 42, VerticalReferencePoint at 44, AnchorPoint
from 46 and 50, HorizontalAlignment at 48, VerticalAlignment at 52,
SpineRelative at 54 and PinPosition at 56, with the two offsets at 0 and
16. The grouped tables written before were the combinations the smaller
corpus showed. Every field matches the 6,363 object styles, 1,076
anchored items and 1,295 preference chunks of the corpus.
An anchored item gets every value when its object style turns the
anchored object category off (670 items, with LockPosition and
AnchorSpaceAbove from the value files), else the values that differ
from the style's. Preferences and object styles without the chunk get
the values every such file has.
compare.py --exclude own/, trustworthy pairs: 98.46 % -> 98.51 %
(+11,653 values, 0 lost on any key); ObjectStyle SpineRelative
2,031 -> 5,556 of 5,556, preference SpineRelative 0 -> 1,251 of 1,251,
anchored text frames, rectangles, ovals, polygons, lines and groups
gain their settings (TextFrame PinPosition 2 -> 556 of 556). Extra
values unchanged (93,502).
Buttons, check boxes, radio buttons and multi-state objects list their
states in chunk 0x14521 and the items each state shows in chunk 0x14526;
their transform is chunk 0x14527. Text boxes, combo boxes and signature
fields hold their items in chunk 0x15B. The converter writes the
elements with their State elements (type from the state ID), the groups
of states named by their type, PathBoundingBox from the items' paths,
and the name, description, export value, required flag, text box font
and flags from the attribute list (objects.md, form fields). The value
files gain the constant values of buttons and multi-state objects.
Text boxes anchored in table cells now replace their U+FFFC, so those
stories' text matches.
compare.py (trustworthy pairs, with the commits before it): coverage
97.92 % -> 98.46 %, story text 37,112 -> 37,129 of 37,150; the form
field elements of all 48 documents are written (Button, Group, State,
Rectangle and their children about 150,000 values). Extra values from
anchored fields without a known object style are avoided by leaving
their anchor settings out.
A shade whose chunk 0x5532 holds shading type 5 is written as a
PastedSmoothShade with ContentsType AxialShade: the matrix of the chunk
and the shading data with every field converted to big-endian, by the
grammar in objects.md. A shade is written when something refers to it
or the page item defaults name it, as unnamed colours are, and page
item fills that hold its UID refer to it.
compare.py (trustworthy pairs): all 891 axial shades of 12 documents
(11,574 values) and 884 more Rectangle FillColor values; none of the
2,813 shades IDML leaves out is written.
Where an inserted change run starts at a hyperlink source's first
character and ends before the source does, IDML writes an empty Change
before the source and the first character without one; a paragraph-level
source gets a character range of its own for the deleted text and the
empty Change. The converter now writes the same 1,681 empty changes as
the trustworthy references (385 of 385 in stale pairs).
Deleted text owned by a table cell's terminating return is written as
the cell's last child. All 1,889 deleted-text changes of the trustworthy
pairs are now written.
compare.py (trustworthy pairs), together with the form fields that
follow: TextRange (element not produced) 6,062 -> 758 values,
TextRange (story text differs) 8,417 -> 3,979.
The chunk has 16 bytes before the page size, and its fields from the
facing pages flag on are 20 bytes earlier than in the 146-byte chunk.
The page size, facing pages, intent, binding, bleeds and slugs match
the IDML in 11 of 11 pairs.
compare.py (trustworthy pairs): 135 more DocumentPreference values in
9 documents (15 attributes each), no extra value; the warning
"document preferences of 126 bytes are not known" is gone.
OverprintFill and OverprintStroke come from attributes 0x6E6A and 0x6E67
of the item's attribute list, where the fill or stroke puts ink on the
page, compared with the object style as the fill colour is.
compare.py (trustworthy pairs): all 3,448 OverprintFill and 121
OverprintStroke values of polygons, rectangles, text frames, EPS text,
ovals and graphic lines are reproduced (0 before), with no extra value.
Coverage 97.91 % -> 97.92 %. Group overprint (313 values) is left out.
- Frame fitting: AutoFit 1 (true), FittingAlignment 1 (TopCenterAnchor)
and 3 (LeftCenterAnchor). A rectangle, oval or polygon whose object
style turns the fitting category off (0x1B960 not in chunk 0x1B92E)
gets all seven attributes, the item's or the style's; this holds for
the root [None] too (22 trustworthy documents, 9,560 items; none in
the 883 others). A style with the category on writes only the item's
values that differ. The preference FrameFittingOption writes all seven,
0 or code 0 where the page item defaults list lacks one (1,251 of
1,251 pairs each).
- Object style OverprintFill, OverprintStroke and OverprintGap (0x6E6A,
0x6E67, 0x6E8B) where the colour carries ink (not None, Paper, tint 0
or a white process colour), on every style from 21.4.
- StyleUniqueId on every paragraph and character style when 11.2 saved
the document: the GUID after the name's string (GUID form only), else
$ID/. A 36-character style name is no longer taken for the GUID.
- Chunk 0x28F0 parsed with its export tag maps: SplitDocument, EmitCss
(10.1 on) and IncludeClass (13.0 on) on every text style, root styles
and styles with tag maps included, EmitCss and IncludeClass on object
styles; StyleExportTagMap elements. The WhenWritten values of these
attributes are dropped from element_values.xml.
Evidence in objects.md, preferences.md and idml-values.md.
compare.py --exclude own/, trustworthy pairs: 97.60 % -> 97.84 %
(+71,229 values, 0 lost on any key); Oval/Polygon/Rectangle
FrameFittingOption AutoFit 32,300 -> 44,868 of 44,868, preference
FrameFittingOption 1,251 of 1,251 per attribute, StyleUniqueId 6,424 ->
10,047 of 10,051, ObjectStyle OverprintStroke 0 -> 1,512 of 1,512,
StyleExportTagMap 0 -> 306 of 306 elements, export flags of every
style. Extras 94,157 -> 93,569: 444 FrameFittingOption elements and 168
SplitDocument and EmitCss values gone; 24 StyleUniqueId values are new,
in five 11.4 pairs whose IDML was exported by 11.0.
Section page number styles 0x4C1A SingleLeadingZeros, 0x4C13
DoubleLeadingZeros, 0x4C19 LowerLetters and 0x4C16 UpperRoman, from
pairs that had warned about them (89 sections with an IDML section, no
contradiction). Pages of these sections get names in that style; in the
page Descriptor of a SingleLeadingZeros section IDML writes the section
prefix as a string in place of the style (1,249 of 1,249 descriptors).
An index topic record ends with a u32 count of cross references, their
UIDs and a u32, not with 8 fixed bytes. 29 pairs whose topic has one
cross reference lost the whole index section with a warning; their
topics are now written. CrossReference elements are not written yet.
Also documents the colour model code 3 (mixed inks, written by IDML as
MixedInk elements) and the 104-byte object style frame settings of
InDesign 4.0 to 6.0 (no pair), which stay unconverted.
compare.py --exclude own/, trustworthy pairs: +727 values before the
descriptor rule (Section PageNumberStyle 1,257 -> 1,344 of 1,344, Page
Name 645 -> 553 wrong, 29 Topic elements), 0 lost on any key; with the
rule, the page descriptors of the subsets checked are all reproduced.
Keys that the regeneration dropped because a few documents differ are
read from the INDD instead (evidence in objects.md and attributes.md):
- Spread PageTransitionType and PageTransitionDirection: chunk 0x2050C
(five transition codes, four direction codes; none without it).
- Object style StrokeTint, GapColor, EndCap, EndJoin, LeftLineEnd,
RightLineEnd and ArrowHeadAlignment: page item attribute IDs 0x6E66,
0x6E89, 0x6E6B, 0x6E6C, 0x6E71, 0x6E72 and 0x6E84 of the style's list.
- Paragraph and character style PreviewColor: the u32 14 bytes before
the name, 0 or an interface colour.
- Object style ApplyNextParagraphStyle (chunk 0x1B94D),
KeyboardShortcut "0 0" (key 0 after the name in chunk 0x1B907),
ApplyToMasterPageOnly (wrap chunk u16 at 40), and
UseNoLineBreaksForAutoSizing (chunk 0x1B924 u16 at 140).
- AnchorXoffset and HorizontalReferencePoint of anchored object
settings (chunk 0x2800, f64 at 16 and u16 at 42), for object styles,
anchored frames and the preferences.
compare.py --exclude own/, trustworthy pairs: +58,048 values, 0 lost on
any key; Spread transitions 0 -> 9,509 of 9,509 each, CharacterStyle
PreviewColor 0 -> 3,154 of 3,154, the object style keys 5,550 to 5,556
of 5,556 (KeyboardShortcut 4,262 of 4,305: other keys left out).
Extras 94,249 -> 94,233.
Text frames get BaselineFrameGridOption from chunk 0x2834 of their
multi-column frame, and TextFrameFootnoteOptionsObject from DOM 12, both
by the category rule of TextFramePreference (all values when the
category is off or the frame has no style, else the values that differ
from the style's). FootnotesSpanAcrossColumns comes from chunk 0x22608
u16 at 2. Frames without chunk 0x2CE, or with the 22-byte chunk of 7.0,
now get their TextFramePreference. Object styles write their baseline
grid from chunk 0x1B924 at 82 (code 1 is TopOfMargin). The document's
TextFramePreference reads the column rule and footnote values of the
preferences chunks 0x22646 and 0x22608.
The 13.1 boundary of the Footnotes... values and of
MergeConsecutiveParaBorders counts as reached when the header or the
last session of the save history is 13.1: 109 pairs whose header says
13.0 have them, 93 of which a 13.1 session saved last. Evidence and
re-checked counts in objects.md, footnotes.md and idml-values.md.
compare.py --exclude own/, trustworthy pairs: +48,405 values (+43,158
for the elements, +5,247 for the 13.1 rule), 0 lost on any key.
TextFrameFootnoteOptionsObject 17,002 -> 20,096 of 20,096 on written
frames; BaselineFrameGridOption 0 -> 1,961 of 1,961; TextFramePreference
elements 48,038 -> 48,708; document column rule 0 -> 576 of 576.
Extras: the 82 FootnotesSpanAcrossColumns values are gone, 3
EnableOverrides values are new.
aiongg