🍱 Lunchbox Hands

xml

XML to JSON Has No Canonical Mapping: Attributes, Repeats, and Lost Order

XML and JSON have different data models, so every converter invents a convention: BadgerFish, Parker, GData and fast-xml-parser disagree on attributes, text nodes and repeats. Executed examples of the singleton-vs-array trap, mixed content, type coercion, and why element order cannot survive the trip.

Paste the same XML into three converters and you get three different JSON documents: attributes are @id in one, @_id in another, gone from the third, and a repeated <item> is an array unless there was only one. Nobody is buggy here, because there is nothing to be buggy against: XML and JSON have different data models, and no standard says how to map one onto the other. XML is an ordered tree of elements with attributes, text, comments, and processing instructions. JSON is objects, arrays, strings, numbers, booleans, and null. Every converter has to invent a convention for the parts that don’t line up, and they all invented different ones.

This is the same “no spec, only dialects” problem that makes INI to JSON round-trips lossy, except XML does have a rigorous spec — the gap is entirely on the mapping side. Below, every output labeled “ours” is real output from the library behind our XML ↔ JSON converter, which is fast-xml-parser 5.5.1 constructed with three non-default options: ignoreAttributes: false, attributeNamePrefix: '@_', and numberParseOptions: { hex: false, leadingZeros: false, eNotation: false } (section 5 explains that last one). Everything else is the library default, and several of those defaults are the story.

Four conventions, four answers

The three named conventions that predate every modern library are BadgerFish (David Sklar), Parker (an XSLT stylesheet whose rules are documented in the Open311 project), and GData (Google’s feed API). fast-xml-parser’s default is a fourth. For <book id="42">Dune</book>:

BadgerFishParkerGDatafast-xml-parser (ours)
Attributes"@id": "42"dropped — “lossy, it simply ignores XML attributes”"id": "42" (no prefix, “converted to String properties”)"@_id": "42" (prefix is configurable; the library default is to ignore attributes entirely)
Text when attrs/children present"$": "Dune"text only, attributes gone"$t": "Dune""#text": "Dune"
Text with no attrs/childrenstill {"$": "Dune"}"Dune" directlystill {"$t": "Dune"}"Dune" directly
Repeated elements”Multiple elements at the same level become array elements""If all sibling elements have the same name, they become an array""Elements that may appear more than once are converted to Array properties” (schema-aware)array only when two or more siblings share a name
Namespaces@xmlns object per elementprefix kept in the key: "ding:dong"ns:element → ns$elementprefix kept literally: "xs:element"
Root elementkeptabsorbed — <root>test</root> becomes "test"kept, plus the XML declarationkept, plus "?xml"

Read the “repeated elements” row carefully. BadgerFish and fast-xml-parser decide from the instance (how many siblings are actually there); GData decides from the schema (whether the element may repeat). Only the schema-aware answer gives consumer code a stable shape, and a generic converter has no schema. That’s the next section.

1. The singleton-vs-array trap

This is the one that ships to production and breaks a week later. Ours:

<order><item>Book</item><item>Pen</item></order>
→ { "order": { "item": ["Book", "Pen"] } }

<order><item>Book</item></order>
→ { "order": { "item": "Book" } }

Two items: array. One item: string. Code written against the first payload — order.item.map(...), order.item.length — throws or misbehaves on the second ("Book".length is 4). The converter can’t know <item> is a list element, because the XML never says so.

fast-xml-parser’s docs are candid: “Whether a single tag should be parsed as an array or an object, it can’t be decided by FXP.” The fix is the isArray callback, which forces named tags to always be arrays:

new XMLParser({ isArray: (name) => name === 'item' })
  .parse('<order><item>Book</item></order>')
// { "order": { "item": ["Book"] } }

Honest disclosure: our tool does not expose isArray. It uses the library default (isArray: () => false), so a singleton always collapses. If you’re generating JSON for code that expects lists, either normalize on the consuming side ([].concat(x)) or run the library yourself with the callback.

2. Attributes vs child elements

XML gives you two ways to say the same thing — <book id="42"/> and <book><id>42</id></book> — and JSON has one. Ours keeps them distinguishable with the prefix:

<book id="42" lang="en"><title>Dune</title></book>
→ { "book": { "title": "Dune", "@_id": "42", "@_lang": "en" } }

That @_ is fast-xml-parser’s attributeNamePrefix, and we set it explicitly along with ignoreAttributes: false because the library’s default is ignoreAttributes: true — out of the box it behaves like Parker and silently drops every attribute. BadgerFish uses a bare @; GData uses nothing at all, so <book id="42"><id>7</id></book> collides. There is no right answer; there is only “which one does the code on the other end expect.”

Notice too that "@_id" is the string "42". Attribute values are not type-coerced (parseAttributeValue defaults to false) while element text is (parseTagValue: true), so <r n="42"/> gives "@_n": "42" but <n>42</n> gives 42 as a number. Same data, different type, depending on which XML idiom the author picked.

3. Mixed content: text and elements interleaved

Document-style XML — anything HTML-shaped — puts text and child elements side by side. JSON objects can’t interleave a string with keyed children, so something has to give:

<p>Hello <b>world</b>!</p>
→ { "p": { "b": "world", "#text": "Hello!" } }

Two things were lost. The position of <b> relative to the text is gone — was it “Hello world!” or “world Hello!”? — and the text fragments Hello and ! were concatenated into a single #text. The space disappeared because trimValues: true (the default) trims each text chunk before joining; with trimValues: false you get "Hello !", which is still not the original. Convert it back and ours emits <p>\n <b>world</b>\nHello!</p> — the bold word now comes first.

BadgerFish’s own author flagged this on the convention’s own page: the convention “doesn’t provide a way to get the different text bits in an element whose content is a mix of text and children.” Parker documents that “mixed mode text-nodes, comments and attributes get absorbed.” If your XML is prose, an object-shaped JSON mapping is the wrong target; you want an ordered node list (see next section) or HTML to Markdown.

4. Document order cannot round-trip

In XML, sibling order is significant — a document is a sequence, and <a/><b/><a/> is a different document from <a/><a/><b/>. In JSON, it isn’t. RFC 8259 §1 defines an object as “an unordered collection of zero or more name/value pairs,” and §4 warns that “JSON parsing libraries have been observed to differ as to whether or not they make the ordering of object members visible to calling software.” So even when a converter preserves order in its output text, nothing downstream is obliged to keep it.

Worse, keyed objects can’t represent interleaving at all. Ours:

<r><a>1</a><b>2</b><a>3</a></r>
→ { "r": { "a": [1, 3], "b": 2 } }
→ back to XML: <r><a>1</a><a>3</a><b>2</b></r>

The two <a> elements were merged into one array key, and the reverse trip emits them adjacent. a, b, a became a, a, b. No object-shaped convention can fix this; it’s structural.

The escape hatch is fast-xml-parser’s preserveOrder: true, which abandons objects for an array of single-key nodes:

[{"r":[{"a":[{"#text":1}]},{"b":[{"#text":2}]},{"a":[{"#text":3}]}]}]

The docs describe this as “some long ugly result,” which is fair, but it’s the only lossless shape. Our tool doesn’t expose it; if you need a faithful round-trip of document-order XML, that’s the option to use.

5. Type coercion: which text becomes a number

JSON distinguishes 7 from "7"; XML text is just characters. fast-xml-parser’s parseTagValue: true default runs every element’s text through the strnum library, and also turns the literal strings true/false into booleans. strnum’s own defaults are { hex: true, leadingZeros: true, eNotation: true }, which is how a converter turns <sku>0x1A</sku> into 26 and <id>007</id> into 7 — identifiers quietly rewritten. Our tool overrides them to { hex: false, leadingZeros: false, eNotation: false }, keeping the plain-number and boolean coercion and dropping the identifier-mangling parts. Every line below is our real output:

XML textOursWhy
<n>007</n>"007"leadingZeros: false — the zeros survive, and so does the round-trip
<v>true</v>true (boolean)lowercase only
<t>TRUE</t>"TRUE"not recognized, stays a string
<hex>0x1A</hex>"0x1A"hex: false — a part number stays a part number
<id>1e5</id>"1e5"e-notation is not parsed; see the trade-off below
<f>3.10</f>3.1still a number, and the trailing zero is still lost
<p>+5</p>5sign consumed
<big>12345678901234567890</big>"12345678901234567890"kept as string because it exceeds safe integer precision
<inf>Infinity</inf>"Infinity"string
<s> 42 </s>42trimmed, then parsed

The e-notation row is a trade-off, not an oversight. In strnum 2.2.0 the two options are coupled: resolveEnotation() returns a number only when leadingZeros is true, so 1e5 → 100000 is unreachable unless you also accept 007 → 7. Given the choice, we keep identifiers intact — <id>1e5</id> stays the string "1e5", and if you need it as a number, Number(value) on the consuming side is one call.

What still coerces is worth knowing: <f>3.10</f> is 3.1 and <p>+5</p> is 5, so a version string or a formatted decimal is still altered. If your document is entirely identifiers, parseTagValue: false in the library keeps every value a string — our tool does not expose that switch. And the reverse trip is now clean for the leading-zero case: <n>007</n> → {"n":"007"} → <n>007</n>, where it used to come back as <n>7</n>. The INI leading-zeros hazard is the same class of bug in a different file extension.

6. Namespaces

<xs:element> is, to an XML parser, an element named element in the namespace bound to the prefix xs. To a JSON converter it’s a string with a colon in it:

<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"><xs:element name="a"/></xs:schema>
→ { "xs:schema": { "xs:element": { "@_name": "a" }, "@_xmlns:xs": "http://www.w3.org/2001/XMLSchema" } }

Ours keeps the prefix literally (removeNSPrefix: false) and treats the xmlns:xs declaration as an ordinary attribute. With removeNSPrefix: true the library produces { "schema": { "element": ... } } and drops the declaration — cleaner keys, but two elements from different namespaces with the same local name now collide. GData rewrites the colon to $ (xs$element); BadgerFish hoists declarations into a per-element @xmlns object. Four conventions, four spellings of the same qualified name.

7. CDATA, comments, processing instructions, and the declaration

What survives the trip, in ours:

XML constructOursLibrary option that changes it
<r><![CDATA[<b>raw</b> & stuff]]></r>"r": "<b>raw</b> & stuff" — merged into text, CDATA-ness forgottencdataPropName: '#cdata' keeps it separate
<r><!-- note --><a>1</a></r>{ "r": { "a": 1 } } — comment gonecommentPropName: '#comment'
<?xml version="1.0" encoding="UTF-8"?>"?xml": { "@_version": "1.0", "@_encoding": "UTF-8" } — kept as a pseudo-elementignoreDeclaration: true
<?xml-stylesheet href="s.xsl"?>"?xml-stylesheet": { "@_href": "s.xsl" }ignorePiTags: true
&lt;b&gt; &amp; &quot;"<b> & \"" — decoded—
&#169;"&#169;" — not decoded—

The declaration surviving as a "?xml" key surprises people, but it does round-trip (the builder re-emits it). Comments do not survive at all with our options. And the last row — named entities decoded, numeric character references left as literal text — is a quirk of the library’s default entity handling worth knowing if your XML uses &#...; escapes.

8. Empty elements

XML has three ways to say “nothing here,” and JSON has three different nothings ("", null, {}). Ours flattens all of them:

<r><a/><b></b><c> </c></r>
→ { "r": { "a": "", "b": "", "c": "" } }

Self-closing, explicitly empty, and whitespace-only all become the empty string. Parker says “empty elements will become null.” BadgerFish “throws away the text content of elements if the text content is entirely whitespace,” leaving {}. In the reverse direction, ours turns null into <a/>, "" into <d></d>, and {} into <e></e> — three JSON values, two XML spellings, and all three come back as "".

9. The reverse direction: JSON that isn’t XML-shaped

XML requires exactly one root element — the spec’s phrasing is “there is exactly one element, called the root, or document element” (XML 1.0 §2.1) — and element names cannot start with a digit or contain spaces. JSON has no such rules. fast-xml-parser’s builder happily emits all of it — two roots, a tag with a space in its name, a tag starting with a digit, array indices as tag names — none of which is well-formed XML. So jsonToXml checks the JSON before building: exactly one top-level key, and every object key (and attribute name after the @_) tested against the XML Name production. The built string is then re-checked with XMLValidator as a backstop. All four are now errors instead of malformed output, with the fix stated in the message:

{"a":1,"b":2}                → JSON must have exactly one top-level key to become an XML
                               document (found: a, b) — wrap it in a root object
[1,2]                        → JSON must have exactly one top-level key to become an XML
                               document (found a top-level array) — wrap it in a root object
{"root":{"first name":"Al"}} → Invalid XML element name: "first name" (names must start
                               with a letter, "_" or ":" and cannot contain spaces)
{"root":{"1st":"x"}}         → Invalid XML element name: "1st" (names must start with a
                               letter, "_" or ":" and cannot contain spaces)

Names that only look wrong still work: xs:schema, my-elem.v2, über, and the ?xml declaration and #text pseudo-keys all pass, because the Name production allows colons, dots, hyphens after the first character, and non-ASCII letters. Both directions are now checked, which is what makes the tool’s ⇄ swap button safe to lean on. Arrays of objects do work — {"r":{"item":[{"n":1},{"n":2}]}} gives two <item> elements — so a one-element array becomes one <item>, which parses back as a singleton. The trap from section 1 is symmetric.

What to do about it

There’s no fix, only a set of choices to make explicitly rather than by accident:

  1. Know which convention your consumer expects before you convert. @_ vs @ vs $t is a one-line config change in fast-xml-parser but a runtime error in whatever reads the output.
  2. Never trust the singleton case. Normalize with isArray at parse time or [].concat() at read time.
  3. If order or mixed content matters, don’t produce objects. Use preserveOrder and accept the verbose shape.
  4. If values are identifiers, check what your converter’s coercion does. strnum’s defaults lose leading zeros and reinterpret 0x prefixes; ours turns both off, but 3.10 still becomes 3.1 everywhere, and only parseTagValue: false keeps every value a string.
  5. Validate the XML you produce. fast-xml-parser’s XMLBuilder emits whatever your keys say; if you call it yourself, run XMLValidator.validate() on the result the way our tool does.

For the quick look, our XML ↔ JSON converter runs in your browser with the options described above — validation in both directions, @_ attributes, numbers and booleans coerced but identifiers left alone — and the ⇄ button is a cheap way to see what a round-trip loses. If you only need a few values out of the document, you may not need JSON at all: an expression in the XPath tester selects them from the XML directly. Tidy the input first with the XML formatter, inspect the output in the JSON formatter, and if the destination is a spreadsheet, JSON to CSV has its own flattening rules, covered in flattening JSON for CSV export. Coming from YAML instead? YAML ↔ JSON has a far smaller gap to cross, but its own coercion table — a different post.