jpt 1.0 and JSON5 rehab

JSON5 is not JSON nor is an IETF standard but jpt 1.0 can rehabilitate it back to standards compliant JSON. Let’s take the example from its web page.

{
  // comments
  unquoted: 'and you can quote me on that',
  singleQuotes: 'I can use "double quotes" here',
  lineBreaks: "Look, Mom! \
No \\n's!",
  hexadecimal: 0xdecaf,
  leadingDecimalPoint: .8675309, andTrailing: 8675309.,
  positiveSign: +1,
  trailingComma: 'in objects', andIn: ['arrays',],
  "backwardsCompatible": "with JSON",
}

Now let’s send it through jpt

% jpt json5_example.txt        
{
  "unquoted": "and you can quote me on that",
  "singleQuotes": "I can use \"double quotes\" here",
  "lineBreaks": "Look, Mom! No \\n's!",
  "hexadecimal": 912559,
  "leadingDecimalPoint": 0.8675309,
  "andTrailing": 8675309,
  "positiveSign": 1,
  "trailingComma": "in objects",
  "andIn": [
    "arrays"
  ],
  "backwardsCompatible": "with JSON"
}

That last line is a bit misleading. JSON5 texts are totally not backward compatible with JSON parsers. What they mean is that JSON5 parsers are backward compatible with JSON. FYI – Every single JSON5 addition is not compatible with JSON, I should know I had to write parsing rules and behaviors to deal with it! Let’s use the -c flag to comment on all the work that went into un-junking the above example

% jpt -c ./json5_example.txt 
{
  "unquoted": "and you can quote me on that",
  "singleQuotes": "I can use \"double quotes\" here",
  "lineBreaks": "Look, Mom! No \\n's!",
  "hexadecimal": 912559,
  "leadingDecimalPoint": 0.8675309,
  "andTrailing": 8675309,
  "positiveSign": 1,
  "trailingComma": "in objects",
  "andIn": [
    "arrays"
  ],
  "backwardsCompatible": "with JSON"
}
--> Original error: SyntaxError: JSON Parse error: Unrecognized token '/'
--> Byte 5: // line comment (JSON5)
--> Byte 19: unquoted object key name (JSON5)
--> Byte 29: single quoted string (JSON5)
--> Byte 63: unquoted object key name (JSON5)
--> Byte 77: single quoted string (JSON5)
--> Byte 113: unquoted object key name (JSON5)
--> Byte 138: \ escaped newline in string (JSON5)
--> Byte 153: unquoted object key name (JSON5)
--> Byte 164: 0x hex value (JSON5)
--> Byte 177: unquoted object key name (JSON5)
--> Byte 198: decimal point wihtout preceding digit (JSON5)
--> Byte 208: unquoted object key name (JSON5)
--> Byte 228: trailing decimal lacking mantissa (JSON5)
--> Byte 233: unquoted object key name (JSON5)
--> Byte 247: explicit plus + for value (JSON5)
--> Byte 253: unquoted object key name (JSON5)
--> Byte 268: single quoted string (JSON5)
--> Byte 282: unquoted object key name (JSON5)
--> Byte 290: single quoted string (JSON5)
--> Byte 296: trailing comma (JSON5)
--> Byte 336: trailing comma (JSON5)

Yeah there’s so much wrong with JSON5 vs JSON its not funny. It’s baffling that Apple is acknowledging it by adding JSON5 parsing to their frameworks! At least they don’t generate it! Despite not being funny, let’s try anyway, here’s some more JSON5 example to show some of the other ways it can lead you astray.

{
	"jpt_lineBreakHandling": "Extended parsing to ignore tabs \
	after escaped newlines.\nIf JSON5 is supposed to be human readable \
	shouldn't tabs be included in the escaping?",
	"leadingDecimalPoint": .8675309, 
	"andTrailingDecimal": 8675309.,
	"negativeLeadingDecimalPoint": -.1234, 

	//Infinity converts to null, use -I for string
	"+Infinity": +Infinity,
	"-Infinity": -Infinity,
	"Infinity": Infinity,
	//null is not a number now is it? done.
	"NaN": NaN,
}

Running this through jpt we’ll get pure JSON output

% jpt ./json5_examples_brunerd.txt 
{
  "jpt_lineBreakHandling": "Extended parsing to ignore tabs after escaped newlines.\nIf JSON5 is supposed to be human readable shouldn't tabs be included in the escaping?",
  "leadingDecimalPoint": 0.8675309,
  "andTrailingDecimal": 8675309,
  "negativeLeadingDecimalPoint": -0.1234,
  "+Infinity": null,
  "-Infinity": null,
  "Infinity": null,
  "NaN": null
}


#-I to convert to signed Infinity strings instead of null
#-8 to convert NaN to string "NaN" (-N was taken, like NaN 8 is like Infinity but not quite :)
% jpt -I8 ./json5_examples_brunerd.txt 
{
  "jpt_lineBreakHandling": "Extended parsing to ignore tabs after escaped newlines.\nIf JSON5 is supposed to be human readable shouldn't tabs be included in the escaping?",
  "leadingDecimalPoint": 0.8675309,
  "andTrailingDecimal": 8675309,
  "negativeLeadingDecimalPoint": -0.1234,
  "+Infinity": "+Infinity",
  "-Infinity": "-Infinity",
  "Infinity": "Infinity",
  "NaN": "NaN"
}

The fact that they added the data type of +/- Infinity and NaN really complicates the conversion since JSON doesn’t have equivalents. By default Infinity converts to null but -I will convert it to the string "Infinity" with signedness if specified. NaN is also converted to null by default but the -8 flag will convert it to the string "NaN". Why 8? Because it’s sort like infinity but it’s not, just like NaN.

If you have need to rehab JSON5 back to JSON then jpt might be able to help you out. It can run on Mac all the way back to OS X Tiger (10.4) all the way up to the newest macOS Monterey (12) and on other *nixes with jsc installed. Check the jpt Releases page