Helping truncated JSON data with jpt 1.0

jpt 1.0 now has the ability to detect and help truncation in strings, arrays and objects. The -D option can detect this in newline delimited and concatenated JSON and -H will help the JSON by closing up strings, arrays, and objects. For example, here’s two JSON text concatenated both with truncation:

jpt -i0 -DH <<< $'{"a":[1,2 {"b":[3,4,'
{"a":[1,2]}
{"b":[3,4,null]}

#heredoc to more clearly show the input data
% jpt -i0 -DH <<EOF
heredoc> {"a":[1,2
heredoc> {"b":[3,4,
heredoc> EOF
{"a":[1,2]}
{"b":[3,4,null]}

If there is a trailing comma, a null will be put there to signify the fact there was an additional value before truncation occured, in all cases one should assume some data has been lost. The -i0 option is for zero indents, single line JSON however it is not NDJSON. When multiple JSON texts are input, as above, JSON Sequence (RFC 7464) are the default output. JSON Sequences are simply a record separator (RS) character at the beginning of JSON text. RS characters do not appear in the Terminal. If you pipe the output to bbedit or redirect to a file you’ll see them. To get some commentary on the repairs, use the -c option:

jpt -i0 -cDH <<< $'{"a":[1,2 {"b":[3,4,'         
{"a":[1,2]}
{"b":[3,4,null]}
--> Original error: SyntaxError: JSON Parse error: Expected ']'
--> Byte 10: Truncation Detected
--> Truncation help for JSON Text (1), bytes: 1-10
--> Truncation help after byte 10, array end: ]
--> Truncation help after byte 10, object end: }
--> Truncation help for JSON Text (2), bytes: 11-21
--> Truncation help after byte 21, missing value and array end: null]
--> Truncation help after byte 21, object end: }
--> Bytes 1-10: JSON Text (1)
--> Bytes 11-21: JSON Text (2)

echo $?                                 
1

As you can see -c can get pretty verbose depending on the issues. If there is any output with -c the exit code of jpt will be 1, if there is nothing to fix it will exit 0. This can be useful if you need to know if the source JSON needed any help or not.

jpt 1.0 is there when your JSON needs some help, not just when it’s perfectly formatted. If that sounds cool give it a try, download the Mac .pkg at github.com/brunerd/jpt/releases