Determining “iCloud Private Relay” and “Limit IP tracking” status in macOS

If you are a Mac admin you might have noticed some Apple plists are more complex than others these days. As evidenced in my post Don’t Be a Jerk, getting the status of Do Not Disturb in Big Sur was a multi-step exercise. Check out this modified code excerpt from doNotDisturb‘s Big Sur handling

#!/bin/sh

#Big Sur DnD status, returns "true" or [blank] (to be run as console user)
dndStatus="$(/usr/libexec/PlistBuddy -c "print :userPref:enabled" /dev/stdin 2>/dev/null <<< "$(plutil -extract dnd_prefs xml1 -o - /dev/stdin <<< "$(defaults export com.apple.ncprefs.plist -)" | xmllint --xpath "string(//data)" - | base64 --decode | plutil -convert xml1 - -o -)")"

#if we have ANYTHING it is ON (return 0) otherwise fail (return 1)
[ -n "${dndStatus}" ] && { echo ON; exit 0; } || { echo OFF; exit 1; }

Why is it so complex? Well because with the user preference domain of com.apple.ncprefs there is a base64 encoded plist embedded in the dnd_prefs key and requires some massaging to get it to a state where PlistBuddy can read and then extract the status from the :userPref:enabled key. See it’s just that easy! πŸ˜…

In macOS Monterey and higher Apple is now using JSON for Focus status which is great because it can can be parsed much easier (perhaps using my ljt or jpt tools) but sometimes not even that is needed! Sometimes file presence or awking is sufficient to get a reliable result too. See the evolution of doNotDisturb.sh for macOS for the myriad techniques one can use.

Now, here we are at Monterey on the verge of Ventura and not all areas of macOS have seen the JSON light and are still using nested base64-encoded Plist blobs to store the states of new features like iCloud Private Relay but luckily not the per interface setting of Limit IP Tracking.

iCloud Private Relay Status

Now why would you want to know these statuses? Perhaps you have a VPN product that doesn’t work when Private Relay is turned on or if Limit IP Tracking is turned on for a particular interface and you need to alert the user. Additionally maybe there’s an issue if your VPN browser handshake fails when Safari is used, if so see: Determining URL scheme handlers in macOS. Back to the task at hand though let’s turn some prefs inside out and look at iCloud Relay status with iCloudPrivateRelayStatus.sh

#!/bin/bash
: <<-LICENSE_BLOCK
iCloud Private Relay Status Checker (20250204) - (https://github.com/brunerd)
Copyright (c) 2025 Joel Bruner (https://github.com/brunerd)
Licensed under the MIT License

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
LICENSE_BLOCK

#############
# FUNCTIONS #
#############

#include this self-contained function in your script
function iCloudPrivateRelay()(

	#only for Moneterey and up, 11 and under need not apply
	[ "$(sw_vers -productVersion | cut -d. -f1)" -le 11 ] && return 1
	
	#parent pref domain
	domain="com.apple.networkserviceproxy"
	#key that contains base64 encoded Plist within parent domain
	key="NSPServiceStatusManagerInfo"
	#path within base64 embedded plist, there are multiple PrivacyProxyServiceStatus entries but this one seems primary
	targetPath=':$objects:1:PrivacyProxyServiceStatus'

	#get the top level data from the main domain
	parentData=$(launchctl asuser "$(stat -f %u /dev/console)" sudo -u "$(stat -f %Su /dev/console)" defaults export "${domain}" -)

	#if domain does not exist, fail
	[ -z "${parentData}" ] && return 1

	#export and decode the base64 data as plist XML and look for 
	keyStatusCF=$(/usr/libexec/PlistBuddy -c "print '${targetPath}'" /dev/stdin 2>/dev/null <<< "$(plutil -extract "${key}" xml1 -o - /dev/stdin <<< "${parentData}" | xmllint --xpath "string(//data)" - | base64 --decode | plutil -convert xml1 - -o -)")
	#if no value, fail
	[ -z "${keyStatusCF}" ] && return 1
	
	#if true/1 it is on, 0/off (value is integer not boolean BTW)
	[ "${keyStatusCF}" = "1" ] && return 0 || return 1
)

########
# MAIN #
########

#example - multi-line if/else calling
if iCloudPrivateRelay; then
	echo "iCloud Private Relay is: ON"
else
	echo "iCloud Private Relay is: OFF"
fi

Even though we work really hard to get the plist data extracted we don’t need to walk the entire XML document (you will find PlistBuddy balks at exporting JSON for a number of reasons). Instead we look for the presence of the key name PrivacyProxyServiceStatus and that is sufficient to reliably detect the state. UPDATE: This ambiguity has been resolved and it will now look at the path :$objects:1:PrivacyProxyServiceStatus for the status, which seems to be consistent (LMK otherwise). I have 2 versions in my GitHub iCloudPrivateRelayStatus.sh and the minified one line version iCloudPrivateRelayStatus.min.sh

Limit IP Tracking Status

Limit IP Tracking, is a per-interface setting that is on by default, however has no effect unless Private Relay is enabled. For this we will use ljt my Little JSON Tool to retrieve the value from the massaged JSON conversion of com.apple.wifi.known-networks.plist if on WiFi or if on Ethernet .../SystemConfiguration/preferences.plist. (Update: Now macOS Sequoia compatible after Apple broke the usual way of getting the SSID name for WiFi connections)

#!/bin/bash
: <<-LICENSE_BLOCK
Limit IP Tracking Status Checker (20250204) - (https://github.com/brunerd)
Copyright (c) 2025 Joel Bruner (https://github.com/brunerd)
Licensed under the MIT License

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
LICENSE_BLOCK

#############
# FUNCTIONS #
#############

#include this self-contained function in your script
function limitIPTracking()(

	#only for Moneterey and up, 11 and under need not apply
	[ "$(sw_vers -productVersion | cut -d. -f1)" -le 11 ] && return 1

	#Little JSON Tool (ljt) - https://github.com/brunerd/ljt - MIT License
	function ljt () ( #v1.0.8 ljt [query] [file]
	{ set +x; } &> /dev/null; read -r -d '' JSCode <<-'EOT'
	try{var query=decodeURIComponent(escape(arguments[0]));var file=decodeURIComponent(escape(arguments[1]));if(query===".")query="";else if(query[0]==="."&&query[1]==="[")query="$"+query.slice(1);if(query[0]==="/"||query===""){if(/~[^0-1]/g.test(query+" "))throw new SyntaxError("JSON Pointer allows ~0 and ~1 only: "+query);query=query.split("/").slice(1).map(function(f){return"["+JSON.stringify(f.replace(/~1/g,"/").replace(/~0/g,"~"))+"]"}).join("")}else if(query[0]==="$"||query[0]==="."&&query[1]!=="."||query[0]==="["){if(/[^A-Za-z_$\d\.\[\]'"]/.test(query.split("").reverse().join("").replace(/(["'])(.*?)\1(?!\\)/g,"")))throw new Error("Invalid path: "+query);}else query=query.replace("\\.","\udead").split(".").map(function(f){return"["+JSON.stringify(f.replace("\udead","."))+"]"}).join("");if(query[0]==="$")query=query.slice(1);var data=JSON.parse(readFile(file));try{var result=eval("(data)"+query)}catch(e){}}catch(e){printErr(e);quit()}if(result!==undefined)result!==null&&result.constructor===String?print(result):print(JSON.stringify(result,null,2));else printErr("Path not found.")
	EOT
	queryArg="${1}"; fileArg="${2}";jsc=$(find "/System/Library/Frameworks/JavaScriptCore.framework/Versions/Current/" -name 'jsc');[ -z "${jsc}" ] && jsc=$(which jsc);[ -f "${queryArg}" -a -z "${fileArg}" ] && fileArg="${queryArg}" && unset queryArg;if [ -f "${fileArg:=/dev/stdin}" ]; then { errOut=$( { { "${jsc}" -e "${JSCode}" -- "${queryArg}" "${fileArg}"; } 1>&3 ; } 2>&1); } 3>&1;else [ -t '0' ] && echo -e "ljt (v1.0.8) - Little JSON Tool (https://github.com/brunerd/ljt)\nUsage: ljt [query] [filepath]\n  [query] is optional and can be JSON Pointer, canonical JSONPath (with or without leading $), or plutil-style keypath\n  [filepath] is optional, input can also be via file redirection, piped input, here doc, or here strings" >/dev/stderr && exit 0; { errOut=$( { { "${jsc}" -e "${JSCode}" -- "${queryArg}" "/dev/stdin" <<< "$(cat)"; } 1>&3 ; } 2>&1); } 3>&1; fi;if [ -n "${errOut}" ]; then /bin/echo "$errOut" >&2; return 1; fi
	)
	
	#sequoia broke the old ways of getting SSID
	function getCurrentWiFiSSIDforInterface(){
		#no interface, error
		[ -z "${1}" ] && return 1

		#get the SSID _quickly_ thanks MacAdmins @jby
		local currrent_SSID=$(ipconfig getsummary "${1}" | awk -F ' SSID : ' '/ SSID : / {print $2}')
		
		[ -z "${currrent_SSID}" ] && return 1
		
		echo "${currrent_SSID}"; return 0
	}

	#test interface specified or fallback to default interface
	interfaceID=${1:-$(route get 0.0.0.0 2>/dev/null | awk '/interface: / {print $2}')}

	#WIFI: key: PrivacyProxyEnabled, file: /Library/Preferences/com.apple.wifi.known-networks.plist
	#if no error getting WiFi SSID, then we are WiFi
	if wifiSSID=$(getCurrentWiFiSSIDforInterface "${interfaceID}"); then

		#key name inside plist
		keyName="wifi.network.ssid.${wifiSSID}"

		#oddly this file is only read-able by root
		if [ ! -r /Library/Preferences/com.apple.wifi.known-networks.plist ]; then
			echo "Insufficient preferences to determine WiFi state, run as root" >/dev/stderr
			exit 1
		fi

		#get JSON version, so much easier to get around, convert date and data types to strings
		wifiKnownNetworks_JSON=$(defaults export /Library/Preferences/com.apple.wifi.known-networks.plist - | sed -e 's,<date>,<string>,g' -e 's,</date>,</string>,g' -e 's,<data>,<string>,g' -e 's,</data>,</string>,g' | plutil -convert json -o - -)
		
		#if there is NO entry, then it is active, it is opt-out designed
		PrivacyProxyEnabled=$(ljt "/wifi.network.ssid.${wifiSSID}/PrivacyProxyEnabled" 2>/dev/null <<< "${wifiKnownNetworks_JSON}")
		
		if [ "${PrivacyProxyEnabled}" = "false" ]; then
			return 1
		fi
	#ETHERNET: key: DisablePrivateRelay, file: /Library/Preferences/SystemConfiguration/preferences.plist, 
	else
		#get JSON, easily converts with not data or date types within
		systemConfigPrefsJSON=$(plutil -convert json -o - /Library/Preferences/SystemConfiguration/preferences.plist)
		#get current set UUID
		currentSet=$(echo "${systemConfigPrefsJSON}" | ljt /CurrentSet 2>/dev/null)
		#get value for current default interface of current Location set
		DisablePrivateRelay=$(ljt "${currentSet}"/Network/Interface/${interfaceID}/DisablePrivateRelay  2>/dev/null <<< "${systemConfigPrefsJSON}")

		#if it is TRUE we are Disabled, then we are NOT ON, return fail code
		if [ "${DisablePrivateRelay}" = "1" ]; then
			return 1
		fi
	fi
)

########
# MAIN #
########

if [ ! -r /Library/Preferences/com.apple.wifi.known-networks.plist ]; then
	echo "Insufficient privileges to determine WiFi state, run as root" >/dev/stderr
	exit 1
fi

#use default interface if nothing is specified for argument $1
interface=${1:-$(route get 0.0.0.0 2>/dev/null | awk '/interface: / {print $2}')}

#returns 0 if ON and 1 if OFF, you may supply an interface or it will fall back to the default
limitIPTracking "${interface}" && echo "Limit IP tracking is ON for: ${interface}" || echo "Limit IP tracking is OFF for: ${interface}"

You can specify an interface or it will use the default interface (there’s a good one-liner to use route get 0.0.0.0 to figure that out). plutil is not initially used to convert com.apple.wifi.known-networks.plist to JSON as it will fail due to XML/plist data types like <date> and <data> that do not have JSON equivalents. First we use sed to change them to <string> types then plutil can convert to JSON. After that it’s a cake walk for ljt to get the value and report back. If the interface is not WiFi then /Library/Preferences/SystemConfiguration/preferences.plist has none of those conversion issues. Check out limitIPTrackingStatus.sh and the minified limitIPTrackingStatus.min.sh at my GitHub.

Thanks for reading! P.S. I won’t be at JNUC 2022, better luck next year!

Determining URL scheme handlers in macOS

If you’ve worked at a place that uses Outlook for email or Chrome for browsers, knowing is half the battle when it comes to helping out your users. For a long while you could easily determine the URL scheme handlers via the built-in Python or JXA, however things have changed.

For example this JXA used to work brilliantly in Mojave and under:

#!/bin/sh
URLScheme="${1}"

if [ -z "${URLScheme}" ]; then
    echo "Please provide a URL scheme name"
    exit 1
fi

osascript -l JavaScript <<< "ObjC.import('Cocoa'); $.LSCopyDefaultHandlerForURLScheme(\"${URLScheme}\")"

Nothing lasts forever though and all that broke in 10.15 when the result of [object Ref] became the taunting reply. Even if you could hit the juke box like the Fonz and recast the CF value you were getting back into an NS value (thanks for the link Pico), it was a moot point by Big Sur. Apple has killed off about every useful API to figure out what app handles a URL as others have noted.

So after you look at LSCopyDefaultHandlerForURLScheme at the ο£Ώ Dev Docs and then see the scores of other deprecated LaunchServices functions, you might make some joke like “I see dead people APIs” (a la Sixth Sense), but still you wonder: Is there a reliable way to divine this information without resorting to a hacky grep?

Why, yes there is, and it just so happens to use my tool ljt too! While definitely a step up from grepping the output it’s still kinda sad. What was once a few lines must now be accomplished by many more in getDefaultRoleHandler.sh and the minified version getDefaultRoleHandler.min.sh.

#!/bin/sh
: <<-LICENSE_BLOCK
getDefaultRoleHandler - (https://github.com/brunerd)
Copyright (c) 2022 Joel Bruner (https://github.com/brunerd)
Licensed under the MIT License

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
LICENSE_BLOCK

#############
# FUNCTIONS #
#############

#use this self-contained function in your script to detect the default role handle
function getDefaultRoleHandler() (
	#provide a URL scheme like: http, https, ftp, etc...
	URLScheme=${1}
	
	#fail quickly
	if [ -z "${URLScheme}" ]; then
		>/dev/stderr echo "No URL scheme specified"
		return 1
	fi

	#Little JSON Tool (ljt) v1.0.7 - https://github.com/brunerd/ljt - MIT License
	function ljt () ( 
	[ -n "${-//[^x]/}" ] && set +x; read -r -d '' JSCode <<-'EOT'
	try{var query=decodeURIComponent(escape(arguments[0])),file=decodeURIComponent(escape(arguments[1]));if("/"===query[0]||""===query){if(/~[^0-1]/g.test(query+" "))throw new SyntaxError("JSON Pointer allows ~0 and ~1 only: "+query);query=query.split("/").slice(1).map(function(a){return"["+JSON.stringify(a.replace(/~1/g,"/").replace(/~0/g,"~"))+"]"}).join("")}else if("$"===query[0]||"."===query[0]||"["===query[0]){if(/[^A-Za-z_$\d\.\[\]'"]/.test(query.split("").reverse().join("").replace(/(["'])(.*?)\1(?!\\)/g,"")))throw Error("Invalid path: "+query);}else query=query.replace("\\.","\udead").split(".").map(function(a){return"["+JSON.stringify(a.replace("\udead","."))+"]"}).join("");"$"===query[0]&&(query=query.slice(1,query.length));var data=JSON.parse(readFile(file));try{var result=eval("(data)"+query)}catch(a){}}catch(a){printErr(a),quit()}void 0!==result?null!==result&&result.constructor===String?print(result):print(JSON.stringify(result,null,2)):printErr("Path not found.");
	EOT
	queryArg="${1}"; fileArg="${2}";jsc=$(find "/System/Library/Frameworks/JavaScriptCore.framework/Versions/Current/" -name 'jsc');[ -z "${jsc}" ] && jsc=$(which jsc);[ -f "${queryArg}" -a -z "${fileArg}" ] && fileArg="${queryArg}" && unset queryArg;if [ -f "${fileArg:=/dev/stdin}" ]; then { errOut=$( { { "${jsc}" -e "${JSCode}" -- "${queryArg}" "${fileArg}"; } 1>&3 ; } 2>&1); } 3>&1;else [ -t '0' ] && echo -e "ljt (v1.0.7) - Little JSON Tool (https://github.com/brunerd/ljt)\nUsage: ljt [query] [filepath]\n  [query] is optional and can be JSON Pointer, canonical JSONPath (with or without leading $), or plutil-style keypath\n  [filepath] is optional, input can also be via file redirection, piped input, here doc, or here strings" >/dev/stderr && exit 0; { errOut=$( { { "${jsc}" -e "${JSCode}" -- "${queryArg}" "/dev/stdin" <<< "$(cat)"; } 1>&3 ; } 2>&1); } 3>&1; fi;if [ -n "${errOut}" ]; then /bin/echo "$errOut" >&2; return 1; fi
	)

	#in case being run as root get the current console user
	consoleUserHomeFolder=$(sudo -u "$(stat -f %Su /dev/console)" sh -c 'echo ~')
	#get the LaunchServices LSHandlers JSON of the console user
	launchServicesJSON=$(launchctl asuser "$(stat -f %u /dev/console)" sudo -u "$(stat -f %Su /dev/console)" plutil -extract LSHandlers json -o - "${consoleUserHomeFolder}"/Library/Preferences/com.apple.LaunchServices/com.apple.launchservices.secure.plist)

	#loop through JSON and try and find matching URLScheme within
	for ((i=0;;i++)); do
		#if we are at the END of the array or nothing exists bail
		if ! ljt "/$i" <<< "${launchServicesJSON}" &>/dev/null; then
			return 1
		elif [ "$(ljt "/$i/LSHandlerURLScheme" <<< "${launchServicesJSON}" 2>/dev/null)" = "$URLScheme" ]; then
			#run query, print result, errors go to /dev/null, if ljt fails to find something return non-zero
			if ! ljt "/$i/LSHandlerRoleAll" <<< "${launchServicesJSON}" 2>/dev/null; then
				#error
				return 1
			else
				#success
				return 0
			fi
		fi
	done
	
	#if we are here, we did NOT find a match
	return 1	
)

########
# MAIN #
########

getDefaultRoleHandler "$@"

We use plutil to extract the contents of the LSHandlers key as JSON from ~/Library/Preferences/com.apple.LaunchServices/com.apple.launchservices.secure.plist then we iterate over the array using a for loop and ljt until it finds a matching LSHandlerURLScheme and then it prints out the value of LSHandlerRoleAll. The operation and output are simple and no nonsense. Give it a URL scheme and it gives you the Bundle ID of the app which will open it or if not found, nothing. (Note: mailto will be blank by default, only after switching to another client will it return a result)

You can use this function in your scripts. Head over to my GitHub to download either getDefaultRoleHandler.sh or the minified getDefaultRoleHandler.min.sh. Perhaps you need to alert a user to change their application preferences after installation of an email client or web browser. You can use my shui function to alert them, btw. But I’m not getting into setting handlers via script in this post, writing to the plist can be a trick but the real trick is getting LaunchService to read from that plist (without asking for a reboot or logout, ick)! So for now “Knowing Is Half The Battleβ„’

Post Posting Update

Armin Briegel, over at the wonderful wealth of aggregated MacAdmin knowledge that is scriptingosx.com, is always kind enough to post my articles and this very one appears in the Week of 2022-09-22. Along with this post is a link to a MacAdmins Slack thread where he kindly provided some JXA that returns the file path for a URL scheme handler (vs. the above Bundle ID), then Pico jumped in and crushed it down to a 1 liner! I’ve added the thinnest of function wrappers over top:

#!/bin/sh
#getDefaultRoleHandlerPath - Joel Bruner with πŸ™ thanks to a couple stalwarts of the MacAdmin Slack Armin Briegel and Pico

function getDefaultRoleHandlerPath()(urlScheme="${1}"; [ -z "${urlScheme}" ] && return 1; osascript -l 'JavaScript' -e "ObjC.import('AppKit'); $.NSWorkspace.sharedWorkspace.URLForApplicationToOpenURL($.NSURL.URLWithString('${urlScheme}:')).path.js")

getDefaultRoleHandlerPath "$@"
exit $?

So it seems Apple has spared an API in AppKit’s NSWorkspace by the name of URLForApplicationToOpenURL, which has been around since macOS 10.6! Perhaps when I get over my trust issues with JXA, I’ll explore these APIs to see what else might be useful. For now though this does the trick of returning a filepath (not Bundle ID)

Listing All Schemes for User

I got a question about listing all the schemes a user might have registered. This is an example one-liner (granted jpt is embedded in the script or installed locally) to be run by the current user. If you want to run as root, cannibalize getDefaultRoleHandler.sh for code)

jpt -T '$.*.LSHandlerURLScheme' <<< $(plutil -extract LSHandlers json -o - ~/Library/Preferences/com.apple.LaunchServices/com.apple.launchservices.secure.plist)

The -T option will output text (vs. an array of double quoted JSON strings) .* looks at every array member to print out every LSHandlerURLScheme entry for the current console user. It’ll look something like this

brunerd tools summer 2022 updates

Hey there, it’s summer 2022 and I wanted to share some updates on my shell tools for Mac admins.

No breaking changes or anything huge, but some nice incremental improvements all the same. When the JSONPath spec is finalized I’ll give jpt a more considerable under the hood improvement. For now though, I’m pretty happy with tweaking their behaviors to act like you’d expect (print help and exit if no input detected), to get out of your way when you need to debug your scripts (xtrace is now disabled inside them) and to try to understand what you mean when you ask them a question (key path queries now accepted). Because the tools and technologies we make should serve us, not the other way around. I hope these tools can save you time, energy, sanity or all the above!

β€’ jpt the “JSON Power Tool” has been updated to v1.0.2 with all the nice additions mentioned. If you don’t know already jpt can parse, query, and modify JSON every which way. It can be used as both a standalone utility as well as a function to be embedded into your shell scripts, without any other dependencies.

β€’ ljt the “little JSON tool” is jpt‘s smaller sibling weighing in at only 2k. While it lacks the transformational powers of jpt it can easily pluck a value out of JSON up to 2GB big and output results up to 720MB. It has been updated to v1.0.7 because dang it, even a small script can have just as many bugs as the big ones. Sometimes more if you’re not careful!

Both jpt and ljt now accept ye olde NextStep “keypath” expressions to help win over masochists who might still use plutil to work with JSON! 😁 (I kid, I kid!)

β€’ jse, the “JSON string encoder” does this one thing and does it well. It can be used in conjunction with jpt when taking in arbitrary data that needs encoding only. It’s a svelte 1.4k (in minified form) and runs considerably faster that jpt, so it’s ideal for multiple invocations, such as when preparing input data for a JSON Patch document. v1.0.2 is available here

β€’ shui lets you interact with your users with AppleScript but in the comfort of familiar shell syntax! You don’t need to know a line of AppleScript, although it does have the option to output it if you want to learn. Save time, brain cells and hair with shui, while you keep on trucking in shell script. It has been updated to v20220704.

P.S. All these tools still work in the macOS Ventura beta πŸ˜…

Improvisational jpt: Processing Apple Developer JSON transcript files

If you’ve downloaded the Developer app for the Mac there’s a trove of JSON transcripts cached in your home folder at ~/Library/Group Containers/group.developer.apple.wwdc/Library/Caches/Transcripts

Being curious I took a look at them using my JSON Power Tool jpt. In it’s default mode it will “pretty print” JSON or in Javascript parlance “stringify” them with a two space indent per nesting level. Inside it can be seen the transcripts are arrays of arrays inisde a uniquely named object. The 1st entry of the array is the time in seconds and the 2nd entry is the string we want.

Arrays of arrays

One of jpt’s cool features is that it supports the venerable yet nascent JSONPath query syntax. Using JSONPath we can use the recursive operator .. to go straight to the transcript object without needing determine the unique name of the parent object, then we want all the array within there [*]and inside those array we want the second entry of the 0 based array [1]. The query looks like this $..transcript[*][1]

Just text please

The query is single quoted so the shell doesn’t interpret the $ as the beginning of a variable name. The -T option for jpt it outputs text without quotes. The default output mode for jpt is JSON (double quoted strings). I added all sorts of other niceties to the script, as you’ll see below. The results are output to your ~/Desktop in a folder called Developer Transcripts

#!/bin/bash
: <<-LICENSE_BLOCK
Developer Transcript Extractor Copyright (c) 2022 Joel Bruner. Licensed under the MIT License. Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
LICENSE_BLOCK

############
# VARIABLES #
############

destinationFolderName="Developer Transcripts"
destinationPath="${HOME}/Desktop/${destinationFolderName}"

#you'll need to download the Developer app and launch it: https://apps.apple.com/us/app/apple-developer/id640199958
transcriptPaths=$(find "${HOME}/Library/Group Containers/group.developer.apple.wwdc/Library/Caches/Transcripts/ByID" -name '*json')
contentsJSON="${HOME}/Library/Group Containers/group.developer.apple.wwdc/Library/Caches/contents.json"

########
# MAIN #
########

#either jpt should be installed or the function jpt.min can be pasted in here
if ! which jpt &>/dev/null; then
	echo "Please install jpt or embed jpt.min in this script: https://github.com/brunerd/jpt"
	exit 1
fi

#ensure the destination folder exists
[ ! -d "${destinationPath}" ] && mkdir "${destinationPath}"

#ignore spaces in file paths
IFS=$'\n'

#loop through each transcript json file
for transcriptPath in ${transcriptPaths}; do
	#id is just the file name without the path and extension
	id=$(cut -d. -f1 <<< "${transcriptPath##*/}")
	#a couple of nice-to-haves
	title=$(jpt -T '$.contents[?(@.id == "'"${id}"'")].title' "${contentsJSON}")
	description=$(jpt -T '$.contents[?(@.id == "'"${id}"'")].description' "${contentsJSON}")
	#change \ (disallowed in Unix) to : (Disallowed in Finder byt allowed in Unix)
	title=${title//\//:}
	url=$(jpt -T '$.contents[?(@.id == "'"${id}"'")].webPermalink' "${contentsJSON}")

	#"wwdc" always has the year in the id but not tech-talks or insights
	if ! grep -q -i wwdc <<< "$id"; then
		year="$(jpt '$.contents[?(@.id == "'"${id}"'")].originalPublishingDate' "${contentsJSON}" | date -j -r 1593018000 +"%Y")-"
		filename="${year}${id} - ${title}.txt"
	else
		filename="${id} - ${title}.txt"
	fi

	#put the ID and Title, the URL and Description at the top of the transcript
	echo "${id} - ${title}" > "${destinationPath}"/"${filename}"
	echo -e "${url}\n" >> "${destinationPath}"/"${filename}"
	echo -e "Description:\n${description}\n\nTranscript:" >> "${destinationPath}"/"${filename}"
	
	#append the transcript extract
	jpt -T '$..transcript[*][1]' "${transcriptPath}" >> "${destinationPath}"/"${filename}"

	#just echo out our progress
	echo "${destinationPath}"/"${filename}"
done

The final result is sortable folder of text files that you can easily QuickLook through.

Some serviceably readable contents!

So there you go! Some surprise JSON transcript files from the Apple Developer app, made me wonder how someone would turn them into human readable files. It turned out it was a fun and practical use of jpt and it’s support for JSONPath. You can download an installer package from the Releases page to try it out.

Respecting Focus and Meeting Status in Your Mac scripts (aka Don’t Be a Jerk)

In four words Barbara Krueger distills the Golden Rule into an in-your-face admonishment: Don’t be a jerk. It makes for a great coffee mug but how does this relate to shell scripting for a Mac admin and engineer? Well, sometimes a script can only go so far and you need user consent and cooperation to get the job done. Not being a jerk about it can pay off!

A good way to get cooperation from your users is to build upon a foundation of trust and respect. While IT has a long list of to-dos for users: “Did you reboot? Did you run updates? Did you open a ticket? Did you really reboot?” the users might have one for us: “Respect our screens when we’re in the middle of a client meeting and respect when Focus mode is turned on.” And they are right. That’s also two things. “Give an inch and they take a mile” I tell ya!

So how can we in IT can be considerate of our users? First, don’t do anything Nick Burns from SNL does. Second, use the functions below (and in my GitHub) to check if a Zoom, Teams, Webex, or Goto meeting is happening or if Do Not Disturb/Focus mode is on. When non-user-initiated scripts (i.e. daily pop-ups/nags/alerts) run they can bail or wait if the user is busy. If it were real life (and it is!) we wouldn’t walk into a meeting and bug a user, in front of everyone, to run update. If we did, they’d be more likely to remember how rude we were rather than actually running the updates. So let’s get their attention when they will be most receptive to what we have to say.

Detecting Online Meetings Apps

First up is inMeeting_Zoom which simply checks for the CptHost process and returns success or fail. Notice how this simple behavior can be used with an if/then statement. The return code is evaluated by the if, a zero is success and a non-zero is a failure. && is a logical AND and || is a logical OR

#!/bin/sh
#inMeeting_Zoom (20220227) Copyright (c) 2022 Joel Bruner (https://github.com/brunerd)
#Licensed under the MIT License

function inMeeting_Zoom {
	#if this process exists, there is a meeting, return 0 (sucess), otherwise 1 (fail)
	pgrep "CptHost" &>/dev/null && return 0 || return 1
}

if inMeeting_Zoom; then
	echo "In Zoom meeting... don't be a jerk"
else
	echo "Not in Zoom meeting"
fi

Next is another process checker for Webex: inMeetng_Webex. What is a bit more unique is the process appears in ps in parentheses as (WebexAppLauncher)however pgrep cannot find this process (because the actual name has been rewritten by the Meeting Center process). We instead use a combination of ps and grep. A neat trick with grep is to use a [] regex character class to surround a single character, this keeps grep from matching itself in the process list. That way you don’t need to have an extra grep -v grep to clean up the output.

#!/bin/sh
#inMeeting_Webex (20220227) Copyright (c) 2022 Joel Bruner (https://github.com/brunerd)
#Licensed under the MIT License

function inMeeting_Webex {
	#if this process exists, there is a meeting, return 0 (sucess), otherwise 1 (fail)
	ps auxww | grep -q "[(]WebexAppLauncher)" && return 0 || return 1
}

if inMeeting_Webex; then
	echo "In Zoom meeting... don't be a jerk"
else
	echo "Not Webex in meeting"
fi

Goto Meeting is more straightforward, although it should be noted that regardless of quote type, single or double, the parentheses must be escaped with a backslash. Otherwise, it’s the same pattern, look for the process name which only appears during a meeting or during the meeting preview and return 0 or 1 for if to evaluate, find it here: inMeeting_Goto

#!/bin/sh
#inMeeting_Goto (20220227) Copyright (c) 2022 Joel Bruner (https://github.com/brunerd)
#Licensed under the MIT License

function inMeeting_Goto() {
	#if this process exists, there is a meeting, return 0 (sucess), otherwise 1 (fail)
	pgrep "GoTo Helper \(Plugin\)" &>/dev/null && return 0 || return 1
}

if inMeeting_Goto; then
	echo "In Goto meeting... don't be a jerk"
else
	echo "Not in Goto meeting"
fi

Lastly, Teams is a bit more complex, rather than looking for the presence of a process, we instead look for a JSON file in the user’s /Library/Application Support/Microsoft/Teams folder which has the current call status for both the app and the web plugin (the other methods above are for the app only). We’ll use the ljt to extract the value from the JSON. In fact I wrote ljt after starting to write this blog last week and realizing that jpt (weighing in at 64k) was just overkill. As a bonus to doing that, I just realized that bash functions can contain functions! Long ago I ditched using () in shell function declarations and just used the function keyword. Empty parentheses seemed decorative rather than functional since it’s not like it’s a C function that needs parameter names and types. However the lack of parentheses () apparently, prevents a function from being declared inside a function! Below I just wanted to make sure ljt doesn’t get separated from inMeetings_Teams

#!/bin/bash
#inMeeting_Teams (20220227) Copyright (c) 2022 Joel Bruner (https://github.com/brunerd)
#Licensed under the MIT License

function inMeeting_Teams ()(

	function ljt () ( #v1.0.3
		[ -n "${-//[^x]/}" ] && set +x; read -r -d '' JSCode <<-'EOT'
		try {var query=decodeURIComponent(escape(arguments[0]));var file=decodeURIComponent(escape(arguments[1]));if (query[0]==='/'){ query = query.split('/').slice(1).map(function (f){return "["+JSON.stringify(f)+"]"}).join('')}if(/[^A-Za-z_$\d\.\[\]'"]/.test(query.split('').reverse().join('').replace(/(["'])(.*?)\1(?!\\)/g, ""))){throw new Error("Invalid path: "+ query)};if(query[0]==="$"){query=query.slice(1,query.length)};var data=JSON.parse(readFile(file));var result=eval("(data)"+query)}catch(e){printErr(e);quit()};if(result !==undefined){result!==null&&result.constructor===String?print(result): print(JSON.stringify(result,null,2))}else{printErr("Node not found.")}
		EOT
		queryArg="${1}"; fileArg="${2}";jsc=$(find "/System/Library/Frameworks/JavaScriptCore.framework/Versions/Current/" -name 'jsc');[ -z "${jsc}" ] && jsc=$(which jsc);[ -f "${queryArg}" -a -z "${fileArg}" ] && fileArg="${queryArg}" && unset queryArg;if [ -f "${fileArg:=/dev/stdin}" ]; then { errOut=$( { { "${jsc}" -e "${JSCode}" -- "${queryArg}" "${fileArg}"; } 1>&3 ; } 2>&1); } 3>&1;else { errOut=$( { { "${jsc}" -e "${JSCode}" -- "${queryArg}" "/dev/stdin" <<< "$(cat)"; } 1>&3 ; } 2>&1); } 3>&1; fi;if [ -n "${errOut}" ]; then /bin/echo "$errOut" >&2; return 1; fi
	)

	consoleUser=$(stat -f %Su /dev/console)
	consoleUserHomeFolder=$(dscl . -read /Users/"${consoleUser}" NFSHomeDirectory | awk -F ': ' '{print $2}')
	storageJSON_path="${consoleUserHomeFolder}/Library/Application Support/Microsoft/Teams/storage.json"
	
	#no file, no meeting
	[ ! -f "${storageJSON_path}" ] && return 1

	#get both states
	appState=$(ljt /appStates/states "${storageJSON_path}" | tr , $'\n' | tail -n 1)
	webappState=$(ljt /webAppStates/states "${storageJSON_path}"| tr , $'\n' | tail -n 1)
	
	#determine app state
	if [ "${appState}" = "InCall" ]	|| [ "${webAppState}" = "InCall" ]; then
		return 0
	else
		return 1
	fi
)


if inMeeting_Teams; then
	echo "In Teams Meeting... don't be a jerk"
else
	echo "Not in Teams Meeting"
fi

Detecting Focus (formerly Do Not Disturb)

Last but not least is determining Focus (formerly Do Not Disturb) with doNotDisturb. As you can see there’s been a few different ways this has been implemented over the years. In macOS 10.13-11 the state was stored inside of a plist. For macOS 12 Monterey they’ve switched from a plist to a JSON file. A simple grep though is all that’s needed to find the key name storeAssertionRecords. If it is off, that string is nowhere to be find, when it’s on it’s there. Simple (as in Keep it Simple Stoopid)

#!/bin/bash
#doNotDisturb (grep) (20220227) Copyright (c) 2022 Joel Bruner (https://github.com/brunerd)
#Licensed under the MIT License

#An example of detecting Do Not Disturb (macOS 10.13-12)

function doNotDisturb()(

	OS_major="$(sw_vers -productVersion | cut -d. -f1)"
	consoleUserID="$(stat -f %u /dev/console)"
	consoleUser="$(stat -f %Su /dev/console)"
	
	#get Do Not Disturb status
	if [ "${OS_major}" = "10" ]; then
		#returns c-cstyle boolean 0 (off) or 1 (on)
		dndStatus="$(launchctl asuser ${consoleUserID} sudo -u ${consoleUser} defaults -currentHost read com.apple.notificationcenterui doNotDisturb 2>/dev/null)"

		#eval c-style boolean and return shell style value
		[ "${dndStatus}" = "1" ] && return 0 || return 1
	#this only works for macOS 11 - macOS12 does not affect any of the settings in com.apple.ncprefs
	elif [ "${OS_major}" = "11" ]; then
		#returns "true" or [blank]
		dndStatus="$(/usr/libexec/PlistBuddy -c "print :userPref:enabled" /dev/stdin 2>/dev/null <<< "$(plutil -extract dnd_prefs xml1 -o - /dev/stdin <<< "$(launchctl asuser ${consoleUserID} sudo -u ${consoleUser} defaults export com.apple.ncprefs.plist -)" | xmllint --xpath "string(//data)" - | base64 --decode | plutil -convert xml1 - -o -)")"

		#if we have ANYTHING it is ON (return 0) otherwise fail (return 1)
		[ -n "${dndStatus}" ] && return 0 || return 1
	elif [ "${OS_major}" -ge "12" ]; then
		consoleUserHomeFolder=$(dscl . -read /Users/"${consoleUser}" NFSHomeDirectory | awk -F ': ' '{print $2}')
		file_assertions="${consoleUserHomeFolder}/Library/DoNotDisturb/DB/Assertions.json"

		#if Assertions.json file does NOT exist, then DnD is OFF
		[ ! -f "${file_assertions}" ] && return 1

		#simply check for storeAssertionRecords existence, usually found at: /data/0/storeAssertionRecords (and only exists when ON)
		! grep -q "storeAssertionRecords" "${file_assertions}" 2>/dev/null && return 1 || return 0
	fi
)
if doNotDisturb; then
	echo "DnD/Focus is ON... don't be a jerk"
else
	echo "DnD/Focus is OFF"
fi

Since Focus can remain on indefinitely an end user may never see your pop-up. If so, build a counter with a local plist to record and increment the number of attempts. After a threshold has been reached you can then break through to the user (I certainly do).

Detecting Apps in Presentation Mode

A newer addition to this page is some code to detect fullscreen presentation apps that I cannot take credit for. Adam Codega, one of the contributors to Installomator hipped me to a cool line of code that was added in PR 268. It leverages pmset to see what assertions have been made to the power management subsystem. It uses awk to look for the IOPMAssertionTypes named NoDisplaySleepAssertion and PreventUserIdleDisplaySleep with some additional logic to throw out false positives from coreaudiod. In testing I’ve found this able to detect the presentation modes of Keynote, Powerpoint and Google Slides in Slideshow mode in Chrome (but not Safari), your mileage may vary for other apps. Another caveat is that when a YouTube video is playing in a visible tab, it will assert a NoDisplaySleepAssertion, however these will be named “Video Wake Lock” whereas a Slideshow presentation mode will have its name assertions named “Blink Wake Lock”. So I am adding an additional check to throw our “Video Wake Locks”. This may be more of a can of worms than you’d like, if so, user education to set Focus mode may be the way to go. A functionalized version can be found here: inPresentationMode

#!/bin/sh
#inPresentationMode (20220319) Copyright (c) 2022 Joel Bruner (https://github.com/brunerd)
#with code from Installomator (PR 268) (https://github.com/Installomator/Installomator) Copyright 2020 Armin Briegel
#Licensed under the MIT License

function inPresentationMode {
	#Apple Dev Docs: https://developer.apple.com/documentation/iokit/iopmlib_h/iopmassertiontypes
	#ignore assertions without the process in parentheses, any coreaudiod procs, and "Video Wake Lock" is just Chrome playing a Youtube vid in the foreground
	assertingApps=$(/usr/bin/pmset -g assertions | /usr/bin/awk '/NoDisplaySleepAssertion | PreventUserIdleDisplaySleep/ && match($0,/\(.+\)/) && ! /coreaudiod/ && ! /Video\ Wake\ Lock/ {gsub(/^.*\(/,"",$0); gsub(/\).*$/,"",$0); print};')
	[ -n "${assertingApps}" ] && return 0 || return 1
}

if inPresentationMode; then
	echo "In presentation mode... don't be a jerk"
else
	echo "Not in presentation mode..."
fi

All together now

Putting it all together here’s how you can test multiple functions in a single if statement, just chain them together with a bunch of || ORs

#!/bin/bash
#Joel Bruner - demo of meeting/focus aware functions for your script

#pretend we've declared all the functions above and copy and pasted them in here
function doNotDisturb()(:)
function inMeeting_Teams()(:)
function inMeeting_Zoom(){:}
function inMeeting_Goto(){:}
function inMeeting_Webex(){:}
function inPresentationMode(){:}

#test each one with || OR conditionals
#the FIRST successful test will "short-circuit" and no more functions will be run
if doNotDisturb || inPresentationMode || inZoomMeeting || inMeeting_Goto || inMeeting_Webex || inMeeting_Teams; then
	echo "In a meeting, presentation, or Focus is On... don't be a jerk"
	#do something else, like wait 
else
	echo "Not in a meeting..."
	#alert the user or do whatever you needed to do that might impact them
fi

Using these functions in your scripts can help respect your users’ online meeting times and Focus states. Also it doesn’t hurt to document it somewhere and toot your own horn in a user facing KB or wiki. If and when a user complains about that pop-up that destroyed their concentration and their world, you can show them the forethought and effort you’ve taken to be as considerate as possible regarding this perceived incursion. This usually has the effect of blowing their mind 🀯 that someone in IT is actually trying to be considerate!

P.S. I’m pretty stoked that Prism.js can really jazz up my normally dreary grey code blocks! πŸ˜πŸ€“ I found a good WordPress tutorial here

ljt, a little JSON tool for your shell script

ljt, the Little JSON Tool, is a concise and focused tool for quickly getting values from JSON and nothing else. It can be used standalone or embedded in your shell scripts. It requires only macOS 10.11+ or a *nix distro with jsc installed. It has no other dependencies.

You might have also seen my other project jpt, the JSON Power Tool. It too can be used standalone or embedded in a script however its features and size (64k) might be overkill in some case and I get it! I thought the same thing too after I looked at work of Mathew Warren, Paul Galow, and Richard Purves. Sometimes you don’t need to process JSON Text Sequences, use advanced JSONPath querying, modify JSON, or encode the output in a myriad of ways. Maybe all you need is something to retrieve a JSON value in your shell script.

Where jpt was an exercise in maximalism, ljt is one of essential minimalism – or at least as minimal as this CliftonStrengths Maximizer can go! πŸ€“ The minified version is mere 1.2k and offers a bit more security and functionality than a one-liner.

ljt features:
β€’ Query using JSON Pointer or JSONPath (canonical only, no filters, unions, etc)
β€’ Javascript code injection prevention for both JSON and the query
β€’ Multiple input methods: file redirection, file path, here doc, here string and Unix pipe
β€’ Output of JSON strings as regular text and JSON for all others
β€’ Maximum input size of 2GB and max output of 720MB (Note: functions that take JSON data as an environment variable or an argument are limited to a maximum of 1MB of data)
β€’ Zero and non-zero exit statuses for success and failure, respectively

Swing by the ljt Github page and check it out. There are two versions of the tool one is fully commented for studying and hacking (ljt), the other is a “minified” version without any comments meant for embedding into your shell scripts (ljt.min). The Releases page has a macOS pkg package to install and run ljt as a standalone utility.

Thanks for reading and happy scripting!

jpt 1.0 text encoding fun

Besides JSON, jpt (the JSON power tool) can also output strings and numbers in a variety of encodings that the sysadmin or programmer might find useful. Let’s look at the encoding options from the output of jpt -h

% jpt -h
...
-T textual output of all data (omits property names and indices)
  Options:
	-e Print escaped characters literally: \b \f \n \r \t \v and \\ (escapes formats only)
	-i "<value>" indent spaces (0-10) or character string for each level of indent
	-n Print null values as the string 'null' (pre-encoding)

	-E "<value>" encoding options for -T output:

	  Encodes string characters below 0x20 and above 0x7E with pass-through for all else:
		x 	"\x" prefixed hexadecimal UTF-8 strings
		O 	"\nnn" style octal for UTF-8 strings
		0 	"\0nnn" style octal for UTF-8 strings
		u 	"\u" prefixed Unicode for UTF-16 strings
		U 	"\U "prefixed Unicode Code Point strings
		E 	"\u{...}" prefixed ES2016 Unicode Code Point strings
		W 	"%nn" Web encoded UTF-8 string using encodeURI (respects scheme and domain of URL)
		w 	"%nn" Web encoded UTF-8 string using encodeURIComponent (encodes all components URL)

		  -A encodes ALL characters
	
	  Encodes both strings and numbers with pass-through for all else:
		h 	"0x" prefixed lowercase hexadecimal, UTF-8 strings
		H 	"0x" prefixed uppercase hexadecimal, UTF-8 strings
		o 	"0o" prefixed octal, UTF-8 strings
		6 	"0b" prefixed binary, 16 bit _ spaced numbers and UTF-16 strings
		B 	"0b" prefixed binary, 8 bit _ spaced numbers and UTF-16 strings
		b 	"0b" prefixed binary, 8 bit _ spaced numbers and UTF-8 strings

		  -U whitespace is left untouched (not encoded)

Strings

While the above conversion modes will do both number and string types, these options will work only on strings (numbers and booleans pass-through). If you work with with shell scripts these techniques may be useful.

If you store shell scripts in a database that’s not using utf8mb4 table and column encodings then you won’t be able to include snazzy emojis to catch your user’s attention! In fact this WordPress install was so old (almost 15 years!) the default encoding was still latin1_swedish_ci, which an odd but surprisingly common default for many old blogs. Also if you store your scripts in Jamf (still in v10.35 as of this writing) it uses latin1 encoding and your 4 byte characters will get mangled. Below you can see in Jamf things look good while editing, fails once saved, and the eventual workaround is to use an coding like \x escaped hex (octal is an alternate)

Let’s use the red “octagonal sign” emoji, which is a stop sign to most everyone around the world, with the exception of Japan and Libya (thanks Google image search). Let’s look at some of the way πŸ›‘ can be encoded in a shell script

#reliable \x hex notation for bash and zsh
% jpt -STEx <<< "Alert πŸ›‘"
Alert \xf0\x9f\x9b\x91

#above string can be  in both bash and zsh
% echo $'Alert \xf0\x9f\x9b\x91'
Alert πŸ›‘

#also reliable, \nnn octal notation
% jpt -STEO <<< "Alert πŸ›‘"
Alert \360\237\233\221

#works in both bash and zsh
% echo $'Alert \360\237\233\221'
Alert πŸ›‘

#\0nnn octal notation
% jpt -STE0 <<< "Alert πŸ›‘"
Alert \0360\0237\0233\0221

#use with shell builtin echo -e and ALWAYS in double quotes
#zsh does NOT require -e but bash DOES, safest to use -e
% echo -e "Alert \0360\0237\0233\0221"
Alert πŸ›‘

#-EU code point for zsh only
% jpt -STEU <<< "Alert πŸ›‘"
Alert \U0001f6d1

#use in C-style quotes in zsh
% echo $'Alert \U0001f6d1'
Alert πŸ›‘

The -w/-W flags can encode characters for use in URLs

#web/percent encoded output in the case of non-URLs -W and -w are the same
% jpt -STEW <<< πŸ›‘  
%F0%9F%9B%91

#-W URL example (encodeURI)
jpt -STEW <<< http://site.local/page.php?wow=πŸ›‘
http://site.local/page.php?wow=%F0%9F%9B%91

#-w will encode everything (encodeURIComponent)
% jpt -STEw <<< http://site.local/page.php?wow=πŸ›‘
http%3A%2F%2Fsite.local%2Fpage.php%3Fwow%3D%F0%9F%9B%91

And a couple other oddballs…

#text output -T (no quotes), -Eu for \u encoding
#not so useful for the shell scripter
#zsh CANNOT handle multi-byte \u character pairs
% jpt -S -T -Eu <<< "Alert πŸ›‘"
Alert \ud83d\uded1

#-EE for an Javascript ES2016 style code point
% jpt -STEE <<< "Alert πŸ›‘"
Alert \u{1f6d1}

You can also \u encode all characters above 0x7E in JSON with the -u flag

#JSON output (not using -T)
% jpt <<< '"Alert πŸ›‘"'
"Alert πŸ›‘"

#use -S to treat input as a string without requiring hard " quotes enclosing
% jpt -S <<< 'Alert πŸ›‘'
"Alert πŸ›‘"

#use -u for JSON output to encode any character above 0x7E
% jpt -Su <<< 'Alert πŸ›‘'
"Alert \ud83d\uded1"

#this will apply to all strings, key names and values
% jpt -u <<< '{"πŸ›‘":"stop", "message":"Alert πŸ›‘"}' 
{
  "\ud83d\uded1": "stop",
  "message": "Alert \ud83d\uded1"
}

Whew! I think I covered them all. If there are newlines, tabs and other invisibles you can choose to output them or leave them encoded when you are outputting to text with -T

#JSON in, JSON out
jpt <<< '"Hello\n\tWorld"'
"Hello\n\tWorld"

#ANSI-C string in, -S to treat as string despite lack of " with JSON out
% jpt -S <<< $'Hello\n\tWorld' 
"Hello\n\tWorld"

#JSON in, text out: -T alone prints whitespace characters
% jpt -T <<< '"Hello\n\tWorld"'
Hello
	World

#use the -e option with -T to encode whitespace
% jpt -Te <<< '"Hello\n\tWorld"'
Hello\n\tWorld

Numbers

Let’s start simply with some numbers. First up is hex notation in the style of 0xhh and 0XHH. This encoding has been around since ES1, use the -Eh and -EH respectively to do so. All alternate output (i.e. not JSON) needs the -T option. In shell you can combine multiple options/flags together except only the last flag can have an argument, like -E does below.

#-EH uppercase hex
% jpt -TEH <<< [255,256,4095,4096] 
0xFFa
0x100
0xFFF
0x1000

#-Eh lowecase hex
% jpt -TEh <<< [255,256,4095,4096]
0xff
0x100
0xfff
0x1000

Next up are ye olde octals. Use the -Eo option to convert numbers to ye olde octals except using the more modern 0o prefix introduced in ES6

-Eo ES6 octals
% jpt -TEo <<< [255,256,4095,4096]    
0o377
0o400
0o7777
0o10000

Binary notation debuted in the ES6 spec, it used a 0b prefix and allows for _ underscore separators

#-E6 16 bit wide binary
% jpt -TE6 <<< [255,256,4095,4096]
0b0000000011111111
0b0000000100000000
0b0000111111111111
0b0001000000000000

#-EB 16 bit minimum width with _ separator per 8
% jpt -TEB <<< [255,256,4095,4096]
0b00000000_11111111
0b00000001_00000000
0b00001111_11111111
0b00010000_00000000

#-Eb 8 bit minimum width with _ separator per 8
% jpt -TEb <<< [15,16,255,256,4095,4096]
0b00001111
0b00010000
0b11111111
0b00000001_00000000
0b00001111_11111111
0b00010000_00000000

If you need to encode strings or numbers for use in scripting or programming, then jpt might be a handy utility for you and your Mac and if your *nix has jsc then it should work also. Check the jpt Releases page for Mac installer package download.

jpt 1.0, the JSON power tool in 2022

I’m happy to announce that jpt, the JSON power tool, has been updated to 1.0 and is available on my GitHub Releases page! It’s been over a year since the last release and I’ve been of working on, learning, and pondering what a really useful JSON tool could be and then working like heck to make it a reality.

jpt is a command line utility for formatting, querying, and modifying JSON. It can run on any version of macOS and most any Unix/Linux with jsc (JavaScriptCore) installed, otherwise it’s dependency free! For systems administrators and MacAdmins who need first class JSON tooling in their own shell scripts it can be embedded as a shell function in either bash or zsh. jpt was built to stand the test of time, needing only jsc and a shell, it will work on the newest macOS all the way back to OS X 10.4 Tiger!

There’s a year’s worth of bug fixes, improvements, and new features, let’s take a look at some of them.

Notable Features and Improvement

β€’Β jpt can now input and output multi-JSON documents like JSON Text Sequences (RFC 7464) plus other non-standard formats like NDJSON and JSON Lines and concatenated JSON. See: jpt 1.0 can deal with multiple JSON texts

β€’ Truncated JSON can be de detected in concatenated/lines JSON and repaired by closing up open strings, arrays and objects. See: Helping truncated JSON data with jpt 1.0

β€’ jpt can now fully rehabilitate JSON5 back to standard JSON. As I learned last year, there’s more mutant JSON than I realized! See: jpt 1.0 and JSON5 rehab

β€’ jpt can query JSON using both JSON Pointer, IETF RFC 6901 and JSONPath, a nascent draft RFC with powerful recursive search and filtering features.

β€’ jpt can manipulate JSON using JSONPatch, JSON Merge Patch, and some new merge modes not found anywhere else. See: merging JSON objects with jpt 1.0

β€’ shell scripters and programmers may find it useful that jpt has extensive string and number encoding capabilities. From classic encodings like hex, octal, and binary to more recent ones like URL/”percent” encoding and Unicode code points. See: jpt 1.0 text encoding fun

β€’ Need help finding visualizing JSON in new ways? jpt can output the structure of JSON in ways not seen, like JSONPath Object Literals which are simply the JSONPath, an equal sign, and the JSON value (e.g. $.ok="got it"). This simple and powerful declarative syntax can help find the exact path to a particular value and can also be used to quickly prototype a JSON object. See: jpt: see JSON differently

Try it out

There’s a lot to explore in jpt 1.0, download it from my GitHub with a macOS installer package in the Releases page. Check out my other jpt blog entries here at brunerd. As the IETF JSONPath standard takes shape I’ll be updating jpt to accomodate that standard. Until then, I hope this tool helps you in your JSON work, play, or research!


Decoding macOS automatic login details

In my previous post, Automating automatic login, we looked at how to create the /etc/kcpassword file used for automatic login by using only shell script and built-in command line tools. Why shell only? In preparation for the great scripting runtime deprecation yet to come, I say! Now it’s time to do the reverse for auto login. Let’s get those details back out! Who would need to do such a thing? Imagine a scenario where you the hapless Mac admin have inherited a bunch of Zoom Room Mac minis with auto-login enabled yet no one has documented the passwords used for them! If they are enrolled in Jamf there’s no need to guess what annoying l33t sp3@k password was used, let’s leverage our XOR’ing skills and knowledge of how kcpassword works to send those details back to Jamf.

To get the password back out of /etc/kcpassword we XOR the password again with the same cipher used to obfuscate it originally however but instead of padding it in multiples of 12, we will stop when a character is the same as the current cipher character. FYI when you XOR a value with itself the result is 00 but that’s an unnecessary operation, we can just compare the characters. VoilΓ‘, that’s it.

Here’s the gist of the kcpasswordDecode routine:

Now for something a bit more useful to those with Jamf or other management tools: getAutoLogin. It reports the auto login username, if set, and the decodes the /etc/kcpassword file, if present. Note that until macOS 12 Monterey /etc/kcpassword was not removed when Automatic Login was turned off in System Preferences! Here’s what getAutoLogin looks in the Jamf policy logs:

Plaintext passwords in your logs are probably not the best, but hey, how else you gonna figure out your dang Zoom Room passwords? After retrieving the credentials and storing somewhere more secure, like a password manager, make sure to Flush the policy logs! Thanks for reading, I hope this comes in handy or at the very least was informative and mildly entertaining. πŸ€“

Gist: kcpasswordDecode
Github: getAutoLogin

Automating automatic login for macOS

I recently had some Zoom Room Macs that needed some automation love and I thought I’d share how you can enable Automatic Login via script, perhaps you have several dozen and use Jamf or some other Mac management tool? Interested? Read on! Currently what’s out there are either standalone encoders only or part of larger packaging workflow to create a new user. The standalone encoders lacked some niceties like logging or account password verification and the package method added the required dependency of packaging if any changes were required. Above all, every script required Python, perl or Ruby, which are all on Apple’s hit list for removal in an upcoming OS release. For now macOS Monterey still has all of these runtimes but there will come a day when macOS won’t and will you really want to add 3rd party scripting runtimes and weaken your security by increasing attack surface when you can weaken your security using just shell? 😜 So for some fun, I re-implemented the /etc/kcpassword encoder in shell so it requires only awk, sed, and xxd, all out of the box installs. I also added some bells and whistles too.

Some of the features are:

  • If the username is empty, it will fully disable Automatic Login. Since turning it off via the System Preferences GUI does not remove the /etc/kcpassword file if it has been enabled (!)
  • Ensures the specified user exists
  • Verifies the password is valid for the specified user
  • Can handle blank passwords
  • Works on OS X 10.5+ including macOS 12 Monterey

For the Jamf admin the script is setAutomaticLogin.jamf.sh and for standalone usage get setAutomaticLogin.sh, both take a username and password, in that order and then enable Automatic Login if it all checks out. The difference with the Jamf script is that the first parameter is ${3} versus ${1} for the standalone version.

Also here’s a well commented Gist as a little show and tell for what the shell only version of the kcpassword encoder looks like. Enjoy!

Update: I meant to expound on how it didn’t seem that padding kcpassword to a length multiple of 12 was necessary, since it had been working fine on the versions I was testing with but then I tested on Catalina (thinking of this thread) and was proven wrong by Catalina. I then padded with 0x00 with HexFiend in a successful test but was reminded that bash can’t handle that character in strings, instead I padded with 0x01, which worked on some macOS versions but not others. Finally, I settled on doing on what macOS does somewhat, whereas it pads with the next cipher character then seemingly random data, I pad with the cipher characters for the length of the padding. This works for Catalina and other of macOS versions. πŸ˜…

Big Hat Tip to Pico over at MacAdmins Slack for pointing out this issue in pycreateuserpkg where it’s made clear that passwords with a lengths of 12 or multiples thereof, need another 12 characters of padding (or in some newer OSes at least one cipher character terminating the kcpassword data, thanks! πŸ‘