Tag: jpt

Determining eligible macOS versions via script

brunerd

Every year Mac admins wonder which Macs will make the cut for the new MacOS. While it’s no mystery which models those are, if you’ve got Jamf you’ll be wondering how to best scope to those Macs so you can perhaps offer the upgrade in Self Service or alert the user to request a new Mac! One way to scope a Smart Group is with the Model Identifier criteria and the regex operator, like this one (I even chipped in!). It doesn’t require an inventory and results are near instant. Before I was any good at regex though, I took another route and made an Extension Attribute macOSCompatibility.sh, where the Mac reports back to Jamf. (It also has a CSV output mode for nerdy fun!) Both methods however require manual upkeep and are now somewhat complicated by Apple’s new use of the very generic Macxx,xx model identifier which doesn’t seem to follow the usual model name and number scheme of major version and minor form factor variants (on purpose me-thinks!). Let’s look at some new methods that don’t require future upkeep.

Using softwareupdate –list-full-installers

macOS Big Sur (11) introduced a new command softwareupdate --list-full-installers which shows all eligible installers available for download by the Mac running that command. The funny thing about this is that even though it is a Big Sur or newer feature, if the hardware is old enough, like a 2017, it offer versions all the way back to 10.13 High Sierra! Monterey added build numbers to the output and it can be easily reduced to just versions with awk:

softwareupdate --list-full-installers | awk -F 'Version: |, Size' '/Title:/{print $2}'

This one-liner can be made into a simple function. I’ve added a uniq at the end for case where two differing builds have the same version, like 10.15.7. Here’s that function in a script with a little version check: getSupportedMacOSVersions_SWU.sh

#!/bin/sh
: <<-LICENSE_BLOCK
getSupportedMacOSVersions_SWU - Copyright (c) 2022 Joel Bruner
Licensed under the MIT License
LICENSE_BLOCK

function getSupportedMacOSVersions_SWU()( 
#getSupportedMacOSVersions_SWU - uses softwareupdate to determine compatible macOS versions for the Mac host that runs this
	if [ "$(sw_vers -productVersion | cut -d. -f1)" -lt 11 ]; then echo "Error: macOS 11+ required" >&2; return 1; fi
	#get full installers and strip out all other columns
	softwareupdate --list-full-installers 2>/dev/null | awk -F 'Version: |, Size' '/Title:/{print $2}' | uniq
)

getSupportedMacOSVersions_SWU
Output from Apple Silicon will never include 10.x versions

Software Update (SWU) Based Extension Attribute for Jamf

The possible inclusion of 10.x versions in the output complicates things a bit. In ye olden OS X days, the “minor version” (after the first period) acted more like the major versions of today! Still it can be done, and we will output any macOS 10.x versions, as if they are major versions like macOS 11, 12, 13, etc. Here’s getSupportedMacOSVersions-SWU-EA.sh

#!/bin/sh
: <<-LICENSE_BLOCK
getSupportedMacOSVersions-SWU-EA (Extension Attribute) - Copyright (c) 2022 Joel Bruner
Licensed under the MIT License
LICENSE_BLOCK

function getSupportedMacOSVersions_SWU()( 
#getSupportedMacOSVersions_SWU - uses softwareupdate to determine compatible macOS versions for the Mac host that runs this
	#[ "$(sw_vers -productVersion | cut -d. -f1)" -lt 11 ] && return 1
	if [ "$(sw_vers -productVersion | cut -d. -f1)" -lt 11 ]; then echo "Error: macOS 11+ required" >&2; return 1; fi
	#get full installers and strip out all other columns
	softwareupdate --list-full-installers 2>/dev/null | awk -F 'Version: |, Size' '/Title:/{print $2}'
)

#get our version
all_versions=$(getSupportedMacOSVersions_SWU)

#depending on the model (2020 and under) we might still get some 10.x versions 
if grep -q ^10 <<< "${all_versions}" ; then versions_10=$(awk -F. '/^10/{print $1"."$2}' <<< "${all_versions}")$'\n'; fi
#all the other major versions
version_others=$(awk -F. '/^1[^0]/{print $1}' <<< "${all_versions}")

#echo without double quotes to convert newlines to spaces
echo "<result>"$(sort -V <<< "${versions_10}${version_others}" | uniq)"</result>"
The venerable 2017 MacBook Pro has quite a span

Now if you wanted to make a Jamf Smart Group for those that could run macOS 13 you wouldn’t want to match 10.13 by accident. You could comment out the line in the script that matches versions beginning with ^10 or you could enclose everything in double quotes for the echo on the last line, so the newlines remained or you could use regex to match ([^.]|^)13 that is: not .13 or if the hardware is so new ^13 is at the very beginning of the string. As 10.x capable hardware fades away such regex sorcery shouldn’t be needed.

Using the Apple Software Lookup Service

“What’s the Apple Software Lookup Service?!”, you may be asking? I myself asked the same question! It’s a highly available JSON file that MDM servers can reference. If softwareupdate is acting up or hanging (and it’s been known to do so!), you have all you need in this JSON file to do a little sanity checking of softwareupdate too if you’d like. The URL is found in the Apple MDM Protocol Reference and it contains versions, models and their compatibility.

This method has far fewer patch and point versions than the softwareupdate method above and the OSes start with Big Sur (11). No 10.x versions are in the ASLS. There are two “sets” in ASLS, PublicAssetSets, which has only the newest release of each major version and AssetSets which has additional point releases (use the -a option for this one). plutil has quirky (IMO) rules for what it will and will not output as json but the raw output type can get around. It was introduced in Monterey and it can also be used to count array members, it’s goofy but manageable. Richard Purves has an article on that here. The code is generously commented, so I won’t expound upon it too much more, here’s getSupportedMacOSVersions_ASLS.sh

#!/bin/sh
: <<-LICENSE_BLOCK
getSupportedMacOSVersions_ASLS - Copyright (c) 2022 Joel Bruner
Licensed under the MIT License...
LICENSE_BLOCK

function getSupportedMacOSVersions_ASLS()( 
#getSupportMacOSVersions - uses Apple Software Lookup Service to determine compatible macOS versions for the Mac host that runs this
#  Options:
#  [-a] - to see "all" versions including prior point releases, otherwise only newest of each major version shown

	if [ "${1}" = "-a" ]; then
		setName="AssetSets"
	else
		setName="PublicAssetSets"
	fi

	#get Device ID for Apple Silicon or Board ID for Intel
	case "$(arch)" in
		"arm64")
			#NOTE: Output on ARM is Device ID (J314cAP) but on Intel output is Model ID (MacBookPro14,3)
			myID=$(ioreg -arc IOPlatformExpertDevice -d 1 | plutil -extract 0.IORegistryEntryName raw -o - -)
		;;
		"i386")
			#Intel only, Board ID (Mac-551B86E5744E2388)
			myID=$(ioreg -arc IOPlatformExpertDevice -d 1 | plutil -extract 0.board-id raw -o - - | base64 -D)
		;;
	esac	

	#get JSON data from "Apple Software Lookup Service" - https://developer.apple.com/business/documentation/MDM-Protocol-Reference.pdf
	JSONData=$(curl -s https://gdmf.apple.com/v2/pmv)

	#get macOS array count
	arrayCount=$(plutil -extract "${setName}.macOS" raw -o - /dev/stdin <<< "${JSONData}")

	#look for our device/board ID in each array member and add to list if found
	for ((i=0; i<arrayCount; i++)); do
		#if found by grep in JSON (this is sufficient)
		if grep -q \"${myID}\" <<< "$(plutil -extract "${setName}.macOS.${i}.SupportedDevices" json -o - /dev/stdin <<< "${JSONData}")"; then
			#add macOS version to the list
			supportedVersions+="${newline}$(plutil -extract "${setName}.macOS.${i}.ProductVersion" raw -o - /dev/stdin <<< "${JSONData}")"
			#only set for the next entry, so no trailing newlines
			newline=$'\n'
		fi
	done

	#echo out the results sorted in descending order (newest on top)
	sort -rV <<< "${supportedVersions}"
)

#pass possible "-a" argument
getSupportedMacOSVersions_ASLS "$@"
PublicAssetSets (top) vs. AssetSets (bottom)

The fact that this method requires Monterey for the plutil stuff didn’t agree with me, so I made a version that uses my JSON power tool (jpt) so it will work on all earlier OSes too. It’s a tad large (88k) but still runs quite fast: getSupportedMacOSVersions_ASLS-legacy.sh

Just as with the softwareupdate based function, the same can be done to reduce the output to only major versions and since it is v11 and up, a simple cut will do!

#major versions only, descending, line delimited
getSupportedMacOSVersions_ASLS | cut -d. -f1 | uniq

#major versions only ascending
echo $(getSupportedMacOSVersions_ASLS | cut -d. -f1 | sort -n | uniq)

ASLS Based Extension Attribute

The Apple Software Lookup Service (ASLS) JSON file itself doesn’t care what version of macOS a client is on, but the methods in plutil to work with JSON aren’t available until Monterey. So here’s the ASLS based Extension Attribute a couple ways: getSupportedMacOSVersions-ASLS-EA.sh and getSupportedMacOSVersions-ASLS-legacy-EA.sh both get the job done.

Same. Same.

Bonus Methods for Determining Board ID and Device ID

The ASLS method requires either the Board ID or the Device ID and in a way that worked across all macOS versions and hardware architectures. I’ve updated my gist for that (although gists are a worse junk drawer than a bunch of scripts in a repo if only because it’s hard to get an overall listing) and here’s a few callouts for what I came up with

#DeviceID - ARM, UNIVERSAL - uses xmllint --xpath
myDeviceID=$(ioreg -arc IOPlatformExpertDevice -d 1 | plutil -extract 0.IORegistryEntryName xml1 -o - - | xmllint --xpath '/plist/string/text()' - 2>/dev/null)
#DeviceID - ARM, macOS 12+ only, uses plutil raw output
myDeviceID=$(ioreg -arc IOPlatformExpertDevice -d 1 | plutil -extract 0.IORegistryEntryName raw -o - -)
#NOTE: Different output depending on platform! 
# ARM gets the Device ID - J314cAP
# Intel gets the Model ID - MacBookPro14,3

#Board ID - Intel ONLY, Mac-551B86E5744E2388
#Intel, UNIVERSAL - uses xmllint --xpath
myBoardID=$(ioreg -arc IOPlatformExpertDevice -d 1 | plutil -extract 0.board-id xml1 -o - - | xmllint --xpath '/plist/data/text()' - | base64 -D)
#Intel, macOS 12+ only - uses plutil raw output 
myBoardID=$(ioreg -arc IOPlatformExpertDevice -d 1 | plutil -extract 0.board-id raw -o - - | base64 -D)

Wrapping Up

This was all really an excuse to play around with the Apple Software Lookup Service JSON file, what can I say! And not just to plug jpt either! It was fun to use the new plutil raw type too (is fun the right word?) and “live off the land”. Be aware that the newest macOS version only appears once it’s publicly released, so keep that in mind when scoping. You can still keep scoping in Jamf via Model Identifier Regex or by my older extension attribute just keep in mind you’ll need to update them yearly. Whereas, these newer EAs based on either softwareupate (getSupportedMacOSVersions-SWU-EA.sh) or ASLS (getSupportedMacOSVersions-ASLS-EA.sh, getSupportedMacOSVersions-ASLS-legacy-EA.sh) should take care of themselves into the future.

Determining iCloud Drive and Desktop and Documents Sync Status in macOS

brunerd

I’m on a roll with iCloud stuff. In this post I’d like to show you how you can determine if either iCloud Drive is enabled along with the “Desktop and Documents Folders” sync feature. While you can use MDM to turn these off, perhaps you like to know who you’d affect first! Perhaps the folks in your Enterprise currently using these features are in the C-Suite? I’m sure they’d appreciate a heads up before all their iCloud docs get removed from their Macs (when MDM disallowance takes affect it is swift and unforgiving).

iCloud Drive Status

When it comes to using on-disk artifacts to figure out the state of macOS I like to use the analogy of reading tea leaves. Usually it’s pretty straightforward but every now and then there’s something inscrutable and you have to take your best guess. For example iCloud Drive status is stored in your home folder at ~/Library/Preferences/MobileMeAccounts.plist but yet the Accounts key is an array. The question is how and why you could even have more than one iCloud account signed-in?! Perhaps a reader will tell me when and how you would ever have more than one? For now it seems inexplicable.

Update: It seems quite obvious now but as many folks did point out, of course you can add another iCloud account to Internet Accounts and it can sync Mail, Contacts, Calendars, Reminders, and Notes, just not iCloud Drive. Only one iCloud Drive user per user on a Mac. While I do have another AppleID I only use it for Media and Purchases and none of the other iCloud services. The code below stays the same as it is looking at all array entries. Aside: Boy, do I wish I could merge or transfer my old iTunes purchasing/media Apple ID with my main iCloud Apple ID! <weakly shakes fist at faceless Apple bureaucracy that for some reason hasn't solved the problem of merging Apple IDs>

Regardless of that mystery jpt can use the JSONPath query language to get us an answer in iCloudDrive_func.sh and the minified iCloudDrive_func.min.sh. Below is an edited excerpt: 

#!/bin/bash
#Joel Bruner - iCloudDrive.func.sh - gets the iCloud Drive status for a console user

#############
# FUNCTIONS #
#############

function iCloudDrive()(

	#for brevity pretend we've pasted in the minified jpt function:
	#https://github.com/brunerd/jpt/blob/main/sources/jpt.min
	#for the full function see https://github.com/brunerd/macAdminTools/tree/main/Scripts

	consoleUser=$(stat -f %Su /dev/console)

	#if root grab the last console user
	if [ "${consoleUser}" = "root" ]; then
		consoleUser=$(/usr/bin/last -1 -t console | awk '{print $1}')
	fi

	userPref_json=$(sudo -u $consoleUser defaults export MobileMeAccounts - | plutil -convert json - -o -)

	#pref domain not found an empty object is returned
	if [ "${userPref_json}" = "{}" ]; then
		return 1
	else
		#returns the number paths that match
		matchingPathCount=$(jpt -r '$.Accounts[*].Services[?(@.Name == "MOBILE_DOCUMENTS" && @.Enabled == true)]' <<< "${userPref_json}" 2>/dev/null | wc -l | tr -d "[[:space:]]")
	
		if [ ${matchingPathCount:=0} -eq 0 ]; then
			return 1
		else
			return 0
		fi
	fi
)
########
# MAIN #
########

#example function usage, if leverages the return values
if iCloudDrive; then
	echo "iCloud Drive is ON"
	exit 0
else
	echo "iCloud Drive is OFF"
	exit 1
fi

The magic happens once we’ve gotten the JSON version of MobileMeAccount.plist I use jpt to see if there are any objects within the Accounts array with Services that have both a Name that matches MOBILE_DOCUMENTS and have an Enabled key that is set to true, the -r option on jpt tells it to output the the JSON Pointer path(s) the query matches. I could have used the -j option to output JSONPath(s) but either way a line is a line and that’s all we need. Altogether it looks like this: jpt -r '$.Accounts[*].Services[?(@.Name == "MOBILE_DOCUMENTS" && @.Enabled == true)]

Again because Accounts is an Array we have it look at all of them with $.Accounts[*]and in the off chance we get more than one we simply say if the number of matches is greater than zero then it’s on. This works very well in practice. This function could best be used as a JAMF Extension Attribute. I’ll leave that as a copy/paste exercise for the reader. Add it to your Jamf Pro EAs, let sit for 24-48 hours and check for results! ⏲ And while some of you might balk at a 73k Extensions Attribute, the execution time is on average a speedy .15s!

#!/bin/bash
#a pretend iCloudDrive Jamf EA 
#pretend we've pasted in the function iCloudDrive() above

if iCloudDrive; then
	result="ON"
else
	result="OFF"
fi

echo "<result>${result}</result>"

iCloud Drive “Desktop and Documents” status

Thankfully, slightly easier to determine yet devilishly subtle to discover, is determining the status of the “Desktop and Documents” sync feature of iCloud Drive. After searching in vain for plist artifacts, I discovered the clue is in the extended attributes of your Desktop (and/or Documents) folder! You can find the scripts at my GitHub here iCloudDriveDesktopSync_func.sh and the minified version iCloudDriveDesktopSync_func min.sh

#!/bin/bash
#Joel Bruner - iCloudDriveDesktopSync - gets the iCloud Drive Desktop and Document Sync Status for the console user

#############
# FUNCTIONS #
#############

#must be run as root
function iCloudDriveDesktopSync()(
	consoleUser=$(stat -f %Su /dev/console)

	#if root (loginwindow) grab the last console user
	if [ "${consoleUser}" = "root" ]; then
		consoleUser=$(/usr/bin/last -1 -t console | awk '{print $1}')
	fi

	#if this xattr exists then sync is turned on
	xattr_desktop=$(sudo -u $consoleUser /bin/sh -c 'xattr -p com.apple.icloud.desktop ~/Desktop 2>/dev/null')

	if [ -z "${xattr_desktop}" ]; then
		return 1
	else
		return 0
	fi
)

#example function usage, if leverages the return values
if iCloudDriveDesktopSync; then
	echo "iCloud Drive Desktop and Documents Sync is ON"
	exit 0
else
	echo "iCloud Drive Desktop and Documents Sync is OFF"
	exit 1
fi

The operation is pretty simple, it finds a console user or last user, then runs the xattr -p command as that user (anticipating this being run as root by Jamf) to see if the com.apple.icloud.desktop extended attribute exists on their ~/Desktop. In testing you’ll find if you toggle the “Desktop and Documents” checkbox in the iCloud Drive options, it will apply this to both of those folders almost immediately without fail. The function can be used in a Jamf Extension Attribute in the same way the iCloudDrive was above. Some assembly required. 💪

So, there you go! Another couple functions to read the stateful tea leaves of iCloud Drive settings. Very useful if you are about to disallow iCloud Drive and/or Desktop and Document sync via MDM but need to know who you are going to affect and let them know beforehand. Because I still stand by this sage advice: Don’t Be A Jerk. Thanks for reading you can find these script and more at my GitHub repo. Thanks for reading!

jpt + jamf uapi = backupJamf-scripts

brunerd

Jamf UAPI: JSON Only

I am a creature of habit, no doubt, however sometimes you must get out of your comfort zone. The Jamf Universal API (UAPI) is one such case, it is JSON only and not XM. Those tried and true xpath snippets will no longer work with JSON, in fact what tool do you use with JSON? macOS really doesn’t have a good built-in JSON tool and if your scripts are client side do you really want to have jq as a dependency? Good thing I wrote a JSON parser you can embed in your scripts this summer! In fact, when I finished writing my JSON power tool jpt, I needed to find some practical examples to demonstrate its utility. Looking at the UAPI it’s clear some parts are still a work-in-progress, however the endpoint for scripts is actually really good. It gives you everything in one go almost like a database query. That should made backing up scripts a breeze!

backupJamf-scripts. boom.

If you’ve used the Classic API from Jamf, it is a 1:1 ratio of scripts to API calls: 2 scripts? 2 API calls via curl. 200 scripts? 200 API calls via curl. The new Universal API reduces that down to 1 call to get everything (plus one call to get the token), it’s super fast and I love it. Check out backupJamf-scripts.command in my newly minted jamfTools repo on GitHub for a working demostration of both the Jamf UAPI and jpt’s in-script JSON handling. I hope you like it!

jpt + jamf uapi = scripts downloading scripts

jpt: jamf examples

brunerd

jpt has some practical applications for the Jamf admin

Sanitizing Jamf Reports

Let’s say you’ve exported an Advanced Search, it’s got some interesting data you’d like to share, however there is personal data in it. Rather than re-running the report, why not blank out or remove those fields?

Here’s a sample file: advancedcomputersearch-2-raw.json

This an excerpt from one of the computer record:

{
"name": "Deathquark",
"udid": "2ca8977b-05a1-4cf0-9e06-24c4aa8115bc",
"Managed": "Managed",
"id": 2,
"Computer_Name": "Deathquark",
"Last_Inventory_Update": "2020-11-04 22:01:09",
"Total_Number_of_Cores": "8",
"Username": "Professor Frink",
"FileVault_2_Status": "Encrypted",
"JSS_Computer_ID": "2",
"Number_of_Available_Updates": "1",
"Model_Identifier": "MacBookPro16,1",
"Operating_System": "Mac OS X 10.15.7",
"Model": "MacBook Pro (16-inch, 2019)",
"MAC_Address": "12:34:56:78:9A:BC",
"Serial_Number": "C02K2ND8CMF1",
"Email_Address": "frink@hoyvin-glavin.com",
"IP_Address": "10.0.1.42",
"FileVault_Status": "1/1",
"Processor_Type": "Intel Core i9",
"Processor_Speed_MHz": "2457",
"Total_RAM_MB": "65536"
}

Now let’s say the privacy standards for this place is GDPR on steroids and all personally identifiable information must be removed, including serials, IPs, UUIDs, almost everything (but you don’t want to run the report again because it took ages to get the output)!

Here’s what that command would look like: jpt -o replace -v '"REDACTED"' -p '$["advanced_computer_search"]["computers"][*]["name","udid","Username","Computer_Name","MAC_Address","Serial_Number","Email_Address","IP_Address"]' ./advancedcomputersearch-2-raw.json

Here’s what that same computer looks like in the resulting output (as well as all others in the document):

  {
    "name": "REDACTED",
    "udid": "REDACTED",
    "Managed": "Managed",
    "id": 2,
    "Computer_Name": "REDACTED",
    "Architecture_Type": "i386",
    "Make": "Apple",
    "Service_Pack": "",
    "Last_Inventory_Update": "2020-11-04 22:01:09",
    "Active_Directory_Status": "Not Bound",
    "Total_Number_of_Cores": "8",
    "Username": "REDACTED",
    "FileVault_2_Status": "Encrypted",
    "JSS_Computer_ID": "254",
    "Number_of_Available_Updates": "1",
    "Model_Identifier": "MacBookPro16,1",
    "Operating_System": "Mac OS X 10.15.7",
    "Model": "MacBook Pro (16-inch, 2019)",
    "MAC_Address": "REDACTED",
    "Serial_Number": "REDACTED",
    "Email_Address": "REDACTED",
    "IP_Address": "REDACTED",
    "FileVault_Status": "1/1",
    "Processor_Type": "Intel Core i9",
    "Processor_Speed_MHz": "2457",
    "Total_RAM_MB": "65536"
  }

What we did was use the -o replace operation with a -v <value> of the JSON string "REDACTED" to all the paths matched by the JSONPath union expression (the comma separated property names in brackets) of the -p option. JSON Pointer can only act on one value at a time, this is where JSONPath can save you time and really shines.

jpt is fast because WebKit’s JavascriptCore engine is fast, for instance there is a larger version of that search that has 15,999 computers, it took only 11 seconds to redact every record.

jpt -l $.advanced_computer_search.computers ./advancedcomputersearch.json
15999

time jpt -o replace -v '"REDACTED"' -p '$.advanced_computer_search.computers[*]["name","udid","Username","Computer_Name","MAC_Address","Serial_Number","Email_Address","IP_Address"]' ./advancedcomputersearch.json > /dev/null 

11.14s user 0.35s system 104% cpu 11.030 total

Put Smart Computer Groups on a diet

When you download Smart Groups via the API, you will also get an array of all the computers objects that match at that moment in time. If you just want to back up the logic or upload to another system, you don’t want all those computers in there.

Sample file: smartgroup-1-raw.json

{
  "computer_group": {
    "id": 12,
    "name": "All 10.15.x Macs",
    "is_smart": true,
    "site": {
      "id": -1,
      "name": "None"
    },
    "criteria": [
      {
        "name": "Operating System Version",
        "priority": 0,
        "and_or": "and",
        "search_type": "like",
        "value": "10.15",
        "opening_paren": false,
        "closing_paren": false
      }
    ],
    "computers": [
      {
        "id": 1,
        "name": "mac1",
        "mac_address": "12:34:56:78:9A:BC",
        "alt_mac_address": "12:34:56:78:9A:BD",
        "serial_number": "Z18D132XJTQD"
      },
      {
        "id": 2,
        "name": "mac2",
        "mac_address": "12:34:56:78:9A:BE",
        "alt_mac_address": "12:34:56:78:9A:BF",
        "serial_number": "Z39VM86X01MZ"
      }
    ]
  }
}

Lets’ remove those computers with jpt and the JSON Patch remove operartion:
jpt -o remove -p '$.computer_group.computers' ./smartgroup-1-raw.json

Since the target is a single property name, JSON Pointer can also be used:
jpt -o remove -p /computer_group/computers ./smartgroup-1-raw.json

{
  "computer_group": {
    "id": 12,
    "name": "All 10.15.x Macs",
    "is_smart": true,
    "site": {
      "id": -1,
      "name": "None"
    },
    "criteria": [
      {
        "name": "Operating System Version",
        "priority": 0,
        "and_or": "and",
        "search_type": "like",
        "value": "10.15",
        "opening_paren": false,
        "closing_paren": false
      }
    ]
  }
}

Further Uses

Using only JSON Patch replace and remove operations this JSON could have its id removed to prep it for an API POST on another JSS to create a new group or the name and value could be modified in a looping script to create multiple JSON files for every macOS version. The jpt is flexible enough to handle most anything you throw at it, whether you are using the standalone version or have embedded it in your scripts.

Stop by the jpt GitHub page to get your copy

secret origins: the jpt

brunerd

On building a JSON tool for macOS without using Python, perl, or Ruby.

In my work as a Macintosh engineer and administrator I’ve noticed macOS has lacked a bundled tool for working with JSON at the command line. Where XML has its xpath, if your shell script needs some JSON chops, it’ll require an external binary like jq or something else scripted in Python, Ruby or perl using their JSON modules. The problem is, those runtimes have been slated for removal from a future macOS. So I took that as challenge to devise a method to query and modify JSON data within shell scripts, that didn’t use one of those deprecated scripting runtimes and didn’t require an external binary dependency either. Could I achieve robust and native JSON parsing on a Mac by simply “living off the land”?

Why not just re-install the runtimes when Apple deprecates them?

“Why limit yourself like this? Just re-install the runtimes and move on”, you may ask. Well, I’d like to think that limitations can inspire creativity but we should also consider there may be some other reasons why Apple is discontinuing the inclusion of those runtimes. Some may say, “Apple Silicon + macOS 11.0 is the perfect time for them to clean house”, to which I’d have to agree that’s a very good reason and likely a factor. Others could say they are looking to tighten the screws to keep out unsigned code: maybe, they do like to glue things shut! But really, I think it’s more akin to the web-plugins of yore like Java and Flash. Apple does not want to be the conduit for deploying 3rd party party runtimes which increase the attack surface of macOS. This seems like the most reasonable of the explanations. So, if you accept that Apple is attempting to reduce attack surface, why increase it by re-installing Python, Ruby, or perl, just so a transient script (like a Jamf Extension Attribute) can parse a JSON file? My answer to that, is you don’t. You play the hand you’re dealt. Game on!

Looking for truffles (in a very small back yard)

Despite having another project (shui) that can output and invoke Applescript from within a shell scripts for generating user interfaces, I definitely knew that Applescript was not the way to go. Apple however, added to the languages Open Scripting Architecture (OSA) supports back in 2014 with OS X Yosemite (10.10), they added Javascript along with a bridge to the Cocoa Objective-C classes and they called it JXA: Javascript for Automation. This seemed like a promising place, so I started playing with osascript and figured out how to load files and read /dev/stdin using JXA, and while looking for an answer for garbled input from stdin I came upon a Japanese blog that mentioned jsc the JavaScriptCore binary which resides in the /System/Library/Frameworks/ JavaScriptCore.framework. Arigato! Pay dirt! 🤑 jsc does exactly what we need it to do: It can interpret Javascript passed as an argument, can access the filesystem and read from /dev/stdin, and best of all is in non-Private System level Framework that exists all the way back to OS X 10.4! Just the kind of foundation on which to build the tool.

Homesteading jsc

The existence of jsc goes back all the way to OS X Tiger and it’s functionality has evolved over the years. In order to have a consistent experience in jpt from macOS 10.4 – 11.x+ a few polyfills had to be employed for missing functions, along with a few other workarounds regarding file loading, printing and exit codes (or lack thereof). Once those were addressed the jsc proved to be a highly optimized Javascript environment that’s blazingly fast. It spans 13 macOS releases and is even present in many Linux distros out-of-the-box (Ubuntu and CentOS) and can even be run on Windows when the Linux Subsytem is installed.

With the host environment sorted, I began working with the original JSONPath code by Stefan Goessner as the query language. I didn’t know about JSON Pointer yet so this strange beast was all I knew! It worked out really. I went full throttle into developing the “swiss army knife” of JSON tools. I really leaned into the Second System Effect, as described in the Mythical Man-month, it’s when you put every doodad, gizmo and doo-hicky in your 2nd product (my first simple JSON pretty-printer built in JS on jsc). Eventually though, after I reached a feature plateau, I came back around to the address the quirks of the original JSONPath code. I ended up rewriting signifigant chunks of JSONPath and released it as it’s own project: brunerd JSONPath. But I digress, let’s get back to the JSON Power Tool.

jpt: powers and abilities

At it’s most basic, jpt will format or “pretty print” any JSON given. jpt can also handle data retrieval from JSON document using either JSONPath or JSON Pointer syntax. JSONPath, while not a standard, is a highly expressive query language akin to XPath for XML, with poweful features like recursive search, filter expressions, slices, and unions. JSON Pointer on the other hand is narrower in focus, succint and easily expressed, and standardized but it does not offer any of the interogative features that I feel make JSONPath so intriguing. Finally, jpt can also modify values in a JSON document using standardized JSON Patch operations like: add, remove, replace, copy, move, and test, as well as the also standardized JSON Merge Patch operation. Altogether, the jpt can format, retrieve and alter JSON documents using only a bit of outer shell script plus a lot more Javascript on any macOS since 10.4! 😅

Where can I get the jpt?

Stop by the project’s GitHub page at: https://github.com/brunerd/jpt

There you will find the full source and also a minified version of the jpt for inclusion within your shell scripts. Since it’s never compiled you can always peer inside and learn from it, customize it, modify it, or just tinker around with it (usually the best teacher).

Future Plans

There will undoubtedly be continued work on the jpt. Surely there are less than optimal routines, un-idiomatic idioms, edge cases not found, and features yet to be realized. But as far as the core functionality goes though, it’s fairly feature complete in my opinion. Considering that one of my top 10 StrengthsFinder qualities is “Maximizer”, the odds are pretty good, I’ll keep honing the jpt‘s utility, size (smaller), and sharing more articles with examples on the kinds of queries and data alteration operations the jpt can so perform. Stay tuned!

New Projects: jpt and shui, Now Available

brunerd

Between March and October 2020 I had some great ideas for command line Mac utilities the MacAdmin could apprecite and I had the time to devote to their realization. I’m excited to present these two open source projects, available on GitHub: jpt and shui. I hope they can add richness to your shell scripts’ presentation and capabilities without requiring additional external dependancies.

jpt – the “JSON Power Tool” is a Javascript and shell script polyglot that leverages jsc, the JavascriptCore binary that is standard on every Mac since 10.4 and since the jpt is purposefully written in ES5 to maintain maximum compatibility, why yes, this tool does run on both PPC and Intel Macs all the way back to OS X Tiger and then all the way forward to the latest 11.0 macOS Big Sur! Many Linux distros like CentOS and Ubuntu come with jsc pre-installed also, even Windows with the Linux Subsystem installed can run jsc and therefore can run the jpt!

What you can do with the jpt? Query JSON documents using either the simple yet expressive JSONPath syntax or the singular and precise JSON Pointer (RFC6901) syntax. The output mode is JSON but additional creative output modes can render JSONPaths, JSON Pointer paths, or even just the property names with their “constructor” types (try -KC with -J or -R) Textual output can be encoded in a variety of formats (hex/octal/URI encoding, Unicode code points, etc…), data can be modified using both JSON Patch (RFC6902) operations (add, replace, remove, copy, move, test) and also JSON Merge Patch (RFC7386) operations. JSON can be worked with in new ways, try -L for “JSONPath Object Literal” output to see what I mean. Or you simply feed jsc a file to pretty-print (stringify) to /dev/stdout. I’ll be writing more about this one for sure.
Github project page: jpt
Tagged blog posts: scripting/jpt

shui – first-class Applescript dialog boxes in your shell scripts without needing to remember esoteric Applescript phrasings! If you think it’s odd for code to have possessive nouns and are more comfortable in shell, you’re not alone. shui can be embedded in either bash or zsh scripts but it can also output Applescript if you really want to know how the sausage is made or want to embed in your script without shui. Hopefully shui will let you forget those awkward Applescript phrasing and focus on your shell script’s features and functionality. It uses osascript to execute the Applescript and launchctl to invoke osascript in the correct user context so user keyboard layouts are respected (vs. root runs). Check out the project page for demo videos and then give shui a try.
Project page: shui
Tagged blog posts: scripting/shui