Category: bash

Jamf & FileVault 2: Tips & Tricks (and more)

brunerd

Raiders of the Lost Feature Requests

So there’s this old feature request at Jamf Nation (stop me if you’ve heard this one…) it’s almost 6 years old: Add ability to report on FV2 Recovery Keys (and/or access them via API) In fact, maybe you came here from there, watch out don’t loop! Continue!

The pain point is this: Keys are sent back to Jamf Pro (JSS) but then can only be gotten at manually/interactively through the web interface, not via API nor another method. For cases of mass migration to another JSS it sure would be nice to move those keys over rather than decrypt/re-encrypt. Well, I’ve got a few insights regarding this that I’d like to share that may help. ‘Cuz hey it’s 2020 and we’ve learned that hoarding is just silly.

Firstly, it should be pointed out that neither ye olde “Recovery Key Redirection” payload nor it’s replacement “Recovery Key Escrow” are needed to get keys to the JSS. There is another method and it’s what is used by the built-in “Filevault Encryption” policy payload to get the keys back to your JSS. Jamf references this method in this old script at their GitHub. I revamped the core bits a couple years ago in a (nearly 7 year old) feature request: Manually Edit FileVault 2 Recovery Key

Telling the JSS Your Secrets

The takeaway from that is to realize we have a way to explicitly send keys to the JSS by placing 2 XML files in the /Library/Application Support/JAMF/run folder: file_vault_2_id.xml and file_vault_2_recovery_key.xml. Also note, Jamf has updated the process for the better in the last two years: a jamf recon (or two) is no longer required to send the key and validate it, instead JamfDaemon will send it immediately when both the files are detected. Which is nice, but it’s the subsequent recon validations where we have an opportunity to get grabby.

Cold Lamping, Hard Linking

So here’s the fun part: When recon occurs there’s lots of file traffic in /Library/Application Support/JAMF/tmp all sorts of transient scripts hit this folder. What we can do is make hard links to these files as they come in so when the link is removed in tmp another exists elsewhere and the file remains (just in our new location). EAGrabber.sh does exactly that (and a little bit more)

EAGrabber.sh can be easily modified to narrow it’s focus to the FileVault 2 key only, deleting the rest. What you do with the key is up to you: Send it somewhere else for safe keeping or keep it on device temporarily for a migration to another Jamf console. A script on the new JSS could then put that key on-disk into file_vault_2_recovery_key.xml file which will then import and validate, no decrypt/recrypt necessary. Hope this helps.

Cuidado ¡ Achtung ! Alert

Jamf admins take note: Do you have hard coded passwords in your extension attributes or scripts? If so, then all your scripts are belong to us. Now, go read Obfuscation vs. Encryption from Richard Purves. Read it? OK, now consider what happens if you were to add a routine to capture the output of ps aww along with a hard-linking loop like in EAGraber.sh. If you are passing API credentials from policies via parameter, then ps can capture those parameters and even if you try and obscure them, if we’ve captured the script we can de-obfuscate them. This is a good reason to be really careful with what your API accounts can do. If you have an API account with Computer record Read rights that gets passed into a script via policy and you use LAPS, then captured API credentials could be used to harvest LAPS passwords via API. Keep this in mind and we’ll see if any meaningful changes will occur in recon and/or the script running process in the future (if you open a ticket you can reference PI-006270 regarding API credentials in the process list). In the meantime make API actions as short lived as possible and cross your fingers that only you, good and noble #MacAdmins read this blog. 🤞

jpt: jamf examples pt. 2

brunerd

Over in the MacAdmins’ #bash channel I saw a I question regarding how to get the Sharing states of Bluetooth devices from system_profiler. The most succinct answer was to awk out the values:

system_profiler SPBluetoothDataType 2> /dev/null | awk '/State: / {print $2}'
Disabled
Disabled
Disabled

If you are using this for a Jamf Extension Attribute, I suppose it’ll do if you never want to allow any of them to be Enabled, but what if Internet Sharing was OK but not File Sharing? How would you match your Smart Group to multiple lines of unlabeled values? How would you match the first two but not the last two… and what if there was another USB Bluetooth device, that would add extra rows. Hmmm…

The answer for me, outputting the service name and the state on the same line. Since there isn’t a consistent line count from State: going back the service name, using something like grep -B n to include n lines of preceding data isn’t going to work.

      Services:
          Bluetooth File Transfer:
              Folder other devices can browse: ~/Public
              When receiving items: Accept all without warning
              State: Disabled
          Bluetooth File Exchange:
              Folder for accepted items: ~/Downloads
              When other items are accepted: Save to location
              When receiving items: Accept all without warning
              State: Disabled
          Bluetooth Internet Sharing:
              State: Disabled

So you know what I say the answer to that is? That’s right, jpt the JSON Power Tool! It can parse the -json output from system_profiler in a more structured way and it allows for the discovery of as many applicable Bluetooth devices might be on the system.

Here’s a sample run with Internet Sharing turned On as well as Bluetooth Sharing turned On

file_browsing: disabled
object_push: enabled
internet_sharing: enabled

File Browsing is set to “Never Allow” but File Receiving is in the affirmative (Accept and Open, Accept and Save, or Ask). The addition of labels gives us the ability to create a Smart Group to match specific services like “file_browing: enabled” or any other combination thereof (perhaps internet_sharing should always be enabled, who am I to say what your requirements are!).

About the jpt

The JSON Power Tool (jpt) is a parser/manipulator for JSON documents written in Javascript and shell and can run standalone or embedded in your scripts bash or zsh and all the way back to OS X 10.4 Tiger! Check it out at: https://github.com/brunerd/jpt

secret origins: the jpt

brunerd

On building a JSON tool for macOS without using Python, perl, or Ruby.

In my work as a Macintosh engineer and administrator I’ve noticed macOS has lacked a bundled tool for working with JSON at the command line. Where XML has its xpath, if your shell script needs some JSON chops, it’ll require an external binary like jq or something else scripted in Python, Ruby or perl using their JSON modules. The problem is, those runtimes have been slated for removal from a future macOS. So I took that as challenge to devise a method to query and modify JSON data within shell scripts, that didn’t use one of those deprecated scripting runtimes and didn’t require an external binary dependency either. Could I achieve robust and native JSON parsing on a Mac by simply “living off the land”?

Why not just re-install the runtimes when Apple deprecates them?

“Why limit yourself like this? Just re-install the runtimes and move on”, you may ask. Well, I’d like to think that limitations can inspire creativity but we should also consider there may be some other reasons why Apple is discontinuing the inclusion of those runtimes. Some may say, “Apple Silicon + macOS 11.0 is the perfect time for them to clean house”, to which I’d have to agree that’s a very good reason and likely a factor. Others could say they are looking to tighten the screws to keep out unsigned code: maybe, they do like to glue things shut! But really, I think it’s more akin to the web-plugins of yore like Java and Flash. Apple does not want to be the conduit for deploying 3rd party party runtimes which increase the attack surface of macOS. This seems like the most reasonable of the explanations. So, if you accept that Apple is attempting to reduce attack surface, why increase it by re-installing Python, Ruby, or perl, just so a transient script (like a Jamf Extension Attribute) can parse a JSON file? My answer to that, is you don’t. You play the hand you’re dealt. Game on!

Looking for truffles (in a very small back yard)

Despite having another project (shui) that can output and invoke Applescript from within a shell scripts for generating user interfaces, I definitely knew that Applescript was not the way to go. Apple however, added to the languages Open Scripting Architecture (OSA) supports back in 2014 with OS X Yosemite (10.10), they added Javascript along with a bridge to the Cocoa Objective-C classes and they called it JXA: Javascript for Automation. This seemed like a promising place, so I started playing with osascript and figured out how to load files and read /dev/stdin using JXA, and while looking for an answer for garbled input from stdin I came upon a Japanese blog that mentioned jsc the JavaScriptCore binary which resides in the /System/Library/Frameworks/ JavaScriptCore.framework. Arigato! Pay dirt! 🤑 jsc does exactly what we need it to do: It can interpret Javascript passed as an argument, can access the filesystem and read from /dev/stdin, and best of all is in non-Private System level Framework that exists all the way back to OS X 10.4! Just the kind of foundation on which to build the tool.

Homesteading jsc

The existence of jsc goes back all the way to OS X Tiger and it’s functionality has evolved over the years. In order to have a consistent experience in jpt from macOS 10.4 – 11.x+ a few polyfills had to be employed for missing functions, along with a few other workarounds regarding file loading, printing and exit codes (or lack thereof). Once those were addressed the jsc proved to be a highly optimized Javascript environment that’s blazingly fast. It spans 13 macOS releases and is even present in many Linux distros out-of-the-box (Ubuntu and CentOS) and can even be run on Windows when the Linux Subsytem is installed.

With the host environment sorted, I began working with the original JSONPath code by Stefan Goessner as the query language. I didn’t know about JSON Pointer yet so this strange beast was all I knew! It worked out really. I went full throttle into developing the “swiss army knife” of JSON tools. I really leaned into the Second System Effect, as described in the Mythical Man-month, it’s when you put every doodad, gizmo and doo-hicky in your 2nd product (my first simple JSON pretty-printer built in JS on jsc). Eventually though, after I reached a feature plateau, I came back around to the address the quirks of the original JSONPath code. I ended up rewriting signifigant chunks of JSONPath and released it as it’s own project: brunerd JSONPath. But I digress, let’s get back to the JSON Power Tool.

jpt: powers and abilities

At it’s most basic, jpt will format or “pretty print” any JSON given. jpt can also handle data retrieval from JSON document using either JSONPath or JSON Pointer syntax. JSONPath, while not a standard, is a highly expressive query language akin to XPath for XML, with poweful features like recursive search, filter expressions, slices, and unions. JSON Pointer on the other hand is narrower in focus, succint and easily expressed, and standardized but it does not offer any of the interogative features that I feel make JSONPath so intriguing. Finally, jpt can also modify values in a JSON document using standardized JSON Patch operations like: add, remove, replace, copy, move, and test, as well as the also standardized JSON Merge Patch operation. Altogether, the jpt can format, retrieve and alter JSON documents using only a bit of outer shell script plus a lot more Javascript on any macOS since 10.4! 😅

Where can I get the jpt?

Stop by the project’s GitHub page at: https://github.com/brunerd/jpt

There you will find the full source and also a minified version of the jpt for inclusion within your shell scripts. Since it’s never compiled you can always peer inside and learn from it, customize it, modify it, or just tinker around with it (usually the best teacher).

Future Plans

There will undoubtedly be continued work on the jpt. Surely there are less than optimal routines, un-idiomatic idioms, edge cases not found, and features yet to be realized. But as far as the core functionality goes though, it’s fairly feature complete in my opinion. Considering that one of my top 10 StrengthsFinder qualities is “Maximizer”, the odds are pretty good, I’ll keep honing the jpt‘s utility, size (smaller), and sharing more articles with examples on the kinds of queries and data alteration operations the jpt can so perform. Stay tuned!

Track and Tackle com.apple.macl

brunerd

UPDATE: As of Big Sur (macOS 11) the com.apple.macl extended attribute can be removed using: xattr -rd com.apple.macl <file/folder>

Starting in macOS Catalina, an extended attribute (XA) named com.apple.macl is being added to the files and folder you work with. What does it do? How does it work? When does it get added? Where is the documentation?

All these are good questions, but there’s no official documentation. A good start is this assemblage of articles, here’s the highlights from that page and a few more…

Apple talked about this vaguely in their WWDC 2019 session 701, “Advances in macOS Security“, the Files and Folders fun starts at 21:21 but there’s nothing really about the implementation

Tom Bridge talked about com.apple.macl to the Penn State Macadmins back in August before Catalina’s release. He’s one of the first to notice this XA (and talk about it). Although, he says that curl will add this XA to downloads, it thankfully does not. Most likely it happened when he dragged it to Terminal or performed some other action on it.

Howard Oakley of the prolific Eclectic Light Company blog kicks the tires a bit more and finds some interesting quirks, as he always does.

Jeff Johnson though, really hits the nail on the head in illustrating what’s going on when you simply drag and drop a file or folder into Terminal and com.apple.macl is appended. He dropped this, December 18th and well… the holidays were around the corner and we all had better stuff to do!

It’s 2020 now and time to come back around to this. A very helpful post in the Apple Dev forums from an Apple employee, Quinn “the Eskimo”, decodes the data structure a bit more as well as what some of the conditions for having this XA appended are:

When a user selects a “protected” file or folder in an NSOpenPanel in a non-sandboxed app on Catalina, consent is inferred and the app can access it.
…
It seems that 01 00 is a header of some form and … is a UUID associated with my test app.  I dug into how that UUID is set up and, well, it’s complex, and more of an implementation detail than I care to go into here on DevForums.

https://forums.developer.apple.com/thread/124121

Well, I would have loved the complex implementation detail! But I’ll tell you this: The app UUID in com.apple.macl is unique to EVERY computer. Jeff Johnson was right when he said: “The macl is effectively untraceable“.

However you can begin to see the shape of things when you write a tool to output the UUID in CSV like I have. Behold maclTrack.command

Run it in Terminal, give it files or folders as arguments, it’ll report on them, you can even specify a maximum depth for folders (-d) and silence reporting on items lacking com.apple.macl (-s), pipe it into tee to see the output and write it to a file.

This poor file got clobbered by multiple app UUIDs

Something interesting I noticed from using this tool is seeing the “Header” as Quinn called it, differ from 0100 to 0200. I believe 0200 is for drag and drop operations vs. a regular save which results in 0100.

Again, the app UUIDs you see on the files on your Mac will never be seen on another Mac ever – why? Because UUIDs are meant to be truly unique. Have you read the man page for uuidgen? They ain’t eff’in around!

 The uuidgen command generates a Universally Unique IDentifier (UUID), a 128-bit value guaranteed to be unique over both space and time.

Apple man page for uuidgen

So what’s the point of having a long lived XA that you can’t get rid of and that’s only useful on your local Mac? If it’s a permissions granting XA (the opposite of com.apple.quarantine which imposes restrictions) then why not let the user remove it? In some ways it’s a cross computer file tracking mechanism, albeit “anonymized”. Perhaps allowing it’s removal though (aka writing “nothing” to the XA) also allows for something to be written? This is where I could get in trouble speculating, it’s just a guess.

Perhaps you’ll forgive me that, since I have found a way to clear com.apple.macl without disabling SIP! Zip it. No, not the one spawned Finder’s “Compress” menu item, I mean /usr/bin/zip! I noticed command line zip would obliterate XAs during a project many years ago and it still does my friends! So behold maclTackle.command

This script has considerably less engineering to it because it’s a PoC and I’m really don’t want to take responsibility for something going awry should you try to clear com.apple.macl on your entire Desktop folder (no I haven’t tested that). But it will strip the XAs off a file that’s for sure! NOTE: USE A NON-IMPORTANT FILE with this script (make a duplicate). The script will overwrite the original (don’t like the behavior? It’s a script: Comment it out!)

🎶 Gonna wash that macl right outta my file… 🎶

Apple may come along and close this loophole now that it’s been pointed out or perhaps they won’t since they’d have to publish the modfication? Perhaps they’ll continue to purge “undesirable” binaries from macOS? They say they will stop shipping Python, perl and ruby in macOS some day and you know what xattr is written in? That’s right Python! ¯\_(ツ)_/¯

Alright, come and get it while the gettin’s good! Thanks for reading!

macOS shell games: long live bash

brunerd

TL;DR – Bash ain’t goin’ nowhere on Mac, both version-wise and in terms of its presence. Looking at the longevity of other shells on the system, it will likely be around for a good while longer.

There’s been a lot of hand wringing and angst online about bash and zsh becoming the new default shell. Some folks feel Apple is signaling deprecation and removal and have the crushing feeling they must convert all their bash script to zsh. I think that’s a bit unnecessary.

True, the default shell is changing from bash to zsh, as Apple notes here. This is indeed a Good Thing™ as zsh shell has been one of the most frequently updated shells on macOS. Bash, on the other hand, has been stuck at varying versions of 3.2 for 12 years now! On the plus side, sysadmins have “enjoyed” predictable and stable behavior from bash during this time. Sure, you’d love new features but when you are scripting for the enterprise, across multiple OS versions, this is just the sort of thing you want: boringness and dependability.

As far as deprecations go, the only thing Apple has signaled as being deprecated eventually are scripting language runtimes (not shells):

Scripting language runtimes such as Python, Ruby, and Perl are included in macOS for compatibility with legacy software. Future versions of macOS won’t include scripting language runtimes by default, and might require you to install additional packages. If your software depends on scripting languages, it’s recommended that you bundle the runtime within the app. (49764202)

Use of Python 2.7 isn’t recommended as this version is included in macOS for compatibility with legacy software. Future versions of macOS won’t include Python 2.7. Instead, it’s recommended that you run python3 from within Terminal. (51097165)

mac OS Catalina 10.15 Release Notes

I’ve done some digging and culled the shell versions from OS X 10.0 to macOS 10.15, along with their respective release dates. I think it shows that shells, no matter how old and crusty, tend to be long lived and not soon removed on macOS.

Here’s a quick way to check your shell versions (except for dash):

macOSzshbash/shcsh/tcshkshdash
10.03.0.8 (2000-05-16)3.0.8 (zsh, no bash)6.08.00 (1998-10-02)––
10.13.0.8 (2000-05-16)3.0.8 (zsh, no bash)6.10.00 (2000-11-19)––
10.24.0.4 (2001-10-26)2.05b.0 (2002-07-17)6.10.00 (2000-11-19)––
10.34.1.1 (2003-06-19)2.05b.0 (2002-07-17)6.12.00 (2002-07-23)––
10.44.2.3 (2005-03-00)2.05b.0 (2002-07-17)6.12.00 (2002-07-23)M p (1993-12-28)–
10.54.3.4 (2007-04-19)3.2.17 (2007-05-01)6.14.00 (2005-03-23)M s+ (1993-12-28)–
10.64.3.4 (2008-11-03)3.2.48 (2008-11-18)6.15.00 (2007-03-03)M s+ (1993-12-28)–
10.74.3.11 (2010-12-20)3.2.48 (2008-11-18)6.17.00 (2009-07-10)M s+ (1993-12-28)–
10.84.3.11 (2010-12-20)3.2.48 (2008-11-18)6.17.00 (2009-07-10)JM 93u (2011-02-08)–
10.95.0.2 (2012-12-12)3.2.51 (2010-03-17)6.17.00 (2009-07-10)JM 93u (2011-02-08)–
10.105.0.5 (2014-01-06)3.2.57 (2014-11-07)6.17.00 (2009-07-10)AJM 93u+ (2012-08-01)–
10.115.0.8 (2015-05-31)3.2.57 (2014-11-07)6.18.01 (2012-02-14)AJM 93u+ (2012-08-01)–
10.125.2 (2015-12-02)3.2.57 (2014-11-07)6.18.01 (2012-02-14)AJM 93u+ (2012-08-01)–
10.135.3 (2016-12-12)3.2.57 (2014-11-07)6.18.01 (2012-02-14)AJM 93u+ (2012-08-01)–
10.145.3 (2016-12-12)3.2.57 (2014-11-07)6.18.01 (2012-02-14)AJM 93u+ (2012-08-01)–
10.155.7.1 (2019-02-03)3.2.57 (2014-11-07)6.21.00 (2019-05-08)AJM 93u+ (2012-08-01)dash-9 (1993)

As you can see, zsh has has been updated with almost every new release of macOS. Bash really hit a wall with 3.2 and as many have noted, it was v4’s change in licensing to GPLv3 that caused this (sh is really bash in sh compatibility mode so the versions are intertwined). csh/tcsh has the same duality thing going on and took a notably giant 7 year leap in 10.15 to a version from 2019. ksh has remained at the same version just as long as bash yet I don’t think anyone is fretting that ksh will be deprecated or removed. Finally, dash is just a weirdo that apparently disdains versioning! I used a combination of what /bin/dash and man dash to get some sort of crude answer.

So there you go: In my opinion, all signs point to bash being yet another shell on macOS for some time. Removing bash from macOS would break a lot of stuff and while that reason alone hasn’t stopped Apple before, I think they will let sleeping dogs lie. Go ahead and learn zsh, master it, customize it, or make it “sexy” but take the rumors of its demise on macOS with a grain of salt and dose of skepticism.

Determining the current console user in macOS

brunerd

There’s a few methods floating out on the web, but I’d just like to point out a quick and reliable way to get the current console user on macOS using stat. This works great with Fast User Switching and if the mac is at the login screen it will return root as the user

consoleUser=$(stat -f %Su /dev/console)

There you go! Now go forth and script.

Also, if you are using backticks ` to capture output (a.k.a. command substitution) then consider using the more modern and nest-able $(…) method, here’s a good reason why: Why is $(…) preferred over `…` (backticks)?

Nothing (bash style)

brunerd

The brunerd blog has been a whole lot of nothing lately, so let’s commemorate and celebrate with an entry featuring the premiere of the bash tag and category… hmm, which one is more useful? We’ll find out. Now as for nothing, there’s a legit command, special built-in to do nothing in bash!

man page for bash command :

: [arguments]

No effect; the command does nothing beyond expanding arguments and performing any specified redirections.  A zero exit code is returned.

 

Isn’t that lovely? When you need to do nothing, you can call :
Perhaps you are writing a script and aren’t sure what you’ll do for that if statement —

if [ "${this}" == "that" ]; then
#actually not sure what to do here, let's do nothing
:
else
echo "This else, I'll do"
fi

I made a gist here and I think I’ll keep making more. I’ll feature them here with the bash tag, if only as an auxillary backup of my brain and to add to the collective patchwork quilt of code we all use to get our ideas off the ground, when our reach is just a hare shy of grasp and we need a little help.

Also in the gist was a snippet of a novel use by kbeck at Extensis in his UTC removal script to use : to redirect a here document into itself and server as a comment block that doesn’t need # on every line to be treated as a comment —

: <<COMMENTBLOCK

[ ] 2016-0324: enclose me; love me. i am a block of comments

COMMENTBLOCK

Spiffy!