python

Script Update - Mac MRU Parser - Spotlight Shortcuts & BLOB Parsing!

in , , ,

Get the script here!

Added in Spotlight Shortcuts

I've updated my macMRU.py script to parse the Spotlight Shortcuts plist file that I consider to be very MRU-like. This plist file contains what the user typed into the Spotlight search window, what they clicked on, and when this all occurred. You can find artifacts of application usage, documents, emails, photos, etc.

Bookmark and Alias BLOB Parsing

The second part of the update (and probably the most requested!) was to add in arguments to allow the analyst to see the parse out the Bookmark and Alias BLOB data. Previously the script only had an option to show the BLOB hexdump. Using mac_alias [download here], the script can now parse these BLOBs. This comes with a caveat, this script relies on the documented structures by others (with a few minor changes by my own research). These have not been officially documented by Apple therefore YMMV. Many of us are still researching these, hopefully this script will help the process! This update provides three new arguments:

  • --blob_hex = This is the same as what --blob was before. Just a hexdump of the BLOB data.
  • --blob_parse_raw = This shows the raw output from mac_alias, not exactly the easiest to read but good for debugging.
  • --blob_parse_human = This shows the pieces of the BLOBs in a (mostly) human-friendly (example below).

I want to say thank you to a few folks that allowed these additions to be made:

  • Alastair Houghton & Wim Lewis - For mac_alias and the reversing the Alias/Bookmark structures. I took a shot at it years ago and gave up, really appreciate someone else's effort!
  • Phill Moore of thisweekin4n6.com - You'll notice some support for legacy keys in the com.apple.recentitems.plist have also been added in this update thanks to Phil and his oddly old MRUs files! :) I also appreciate his testing of this tool on Windows also. (Sorry Phil, didn't get a chance to test it on Windows yet!) 
  • Shout out to Patrick Olsen for his work on Alias V3 structures. Unfortunately I did not get to use his research - turns out V3 Alias BLOBs do not get used in any of the MRUs that I'm parsing in this script (they are all V2)! Maybe in the future!

Update to MacMRU Parser - Now with Microsoft Office Support!

in , ,

I've updated my MacMRU parser script, located here: https://github.com/mac4n6/macMRU-Parser.

This update includes support for 'Most Recently Used' artifacts for Microsoft Office for Mac 2011 and 2016. I've also added a bit more error checking, so please let me know if you come across a plist that doesn't work or does not contain certain expected keys. It happens, I've run across a few myself - I can't explain why now, but I hope to look into it further.

While researching for this update I noticed something interesting. Microsoft Office used a binary plist format for the 2011 version of the software, but "downgraded" to an XML-based plist format in the 2016 version. Most of the time I see plists headed in the XML->Binary direction.

Got a MacMRU file that I don't parse? Please let me know!

I hope you find this script useful!

New Script! - MacMRU (Most Recently Used) Plist Parser

in , ,

I have been studying the new SFL-based MRU plist files found in OS X 10.11. They make analysis hard because they are binary plist files using the NSKeyedArchiver format – see here for my manual analysis of these files. I’ve also included the ‘older’ format plist files used in OS X 10.10 and older.

In order to analyze them better (and student requests) I wrote a Python script to output the contents of these files in an easier to read format. Nothing fancy, just text printed to standard output.

Get the script here from my Github page. I hope you find the script useful! 

The script is meant to be run on a directory; this can be a directory of extracted plist files from an image, a directory on your own system (ie: ~/Library), or from a mounted image (ie: /Volumes/mounted_image_file/Users/<username>/), you get the idea.

This script parses the following plist files:

  • /Users/<username>/Library/Preferences/<bundle_id>.LSShardFileList.plist   
  • /Users/<username>/Library/Preferences/com.apple.finder.plist   
  •  [10.10-] /Users/<username>/Library/Preferences/com.apple.recentitems.plist   
  • [10.11+] /Users/<username>/Library/Library/Application Support/com.apple.sharedfilelist/com.apple.LSSharedFileList.ApplicationRecentDocuments/<bundle_id>.sfl   
  • [10.11+] /Users/<username>/Library/Library/Application Support/com.apple.sharedfilelist/RecentApplications.sfl   
  •  [10.11+] /Users/<username>/Library/Library/Application Support/com.apple.sharedfilelist/RecentDocuments.sfl   
  •  [10.11+] /Users/<username>/Library/Library/Application Support/com.apple.sharedfilelist/RecentServers.sfl   
  •  [10.11+] /Users/<username>/Library/Library/Application Support/com.apple.sharedfilelist/RecentHosts.sfl 

The script usage is below. The only required argument is the directory, but the output can include binary BLOB hex dump of the Bookmark data (--blob). Most of the Mac MRUs contain a binary Bookmark BLOB of data that can be useful to determine where a certain file was located or where an application was run from. I’ve included it as an option as it can get very, very verbose.

The script also has two dependencies, hexdump.py and ccl_bplist.py. These files can be installed or just simply placed in the same directory you are running the macMRU.py script from. (Installation on OS X 10.11 systems are limited thanks to SIP.)

A few screenshots of example script output:

This example shows the output without the BLOB data of the newer SFL-based MRU files:

This example shows the same output with a sample of the hexdump BLOB data, you can see where this can get quite verbose.

The last example shows the ‘older’ MRU plist files found on 10.10 and older systems. (The com.apple.finder.plist files is the same on 10.11.)

Parsing iOS “Frequent Locations”

in ,

The Artifact:

The phrase “Location, Location, Location” has special meaning for those looking for real estate but can also mean everything to a forensicator looking for locational data. One of the most useful (or creepy if its yours) artifacts on an iOS device is the “Frequent Locations”. iOS will store, for a lengthy amount of time, locations a user’s device has been. This feature came out in iOS 7 and is used to record a pattern of activity for the user. Most often, a user will notice a message like one in the screenshot below - this is the 'routined' process in action on iOS devices. My phone “knew” I was likely headed to Arlington at this specific moment and ‘helped me’ by showing the traffic times. (Way to be creepy, iPhone!)

Users can review and clear their frequent location histories as shown in the next screenshot. 

Settings -> Privacy -> Location Services -> System Services -> Frequent Locations

Got an iPhone? Play along! Take a look at your frequent locations.

Don’t have locations? Location Services must be enabled.  iOS Location Services must be enabled and more specifically the option ‘Frequent Locations’ under ‘System Services’ must be turned on. There are many services that may be configured under ‘Location Services’ this is only one setting. The ‘Frequent Locations’ are enabled by default.

Locations are clustered by general areas. In the example above you can see I live in the Northern Virginia area and visit many cities in and around the area. These clusters also include the number of sub-locations and general timestamps. You can select one of these areas and get a more detailed street-level overview. This level shows that there are a number of ‘visits’ for each location, and again - a general day-based timestamp.

If we keep selecting down, we can get specific visit and timing information. Each location can have multiple visits. (Note the ‘N Nelson St.' example with 49 visits above.) The example below shows one visit on December 13th at around 6:30 – 6:45pm, however the visit radius (shown in blue) can of course be very specific or more general depending on GPS availability. (I believe this was a visit to Whole Foods.)

Going back to the top - I recently traveled to San Francisco to teach my SANS FOR518 course and have locations from there in the broader ‘United States’ cluster history. Since I traveled outside of my normal Northern Virginia area, iOS made a new “cluster” of locations to include those in the San Francisco Area but also some in the NoVA area as well! I’m sure there is an algorithm somewhere to determine this – but sometimes it just doesn’t make sense how it is organized.

The Script:

Available Here

I wanted to study these artifacts in more detail as they have obvious benefit to the forensics community. These ‘frequent locations’ are stored in two binary plists in the following location:

/private/var/mobile/Library/Caches/com.apple.routined/

  • StateModel1.archive
  • StateModel2.archive

(Note: These files are protected and are only accessible in a physical dump or by physical (jailbroken) access to a device. You will not find these files in an iCloud/iTunes backup or a logical dump by forensic tools.)

I was elated to find them in a common file format, however the binary plist format used is the NSKeyedArchiver format which is less than human analyst friendly. (More on these types of binary plists in a future blog article.)

I needed to write a script to truly begin to understand how and why these entries get created as well as put the plist data into context. Enter, the script.

“Big caveat here. I’m the first to tell anyone that I’m not a programmer. In fact – I’ve been avoiding it like the plague for the longest time (I had a not-so-fun experience with it in school – mind you I loved my scripting course.) I have come to realize that as a forensic analyst some times you just have to suck it up and script something to make your life easier - this is one of those times.

I decided to learn and script in Python since there were many forensic-based tools and packages that already use Python – it seems pretty industry standard. I’ve also noticed that I keep saying to myself (and my students) that this would be so much better or easier if there was a tool to do this or to do that! Sometimes you just gotta do it yourself if you want it done! So I will be trying to contribute as needed to the Mac forensics community – I foresee this being the first of many scripts.”

The dump_freq_locs.py script takes in the StateModel#.archive files (where # is 1 or 2).

The script also has two dependencies, hexdump.py and ccl_bplist.py. These files can be installed or just simply placed in the same directory you are running the dump_freq_locs.py script from. (Installation on OS X 10.11 systems are limited thanks to SIP.)

The script parses the StateModel files and dumps the information to standard output so you will likely want to redirect to a file for later analysis.

$ python dump_freq_locs.py StateModel1.archive > SM1_parsed.txt

The script output contains three sections some of which I can describe here, but as stated above – I don’t know the significance of some of these items….yet. Parsing is the first step toward understanding.

  • Metadata – Contains metadata about the file (eg: when it was last updated, last location information, as well as other timestamps.)
  • RTVisitMonitor Data – Location information, timestamps, edge detection, outlier, last visit, and LOI (Location of Interest?) information.
  • Location Data – The ‘meat’ of plist file. Each location entry has the following:
    • A timestamp of when it was last updated (not necessarily visited)
    • Hexdump output of the location BLOB data containing reverse geolocation data
    • Hex Output - so you can input it into your favorite hex editor for additional analysis
    • Location Data -  The decimal latitude and longitude as well as ‘confidence’ and ‘uncertainty’ (I’m assuming these have something to do with how big the blue radius is) and the last update timestamp.
    • Visits – Entry and Exit timestamps for each visit.
    • Transition Data – When the device was ‘out of range’ of a location. The ‘Motion Activity’ may have something to do with if the device was in a vehicle or in a persons pocket while walking. Looking at my own data a value of 0 seems to be when I’m walking and a value of 4 seems to be when I’m in a car. Not sure what value 2 is yet.

This script was tested of a variety of different devices including iPhones and iPads on different iOS’s from iOS 7-9. As usual the format of these StateModel files has changed slightly from version to version. If you see I’m missing certain information from the parsed output please let me know! On that same note, if I’m not doing something Pythonically correct or weird – let me know too! I’m new to this Python/scripting thing.

I plan on adding future capabilities to this script, like CSV and KML outputs. If you have specific output format needs – let me know, I’ll take them into consideration!

Enjoy!

Sample script output is available here.