More Scripting in Swift: Shuffling files and writing a property list

MakeLogarchive tackles a classic scripting problem: copy a set of files from one structure into a different structure, and generate a property list which magically turns them into the undocumented logarchive format. It also catalogues the individual log files within any logarchive.

My code is written in Swift 4.0, in Xcode 9, and aimed at both Sierra and High Sierra, which feature the new unified log. For once, it is a bit better structured, so I am going to present it from the top-level functions down. I hope that you find this a useful approach.

There are three buttons, each of which requires an Action function, and a single checkbox, which has no additional code driven from it.

The first (last) action function simply saves the text from the scrolling text box below the buttons:
@IBAction func theSaveButton(_ sender: Any) {
let fileContentToWrite = outputText.string
let FS = NSSavePanel()
FS.canCreateDirectories = true
FS.allowedFileTypes = ["text", "txt"]
FS.begin { result in
if result.rawValue == NSFileHandlingPanelOKButton {
guard let url = FS.url else { return }
do {
try fileContentToWrite.write(to: url, atomically: false, encoding: String.Encoding.utf8)
} catch {
self.appendOutputText(string: error.localizedDescription)
}}}}

appendOutputText() is a local function which I define below, and just adds the error message to the text in that box.

This is the function to handle the main business button, to generate the logarchive bundle:
@IBAction func copyButton(_ sender: Any) {
let FO = NSOpenPanel()
FO.allowsMultipleSelection = false
FO.canChooseFiles = false
FO.canChooseDirectories = true
FO.showsHiddenFiles = true
FO.begin { result1 in
if result1.rawValue == NSFileHandlingPanelOKButton {
guard let sourceUrl = FO.url else { return }
do {
let theSourceString = "Source: " + (sourceUrl.path as String) + "\n"
self.replaceOutputText(string: theSourceString)
let FS = NSSavePanel()
FS.canCreateDirectories = true
FS.allowedFileTypes = ["logarchive"]
FS.begin { result2 in
if result2.rawValue == NSFileHandlingPanelOKButton {
guard let destUrl = FS.url else { return }
do {
let theDestString = "Destination:" + (destUrl.path as String) + "\n"
self.appendOutputText(string: theDestString)
self.copyToLogarchive(sourceURL: sourceUrl, destURL: destUrl)
self.makeInfoFile(destURL: destUrl)
}}}}}}}

That displays a File Open dialog to prompt the user to select the folder to be used as the source, then displays a File Save dialog to obtain the destination. The first is interesting as its options enable the user to see hidden files and folders, thus to select the /var/db folder in which live logs are collected, and can only select folders, not files.

The final three function calls invoke local functions discussed below. The first writes the destination details to the text box, then copyToLogarchive() copies the files into the new logarchive bundle. Finally, makeInfoFile() generates the all-important Info.plist which is required to validate the logarchive.

insidemla01

The third button generates a catalogue of the selected logarchive bundle:
@IBAction func catalogueButton(_ sender: Any) {
let FO = NSOpenPanel()
FO.allowsMultipleSelection = false
FO.allowedFileTypes = ["logarchive"]
FO.begin { result in
if result.rawValue == NSFileHandlingPanelOKButton {
guard let sourceURL = FO.url else { return }
do {
let theSourceStr = sourceURL.path as String
let theOutStr = "Catalogue listing of " + theSourceStr + " :\n"
self.replaceOutputText(string: theOutStr)
self.theFileArray = []
self.runDirCheck(theURL: sourceURL, theFolder: "")
let thePersistStr = theSourceStr + "/Persist"
let thePersistURL = URL.init(fileURLWithPath: thePersistStr, isDirectory: true)
self.runDirCheck(theURL: thePersistURL, theFolder: "Persist")
let theSpecialStr = theSourceStr + "/Special"
let theSpecialURL = URL.init(fileURLWithPath: theSpecialStr, isDirectory: true)
self.runDirCheck(theURL: theSpecialURL, theFolder: "Special")
self.theFileArray.sort()
let theOut2Str = self.theFileArray.joined()
self.appendOutputText(string: theOut2Str)
}}}}

This re-initialises the variable theFileArray, into which the catalogue will be generated. It then runs runDirCheck() on the top-level of the bundle, and on the two folders Persist and Special within it, which are the only places in which tracev3 log files should be found. Each of these is transformed from a path String to a URL for that purpose.

This function builds the contents of theFileArray as an array of Strings containing the information about individual log files, then sorts that array, joins it up into a single String, and outputs that to the text box.

insidemla02

Here is that function runDirCheck():
func runDirCheck(theURL: URL, theFolder: String) {
if theURL.hasDirectoryPath {
var theOthers: [URL] = []
let FM = FileManager.default
let theOpts = [URLResourceKey.fileSizeKey, URLResourceKey.contentModificationDateKey, URLResourceKey.creationDateKey, URLResourceKey.isRegularFileKey, URLResourceKey.nameKey]
do {
try theOthers = FM.contentsOfDirectory(at: theURL, includingPropertiesForKeys: theOpts, options: [])
for item in theOthers {
self.getTracev3(theItemURL: item, theFolder: theFolder)
}
} catch let error {
self.appendOutputText(string: error.localizedDescription)
}}}

This takes the URL, and checks that it is to a directory path and not a file. It then initialises the default FileManager, required to support FM.contentsOfDirectory(). That call can pre-fetch file properties, which are here defined as the options in theOpts. Because the call to contentsOfDirectory() can throw, it is set inside a do { try {} catch {} } structure, to handle errors.

Having obtained a shallow directory listing using contentsOfDirectory(), the code then iterates through that list of URLs, calling the getTracev3() function below, to actually obtain the information for the catalogue.

func getTracev3(theItemURL: URL, theFolder: String) {
if (theItemURL.pathExtension == "tracev3") {
let theResKeys: Set = [URLResourceKey.fileSizeKey, URLResourceKey.contentModificationDateKey, URLResourceKey.creationDateKey, URLResourceKey.isRegularFileKey, URLResourceKey.nameKey]
do {
var theResVals: URLResourceValues
try theResVals = theItemURL.resourceValues(forKeys: theResKeys)
let theIsFile = theResVals.isRegularFile
if theIsFile! {
let theFileSizeStr = "\(theResVals.fileSize!)"
let theModDate = theResVals.contentModificationDate
let theModDateL = theModDate?.description
let theModDateArray = theModDateL?.components(separatedBy: " ")
let theModDateStr = theModDateArray![0] + " " + theModDateArray![1]
let theCreateDate = theResVals.creationDate
let theCreateDateL = theCreateDate?.description
let theCreateDateArray = theCreateDateL?.components(separatedBy: " ")
let theCreateDateStr = theCreateDateArray![0] + " " + theCreateDateArray![1]
let theDuration = DateInterval.init(start: theCreateDate!, end: theModDate!)
let theDurationStr = String(format:"%.1f", (theDuration.duration / 60.0))
let theNameStr = theFolder + "/" + theResVals.name!
let theOut2Str = theNameStr + "\t" + theCreateDateStr + " " + theModDateStr + " " + theFileSizeStr + " bytes, period " + theDurationStr + " min\n"
theFileArray.append(theOut2Str)
}
} catch let error {
self.appendOutputText(string: error.localizedDescription)
}}}

This function first checks whether the URL represents a file with the .tracev3 extension. If it does, then it creates a Set of URLResourceKeys which will be obtained for the file specified by the URL. It next obtains those values, and converts them into String format to use in the catalogue. The file size is simply converted from integer to string, the dates into their date and time components, which are then concatenated, and the duration is obtained as a DateInterval which is converted to a String using format, as it is a floating point number.

These String components are then concatenated into the final line, containing the information for that tracev3 log file.

insidemla03

My next function does the crucial job of generating the Info.plist file for the logarchive bundle. The format used here has been reversed by studying many examples in ‘proper’ logarchives. A large chunk of the property list is taken from the property list version.plist, part of the log file assembly which ends up inside the Extra folder of a logarchive.

func makeInfoFile(destURL: URL) {
var theImportDict = NSMutableDictionary()
let theVerPath = destURL.path + "/Extra/version.plist"
theImportDict = NSMutableDictionary.init(contentsOfFile: theVerPath)!
if (theImportDict.count > 0) {
theImportDict.removeObject(forKey: "Version")
} else {
if (debugCheckbox.state == .on) {
let theDStr = "theImportDict is empty.\n"
self.appendOutputText(string: theDStr)
}}

This first step reads that version.plist file in as a mutable Dictionary, and then removes its entry for the key “Version”, which doesn’t appear in Info.plist.

Two new pieces of information are required to generate Info.plist: the Mach system time in ticks of the last log entry, and a UUID. I haven’t yet worked out how to obtain the former, but discovered that using the current time is sufficient. Constructing the new mutable dictionary has to take into account its nested structure, so starts from the outermost leaves and works inwards.

let currentTime = mach_absolute_time()
let theUUID = UUID.init()
let theUUIDStr = theUUID.uuidString
let theSupVer = 518.70000000000005
let theBaseEntry = ["ContinuousTime": currentTime, "UUID": theUUIDStr] as [String : Any]
let theOTR = ["OldestTimeRef": theBaseEntry]
let theTTL = theImportDict as Dictionary
let theSpecialM = ["OldestTimeRef": theBaseEntry, "TTL": theTTL] as [String : Any]
let theLot = ["EndTimeRef": theBaseEntry, "LiveMetadata": theOTR, "OSArchiveVersion": 3, "OSLoggingSupportProject": "libtrace-518.70.1", "OSLoggingSupportVersion": theSupVer, "PersistMetadata": theOTR, "SpecialMetadata": theSpecialM] as [String : Any]

The dictionary is now complete, and ready to export to the Info.plist file:
let theExportDict = NSMutableDictionary()
theExportDict.setDictionary(theLot)
let theDest = destURL.path + "/Info.plist"
let outURL = URL.init(fileURLWithPath: theDest)
let theResult = theExportDict.write(to: outURL, atomically: false)
if !theResult {
self.appendOutputText(string: "Couldn't save exported file.")
}}

insidemla04

The next function does the donkey work of copying the files and folders from the old two-folder structure, as found in /var/db, to the more complex structure of a logarchive bundle. Once again it does this using FileManager calls, so that has to be set up first.

func copyToLogarchive(sourceURL: URL, destURL: URL) {
let FM = FileManager.default
do {
let theUuidSourceStr = sourceURL.path + "/uuidtext"
let theUuidDestStr = destURL.path
try FM.copyItem(atPath: theUuidSourceStr, toPath: theUuidDestStr)
let theDiagSourceStr = sourceURL.path + "/diagnostics/Persist"
let theUuidDest2Str = destURL.path + "/Persist"
try FM.copyItem(atPath: theDiagSourceStr, toPath: theUuidDest2Str)
let theDiagSource2Str = sourceURL.path + "/diagnostics/Special"
let theUuidDest3Str = destURL.path + "/Special"
try FM.copyItem(atPath: theDiagSource2Str, toPath: theUuidDest3Str)
let theDiagSource3Str = sourceURL.path + "/diagnostics/timesync"
let theUuidDest4Str = destURL.path + "/timesync"
try FM.copyItem(atPath: theDiagSource3Str, toPath: theUuidDest4Str)
let theMiscDestStr = destURL.path + "/Extra"
try FM.createDirectory(atPath: theMiscDestStr, withIntermediateDirectories: false, attributes: nil)
var theOthers: [String] = []
let theDiagSource5Str = sourceURL.path + "/diagnostics"
try theOthers = FM.contentsOfDirectory(atPath: theDiagSource5Str)
for item in theOthers {
let theTempStr = theDiagSource5Str + "/" + item
let theTempURL = URL.init(fileURLWithPath: theTempStr)
if !(theTempURL.hasDirectoryPath) {
let theTempDestStr = theMiscDestStr + "/" + item
try FM.copyItem(atPath: theTempURL.path, toPath: theTempDestStr)
}}} catch let error {
self.appendOutputText(string: error.localizedDescription)
}}

I have stripped the calls for most of the verbose output from the code above, so it is just a series of FileManager copyItem() calls, together with creation of one additional folder.

insidemla05

Finally, there are the two utility functions to write a String to the text box, first replacing its existing contents, and second appending the string to whatever is already there:

func replaceOutputText(string: String) {
let myString = NSAttributedString(string: string)
outputText.textStorage?.setAttributedString(myString)
}

func appendOutputText(string: String) {
let myString = NSAttributedString(string: string)
outputText.textStorage?.append(myString)
}

And the important String array and two control outlets:
var theFileArray: [String] = []
@IBOutlet var outputText: NSTextView!
@IBOutlet var debugCheckbox: NSButton!

insidemla06

I hope that you find this a useful example as to how to script file actions using Swift 4.0.