hoakley Macs, Technology

What is triald and why is it taking so much disk space?

Over the last year or so Mac users have run into problems that appear related to a background service named triald. Some report it stealing huge amounts of CPU, others associate it with various glitches, and a few have noticed gigabytes of disk space apparently being taken up by its folder at ~/Library/Trial, and in their Time Machine backups. This article explains as much as I know about this mysterious new service, and ponders what’s going on.

Trial, as I’ll refer to it, was introduced to macOS in Big Sur, when it appeared in three new private frameworks named Trial, TrialProto and TrialServer. At that time it was version 1.0, with a build number of 174.4 in macOS 11.0.1, and the latter has since risen to 292.18 in Monterey 12.3. In addition to those private frameworks, there are three essential components:

  • the background service /usr/libexec/triald
  • an XPC service TrialArchivingService.xpc contained in TrialServer.framework
  • a launch agent /System/Library/LaunchAgents/com.apple.triald.plist.

Note that all these are on the SSV, therefore can’t be tampered with in any way. The high build number of the version first shipped in Big Sur suggests that Trial may have already been included in iOS, and it’s listed on some iOS sites as being present, with the suggestion that it has something to do with “Apple A/B testing”.

Storage

After a user Home folder has been created, Trial normally creates its own root directory at ~/Library/Trial, which contains folders of NamespaceDescriptors, Treatments, and an SQLite database in the folder v6 named trialdb.db. None of these appears specially protected, although each folder is a warren of sub-folders which makes extracting information from them very difficult. The Trial root directory can readily occupy substantial amounts of disk space: on my production Mac, it appears to contain 1.47 GB in 148,618 items, which is a hefty chunk in backups. Even on a lightly used Mac, it may amount to more than 1 GB and over 300,000 items. However, most of those may be symbolic links, and its real size may thus be considerably less.

This Trial root directory isn’t included in the standard Time Machine exclusions, so unless explicitly added to the user’s exclusion list, it will normally be backed up automatically with the rest of the contents of the Home folder.

Activation and activity

It has been suggested that Trial is only activated on Macs which have been enrolled in Apple’s beta programmes. I don’t think that’s correct, as my production Mac has never been so enrolled, nor has my M1 Pro MBP, but both are active Trial participants. I’m not aware of any control over or opt-out from Trial, although it could be covered in Apple’s general request to share data including panic logs, if you can remember where to control that.

I first noticed Trial when studying Visual Look Up in the log. At the start of each Look Up, the subsystem com.apple.trial enters the following into the log:
Initializing TRIClient. Trial version: TrialXP-292.18
_PASEntitlement: Entitlement "com.apple.private.security.storage.triald" is not present.
Found entitlement: "com.apple.trial.client" --> <private>
<private> 0x12d9547e0 (no container): using Trial root dir <private>
This suggests that Trial is recording information about Visual Look Up, although nothing more explicit appears in subsequent log entries.

Otherwise Trial is most active in the log when a Mac isn’t in use, but isn’t asleep. Its activities are normally scheduled and dispatched through the DAS-CTS mechanism, and on M1 series Macs run exclusively on their E cores. In addition to what appear to be opportunistic periods during the day, Trial undertakes its most extensive maintenance activity at 0330 daily. Log excerpts detailing active sesssions are provided in the Appendix at the end.

Trial (triald) has CloudKit entitlements, and often communicates over CloudKit, from where it appears to get Experiments, and to upload its data. I have so far been unable to discover whether it has any dedicated container, though.

Intense maintenance periods can occur in the small hours of the morning, in my case 0330, or soon after starting the Mac up if it’s not left on for long periods. These include maintenance on the Trial root directory, CloudKit access, and checking for the availability of new Experiments, which appear to be rolled out through iCloud. Each Experiment appears to have its own history, maintained by Trial.

Investigation

Trial is very visible in the Unified log, even if its entries are cryptic and full of <private>. The two key subsystems to include in predicates are com.apple.trial and com.apple.triald, and they can usefully be extended by processes containing trial.

With limited understanding of Trial, it’s hard to guess the best approaches to dealing with problems. Users who have tried killing triald have inevitably discovered how quickly it’s restarted, and deleting its root directory in ~/Library/Trial also results in its reappearance. For the time being, the most attractive approach is to restart in Safe mode, wait a couple of minutes, and restart back in normal mode. If you want to try deleting its root directory, that too would appear to be safest in Safe mode.

Conclusions

  • Trial is a new and hitherto undocumented service introduced in Big Sur.
  • Although problems with Trial aren’t common, they do occur, and without any documentation they can be mystifying.
  • Trial appears primarily concerned with gathering data from Experiments distributed via CloudKit. These notably include ML features such as Visual Look Up.
  • Although there’s no indication that Trial gathers or uploads sensitive data, its root directory at ~/Library/Trial is unprotected and its activities are opaque. While Apple presumably considers those activities and use of data fall within its existing policies, suspicious users may presume the worst. Silence only fuels suspicion and rumour.
  • Trial’s root directory at ~/Library/Trial should be added routinely to all backup exclusion lists. Apple should add it to Time Machine’s system exclusions at the earliest opportunity.
  • Trial’s periods of intense maintenance normally occur when the user isn’t present. If they are started soon after startup or when the user is active, they can cause concern. This needs to be addressed, at least by informing users.
  • Apple needs to come clean and explain Trial properly.

Appendix: Log excerpts

Entries from a short session occurring with the Mac quiet during the day, showing involvement with CloudKit:

com.apple.triald TrialServer Enqueueing eligible task <TRIFetchMultipleExperimentNotificationsTask:(null)> [tid:9719]
com.apple.triald TrialServer enqueued follow-up task <TRIFetchMultipleExperimentNotificationsTask:(null)> [tid:9719] after execution of [tid:9718]
com.apple.triald TrialServer CKNative checking for new experiments since 2022-03-28 22:07:22 (BST) | 2022-03-28 21:07:22 (UTC)
com.apple.triald TrialServer created cloudkit query with predicate {<private>} zoneID:_defaultZone
com.apple.cloudkit triald CloudKit Starting operation <CKQueryOperation: 0x7f92f481c940; operationID=C09CA75C04DF65C4, resolvedConfig={ timeoutForRequest=60, flags=expensiveAcccess|xpcActivity|preferAnon, discretionary=Infers Disc }, qos=Default, operationGroup={ operationGroupID=A1C149DFD704966A, shortened-name=Periodic }, <private>>
com.apple.cloudkit triald CloudKit Invoking main for operation <CKQueryOperation: 0x7f92f481c940; operationID=C09CA75C04DF65C4>
com.apple.cloudkit triald CloudKit Operation <CKQueryOperation: 0x7f92f481c940; operationID=C09CA75C04DF65C4> enqueuing on container throttle queue
com.apple.cloudkit triald CloudKit Operation <CKQueryOperation: 0x7f92f481c940; operationID=C09CA75C04DF65C4> invoking on container throttle queue
com.apple.CloudKit PointsOfInterest triald CloudKit ID=<private> Container=<private> GroupID=<private> GroupName=__##__signpost.description#____#attribute#_##_#Periodic##__## Behavior=__##__signpost.description#____#attribute#_##_#Infers Disc##__## QoS=__##__signpost.description#____#attribute#_##_#Default##__##
com.apple.CloudKit PointsOfInterest triald CloudKit Finishing
com.apple.triald triald TrialServer finished fetchExperiment query with 0 results cursor (null) error (null)
com.apple.triald triald TrialServer received 0 experiment records from CloudKit
com.apple.triald triald TrialServer Fetch status success but 0 results fetched.
com.apple.triald triald TrialServer register xpc activity handler for com.apple.triald.cellular

When sleep is disabled, these are short excerpts from copious entries made during Trial Maintenance performed at 0330:

com.apple.triald triald TrialServer Enqueueing eligible task <TRIRolloutTargetingTask:61af99aeda72d16a4beb7756.240000067> [tid:9735]
com.apple.triald triald TrialServer enqueued follow-up task <TRIRolloutTargetingTask:61af99aeda72d16a4beb7756.240000067> [tid:9735] after execution of [tid:9724]
com.apple.triald triald TrialServer adding task to task queue: <private>
com.apple.triald triald TrialServer adding task with 0 dependencies to task queue: <private>
com.apple.triald triald TrialServer refCount+ <private>: 1 --> 2
com.apple.triald triald TrialServer Enqueueing eligible task <TRIRolloutTargetingTask:5ffde50ce2aacd000d47a95f.240000128> [tid:9736]
com.apple.triald triald TrialServer enqueued follow-up task <TRIRolloutTargetingTask:5ffde50ce2aacd000d47a95f.240000128> [tid:9736] after execution of [tid:9724]
com.apple.triald triald TrialServer Starting <TRIMaintenanceTask:0x7f92f4a07930> [tid:9725] with tags {}
com.apple.triald triald TrialServer TRIMaintenanceTask scheduled deactivation of 0 experiments
com.apple.triald triald TrialServer TRIMaintenanceTask deregistered 0 dynamic namespaces

com.apple.triald triald TrialServer [LOCK_SH] GC scan found 0 live symlinks into reference-managed directory: /Users/hoakley/Library/Trial/NamespaceDescriptors/v2/rolloutV2/61301e3a61217b3110231469/240000016
com.apple.triald triald TrialServer [LOCK_SH] Removed 0 dead symlinks from reference-managed directory: /Users/hoakley/Library/Trial/NamespaceDescriptors/v2/rolloutV2/61301e3a61217b3110231469/240000016
com.apple.triald triald TrialServer [LOCK_SH] GC scan found a nonzero external reference count for reference-managed directory: /Users/hoakley/Library/Trial/NamespaceDescriptors/v2/rolloutV2/61301e3a61217b3110231469/240000016
com.apple.triald triald TrialServer No deferred deleted items to garbage collect for root directory /Users/hoakley/Library/Trial/NamespaceDescriptors/v2/rolloutV2/61301e3a61217b3110231469/240000016

com.apple.triald triald TrialServer Garbage collection for asset store /Users/hoakley/Library/Trial/v6/AssetStore removed 2 assets.
apsd Looking up connection on peer: 62415d60 found <private>
apsd <private> Getting channels for topic <private> environment production tokenName <private>
apsd <private> Found channels for topic <private>: <private>
com.apple.triald triald TrialServer Expired 0 records from experiment history.
com.apple.triald triald TrialServer Expired 0 records from MLRuntime evaluation history.
com.apple.apsd triald ApplePushService <private> push topic com.apple.triald got registered channels <private>
com.apple.triald triald TrialServer Keeping channels for 8 rollouts + 1 experiments
com.apple.triald triald TrialServer Reconciling 7 actual subscriptions with 9 expected subscriptions