Every app page in the Library and every number in our app study comes from the same procedure, so results can be compared and repeated. This page describes it.
Test setup
- Hardware and system: an Apple silicon Mac on the current release of macOS, with all updates installed. The macOS and app versions are recorded on every app page.
- Fresh account: a new macOS user account for each app, with no other apps installed and iCloud signed out unless the app needs it.
- Recording: LightAtom records every connection the app’s processes make: process, remote address, port, protocol and bytes in each direction. Its DNS helper records the domain name each connection was made to.
- Network: a residential connection with no VPN, filtering DNS or proxy.
The test script
The same five steps for every app:
- Launch the app for the first time and complete any first-run setup with default choices.
- Leave it idle for 10 minutes.
- Use it for 5 minutes following a short, app-specific script of typical actions (for a chat app: open a conversation, send a message, attach an image). The script is published with the results.
- Quit the app.
- Export the recorded connections.
Idle and active data are reported separately, because what an app sends while you’re not using it is often the interesting part.
Classifying domains
Each domain is classified by purpose using a published rulebook. A domain matches a rule when it equals the rule’s domain or is a subdomain of it, and the most specific rule wins:
- core: the app’s own service, needed for it to work.
- updates: checking for and downloading updates.
- analytics: usage measurement, product analytics, attribution and session recording (for example Google Analytics, Firebase, Segment, Mixpanel, Amplitude).
- ads: ad serving and ad measurement (for example doubleclick.net, googlesyndication.com).
- crash_reporting: crash and error reports (for example Crashlytics, Sentry, Bugsnag).
- cdn: content delivery networks serving files (for example CloudFront, Akamai, Fastly).
- unknown: anything not covered, until a person reviews it.
Rules are added only when the domain’s owner documents its purpose. Where a domain is shared, such as a CDN, we classify the role it plays, not the company.
What we report, and what we don’t
- We report facts we measured: domains, their owners and countries, how much data moved, and how many fall into each class.
- We don’t judge intent, and we never call an app spyware or malicious. A connection to an analytics domain means that the app sends usage data, not what the data contains.
- We don’t read traffic contents; LightAtom only sees connection metadata and DNS names.
Corrections and statements
Developers can send corrections or a statement about their app’s results. We re-test when an app changes materially and at least once a year, and every app page shows the version and date tested. Statements from developers are published on the app’s page.