The Apple PSI System: Are iCloud and iMessage CSAM Detection Really a Privacy Nightmare?

A few years ago, Apple announced a bundle of new Child Safety policies. The pitch was simple enough: if a child account tried to send or receive sexually sensitive material, the device would intervene before the image was viewed or sent. If known Child Sexual Abuse Material, or CSAM, appeared in iCloud Photos, Apple would flag the Apple ID, perform human review, and report confirmed material to the National Center for Missing and Exploited Children.

On paper, that sounds like the kind of thing no sane person wants to argue against. CSAM is real. It is horrific. Platforms should not get to pretend the problem vanishes once the file is encrypted or uploaded somewhere inconvenient.

And yet the announcement detonated almost immediately among privacy advocates, cryptographers, and the usual cypherpunk neighborhood watch. The objections had two layers.

First, the child-safety layer. Automated sexual-content detection is blunt by design. A system that decides what a child may view, send, or ask about can easily become a system that narrows access to information, disciplines normal adolescent behavior, or puts queer children in danger. A warning screen is not neutral just because it appears locally on a device.

Second, the architecture layer. If Apple can make a user's phone compare private photos against a hidden database, what exactly stops that database from changing later? Today the set is known CSAM. Tomorrow it could be protest posters, leaked documents, banned books, copyright fingerprints, religious material, or whatever a government manages to place under the label of safety.

Then came the little bonfire that made everything worse: an internal memo appeared to dismiss critics as "screeching voices of the minority". The backlash got louder. Balaji had a thread. Snowden had a thread. Everyone with a threat model had a thread.

Apple eventually backed away from the iCloud Photos part. It paused the rollout in September 2021, then told WIRED in December 2022 that the proposed iCloud Photos CSAM detection tool would not move forward. That system was announced, argued over, and killed. It never shipped.

But that does not make the episode a dead historical curiosity. The iCloud Photos scanner may have died; he endpoint did not.

Apple's Communication Safety feature survived and kept expanding. It now uses on-device analysis to detect nude photos and videos in Messages, AirDrop, Contact Posters, FaceTime, shared photo albums, and some third-party sharing flows. Apple says the analysis happens on the child's device, that Apple does not receive a signal that nudity was detected, and that Apple does not get access to the photos or videos; see Apple's Communication Safety support page. In 2026, Apple previewed another expansion: Communication Safety would also intervene when gore or violent content is detected in shared images or videos; see Apple's 2026 child-safety preview.

The iCloud side changed too. Under Standard Data Protection, iCloud data is encrypted in transit and at rest, but many encryption keys are still stored in Apple data centers. With Advanced Data Protection enabled, Apple says most iCloud categories, including iCloud Backup, Photos, Notes, and more, use end-to-end encryption. See Apple's iCloud data security overview and its Platform Security explanation of Advanced Data Protection.

But even that stronger cloud encryption is not a law of nature. In the United Kingdom, Apple says new users can no longer enable Advanced Data Protection, while iMessage and FaceTime remain end-to-end encrypted globally.

To summarize what's going on:

1. The iCloud Photos CSAM Detection system was proposed and then abandoned.
2. Communication Safety shipped and expanded.
3. iCloud Photos can be end-to-end encrypted when Advanced Data Protection is available and enabled.
4. Stronger cloud encryption does not remove scanning pressure. It tends to move the pressure toward the device.

Before getting drunk on the politics, though, there is one technical question worth answering.

Wasn't iCloud or iMessage supposed to be end-to-end encrypted? How can Apple compare user data against a hidden database without directly reading the data?

Conclusion first: Apple was not simply lying about the cryptography. At least in the proposed iCloud Photos system, Apple was not planning to run a dumb server-side bulk scan over every user's plaintext photo library. The matching was supposed to happen before upload, on the user's device. The device would generate a cryptographically sealed safety voucher and upload that voucher alongside the photo. Apple would not be able to interpret the meaningful contents of those vouchers unless the account crossed a threshold number of matches.

The mind-reading trick was made possible by a cryptographic primitive called Private Set Intersection, or PSI.

And that is exactly why the proposal was unsettling. PSI is real cryptography; it is not a fake privacy sticker slapped onto a backdoor. It can be engineered carefully. It can hide non-matches from the server. It can hide the database from the client. It can even delay disclosure until enough matches accumulate.

The real issue is that the math is good enough to make a dangerous institutional shape easier to defend.

So the right question to ask is NOT "can Apple see my photos?". It is closer to: "what is my device actually computing?".

A server scanning uploaded files is an old privacy problem. A phone quietly comparing local data against a hidden policy database and manufacturing sealed evidence for a third party is a different animal. To see why, we need to separate iMessage from iCloud, then look at PSI itself.

PSI: What Exactly Is It?

A. iMessage encryption in one pass

Let's start with iMessage.

Unlike many messaging systems, iMessage is built as an end-to-end encrypted messaging service. Apple says iMessage messages and attachments are protected so that only sender and receiver can access them, and the detailed mechanics are described in Apple's Platform Security documentation.

When you send an iMessage to a phone number or email address, your device first asks Apple Identity Service, or IDS, for the public keys and APNs addresses of the recipient's devices. Then the sending device roughly performs the following procedure for each receiving device:

1. Generate a random 88-bit value, the seed.
2. Use the seed as an HMAC-SHA-256 key to derive a 40-bit verifier from the sender public key, recipient public key, and plaintext.
3. Concatenate the 88-bit seed and 40-bit verifier into a 128-bit per-message AES key.
4. Encrypt the message with AES in CTR mode.
5. Encrypt the per-message AES key to the recipient device's public key using RSA-OAEP, or ECIES on newer systems.
6. Hash the encrypted message payload plus encrypted message key with SHA-1.
7. Sign that hash with the sender's ECDSA signing key.
8. Send the encrypted payload, encrypted key, and signature through APNs.

Large attachments are handled separately. The attachment is encrypted with a random 256-bit AES key and uploaded to iCloud. The iMessage then carries the attachment URI, key, and hash inside the end-to-end encrypted message payload.

Structurally, plaintext lives at the endpoints. Sender devices and recipient devices can see the message. Apple servers relay it. They do not get the normal message plaintext.

That is iMessage; but iCloud Photos is not the same beast.

For a long time, iCloud Photos lived in a more traditional cloud-security model: encrypted in transit and at rest, but with keys Apple could hold and use for account recovery, service operation, and lawful process. Advanced Data Protection moves Photos into the end-to-end encrypted bucket, but ADP is opt-in and, as the UK case demonstrates, not equally available everywhere.

This distinction matters because the Apple CSAM controversy was often flattened into "Apple is scanning end-to-end encrypted messages." That is not really what the proposed iCloud Photos system is about. What matters is the endpoint matching for photos before iCloud upload. Communication Safety in Messages is different again: local classification, not PSI against a known-CSAM hash database.

Let's keep the boxes separate:

• iMessage transport is end-to-end encrypted.
• iCloud Photos under Standard Data Protection is cloud-encrypted, but not the same thing as end-to-end encrypted messaging.
• iCloud Photos under Advanced Data Protection can be end-to-end encrypted.
• The proposed iCloud Photos CSAM scanner would have matched photos on the device before upload.
• Communication Safety is on-device machine-learning classification, not PSI.

Now we can talk about PSI without mixing all the wires together.

B. Defining the PSI problem

Suppose Alice has a set

$$
A = (a_1, \ldots, a_n)
$$

and Bob has another set

$$
B = (b_1, \ldots, b_n).
$$

We want a protocol satisfying two conditions.

1. Alice and Bob should learn the intersection $A \cap B$.
2. Neither side should learn anything about the other side's non-intersecting elements.

In other words, if $a_i \notin B$, then Bob should not learn useful information about $a_i$. Likewise, Alice should not be able to run little experiments to infer whether Bob has some arbitrary value $z$ unless $z$ appears in the intersection through the allowed protocol output.

Designing such a protocol is the PSI problem.

The stupid solution is obvious: Alice sends all of $A$ to Bob, and Bob computes $A \cap B$. This does not solve PSI. Bob learns every element of $A$, including all non-matches.

The slightly less stupid solution is to hash everything:

1. Pick a one-way hash function $H(x)$.
2. Alice computes

$$
\hat A = (H(a_1), \ldots, H(a_n)).
$$

Bob computes

$$
\hat B = (H(b_1), \ldots, H(b_n)).
$$

1. Bob stores a key-value map $D_B[H(b_i)] = b_i$.
2. Alice sends $\hat A$ to Bob.
3. Bob computes $\hat A \cap \hat B$ and returns the corresponding original values from his database.

Better? Slightly. Safe? Not really.

If the input domain is small or guessable, a hash is not a privacy tool. It is a lookup table waiting to happen. Phone numbers, national IDs, email addresses, short strings, known image hashes, structured identifiers: all of these can be brute-forced.

If the possible input space is $S$ with $N$ elements, Bob can compute

$$
D[H(s_i)] = s_i
$$

for every $s_i \in S$. Now $\hat A$ becomes readable. The hash function did not preserve privacy. It just made Bob do prep work.

Lesson: a bare hash is not a veil. Very often, it is just delayed disclosure.

C. Basic PSI: Server-aided PSI

So how do we actually solve this?

There are many PSI constructions, and some are aggressively optimized. For intuition, start with server-aided PSI.

Add a third actor: Mallory, the server.

The assumptions are as follows.

1. Untrusted third party. Mallory is not trusted. We do not assume the protocol always completes, so liveness is not guaranteed. But if Mallory cheats, the protocol should detect the cheating.
2. No collusion. To simplify the model, assume Mallory does not collude with Alice or Bob to attack the other party.

Now define a pseudorandom function, or PRF:

$$
Y = F(K, X).
$$

For anyone who does not know the key $K$, the output $F(K, X)$ should look random. More formally, $(X, F(K, X))$ should be computationally indistinguishable from $(X, Y)$ where $Y$ is sampled randomly from the output space.

The protocol runs like this:

1. Alice has $A = (a_1, \ldots, a_n)$.
2. Bob has $B = (b_1, \ldots, b_n)$.
3. Alice and Bob agree on the PRF $F$.
4. Alice and Bob agree on a secret PRF key $K$. Mallory must not learn it.
5. For each $a_i$, Alice sends Mallory

$$
\hat a_i = F(K, a_i)
$$

and stores $D_A[\hat a_i] = a_i$ locally.

1. For each $b_i$, Bob sends Mallory

$$
\hat b_i = F(K, b_i)
$$

and stores $D_B[\hat b_i] = b_i$ locally.

1. Mallory computes $\hat A \cap \hat B$ and sends the result to both parties.
2. Alice and Bob map the returned PRF outputs back to original values using their local tables.

What does Mallory learn?

In this simplified version, Mallory learns the sizes of $A$, $B$, and $A \cap B$. That can still leak something in some settings, but leave that aside for a moment. Mallory does not learn the elements themselves. Without $K$, the PRF outputs look random.

What do Alice and Bob learn?

Only the intersection, assuming Mallory returns exactly the intersection and nothing else.

And there is the catch: Mallory is untrusted.

If Mallory follows the protocol, everything is tidy. If Mallory cheats, edge cases appear.

A. Mallory sends different answers to Alice and Bob.

Then Alice and Bob can perform an equality test. For example, they can sort the received intersection, hash the sorted result, and compare hashes. More rigorously, they can use a Private Equality Test, or PET, so the verification itself does not leak extra information.

B. Mallory sends the same answer to Alice and Bob, but the answer is still false.

Now we have a more annoying problem. Mallory is not merely equivocating. Mallory is fabricating or omitting intersection values while staying consistent.

There are several ways to handle this. Two are useful enough to sketch here: redundancy and polynomials.

Scaling Private Set Intersection to Billion-Element Sets (2014): KMRS14

The 2014 paper Scaling Private Set Intersection to Billion-Element Sets uses redundancy to make cheating detectable.

The trick is simple in spirit. Instead of sending one PRF output per element, each party sends several related PRF outputs per element, shuffled so the server cannot tell which values came from the same original input.

Let $t$ be a redundancy parameter.

Define:

$$
\hat a_i^j = F(K, a_i || j)
$$

where $a_i || j$ means concatenation.

Alice does this for each $a_i$:

$$
\hat A_i = (\hat a_i^1, \hat a_i^2, \ldots, \hat a_i^t).
$$

Then she shuffles all of those values before sending them to Mallory. She also stores each value locally as a key for recovering the original $a_i$.

Bob does the same:

$$
\hat B_i = (\hat b_i^1, \hat b_i^2, \ldots, \hat b_i^t).
$$

If the real original intersection has size $q$, the transformed intersection should have size

$$
|\hat A \cap \hat B| = t \cdot q.
$$

This gives Alice and Bob an integrity check. If Mallory returns only part of a redundant block, or returns values that cannot be grouped into a valid redundant structure, something is off. Either Mallory forged the output, or one of the inputs was malformed.

Could Mallory guess the correct groupings? In theory, yes. In practice, once the shuffled set is large and $t$ is chosen sanely, the odds become ridiculous. In a toy set of size 5 with $t = 3$, the chance of guessing the right block is already only 10 percent. At realistic sizes, this attack path is basically decorative.

Still, two edge cases remain.

1. Mallory always returns the empty set.
2. Mallory returns all of $\hat A$ or all of $\hat B$.

The second case can be handled with PET-style checks. The first can be handled by having Alice and Bob secretly agree on a nonempty dummy set $E$ and run the protocol on $A \cup E$ and $B \cup E$. Then the output should never be empty. If Mallory returns an empty set anyway, Mallory is caught.

Two-party PSI with an Untrusted Third Party (2019): LRG19

Now take the less brute-force route.

The 2019 paper Two-party Private Set Intersection with an Untrusted Third Party avoids redundancy and uses polynomial structure instead. The goal is to prove that the server's claimed intersection size is exactly right.

Let

$$
z = |\hat A \cap \hat B|.
$$

The protocol wants to show:

$$
z = |A \cap B|.
$$

Break that into two inequalities:

1. $z \geq |A \cap B|$
2. $z \leq |A \cap B|$

If both hold, equality follows. To prove them without revealing everything, the protocol leans on Shamir's Secret Sharing.

Take a polynomial of degree $k-1$:

$$
f(x) = a_0 + a_1x + a_2x^2 + \ldots + a_{k-1}x^{k-1}.
$$

Let the secret be

$$
a_0 = s.
$$

The remaining coefficients are sampled randomly from a finite field $\mathbb{F}$.

If you have at least $k$ points on the polynomial, you can reconstruct the polynomial and recover

$$
f(0) = s.
$$

Using interpolation:

$$
f(0) = \sum_{j=0}^{k-1} y_j \prod_{m = 0, m \neq j}^{k-1} \frac{x_m}{x_m - x_j}.
$$

If you have fewer than $k$ points, you do not know the secret. Too many polynomials still fit the points you have. There is not enough geometry to force the answer.

That is the whole magic trick.

A. Proving $z \geq |A \cap B|$

Let the claimed union size be

$$
\hat z = m + n - z.
$$

Showing

$$
\hat z \leq |A \cup B|
$$

is equivalent to showing

$$
z \geq |A \cap B|.
$$

So Alice and Bob agree on a secret $s$ and a polynomial $p(\cdot)$ of degree $\hat z - 1$ such that $p(0) = s$.

Alice sends Mallory the points

$$
V = {(\hat a_i, p(\hat a_i)) \mid \hat a_i \in \hat A}.
$$

Bob sends

$$
W = {(\hat b_i, p(\hat b_i)) \mid \hat b_i \in \hat B}.
$$

Mallory can recover $p$ only if there are enough distinct points from the union. If Mallory understated the intersection size, the claimed union size becomes too large. Mallory would need more independent points than the actual union provides. So Mallory cannot reconstruct $p(0)$.

If Mallory can reconstruct the secret, the claimed $z$ cannot be smaller than the true intersection size.

Therefore:

$$
z \geq |A \cap B|.
$$

B. Proving $z \leq |A \cap B|$

The reverse direction is trickier.

For the union proof, Alice and Bob could safely give points for all their elements. For the intersection proof, they cannot. If they handed Mallory full polynomial points for every element, Mallory would learn too much about non-intersection values.

So the protocol splits each useful polynomial point into two shares. Mallory can assemble a valid point only when the same blinded element appears on both sides.

Alice and Bob agree on two PRF keys, $k_1$ and $k_2$.

They compute:

$$
\hat A = {F(k_1, a_i) \mid a_i \in A}
$$

and

$$
\hat B = {F(k_1, b_i) \mid b_i \in B}.
$$

Mallory computes the claimed intersection and says its size is $z$.

Now Alice and Bob choose a secret $s_1$ and a polynomial $p_1(\cdot)$ of degree $z-1$ with $p_1(0) = s_1$.

For an element $\hat x$, the value $p_1(\hat x)$ is split like this:

$$
p_1(\hat x) = F(k_2, \hat x) \oplus (F(k_2, \hat x) \oplus p_1(\hat x)).
$$

The first share alone reveals nothing. The second share alone reveals nothing. Together, and only together, they reconstruct $p_1(\hat x)$.

So the flow becomes:

1. Alice sends Mallory shares of the form $(\hat a_i, F(k_2, \hat a_i))$.
2. Bob sends Mallory shares of the form $(\hat b_i, F(k_2, \hat b_i) \oplus p_1(\hat b_i))$.
3. Mallory can combine shares only when $\hat a_i = \hat b_i$.

That means Mallory gets valid points on $p_1$ only for actual intersection elements.

If Mallory lies and claims

$$
z > |A \cap B|,
$$

then $p_1$ has degree $z-1$, and Mallory needs $z$ independent points to reconstruct $s_1$. But there are fewer than $z$ real intersection points. Mallory cannot recover the secret.

Once Alice and Bob force Mallory to commit before revealing verification material, Mallory cannot wait, learn the answer, and fake success afterward.

Therefore:

$$
z \leq |A \cap B|.
$$

Combine both inequalities:

$$
z = |A \cap B|.
$$

Mallory cannot shrink the intersection. Mallory cannot inflate it. Mallory cannot invent elements outside the intersection without at least one party noticing locally. Mallory cannot omit elements without contradicting the proved cardinality.

Apple's CSAM System Is Not Plain PSI

Plain PSI is a symmetrical problem. Alice and Bob compare sets. Both learn the intersection. Nobody learns non-intersections.

Apple's proposed iCloud Photos system needed something stranger.

Map the roles like this:

• Alice: the user's device
• Bob: the known CSAM hash database
• Mallory: Apple's iCloud server

Already, this is awkward. In ordinary PSI, Alice might learn, "this item is in Bob's set." That would be unacceptable here. If a malicious user could query the system and discover which images are in the CSAM database, the database itself would become an oracle.

Apple also did not want the server to learn about one-off matches immediately. A single accidental collision should not trigger meaningful disclosure. So the system needed threshold behavior.

That gives us four requirements:

1. Server-only output. The device should not learn which local images matched the hidden CSAM database.
2. Threshold disclosure. Apple should not learn useful information from a single match. It should learn more only after enough matches accumulate in the same iCloud Photos account.
3. Dynamic inputs. A user's photo library is not a fixed set. Photos arrive over time.
4. Database consistency. Apple should not be able to quietly ship one database to one user and another database to another user. If it could, targeted scanning would become technically boring.

Benny Pinkas described Apple's construction as a specialized PSI variant that combines PSI-style matching with threshold disclosure. His technical explanation is worth reading if one enjoys staring into protocol machinery until the machinery stares back.

The rough pipeline had three layers:

1. Perceptual hashing. Convert an image into a NeuralHash value.
2. Private Set Intersection. Check whether that hash appears in the blinded CSAM database.
3. Threshold Secret Sharing. Reveal useful information only after enough real matches accumulate.

A. NeuralHash: hashing by visual similarity

Cryptographic hash functions are deliberately brittle. Change one pixel, crop an edge, recompress the file, and a normal hash like SHA-256 changes completely. That is exactly what you want for integrity checking. It is not what you want for known-image detection.

If the goal is to detect the same underlying image after resizing, recompression, color adjustment, or small edits, you need a perceptual hash. A perceptual hash tries to map visually similar images to the same or nearby fingerprints.

Apple's NeuralHash belonged to that family. The device would run a machine-learning model over an image and produce a hash intended to remain stable under ordinary transformations.

Here is the first problem: a perceptual hash is not a cryptographic hash.

It is closer to a classifier or embedding function. It can produce false positives. It can produce false negatives. It can be attacked.

Research on NeuralHash and deep perceptual hashing found that adversaries could manipulate hash values through small image changes, forcing or avoiding collisions in some settings. The authors argued that deep perceptual hashing, at least in the form they studied, was not robust enough for client-side scanning from a privacy perspective. See the NeuralHash paper on arXiv.

PSI does not fix this. PSI can privately compare two sets. It cannot certify that an image fingerprint correctly represents the meaning of an image. If NeuralHash outputs the wrong value, PSI will compare the wrong value very privately.

Garbage in, cryptographically polished garbage out.

B. The hidden database problem

For the user's device to compare a photo against known CSAM hashes, it needs some form of the CSAM database.

But Apple obviously cannot ship the raw database to every iPhone. If it did, attackers could extract the list and learn what is in it. That would harm investigations and victims.

So the database has to be present enough for matching, but hidden enough that the device cannot read it.

Apple's system used a blinded database. In simplified terms:

1. Apple starts with a set of known CSAM perceptual hashes.
2. The database is transformed cryptographically so entries look random to the device.
3. The transformed table is distributed to devices.
4. The device computes NeuralHash for a local photo.
5. The device uses that value to interact with the blinded table.
6. If the photo hash is in the database, the device can form cryptographic material that Apple's server may later interpret.
7. If it is not in the database, the resulting material remains useless noise to Apple.

The device runs NeuralHash, checks it against a table of blinded CSAM hashes, creates an encryption key from the relevant table entry, and uses that to create a safety voucher. Their technical explainer lays out the process.

The neat part is also the unpleasant part. The device can use the database without really knowing the database. Apple can receive outputs without immediately learning every non-match.

C. Safety vouchers: sealed reports uploaded with photos

Before a photo entered iCloud Photos, the device would perform on-device matching against known CSAM hashes. Then it would create a cryptographic safety voucher and upload that voucher with the image. The voucher is encoding the match result together with additional encrypted data about the image. See Lawfare's overview.

Conceptually, a safety voucher included:

1. PSI-related cryptographic material.
2. The photo's NeuralHash.
3. Encrypted image-related data, including a visual derivative for later human review.
4. A secret share used for threshold reconstruction.
5. Encryption layers that keep the contents unreadable unless the threshold condition is satisfied.

The visual derivative is not the same as saying Apple immediately sees the original photo. But it is still a sensitive representation of the image, created in advance, sitting behind cryptographic glass.

The intended flow was:

1. If a photo did not match the known CSAM database, the voucher was effectively unreadable noise to Apple.
2. If a photo did match, the voucher could contribute a valid secret share.
3. One valid share was not enough.
4. Only after the account crossed a threshold number of matches could Apple reconstruct the key needed to interpret the relevant voucher contents.
5. Apple would then perform human review and, if confirmed, report to NCMEC. See this technical explainer.

This is much more privacy-preserving than naive server-side scanning. The server is not simply opening every photo. Non-matches are meant to stay private. One match is meant to be insufficient. Even a pile of apparent matches should not necessarily reveal the useful material.

But "better than naive scanning" is not the same as "not a problem."

A sealed report is still a report. The fact that it opens only after a threshold does not erase the fact that your device produced it.

D. Synthetic match vouchers: hiding the count before the threshold

There is one more subtlety.

Suppose Apple could see that an account had 1 real match, then 2, then 3, then 29. Even if the threshold were 30, that information would already be radioactive. "This account is one match away from review" is not a harmless fact.

Apple tried to blunt this with synthetic match vouchers.

A synthetic match voucher is a fake-looking match. It is not a real CSAM match, but it makes the server's early view noisier. Real match vouchers contain valid secret shares. Synthetic ones contain random shares. Before threshold reconstruction succeeds, Apple should not be able to cleanly distinguish how many apparent matches are real.

So the system stacks protections:

• NeuralHash converts images into perceptual hashes.
• A blinded database hides the CSAM list from the device.
• PSI-like matching lets the device process membership without raw disclosure.
• Safety vouchers seal match-related evidence.
• Threshold Secret Sharing prevents Apple from opening meaningful contents too early.
• Synthetic match vouchers blur the count before the threshold.
• Human review sits at the end before reporting.

That is precisely why people were worried. A clumsy scanner is easy to condemn. A cryptographically polite scanner is harder. It gives the institution a better answer to the first privacy objection while leaving the deeper governance objection alive.

In other words, this was not merely a CSAM scanner. It was a general pattern for endpoint-based, privacy-preserving, thresholded detection.

A nicer machine for doing a very dangerous thing.

Communication Safety Is Not PSI

At this point, it is tempting to throw every Apple child-safety feature into one bucket and call the bucket client-side scanning. That is rhetorically convenient. It is technically sloppy.

The proposed iCloud Photos CSAM Detection system used NeuralHash, PSI-style matching, safety vouchers, and threshold secret sharing.

Communication Safety does not: it uses on-device machine learning to detect whether photos or videos appear to contain nudity. Apple says the device blurs the content, warns the child, and provides resources. Apple also says it does not receive an indication that nudity was detected and does not access the photos or videos. See Apple's Communication Safety page.

The feature expanded beyond Messages. Apple lists support across Messages, AirDrop, Contact Posters, FaceTime calls and video messages, shared photo albums, Apple Watch, Vision Pro, and some third-party app share flows.

In 2026, Apple previewed the next step: Communication Safety would also intervene to block gore or violent content in shared images or videos. See Apple's announcement.

So Is This Really a Privacy Nightmare?

Technically, Apple's defense of the proposed iCloud Photos system was not nonsense. It really was not the same as uploading every photo to a server and running a plaintext scan. It really did try to hide non-matches. It really did try to prevent Apple from learning useful information before a threshold. It really did use serious cryptographic ideas.

Compared with naive server-side scanning, the PSI design is dramatically more privacy-preserving.

But privacy arguments are not everything. Those questions are, perhaps, more important than the technical privacy arguments themselves:

1. Who controls the database?
2. Who audits what enters it?
3. Can different users, countries, or account classes receive different databases?
4. Can a government force Apple to add non-CSAM material?
5. Can users verify what their devices are matching against?
6. Can researchers independently inspect the system?
7. Can the same architecture be reused for documents, audio, video, political imagery, copyrighted media, or biometric signatures?

Cryptography does not answer those questions. Cryptography computes a function safely. It does not tell us which function should exist.

Put CSAM hashes into the database, and the system searches for known CSAM. Put protest posters into the database, and it searches for protest posters. Put leaked documents into the database, and it searches for leaked documents. Put religious material into the database, and it searches for religious material. Put copyright fingerprints into the database, and the same architecture begins to look like a rights-enforcement machine.

The math does not flinch - It just computes.

Apple proposed safeguards. They said the database would be built from hashes supplied by child-safety organizations, that the same database would be shipped broadly to prevent targeted attacks, and that human review would occur before reporting. Those safeguards matter.

But "Apple promises" and "the public can verify" are very different sentences.

Operating-system updates can be region-specific. Feature flags can be jurisdiction-specific. Legal demands can be secret. The database can be blinded, and that blinding, while useful for privacy, can also make external auditing harder.

That is the paradox: the system was designed so users would not have to trust Apple with every photo.

Politically, it asked users to trust Apple more than ever.

What Was Actually Wrong With the Apple PSI Proposal?

The iCloud Photos scanner is dead. Keep that fact fixed in memory, because otherwise the debate turns into ghost hunting.

But the architecture still matters for three reasons.

First, it was a serious prototype of privacy-preserving surveillance. That phrase sounds self-contradictory, but that is the point. Apple tried to design a system that would not reveal non-matches and would not expose matches too early. Surveillance with less data is better than surveillance with more data. It is still surveillance.

Second, it showed the limit of cryptographic trust. PSI, secret sharing, blinding, and thresholds can reduce exposure inside a defined threat model. They cannot decide whether the database is legitimate. They cannot resist every legal order. They cannot guarantee public auditability. They cannot protect against a future product meeting where someone says, "Could we reuse this for a different category?"

Third, the broader endpoint-classification trend did not disappear. Communication Safety is not PSI, but it lives in the same neighborhood: the device analyzes user content and changes behavior based on what it finds. Today's classifier looks for nudity. The new version also touches gore and violence. Some jurisdiction tomorrow may ask for a different category.

That does not mean all on-device safety features are evil. It means the governance questions must be asked at the device layer, not merely at the server layer.

• Is the analysis entirely on-device?
• Does any result leave the device?
• Can the user disable it?
• Is the feature different for children and adults?
• Who controls the model or database?
• Can the model or database vary by region?
• Can independent researchers audit the behavior?
• Is there a public log or commitment scheme for database changes?
• What happens when a government asks for a new detection category?

These are not anti-child-safety questions. They are anti-unaccountable-infrastructure questions.

CSAM should be fought. The real dispute is where the enforcement layer sits, who controls it, and what kind of democratic and technical accountability surrounds it.

Conclusion: Not Black Magic, Not Harmless Magic Either

The Apple PSI system was not black magic.

Under the hood, it was a stack of known ideas: perceptual hashing, blinded databases, PSI-style matching, safety vouchers, threshold secret sharing, synthetic noise, and human review. The proposed system was far more subtle than "Apple reads all your photos." In the narrow cryptographic sense, Apple was not simply lying.

And Apple did not ship the iCloud Photos CSAM Detection system. But the fear was not imaginary.

A sufficiently good privacy-preserving scanner can be more politically dangerous than a bad one, because it makes the scanner easier to justify. Non-matches stay private. Matches stay hidden until a threshold. The database is blinded. The server learns less. The whole thing begins to sound reasonable.

Then the real question slips in through the side door: "should a user's own device be conscripted into comparing private content against a hidden policy database?"

That question cannot be answered by a polynomial. It cannot be settled by a PRF. It cannot be solved by a threshold.

Mathematics can tell us whether $A \cap B$ was computed safely. It cannot tell us what belongs in $B$.

So the most accurate judgment is probably: "Apple was not lying. But the critics were not hallucinating either."

The proposed system did not let Apple casually browse everyone's photos. It also created a general pattern for endpoint-based detection against a database the user could not inspect. That is a powerful precedent. Killing the iCloud Photos scanner did not kill the underlying political tension, because Communication Safety and stronger iCloud encryption both keep pointing back to the same place: the device.

In an encrypted society, the fight does not end when the server loses access. It moves closer to the user. ■