https://sneak.berlin/20230115/macos-scans-your-local-files-now/
macOS ventura 13.1 scans your local images using network API requests to apple when browsing local image files in the finder now.
be advised.
@bot i was browsing images in the finder and it made network API requests, that's how. there's a screenshot of the image and the API access in the article.
@sneak @bot@seal.cafe That doesn't really tell you anything. For example, this could be an attempt to make a one-time download of an ML model used for locally-running analysis, similar to how enabling certain voice processing features on iOS triggers a download of language models that will then be used on-device.
@yProd that's correct. ML models used to classify images. like the images being scanned by mediaanalysisd.
i don't think Apple's planned LEO-assist features were ever going to run anywhere but on-device.
@sneak Well, the CSAM scanning would report to Apple (which would forward to LEOs). From a privacy perspective, local analysis for local use is something completely different, and about as right or wrong as Spotlight “scanning” all your documents (to build a search index).
@yProd spotlight can be turned off and directories excluded. what ML features are in use when using spacebar quicklook in the finder? literally all i asked it to do is display the image. at best it is a bug.
@sneak Live Text would come to mind, as one example. Maybe the image contains text you may want to copy?
@yProd this is 100% apple scanning my files. apple controls my own, local computer via macOS. there's no data available now to support "all results staying local" as we have already established that the process scanning the files is making network API requests.
@sneak Talking about Live Text here, which is documented to be running locally.
If you're 100% this is Apple scanning your files, prove it – which means checking which data is being transmitted, not just establishing that an Apple daemon is trying to connect to Apple somehow, which is very unsurprising.
(And in the end, if you believe Apple is potentially evil and its statements cannot be trusted, you must stop using macOS. Your firewall does not help, it only sees requests through macOS's API, which could absolutely hide internal requests if it wanted! Using an OS always requires a certain level of trust with its developers.)
@yProd scanning does not mean transmitting. transmission could be conditional on specific features! it could be all-local until it detects something it doesn't like, then and only then does it make a network request. we know it is scanning local files that are not involved in icloud/Photos.app, and we know it's making network requests. it's a single line of code to connect those two. you cannot assert that that code is not on my machine.
@yProd all people vulnerable to FBI coercion are potentially evil as they can be forced to do things against their will.
@sneak Again, this being live text is a guess. If it is the cause, apparently, you can uncheck “Select text in images” in macOS's Language & Region settings to disable it.
Keep in mind this is not Apple somehow scanning your files, it is your own, local computer doing it (and with all results staying local). If you believe this is effectively the same, that's OK of course, but I'll have to respectfully disagree.