NN-512 generates custom code for all the operations, custom units of work for the threads, custom code around tensor edges, everything is fused and unrolled and customized. If they can deduce the network graph specification from the AVX-512 code, I will be astonished.
If you can do it, show me. But I know you can't.
Anyone who cares about model privacy will use their own variant of a tool like NN-512. It's security through obscurity, but that's the best you can hope for if you are distributing an executable.
I firmly believe that security by obscurity should be given more credit than is done normally. If you are a pretty uninteresting target and you want to protect your binaries, making them too tough of a nut to crack in comparison to the motivation of the reverse engineer is a valid strategy.
It's not that there's no value in security-by-obscurity. The issue is when it's the only control. I agree that some are too quick to dismiss operational security controls.
it's certainly valid. Obscurity is cheap and easy.
The only problem is when it's the _only_ security for certain types of threat models that require defence in depth - such as credentials in authentication.
I've been out of the cracking scene for over a decade now, but I expect that to be none other than a challenge, having seen how far publicly available decompilers have progressed.
Even if you had the C code available to you, you would have a hard time producing the input graph.
Good luck reverse engineering it after GCC has compiled it!
NN-512 has an incredibly flexible code generator. It can easily be tweaked to produce completely different code for the same convolution, so everyone can apply their own twist to defeat the reverse engineers ("the intellectual property thieves").
You're describing every single obfuscation scheme, they all get defeated. And you don't need to find the original graph either, they may be equivalent ones and that could work too.
Widevine has been broken at least a few times, including by recovering the private key from its white-box implementation: https://github.com/tomer8007/widevine-l3-decryptor/wiki/Reve.... Note that the write-up says it was the "old" version, but that's relative to the date of the write-up. Google overhauled Widevine after he broke it.
I'm less familiar with shielding data like this, but historically things like VMProtect and Themida were the standard for shielding programs. These offer a degree of resistance to automation, but a determined human will eventually figure them out, and then automation usually follows anyway. Syntia did this for VMProtect and Themida: https://www.usenix.org/system/files/conference/usenixsecurit....
Edit: A quick search points to Widevine continuing to have issues. Two more recent write-ups:
It is a lot of work, but I wouldn't say it is exactly difficult... I never bothered to automate it, and so I didn't finish the one I was doing, but I was under the impression that Pod2G's team (which used a photo of me doing it a bit on a blackboard in their presentation) did, however?
You just don't need to is the thing (if you are in a position to not care about copyright law; I did care, sadly): you can almost always just lift the code--with all its obfuscation intact--and run it in isolation on your input, which more directly undermines the entire premise of the technique.
That presentation seems confused since PRISM is not "a mass surveillance program" or "an alliance with American firms", it's a database the government puts the results of subpoenas in. Of course, the protocol is still weak to an evil key server.
Well, the obfustication is still pretty good if it's 9 years ahead of attacks.
> That presentation seems confused since PRISM is not "a mass surveillance program" or "an alliance with American firms", it's a database the government puts the results of subpoenas in.
I feel like this is a terrible mischaracterization of PRISM, even if it is almost true. The NSA deployed hardware (following demand letters) to service providers and collected large swaths of traffic based on various types of keyword and attribute matches. This was then put in a big searchable database.
That's XKeyscore, not PRISM (sorry that sounds like a nitpick…). But no NSA program ever involved secret cooperation from Apple or Google; that's why they were spying on Google datacenter traffic by tapping it. Why telecom companies did cooperate seems like a cultural question.
(Remember they both said explicitly said they never cooperated, and it's illegal for companies to lie to you, or for the government to make them lie to you. If they were lying, you can sue them for securities fraud. They can refuse to answer questions, of course, which is the usual approach when they don't want to talk about something, but that's quite different from explicit denials.)
> But no NSA program ever involved secret cooperation from Apple or Google
This isn't true. PRISM collection involved demands to internet companies ordered by secret courts under section 702 of FISA. XKeyScore involved secret cooperation from telcos.
I invite you to review some of the documents curated by the Washington Post in response to the Snowden disclosures.
Your argument seems like it's just parroting the DNI's factsheet, which is known to whitewash the programs involved (and is even more charitable to the program than the DNI's own factsheet).
I can just separate your obfuscated AI execution into a DLL. And then I call that DLL with lots of randomly generated input data and estimate the numerical gradients from that. And now I have everything I need to copy things over into a similar NN architecture.
Yes, it might take a few days to evaluate everything, but CPU time is cheap compared to research and employees needed to reverse engineer your implementation.
That said, NN-512 is great because it produces optimized CPU code, thus making deployment cheaper.
Convolutions are fused into convolutions, elementwise operations are fused into convolutions, everything is inlined except where function calls are needed for pthread work units (and those work units are all custom/arbitrary).
I don't have a ms windows pc available nor the time to setup cross compilation for one rn. (Assuming you meant an executable file for one of those with 'exe').
However you ahould be able be able to compile one for yourself by downloading, from e.g. https://nn-512.com/browse/DenseNet121, one of the generated C files and compiling it with GCC[0]. It shouldn't require any special dependencies beside AVX support on your CPU.
Edit: Regarding general decompilation for neural networks this project might be interesting[1]
https://NN-512.com
NN-512 generates custom code for all the operations, custom units of work for the threads, custom code around tensor edges, everything is fused and unrolled and customized. If they can deduce the network graph specification from the AVX-512 code, I will be astonished.
If you can do it, show me. But I know you can't.
Anyone who cares about model privacy will use their own variant of a tool like NN-512. It's security through obscurity, but that's the best you can hope for if you are distributing an executable.