This backdoor was basically immediately identified by security researchers the year it was embedded in the standard. As you can read in the Wikipedia article:
Bruce Schneier concluded shortly after standardization that the “rather obvious” backdoor (along with other deficiencies) would mean that nobody would use Dual_EC_DRBG.
I can’t really figure out what you mean by the DES recommended magic numbers. There were some magic numbers in DES that were used for defense against the differential cryptanalysis technique. Which I do agree is probably the single strongest example we have of an NSA lead, though it’s important to note that that technique was developed at IBM, and then given to the NSA, and not developed internally at the NSA.
To be clear, a 30 (!) year lead seems absolutely impossible to me. A 3 year broad lead seems maybe plausible to me, with a 10 year lead in some very narrow specific subset of the field that gets relatively little attention (in the same way research groups can sometimes pull ahead in a specific subset of the field that they are investing heavily in).
I have never talked to a security researcher who would consider 30 years remotely plausible. The usual impression that I’ve gotten from talking to security researchers is that the NSA has some interesting techniques and probably a variety of backdoors, which they primarily installed not by technological advantage but by political maneuvering, but that in overall competence they are probably behind the academic field, and almost certainly not very far ahead.
though it’s important to note that that technique was developed at IBM, and then given to the NSA, and not developed internally at the NSA.
So I think this is actually a really important point. I think by default the NSA can contract out various tasks to industry professionals and academics and on average get results back from them that are better than what they could have done internally. The differential cryptoanalysis situation is a key example of that. IBM could have instead been contracted by some random other group and developed the technology for them instead, which means that the NSA had basically no lead in cryptography over IBM.
I think 30 years is an overstatement, thought it’s hard to quantify. However, I can think of a few things that makes me think this gap is likely to exist, and be significant in cryptography, and even more specifically in cryptanalysis. For hacking, the gap is clearly smaller, but a still nontrivial amount—perhaps 2 years.
I have never heard this and would extremely surprised by this. Like, willing to take a 15:1 bet on this, at least. Probably more.
Do you have a source for this?
Ugh, I’d have to dig things up, but some things that come to mind that could be confirmed by looking that I count as evidence of this:
lag to figuring out the thing about the DES recommended magic numbers vs. when they were given out
NSA lead on public key crypto and sending agents to discourage mathematicians from publishing (this one was likely shorter because it was earlier)
lag on figuring out the problems with elliptic curve during which the NSA encouraged its use
Even if all of these turn out to be quite significant, that would at most imply a lead of something like 5 years.
The elliptic curve one doesn’t strike me at all like the NSA had a big lead. You are probably referring to this backdoor:
https://en.wikipedia.org/wiki/Dual_EC_DRBG
This backdoor was basically immediately identified by security researchers the year it was embedded in the standard. As you can read in the Wikipedia article:
I can’t really figure out what you mean by the DES recommended magic numbers. There were some magic numbers in DES that were used for defense against the differential cryptanalysis technique. Which I do agree is probably the single strongest example we have of an NSA lead, though it’s important to note that that technique was developed at IBM, and then given to the NSA, and not developed internally at the NSA.
To be clear, a 30 (!) year lead seems absolutely impossible to me. A 3 year broad lead seems maybe plausible to me, with a 10 year lead in some very narrow specific subset of the field that gets relatively little attention (in the same way research groups can sometimes pull ahead in a specific subset of the field that they are investing heavily in).
I have never talked to a security researcher who would consider 30 years remotely plausible. The usual impression that I’ve gotten from talking to security researchers is that the NSA has some interesting techniques and probably a variety of backdoors, which they primarily installed not by technological advantage but by political maneuvering, but that in overall competence they are probably behind the academic field, and almost certainly not very far ahead.
So I think this is actually a really important point. I think by default the NSA can contract out various tasks to industry professionals and academics and on average get results back from them that are better than what they could have done internally. The differential cryptoanalysis situation is a key example of that. IBM could have instead been contracted by some random other group and developed the technology for them instead, which means that the NSA had basically no lead in cryptography over IBM.
I think 30 years is an overstatement, thought it’s hard to quantify. However, I can think of a few things that makes me think this gap is likely to exist, and be significant in cryptography, and even more specifically in cryptanalysis. For hacking, the gap is clearly smaller, but a still nontrivial amount—perhaps 2 years.