Myths about /dev/urandom
This article's premise seems to come down to the key argument under the section "What's wrong with blocking?", which seems to summarize as "security should not get in the way of the developer: if it does, they will turn off the security". This frankly just seems silly: if the developer cannot be trusted to not do something insane (seriously? "patching out the call to random()"?!), then they should not be developing :/.This is like saying that using consistent databases for things that require consistency is also "wrong", because the developer is going to add some ludicrous poorly-implemented caching layer in front on production as the performance dips, leading you to have a poorly-controlled inconsistent system anyway. It's an argument I appreciate, but one that indicates a deeper problem of developers thinking they can fix things they can't....and that's the only argument in the entire document for why /dev/random is bad: everything else is a justification for why /dev/urandom should be considered a reasonable alternative to /dev/random, most of which is difficult to confirm or deny unless you are a cryptographer. I mean, from a naive standpoint, it would sound like "putting all your eggs in one basket" could lead you to a problem, but how would I know, right?Helpfully, the article claims to quote cryptographers on the matter. However, and this pretty much damns this article for me: he takes these quotes out of context. If you read the original sources, Daniel Bernstein makes it clear that /dev/urandom is broken on Linux (more on this later), and Thomas Pornin was responding to a user that was simply trying to get unique random values (where /dev/urandom is obvious).As for Thomas Ptacek's emphatic "use /dev/urandom" repetition, his article was about people attempting to use user-land replacements for the kernel implementations, not people who were using /dev/random. In essence, he's trying to make the argument that people shouldn't be so afraid of /dev/urandom so as to use something clearly worse. He does state that /dev/random is overrated, but correctly points out its flaws.Back to this article: after a ton of argumentation that the people suggesting that there is a key difference between /dev/random and /dev/urandom are inherently being misleading, the author gets around to quoting a section of the man page that sounds perfectly reasonable, and even is willing to cede that there is no issue using /dev/random for those use cases. I think the author is just mad because some service is slow. :/Now, here's what I'll leave you with: an argument why this author's article is not just wrong but promoting beliefs that are actively harmful, an argument that can I hope can be believed even if you aren't a cryptographer and does not rely on me being a cryptographer: its an argument based on practical concerns only, with a demonstration of the problem not just in theory, but that happened to real people/developers.[edit: Apparently, I misremembered this Android issue, so when I pulled sources for it I ended up with something that is not actually an issue with /dev/urandom. I have added details on a different situation where /dev/urandom actually was the culprit, but the argument isn't quite as strong. I was hoping for an argument that anyone would believe without invoking even a single hypothetical, but sadly I screwed up: I'm sorry for the confusion :/. I think the new example still makes a strong, though not impervious, point.]So, if the author got his way, then all source code everywhere would be referencing /dev/urandom instead of /dev/random for all use cases (including "long-lived keys"). However, the author also admits that /dev/urandom "isn't perfect", and that on Linux it never blocks, even when it really should. It is then claimed that the correct solution to this problem is that you should seed virtual machines with entropy before using them.Look: that just isn't always possible. I say this not from theory, but from the trenches of information security. On Android, some poor developer made java.security.SecureRandom use /dev/urandom. This meant that all Java libraries that rely on random numbers were getting ones from urandom, even when they claimed that they wanted "cryptographically strong" ones. Put short: the author's dream come true, codified into the library.Sadly, users expect to be able to use Android devices soon after they boot, the user is not in control of the entropy sources, and the people who build the kernel drivers for these devices often didn't include enough entropy sources to fill the pool faster. This meant that all of this code was now broken: instead of blocking until "seriously-random numbers" could be generated, it would get "reasonably random-looking ones".This is not a theoretical issue: people built bitcoin wallet applications sitting on top of these APIs, and some of the people using those wallets ended up with bitcoin addresses secured by keys that were painfully predictable across usages on different devices (the specific concern people bring up using /dev/urandom!). This was looked into, traced back to this design flaw, and Google even commented on the situation.[edit: For clarity, this is the part that is wrong; as pointed out in the author's rebuttle to my comment, SecureRandom was actually doing something ludicrous here--the kind of thing that Thomas Ptacek was angry about ;P--and thereby Google's own article, which I had pulled up as a source for something I had apparently misremembered, claims that developers could consider using /dev/random or /dev/urandom to replace it.]http://android-developers.blogspot.com/2013/08/some-securera...[edit: Thankfully, there is another situation I can cite: the attack on routers from a year or two ago that involved low-entropy keys that were generated at boot. It is the same issue, but what makes this a weaker argument is that the user is now the developer, and the developer is actually in control of the device and could have fixed the entropy pool as stated by both this author and by Daniel Berstein's full comment.][edit:]So, in this new example, the key issue is that a bunch of embedded systems--routers and firewalls from companies such as Cisco--were generating keys to secure their configuration portals and administrator consoles using /dev/urandom. This was done, on first boot (when there would be no entropy available), and the result was that many RSA keys were generating the exact same input prime numbers. If you are trying to factor a large semi-prime, and you have another large semi-prime that shares a factor, you can break it apart very easily with simple math.Some researchers went through to determine why this happened, and their conclusion was that reliance by library developers on /dev/urandom, being used on devices and at moments where /dev/urandom didn't end up working very well, was the "root cause" of the issue, and made a recommendation to library developers that usage of less secure mechanisms should be a non-default option and that any tradeoffs in its selection should be clearly documented, lest security be contingent on factors that do not immediately seem related to security.https://factorable.net/weakkeys12.extended.pdf> In the final component of our study, we experimentally explore the root causes of these vulnerabilities by investigating several of the most common open-source software components from the population of vulnerable devices (Section 5). Based on the devices we identified, it is clear that no one implementation is solely responsible, but we are able to reproduce the vulnerabilities in plausible software configurations. Every software package we examined relies on /dev/urandom to generate cryptographic keys; however, we find that Linux’s random number generator (RNG) can exhibit a boot-time entropy hole that causes urandom to produce deterministic output under conditions likely to occur in headless and embedded devices. In experiments with OpenSSL and Dropbear SSH, we show how repeated output from the system RNG can lead not only to repeated long-term keys but also to factorable RSA keys and repeated DSA ephemeral keys due to the behavior of application-specific entropy pools.> When we disabled entropy sources that might be unavailable on a headless or embedded device, the Linux RNG produced the same predictable stream on every boot. The only variation we observed over 1,000 boots was the position in the stream where sshd read from urandom.> On stock Ubuntu systems, these risks are somewhat mitigated: TLS keys must be generated manually, and OpenSSH host keys are generated during package installation, which is likely to be late in the install process, giving the system time to collect sufficient entropy. However, on the Fedora, Red Hat Enterprise Linux (RHEL), and CentOS Linux distributions, OpenSSH is installed by default, and host keys are generated on first boot. We experimented further with RHEL 5 and 6 to determine whether host keys on these systems might be compromised, and observed that sufficient entropy had been collected at the time of key generation (due to greater disk activity than with Ubuntu Server) by a slim margin. We believe that most server systems running these distributions are safe, particularly since they likely have multiple cores and gather additional entropy from physical concurrency. However, it is possible that other distributions and customized installations do not collect sufficient entropy on startup and generate weak keys on first boot.> For library developers: Default to the most secure configuration. Both OpenSSL and Dropbear default to using /dev/urandom instead of /dev/random, and Dropbear defaults to using a less secure DSA signature randomness technique even though a more secure technique is available as an option. In general, cryptographic libraries should default to using the most secure mechanisms available. If the library provides fallback options, the documentation should make the trade-offs clear.[/edit]Of course, this is covered by the idea that these devices were not properly configured; but that's kind of the point, right? What everyone is saying about /dev/urandom is that it isn't guaranteed to produce anything "seriously" random; it happens to, most of the time, on most operating systems, produce something incredibly random, but some times, on some operating systems, it might produce something that is barely random at all.This means that if everyone switches to using /dev/urandom in their code, a ton of libraries are going to work on some systems, and not on others; they might work on a properly configured server, but not on an Android device, or might work on one cloud provider, but not another... this is insane... if I download "secure off-the-record messaging implementation library for Linux", am I going to expect it to fail like this?Certainly, the developers this article seems to believe are both common and acting reasonably enough to be worked around by people writing code--the ones who are supposedly going around patching random() out their libraries ;P--are not going to realize their code isn't secure, or that they they need to configure their kernel or operating system distribution to make it secure; but this is even going to burn the people reasonable enough to download code and not try to hack it :/.[edit: And, to close the earlier edit: clearly the people who were building these embedded routers didn't realize these tradeoffs. The author of this article is thereby asking developers to be cognizant of how these different pseudo-random number generators operate and design their entire systems in ways that will make specific sub-components, components they probably don't understand well and "just want to be secure", continue to work correctly under potentially-modified assumptions. While they technically can do this--unlike the sadly-flawed Android argument I wanted to make, where users and developers had no control--this seems like a much higher bar than keeping people from taking random() and patching it out entirely due to a performance benefit, and comes down to a simple heuristic.]