The global economy needs true randomness to encrypt messages and make sure elections are honest. But not all randomness is random enough, and humans and computers alike are really bad at generating it. So we turn to natural sources like seismic waves, radioactive decay, and lava lamps (yes, lava lamps) to generate it for us.
Sponsored by American Express
Listen on: Apple Podcasts | Spotify | Google | Stitcher
Featuring
Kira Bindrim is the host of the Quartz Obsession podcast. She is an executive editor who works on global newsroom coverage and email products. She is obsessed with reality TV.
Amanda Shendruk is a visual journalist who explores narrative at the intersections of code, data, and design. She is obsessed with creepy books and forest walks along tiny streams.
Show notes
Amanda Shendruk’s article about Cloudflare’s wall of entropy
League of Entropy public randomness beacon
This episode uses the following sounds from freesound.org:
Office ambience.mp3 by Nightwatcher98
Lava Bubbles by everythingsounds
Slow Pulse #2 by knova
Beep Loop by everythingsounds
bleep bloops by graham_makes
Lottery Ticket Scratch.wav by Student023
Keyboard Typing Sounds by zrrion_the_insect
Data Transfer.wav by Nakhas
Transcript
Kira Bindrim: If you step inside the headquarters of internet security company Cloudflare, you’ll spot something called the wall of entropy. It’s four long rows, each with 20 lava lamps, their hot wax bubbles moving in a soothing rhythm. Overhead, a camera records the wall. But not just the lamps—it captures the change in sunlight over the course of the day and people wandering through the space.
The wall of entropy isn’t just quirky office decor. It’s part of an elaborate system Cloudflare uses to create a resource crucial to its business: randomness, specifically random numbers. Without them, Cloudflare wouldn’t be able to provide security for around 10% of the web. Randomness isn’t only crucial for Cloudflare. It’s also used to keep elections honest, make sure the lottery is fair, produce scientifically accurate simulations, and secure cryptocurrency.
Good randomness is pretty important to the global economy. But it’s also really hard to find. Not all randomness is random enough. And computers, which are good at a lot of things, are not great at creating randomness. That means to effectively power our modern world, we actually have to look to the natural one.
This is the Quartz Obsession, a podcast that explores the fascinating backstories behind everyday ideas and what they tell us about the global economy. I’m your host Kira Bindrim. Today: randomness, and why just enough chaos is critical.
I am joined now by Amanda Shendruk, who is a data and graphics reporter with Quartz based in Edinburgh. Thank you for joining me, Amanda. How is Scotland today?
Amanda Shendruk: Hi, Kira. You know, it’s a lovely sunny day in Edinburgh, I’m in a cozy little studio just down the street from a castle. So I really can’t complain.
Kira Bindrim: So for your job, you spend a lot of time with numbers. So I’m kind of assuming that’s how you got into randomness. But I’d love to hear how this sort of esoteric topic became a point of interest for you?
Amanda Shendruk: Well, it’s funny that you mentioned Cloudflare at the beginning, because the first time I really started thinking about it, I was looking at sort of a guidebook to quirky things to do in San Francisco. And they mentioned that you could go into the Cloudflare headquarters and look at their wall of entropy. And while I never ended up doing that, I was like, ‘Wait, what?’ and started looking around, and then fell down a real rabbit hole into randomness.
Kira Bindrim: That’s amazing. I’m always looking at like, what bars and restaurants can I go to. You’re looking at the right stuff. Okay, so I kind of want to like get into some of the basics. I keep thinking about being a teenager and how often I would say, ‘That’s so random.’ And I hope teenagers still say this. But if they don’t, that was like a big thing. And I never really thought about what it actually meant. So I’m hoping that’s where we can start. What is true randomness?
Amanda Shendruk: Well, true randomness is a lack of pattern, or something that can’t be predicted. And often when we talk about randomness, we talk about something called entropy, or something that has high entropy. And entropy is the degree to which something doesn’t have pattern or that there is no pattern in a system.
Kira Bindrim: What are some examples of true randomness that would be familiar to me or to listeners, if we’re trying to sort of wrap our minds around that?
Amanda Shendruk: Well, true randomness is interesting, actually, because it can’t be created by humans or by computers. The only place you can find true randomness is in nature, or through physical sources, or physical actions. So one place where you might find true randomness is in turbulence. Another place is cosmic background radiation, it’s truly random, or highly entropic. And another one is radioactive decay. So we know a lot about radioactive decay, we know that the probability that each atom will decay in a given time, but we don’t know exactly when that will happen. That’s random.
Kira Bindrim: So really, the only one of the only appropriate times to say ‘That’s so random’ is when there’s turbulence in the in the air in a plane. Just turn to the person next to you… I’m gonna start.
Amanda Shendruk: Yeah, you could.
Kira Bindrim: They’ll get it, for sure
Amanda Shendruk: You’d be so accurate. I’m sure they’ll really get it.
Kira Bindrim: So something can be truly random. And something can be not random at all, obviously. What’s in the middle? Is there something in the middle?
Amanda Shendruk: Well, when we talk about entropy, entropy is kind of a continuum. So, you can think of entropy as sort of the certainty that you have about an outcome. So if you are very certain about an outcome, that is low entropy. If you are very, very uncertain about an outcome, that’s high entropy. For our purposes today, the randomness that we’re talking about is primarily true randomness, which is what we get from natural sources or physical systems. And pseudo randomness is what we get from random number generators on computers.
Is a random number generator actually random?
Kira Bindrim: Okay, so how is a random number generator, not random?
Amanda Shendruk: Random number generators are made from code. They’re algorithms, they run off your computer. And an algorithm is just a recipe. And computers are very good at following recipes. And so the outcome of a random number generator is always going to be the same given the same input. So if you know the input, you can predict the output, or you can reverse engineer it. So that’s pseudo randomness. And that’s what you get from a random number generator—it appears random, but because it can be reproduced, it’s not useful randomness. Now, there are some cases where pseudo randomness works. And we could talk about that in a bit. But the way to get around this, and the way that a lot of people who need true randomness get around this, is they find ways to give a truly random number as the seed for the random number generator.
Kira Bindrim: What are some places in the world that I would see that pseudo randomness, and what’s something that would be familiar to people?
Amanda Shendruk: Shuffle is a really good example of something that we think is random that is very, very far from random. When we listen to an album on random, we don’t actually want random—we want the appearance of random. Because if you listen to your album truly on random, you could play a song, and then the next song that plays, every song has the equal opportunity of being played again. That song, the first song, is not taken out of rotation. So if your shuffle were truly random, you could end up with the same song three times in a row. Or you would end up with your favorite song not playing for a long period of time. What we actually want is the appearance that it’s random. We’re just asking for basically the album to play in a different order.
How ‘random’ is randomness?
Kira Bindrim: And really so much about algorithms in general is, ‘Let me surface you content in a way that appears random to you but is actually sort of highly choreographed.’
Amanda Shendruk: Oh, yeah, absolutely. None of that stuff is random. None of that stuff is even pseudo random. We’re talking different levels of random here. That’s random in the way that you were referring to random when you were in high school, or whatever, ‘That’s so random.’
Kira Bindrim: But I actually want to go back in history first. Because we’ve been talking a lot about computer-generated random numbers, and things that sort of are of a time of the digital age. But presumably, we needed random numbers, before we had computers.
Amanda Shendruk: I love how your voice goes up like that.
Kira Bindrim: You know, that’s when I don’t know.
Amanda Shendruk: You know, we’ve been introducing uncertainty into our decision-making processes for a very long time. You know, dice way back in the day. The Bible talks about drawing lots. In the more modern past, they used to actually produce books have random numbers. So one of the more well known ones in the randomness field, is a book from the RAND Corporation that was published in 1955. And it’s just hundreds and hundreds of pages of random digits. And I need to look at my notes here to remember what it was called, because it’s such a clever name: A Million Random Digits with 100,000 Normal Deviates was that bestseller. And the digits in there were produced with a digital roulette wheel. And it was the first time that so many large numbers were compiled together. And actually, when I was doing some reporting on this a couple years ago, I went to the British Library in London, and they have a copy of this book. And I was all very, very nerdily excited to check it out. And I booked it—you have to book a room to go sit in to look at it—and I spent like two minutes on it. Because it’s very exciting, but it’s just a book of numbers. So, not much in it. But that was a way that, that was used for random number generation back in the day.
Kira Bindrim: So am I right in assuming that if computers are bad at generating random numbers, humans trying to do it consciously at least are worse.
Amanda Shendruk: Oh, absolutely.
Kira Bindrim: After the break, why do we need true randomness?
[[ad break]]
Randomness is crucial to encryption
Kira Bindrim: So we were talking earlier about how it’s difficult to find randomness, or at least naturally. Now I want to talk a little bit about why that matters, or why people should care. So my first question here is: Why is it so important for us to have access to randomness?
Amanda Shendruk: Well, it’s primarily important right now because randomness is crucial to encryption. It’s not actually so difficult to produce true randomness. It’s difficult to produce true randomness on the scale that we need it. Because we need a lot of randomness. I mean, it’s very important in cryptography and encryption, and that underpins a lot of the internet and a lot of what we do on networks nowadays. And so we need it at a scale that you can’t get just from, you know, watching atoms decay with a Geiger counter, or, you know, measuring turbulence flow. It’s just not enough. And so we need to find ways to amplify that true randomness into enough randomness to supply all of our random needs nowadays.
Kira Bindrim: Got it. So it would be fair to say that we need a lot more today, in large part because of the digital world?
Amanda Shendruk: Yeah, I’d say absolutely. I mean, so much of what we do online is encrypted and involves encryption in some way. I mean, truthfully, encryption is sort of like the currency or the language underpinning a lot of the internet. And when you log into an account, you know, there’s encryption there. And that requires random numbers. When you send a message, when you do transactions through your bank app, things like that—they all require encryption, and that all requires reliable random number generation. The scale is huge. If you think about WhatsApp, for example—WhatsApp says that every message is encrypted. And that means that every message you send requires random number generation at its core.
Kira Bindrim: We talked earlier a little bit about this idea of the seed number in a random number generator itself being random. And I think that’s a good segue to kind of come back around to the lava lamps, which I’m super excited to talk about. Let’s like, unpack that example. How exactly do these lava lamps contribute to randomness for Cloudflare?
Amanda Shendruk: So the lava in a lava lamp is kind of hot wax. And the way that it sort of blurbs and glubbles and glurbs or whatever it does, I think those are the technical definitions, that’s unpredictable. So you’ve got a whole wall of them. And they’ve got a video camera that’s taking video of this wall all the time. And it’s also taking video of people walking by, and the light coming in, and plants. So various things that are happening. Now each frame of that video is unique primarily because things are happening, but even just the lamps themselves are bubbling up and down. Now, the info for each of those images is stored as a long string of numbers and letters. And then that is shortened via a cryptographic hashing function which produces a new value. Now they take that value and mix it up with a whole bunch of other sources of randomness. So they have a double pendulum in their London office, they measure background radiation in their Singapore office—they kind of mash that all together, spit out another different number. That seeds another random number generator. And basically, then Cloudflare services from around the world can access that stream of random numbers.
Kira Bindrim: So it’s this sort of elaborate scheme to get to a place where you can seed a random number generator with the most random possible seed?
Amanda Shendruk: Yeah, absolutely.
Kira Bindrim: Is there another example of one of these sort of like pairings of the natural and computer worlds to ultimately end up with a really random number?
Amanda Shendruk: The University of Chile has a randomness beacon—randomness beacon, that just means like, something that spits out a stream of random numbers. And theirs is interesting because they use entropic sources that include data from the seismological center of Chile, I think radio broadcasts from the university radio, random tweets from Twitter, and something from the blockchain, ethereum blockchain data or something. And they mix that all together to create their randomness.
Kira Bindrim: So when we’re talking about some of these, like modern use cases that we’ve been mentioning—encryption, election security—is true randomness, or as close as you can get to true randomness, necessary for all of those? Or, how do you sort of determine the stakes of how much randomness is needed to power something that requires it, if that makes sense?
How do you measure true randomness?
Amanda Shendruk: Basically, if you want something to be cryptographically secure, so, very well encrypted—and most people want, you know, their bank transfers and things like that to be as secure as possible—you do need as as good a randomness as you can get.
Kira Bindrim: So the more things we need to be secured and the more secure we need things to be, the more important randomness is going to be for us.
Amanda Shendruk: And again, it’s a question of scale. Because we can get this, you know. This isn’t like, true randomness doesn’t exist. It’s just slow. It’s very slow to watch a Geiger counter. And if every time we got the number off a Geiger counter, we use that to be our random number for something, I mean, things would take ages. So we have to figure out ways to amplify it. Which places like Cloudflare are doing very well.
Kira Bindrim: Are we humans, and I guess by proxy, computers, getting better at this over time? Are we developing new ways to produce randomness more effectively?
Amanda Shendruk: As far as I understand, we’re doing a pretty good job at that. I mean, there are places like the National Institute for Standards and Technology in the US and they’ve been experimenting with using quantum mechanics to create randomness by tracking particles of light. But it’s a very expensive process. And it’s a slow process. And it requires like a building’s worth of setup. It’s not viable right now. But it’s also doesn’t really seem to be necessary.
Kira Bindrim: Are there any sort of like randomness equity questions here? Like we’re talking about Cloudflare, which is an enormous company that powers, as we said, 10% of internet security. So it stands to reason they have the resources to create this sort of elaborate system. Are their companies or even countries that are at a disadvantage because they might not have the resources to produce those systems, but still need randomness?
Amanda Shendruk: Well, something that I’ve found pretty interesting is, recently, there’s been talk of public randomness beacons. Cloudflare, along with a series of universities and other organizations, got together and have created this public randomness beacon, the League of Entropy. Which is interesting, because it’s a distributed randomness beacon, which means all of these different companies and organizations from around the world, universities, take their own individual sources of randomness—so in Chile, it’s, you know, the seismic waves, in some other place, it’s ambient noise, somewhere else, it’s a Geiger counter—they pull them all together into kind of this whirlpool of entropy. And that provides seeds for a random number generator that you can access publicly. So you can go online and look at this random number generator. It spits out random numbers all of the time. And it’s great for times when we need publicly verifiable randomness. So maybe for election things, or stuff—you don’t want to use this for encryption, because it’s all public. But it’s an interesting public resource that all of these organizations have come together to put to there for free.
Kira Bindrim: Do we think that would be a good Marvel movie, the league of entropy?
Amanda Shendruk: Yes. I think that’s definitely up Marvel’s alley. On the website, they have a slogan somewhere that says, ‘Not all heroes wear capes,’ which I appreciated. And they definitely have cartoon superheroes. One is like a lava girl or something. And it’s based off of the level lamp entropy from Cloudflare. Yeah, they’ve they’ve really gone all out with it.
Kira Bindrim: Oh I could see, okay, so they could do like pendulum man. And like radioactive woman.
Amanda Shendruk: Yeah, yeah, absolutely.
Kira Bindrim: Total possibility here. So hearing all of this reminds me of learning about CAPTCHAs, which are those tests you take to prove to computers that you’re not a robot. Our colleague Nico Rivero has been researching them a lot. One of the takeaways with all the CAPTCHA stuff is that we need humans picking out photos of traffic lights and re-typing blurry numbers and letters to help train computers to see the world the way that humans do. Basically, we’re helping computers decipher things that they can’t decipher on their own. And here, with randomness, it seems like we need the natural world to help computers create randomness in a way that only the natural world can, and computers can’t on their own. And there’s something really comforting to me about this idea that we keep using the defining characteristics of the natural world to advance the digital one. But also something like a little insidious about that, that we’re sort of handing over these characteristics to improve the digital world around us. I do have one last question for you, which is just a fun one. What is your favorite random fact about randomness? What is the thing that when I am done saying to the person next to me on the plane ‘That’s so random’ about turbulence, that I would share with them about about randomness to kind of intrigue them on this topic?
Amanda Shendruk: I think I’ve probably shared all my favorite facts about randomness so far, but kind of a quirky one is, apparently, the city of Portland, Oregon, was named via a coin toss. And if it had not been named Portland, the other side of the coin was Boston. We could have had a Boston, Oregon.
Kira Bindrim: Listen, you know what that is? Amanda?
Amanda: What?
Kira Bindrim: So random.
Amanda Shendruk: Oh don’t even.
Kira Bindrim: Well, thank you, Amanda, for joining today, this is super interesting.
Amanda Shendruk: Thank you for having me.
Kira Bindrim: That’s our Obsession for the week. This episode was produced by Katie Jane Fernelius. Our sound engineer is George Drake, and the theme music is by Taka Yasuzawa and Alex Sugiura. Special thanks to reporter Amanda Shendruk in Edinburgh, and editor Alex Ossola in New York. If you liked what you heard, please subscribe on Apple Podcasts or wherever you’re listening. Tell your friends about us! Pick five names out of your contact list at random, and shoot them a link. Then head to qz.com/obsession to sign up for Quartz’s Weekly Obsession email and browse hundreds of interesting backstories.