Early on a humid Tuesday morning in May, Robert Xiao, a 26-year-old Ph.D. student, stares intently at his MacBook, his eyes darting among his Windows, OS X, and Linux setups, indie K-pop blaring from his headphones. He hasn’t slept in 22 hours.
Around him, in the fancy main ballroom of the InterContinental Seoul Coex hotel, in the Gangnam district of Seoul, South Korea, 10 hacker teams huddle in small groups, talking in hushed voices. Some are furiously typing, while others stare blankly at their monitors, trying to stay awake.
They’ve all gathered to compete in Capture the Flag. But it’s not the light-tackle after-school sport. This CTF is something else: a niche competition in which teams compete to solve computer security problems modeled on real-world vulnerabilities. It sounds esoteric—and it is, layering a specific competitive vocabulary on already impenetrable hacker jargon—but the stakes are high. This morning, Xiao’s competing in Codegate, one of the most prestigious events, with top teams competing for a cash prize of 30,000,000 South Korean won, or about $25,000.
Xiao plays for Carnegie Mellon University’s Plaid Parliament of Pwning (PPP), which took first place at Codegate in 2011 and again in 2014. Picking up another gold is a point of pride for the team, and as a relative newcomer to the team, Xiao’s a little nervous. He needs to prove that he belongs, that he can play alongside the best.
“I had never seen stacks of hundreds before. You would never think doing a programming contest is how I would come across that.”
As one of the best CTF teams, the PPP earns anywhere from $5,000 to $65,000 a year. Usually it’s around $35,000, often in cash. “[It’s] one of the nerdier hobbies you could have with sort of an old-school gangster money distribution,” says Ned Williamson, the team’s soft-spoken captain. He remembers the first time he saw a briefcase stuffed with prize money. “I had never seen stacks of hundreds before. You would never think doing a programming contest is how I would come across that.”
Much of the prize money comes from competitions in Korea and China, where the high payouts underscore those countries’ seriousness about computer security. PPP’s winnings primarily go back to the team, funding expenses and travel to other competitions; the group incorporated in part to formalize its cash distribution. “We realized for tax purposes it’s probably best if we don’t have tens of thousands of dollars in international wire transfers going into random people’s bank accounts,” says Tyler Nighswander, a former captain who continues to play with the team.
Even after incorporating, the PPP remains loosely organized, consisting of about a dozen regular players, including CMU undergrads, post-docs, graduates, and staff. Most study computer science, electrical engineering, or computer engineering; in addition to regular study sessions to go over specific problems or computer security exploit techniques, they meet on Friday nights to review write-ups and go over problems.
The team got its start in 2009, when a student of professor David Brumley, who teaches courses in computer security, malware, and vulnerability analysis, wanted to qualify for another CTF team. Instead, Brumley offered to be a faculty adviser for a new team, providing access to his undergraduate research lab. In addition to giving them a place to meet, his backing gave the team early legitimacy; Brumley even sponsored the team’s competition trips. “I tried to make sure if they wanted to do something, that they could afford it; I would make sure it happened,” he says. That early support paid off: according to CTFTime.org, the PPP has ranked in either first or second place from 2011 to 2015.
As Codegate’s final hour approaches, Xiao finds an exploitable bug. This is what he’s been looking for: all he needs to do now is access a string buried deep in the program’s memory. But there’s a problem. To defend against just such an attack, the memory layout has been randomized: he doesn’t know where to find the string he’s looking for. Normally he’d look for a second bug, one that would reveal the address he’s looking for, but he’s running out of time. PPP is still in first place, as a giant scoreboard shows, but the second-place team is hot on their heels.
Xiao’s exploit would work only once in 10,000 tries, but he doesn’t have the time—or energy—to find a second bug. So he says screw it and opts for the brute force method, running his code until it works, even if that means burning through all 10,000 tries. Running the code repeatedly requires a program that doesn’t exist, so as the clock winds down he frantically builds the program himself.
Next he needs to get his code to work on the contest computer, but it doesn’t—and he can’t figure out why. The clock runs out. He fails. The second-place team solves the problem and overtakes the PPP.
Later, though, the other team is disqualified for receiving outside help. PPP takes the gold, but Xiao isn’t satisfied. Instead of sleeping, he returns to his hotel room to review his code. He finds a single line that he’d missed; Xiao fixes it and runs his program to completion. Only after it finishes 40 minutes later does he let himself crawl into bed.
“A lot of people on our team will get obsessed with a certain problem and keep playing with it until it’s done, and that can take days,” says Nighswander. At a Russian competition, he was stumped by a challenging crypto problem. He solved it only after the contest ended, then spent around 40 more hours perfecting his solution. By the time he finished, what previously took hours to solve could be cracked in ten minutes.
Xiao’s quiet and focused when he’s absorbed with a problem, but when he talks about Capture the Flag, his eyes light up. Although he’s studying human-computer interaction—which has almost nothing to do with security—he’s quickly become one of the best players on the team. His undergraduate degree is in mathematics, and he does puzzle games (like the Microsoft College Puzzle Challenge) for fun, solving cryptograms, word puzzles, and logic problems.
But Xiao attributes his success to playing almost every CTF game that was available after joining, and to being surrounded by highly skilled teammates. Everyone has their specialties: his is binary exploitation, while others focus on reverse engineering, cryptography, or web hacking.
CTF competitions help students develop an unorthodox skill set, Brumley points out, saying that they probably learn as many as 20 new exploitation skills through CTF. “In real life, you never have that breadth of exposure,” Brumley says. It teaches a different kind of thinking, one that pushes players to think about not just the avenues of attack they already know, but more creative vulnerabilities. “They never teach you about how to attack timing information or error messages or padding oracles in a crypto class,” Brumley says, and software developers don’t learn about them in school, either.
These skills are in high demand. “Our group gets group emails from every defense contractor and Apple and Facebook and Google and LinkedIn and lots of places,” says Nighswander. “There are basically people who say, ‘If you’re good at CTF I’m sure you have the technical skills to work at our company.’”
“There are basically people who say, ‘If you’re good at CTF I’m sure you have the technical skills to work at our company.’”
Of course, Carnegie Mellon is one of the country’s top schools, and a CMU education certainly helps in CTF competitions. “I think one of the things that people don’t appreciate is how much their background knowledge that they gained in class is helping,” says Brumley. He points to a sophomore-level introduction to computer systems class that gives students the foundational knowledge on which they can build.
But schoolwork obviously doesn’t have the same thrill as a CTF competition with thousands of dollars at stake. Williamson says that while he approaches problems in school with more vigor, he’s lost his passion for homework. “Funnily enough, I think I’m learning more because I care more about solving problems in class than I actually care about cranking the widgets to get the grades,” he says.
CTF takes players beyond the classroom. As a freshman, one of Williamson’s classes required him to write an emulator—a program that simulates other computer systems. At the time, it seemed like a huge undertaking, one worthy of a final project. Today, he does it frequently, even scouring the code-hosting platform GitHub looking for half-finished emulators that he can revive as weapons in competition. “Before I would’ve thought it’d be insane to write a full emulator just to solve one problem for CTF,” he says, “but I find myself doing that a lot.”
Illustration by J. Longo