The Evolving Landscape of Bug Bounties in the AI Era
“Nation-state issues are very serious and very real, but criminal actors still make up the vast majority of incidents that organizations deal with, and many of these incidents are quite serious,” adds Hultquist. “The use of zero-day devices by criminal actors has been quite limited, and those who use them tend to be very successful. So I think we shouldn’t underestimate the impact of more criminals with a zero-day in their hands.”
The digital landscape is experiencing a significant shift as artificial intelligence (AI) becomes a double-edged sword in cybersecurity. As organizations grapple with threats from both nation-states and criminal actors, the dynamics of bug hunting have evolved, creating new challenges and opportunities for researchers and companies alike.
AI’s Impact on Bug Bounty Programs
However, times are changing for researchers who make money from troubleshooting. Command-line tool Curl ended its bug bounty program (run through third-party service HackerOne) in January after it was inundated with low-quality AI-generated submissions.
The rise of AI-generated bug reports has been a double-edged sword for cybersecurity programs. Curl, a widely used command-line tool, decided to end its bug bounty program due to an influx of low-quality AI-generated submissions. This influx led to an overuse and abuse of the program, prompting Curl to reassess its strategy.
“We concluded the hard way that a bug bounty gives people too strong an incentive to maliciously find and invent ‘issues’ that lead to overuse and abuse,” the group wrote at the time, adding that “we still value and value valid vulnerability reports.”
The Challenges of Managing AI-Generated Reports
Last week, Linux inventor and lead developer Linus Torvalds wrote that the famous Linux security mailing list had become “almost completely unmanageable” due to the large number and duplicate AI bug reports.
The challenges faced by Curl are not unique. Even the Linux security mailing list, a critical resource for developers worldwide, has struggled with the overwhelming number of AI-generated bug reports. This deluge has made management increasingly difficult, highlighting the need for improved filtering and validation mechanisms.
Improvements and Adaptations in Bug Reporting
However, in April, Daniel Stenberg, Curl’s founder and lead developer, said in a LinkedIn post that the quality of submissions had improved. “In recent months, we have stopped receiving AI safety reports in the Curl project,” he wrote. “Instead, we’re getting more and more really good safety reports, almost all of which are generated using AI. They’re being delivered at an unprecedented rate and are putting a huge burden on us.”
Despite initial challenges, there have been positive developments. Daniel Stenberg, Curl’s founder, noted an improvement in the quality of AI-generated bug reports. These reports, while higher in quality, still place a significant burden on developers due to their volume and complexity.
Evolution of Vulnerability Reward Programs
And in late April, Google announced that it would overhaul its vulnerability reward programs for Chrome and Android, lowering payouts for some bug classes and increasing them for others.
Google, recognizing the evolving landscape, announced changes to its vulnerability reward programs. By adjusting payouts, the company aims to incentivize the discovery of more challenging and impactful vulnerabilities, aligning rewards with the current security landscape.
Encouraging Ethical Research and Structural Defenses
“I think insect hunters with special skills that are in the 90th percentile will always be able to gain insights and get payouts from big companies,” says Jonathan Dunn, a cardiologist who is also an insect bounty hunter. “But even with AI, we need to strongly encourage ethical researchers to find things in public infrastructure and other critical systems that might not otherwise get enough attention from defenders.”
While AI has transformed bug hunting, the need for human expertise remains crucial. Ethical researchers play an essential role in identifying vulnerabilities in public infrastructure and critical systems, ensuring these areas receive the necessary attention and protection.
At the moment, most companies seem willing to deploy any possible solution to the problem (and benefit) of accelerated error detection. “This changes the dynamics of the bug hunting industry, but absolutely still requires human time,” says Alex Zenla, chief technology officer at cloud security company Edera.
The Need for Structural Defenses
Earlier this month, Anthropic launched a HackerOne bug bounty for researchers to submit insights into the company’s proprietary systems and Claude AI models. However, some researchers are increasingly arguing that structural defenses are needed to counter the increasing discovery of vulnerabilities. In other words, they design digital solutions for different classes of vulnerabilities that eliminate them or make them significantly less exploitable in practice.
As companies like Anthropic leverage bug bounty programs, there’s a growing call for structural defenses. These defenses aim to address vulnerabilities fundamentally, reducing their exploitability and enhancing overall security.
“You can’t patch yourself out of this,” says long-time safety engineer and researcher Niels Provos. “You have to build an infrastructure that makes as many mistakes as possible irrelevant.”
The evolving bug bounty landscape underscores the need for a balanced approach that combines AI advancements with human expertise and robust structural defenses. As the industry adapts, collaboration between researchers and organizations will be key to navigating the challenges and opportunities that lie ahead.
For more on the AI era’s impact on bug hunting, visit the original article here.
“`
