This article started as a feature request to be posted on the Immunefi discord, but turned into this along the way.
Bug bounties are here, and they’re here to stay. You’ve probably heard plenty of success stories of hackers receiving up to 1 billion worth of loss to protocols.
Unfortunately, there is much room for actors in the bug bounty space to misbehave, largely due to a lack of transparency and accountability. In this article, I want to dive into some of these problems and propose potential solutions.
Problem
First things first, let’s look at these problems I mentioned. I’m primarily interested in the interaction between the reporter (white hat) and the team.
Disclaimer: This is all based on my observations and the experiences that people have shared. Keep in mind that this data is anecdotal! Furthermore, many of these experiences aren’t specific to Immunefi!
Whitehat → Team
beg bounty If you follow @troyhunt then you’ve probably heard about “beg bounties”. These low-quality bounty submissions often use just the output of off-the-shelf tools. Unfortunately, the submitter will continue pressing for a bounty, even though their contribution is worthless.
Don’t get me wrong! If a team promises a (big) bounty for actual security findings, then a white hat has every right to expect the team to fulfil that promise! Otherwise, bounty postings would be a method of exploiting security researchers.
Signal to Noise Hosting a bug bounty program has often been compared to being on the receiving end of a sewage pipe.
It’s super easy to submit a vague report claiming there is a “critical” vulnerability, while it’s much more difficult to discern that it’s, in fact, a bogus finding.
Team → Whitehat
No payout As a white hat, you lose all leverage once you’ve submitted your report. At this point, the incentives for the receiving team are 100% not to pay out or to pay out less than promised.
Slow Response Times I’m pretty sure some teams post a bug bounty as marketing material or because they’re pressured into it.
Why would I think that? Because some teams go radio silent for a while after getting a report.
These teams don’t take their and their users’ security seriously.
Inappropriate Behaviour Besides haggling down bounty payments and being slow to respond, there are several ways in which a development team can misbehave.
To give an example, I’ve submitted one bounty where the development team refused to converse with me entirely!
Solution
There is good news! All of these factors are things that people can excel at too!
Hackers can submit high-quality reports with clear proof of concepts.
Hackers can be helpful to the team and do their best to help guide the team to resolving the issue!
Teams can be fair and quick with their payout!
Teams can be super fast to respond to a disclosure ( I believe the record on Immunefi is 15 minutes ).
Teams, like hackers, can be collaborative, communicate with the hacker, and be open to feedback that helps them improve their system.
🥕 Transparency
I believe that a bit of transparency can help put the right incentives in place.
Hacker Signal Most bug bounty platforms offer up some stats on their hackers. Reputation, signal and impact are the ones used by HackerOne.
These metrics provide an easy way for teams to assess reports quickly. Got a vague report that doesn’t make any sense from a hacker with a low signal? Probably safe to close it!
These metrics provide great incentives for hackers. Having a great signal means that your reports get escalated and reviewed sooner. This means a faster bug fix (win-win-win) and a faster payout.
Competition brings out the best in people! I’d probably put in some extra hours of bug hunting to get higher on that leaderboard!
Team Signal Similarly, we can attach metrics to teams.
- How fast are they to respond? Triage & fix?
- What are the average payouts for crits/high/med/low?
- How many reports have there been to the team in total? And how many were paid?
Putting these metrics in place will incentivise teams to be great and make them accountable.
Whitehats → Will spend their time on reputable teams! Blackhats → Will turn white only for reputable teams! Users → Will know reputable teams take their security seriously!
Publicise Reports Currently, Immunefi publishes post-mortems of findings. These are great! They provide tremendous educational value for novice and experienced hackers alike. They also incentivise good behaviour to some extent!
That said, I think we should move towards publishing (most) vulnerability reports and associated conversations.
Because of the educational value that every report provides and to make everyone accountable for their actions.
Conclusion
This wall of text is essentially a feature request to add stats tracking & public reports to the Immunefi bug bounty platform.
It will keep us accountable and help us celebrate the great teams and hackers that fill this space!
Much ❤️ Joran