Twitter Should Use Random Sample Voting For Abuse Reports

Bennett Haselton writes: Twitter has announced new protocols for filing and handling abuse reports, making it easier to flag specific types of content (e.g. violence or suicide threats). But with the volume of abusive tweets being reported to the company every day, the internal review process will always be a bottleneck. The company could handle more abuse reports properly by recruiting public volunteers. Read what Bennett thinks below.

In August, Twitter user Kristin Puhl made public the fact that another Twitter user had tweeted at her:

f@#king die feminist moron i'm coming after u and raping u.

and when Puhl filed an abuse complaint with Twitter, Twitter responded after two days:

We've investigated the account and reported Tweets for violent threats and abusive behavior, and have found that it's currently not violating the Twitter Rules (https://twitter.com/rules).

(The "rules" linked in the message include the clause "You may not publish or post direct, specific threats of violence against others.") Twitter must have changed their mind eventually, because the account of the user who sent the message is now gone, but why didn't they close it the first time?

Twitter can't effectively adjudicate all the abuse complaints that they get, but I don't blame them. I don't think they publicize numbers for how many abuse complaints they receive every day, but I'm sure that it's more than an internal review panel could handle fairly. Twitter should not be faulted for that. They've created a world-changing tool, and they shouldn't have had to stifle the growth of their platform just because it grew faster than their ability to handle the abuse reports.

But now that they're publicizing their latest tools for handling online harassment, it's fair to ask more of them. And while the tools may streamline the process of categorizing incoming abuse reports, there's always going to be a human review bottleneck, which will get tighter as the Twitter platform continues to grow.

So I'd suggest the same solution that I suggested for Facebook abuse reports: recruit a pool of volunteers from the general public to review "abuse reports". (You would need a "critical mass" of at least tens of thousands of reviewers for my idea to work, but Twitter shouldn't have trouble amassing that many people for a special program.) Then when an abuse report comes in, do the following:

1. Some small number of reviewers -- say, ten -- are randomly recruited from the pool of volunteers.
2. Each of them looks at the reported content and the category of abuse that it was reported under, and votes Yes or No as to whether the content meets the criteria for abuse.
3. If some threshold of users (say, eight) vote that it does, then the report gets bumped up to a higher-level review. This "higher-level review" could mean having a new, larger pool of users (say, twenty) look at the content and vote on it, in case the original eight-out-of-ten vote was a statistical fluke. Or it could mean forwarding the reported tweet to some human review panel at Twitter -- which now has far fewer abuse cases to review, because it only has to look at the reported tweets that cleared the hurdle of getting eight out of ten votes for violating the guidelines.

These numbers are just guesses. I might be over-optimistic about how many reviewers would even respond when Twitter asked them to vote on whether some content was abusive (even though that's what the reviewers signed up to do) -- it might turn out that to get even ten responses, Twitter would have to nag 50 people to come and vote on a piece of content. And the size of the voting initial voting panel should be large enough to avoid statistical flukes most of the time -- if a tweet is inoffensive enough that only 10% of the reviewer population would consider it "abusive", you'd have to be really unlucky to convene a panel of 10 users where 7 out of 10 voted to label the tweet as "abuse".

As long as the size of the reviewer population grows in proportion with the Twitter user base (or, more precisely, as long as it grows in proportion with the volume of abuse reports coming in), this system scales as much as you want it to. (Well, unless the "higher-level review" involves review by an internal panel at Twitter, which still creates a bottleneck.)

Because the voting panel is randomly selected from among the entire pool of volunteers, that means you can't "game the system" by forming a mob with dozens of your friends so that everyone can file an abuse report about the same content at once. As long as your mob only comprises a tiny proportion of the 100,000+ reviewers in the system, there's virtually no change that a randomly selected panel would contain enough of you to swing the vote.

This could also potentially result in an almost-instant turnaround time for handling abuse cases (a matter of reassurance for victims of normal harassment, and a matter of life and death in the case of suicide threats or threats of violence). Twitter could restrict their random sample to only those users who happen to be signed in at the present moment, and who have a minute or two to review a piece of content and vote on whether it violates the guidelines.

Tweets are by definition public, so there wouldn't be any potential privacy violation in taking someone's tweet, putting it before a panel of 10 volunteer reviewers, and asking them to determine if it violated the terms of service. Direct Messages sent via Twitter, on the other hand, are intended only for the recipient, and are not public by default. If a recipient wanted to flag a Direct Message as abusive, they would have to specify whether they want the content to be reviewable by a panel of randomly selected public volunteers. So in the case of the tweet received by Kristin Puhl -- "fucking die feminist moron i'm coming after u and raping u" -- even if she had received it as a Direct Message from someone she was following (you can only receive DMs from someone if you're following them), presumably she would have been OK with showing the tweet to a panel of volunteers, who probably would have voted that it was in fact abusive. On the other hand, sometimes a user might receive abusive DMs where they want to report the abuse, but the DMs might contain sensitive information that they don't want publicized to randomly selected volunteers. So those abuse reports might have to be handled the old-fashioned way at Twitter, by internal review, which still creates a bottleneck. But hopefully the abuse reports about Direct Messages comprise only a small minority of abuse reports that Twitter receives, since most talk about abuse on Twitter comes in the form of public tweets. (If someone is "abusing" you via DMs, you can just unfollow them.)

Twitter could even be completely transparent about the entire voting process: "Your complaint has been reviewed by 10 people. 8 of them agreed that the tweet in question violated our guidelines. This is above our minimum threshold of 7 that triggers a higher-level review of this content." (Twitter presumably wouldn't want to tell the complainer who the voters on the panel were, since the complainer might harass the individual voters if the voting panel as a whole rejected the complaint. But there's no reason not to be transparent about the actual numbers.)

Why would someone sign up to volunteer to review abusive content? Maybe for the glimpse into strangers' lives. Maybe hoping to save copies of some of the porn contained in the tweets that get reported for abuse. (Of course, there are easier ways to get porn online, but maybe they get off on the fact that some particular pornographic image made someone angry and upset enough to report it.) Maybe they altruistically believe it's part of their civic duty towards the Twitter community. Maybe because they're bored.

Whatever people's myriad motivations for signing up, the important thing is that there's still a statistically significant difference between the number of "yes" votes received when content truly is abusive, and when it's not. Even if you have people signing up as reviewers for all kinds of weird reasons, a tweet like "fucking die feminist moron i'm coming after u and raping u" is still going to receive, on average, more "yes" votes than a tweet like "I respectfully disagree, so let's go our separate ways".

If Twitter were nervous about rolling out a system like this, ceding control of the abuse-report-handling process to a pool of volunteers, they could always do their own random sampling of the random-sample-voting system, to see how it was working. An internal auditor could pull 100 of the abuse report cases that have been handled by the random-sample-voting system recently, decide in each case whether the tweet did in fact violate the abuse guidelines, and then look to see if the voting system reached the same answer. As a control in the experiment, look at some abuse reports that were routed to the old-fashioned internal review panel during the same period, see how they handled the reports, and see how they fared in comparison. I would confidently bet money that the random sample voting system would handle the abuse reports more accurately, and faster, as well.

This won't do much to deter abusers who create an endless series of throwaway accounts for harassment purposes, which makes it futile to block or report any particular account. But it would at least get step zero right, which is to correctly adjudicate whether a tweet is abusive or not. And it would do it in a way that is scalable, non-gameable, and transparent. Plus a few volunteers would get an interesting story to tell at dinner.

Comments

We don't care what he thinks!!!

[Anonymous Coward]

We don't care!!!
Got your own blog its not that hard!!!

I don't care what Bennet thinks

[Anonymous Coward]

below, above or even sideways.

Can I Report Bennett for Abuse

[Anonymous Coward]

Because his shit is stupid.