Danielle Citron speaks at WikiConference USA 2015

←

Danielle Citron speaks at WikiConference USA 2015 (2015)
by Danielle Citron

→

"Danielle Citron speaks at WikiConference USA 2015". (October 11, 2015). Danielle Citron. Video available at [1] Questions begin at [2](Uploaded by US National Archives). Washington DC.

This work is released under the Creative Commons Attribution-ShareAlike 4.0 International license, which allows free use, distribution, and creation of derivatives, so long as the license is unchanged and clearly noted, and the original author is attributed—and if you alter, transform, or build upon this work, you may distribute the resulting work only under the same license as this one.

Public domainPublic domainfalsefalse

2035166Danielle Citron speaks at WikiConference USA 20152015Danielle Citron

Hate Crimes in Cyberspace

It’s terrific to be here. I was saying to someone earlier on, like, I wanted everyone to move up, but no, but this just right. [laughter] I can see everyone.

So it’s terrific to be here, especially given that your community is self-policing. So much, I think we always look to at least–everyone is not a lawyer–looks to lawyers and folks in tech to say, “law, you fix it”, right? and lawyers always say, like, “we want the CS or computer scientists, we want the technologists to fix a lot of our problems”, right. We’re all pointing at each other. But I think, you know, you are all engaged in the process of. sort of, sorting through what your norms are on your site.

So, I’m excited to be here to talk to you about my book, which focuses on the problem of online harassment and stalking.

Online harassment begins: the story of Kathy Sierra

What’s really interesting is I started this project, I would say, about seven years ago—not the book, but just working on the issue—when Kathy Sierra was first targeted online. At the time, when I was–and I’m sure everyone know who Kathy Sierra is, right? Like the technologist? No? [unintelligible voice from audience] Really? Okay, I love that. Right.

So Kathy...she is a technologist who had a blog called Creating Passionate Users. And at the time, like in 2004, 2007, I think, it was in the top, like, 50 top blogs – or most trafficked at the time. She was kind of a trailblazer in her own right. She’s written books on Java and how to code, I mean, clearly uncontroversial. That is, what she’s writing about was how can we code and how can we make people happy, you know, in creating software. And she was targeted on her blog, in the comments and also in email that was sent to her, as well as group blogs.

So, rape and death threats, really graphic, on her own blog, and then an email sent to her. In group blogs, her face appeared in a doctored photograph–and it shows a noose beside her neck. And commenters wrote, “the only thing that Kathy Sierra is good for is her neck size”. And another photograph—she’s very distinctive, beautiful blond woman, and you see her face and its covered by — she looks like she’s being suffocated by lingerie. Now at the time, she was supposed to give a talk–this all happened in a two-week period–kind of this perfect storm of what she understood as anonymous abuse. She had no idea who it was coming from. She cancelled her talk at a tech conference in San Francisco, and she blogged about it. And she said “look, I’m terrified. I’m afraid to go to this conference, I’ve gone to the Colorado police”, where she lived in Boulder. And shockingly, they took her seriously, which is a struggle, even now, right. And they said, don’t leave your house, don’t do anything.

And when the press wrote about her reaction, and what was going on, a sort of second wave of abuse followed, led by people like Weev and others at 4chan, who said “stop whining, Kathy Sierra”. And so they doxxed her, spread her social security number all over the internet, her home, all of her personal contact information, like a narrative that was fabricated about her life and career. And she just got offline. And she really hasn’t blogged since.

And so, Kathy’s struggle and story—she’s sort of one of the main three people I talk about in my book—but she’s emblematic of so many people, who experience online stalking, and at the time when I first started talking...this is why I started talking about Kathy Sierra, we got me off my, sort of, script or whatever.

The law recognizes harassment as a civil rights issue

You know, the idea that we would bring law to bear against threats—privacy invasions like the spreading of social security numbers, that can aid and abet identity theft, defamation—the idea that we would bring law to bear against it was...I was told I was going to break the internet, right? The idea that, like, we could regulate some of this behavior, which criminal and tortious, and as I sort of contend, is a civil rights violation, was really heretical to suggest, and I was totally out of the mainstream. I was told, like, I want to break the First Amendment. And what’s interesting is that seven years later, is this completely–not only is it not heretical to suggest that we should bring law to bear against the worst abuses, but the Electronic Frontier Foundation, ...my friends put up a post in January that says, they said “we agree”.

This is a digital rights problem. That is, the victims have a right to speak and express themselves And this...threats and privacy invasions and defamation...are preventing them from expressing themselves. And that was kind of a moment for me, I have to say, I was, like “Ahhh, you know, my goodness,..” It went from being an off the wall idea, that this abuse is unexceptionable, to being something that civil rights liberties, you know, civil liberties organizations were agreeing with, and said yes, this is a problem of civil rights and civil liberties, right? So mainstream that we were on John Oliver. This is so uncontroversial now, it’s kind of amazing to talk about it.

Scope of speech

So, what I thought I’d do is describe the phenomenon of exactly what I’m talking about — define it, sort of clarify it, like who does it impact, its harm. And then talk a bit about the law, and then what law really can’t do, and a bit about what companies are now doing, right, private entities, and the kinds of choices they’re making. And what does that mean for our system of free expression.

Defining internet harassment

[51:03] Okay, so how do we define cyber-harassment and stalking? And I have to say, I read Wikipedia’s description, and I know it’s not necessarily prescriptive, but your kind of...guidelines...and it was really sophisticated. I thought, this is terrific, right?

So what is harassment and stalking? It is a repeated and persistent course of conduct that’s targeted at a specific individual, that is designed to and intended to, and that causes substantial emotional distress, and often the fear of physical harm. And more importantly—like a definition tells us nothing, right—but it’s how it’s perpetrated, and the components of abuse that are really, I think, important to understand.

Three features of harassment

And it’s often a perfect storm of three key features. One, it really attempts to terrorize people, so by threatening them with physical violence, as Kathy Sierra experienced, by impersonating them online, suggesting that they’re interested in sex, and providing their contact information. It involves reputation—harming lies, like defamation, and then often the manipulation of search engines, to ensure that the lies are prominent in a search of someone’s name. It involves privacy invasion, so that privacy invasion may take the form of hacking into someone’s computer to steal very sensitive information, including nude photos, and then the sharing or posting of them online. We can think of...of course–Jennifer Lawrence is a prominent example, but the iCloud hack is not unusual, and it’s not always celebrities. In fact, normally it’s just an everyday person. But it’s also the posting of people’s nude photos, in violation of their confidence and trust, like we see on revenge porn sites. And the last component is using technology to effectively shove people offline, whether it’s a DDOS attack or otherwise, like engaging in manipulation of technologies with brute force to silence people.

Who does cyber-harassment impact?

So, I think that since I gave you a little, sort of, sense of what this abuse looks like, I’m going to give you….

Anita Sarkeesian and Gamergate

So I’m sure you guys know all about Gamergate. [laughter] I know, right, to say that is kind of crazy to this group, right. Normally, like, when I talk at universities and people are like...I’ll have two really shy gamers in the audience who are like, you know [demonstrates hiding face behind hand] willing to admit that they know what it is. So, Anita Sarkeesian is a friend that is in my book, and I think we largely understand the abuse that she faced, but I think it’s important to know it didn’t start this last summer.

So in 2012, Anita announced that she was starting this Kickstarter campaign to have this video series on sexism in video games. And about a week after Anita announced the launch of this project, she started receiving emails and texts with graphic rape and death threats, bomb threats, right? And that same week, as the cyber-mob kind of descends on her, the game “Beat up Anita Sarkeesian” appears online. So if you google it right this second, you will find this game, right, which is, any key that you touch in your computer–she’s a distinctive woman–you know it’s Anita–her face gets increasingly bloodied, and sort of purplish, right?

Now, whoever was doing this, like a set of people, they went after her livelihood, her Kickstarter campaign, right. So her Kickstarter received hundreds of false abuse complaints, suggesting that her campaign was fraudulent, like, her effort to raise money. Facebook, Twitter, and YouTube received reports that her channels were–this is the irony–and I love this moment–“hate speech, spam, and terrorism”. Right? Seriously, friends? Okay, and because she was very well known, they didn’t shut down her profiles. We know happened at Wikipedia, right, to her Wikipedia page, and the kind of monkey business of...kind of...seed it with pornography, and child porn, and it seems like Wikipedians had to shut down the page. And this abuse went on for years.

So I met Anita before all of this happened, and she called me and said “I know you’ve been writing about this, but I have no idea what to do”. I initially urged her to go to law enforcement, and she said, “what do I say? There are thousands of people coming after me?” Like, I can’t imagine their wrapping their heads around getting a warrant for all of these posters, right?

And we know that last summer it reached a fever pitch after the attacks on Zoe Quinn, and I sort of...one example beyond just the sort of avalanche of rape and death threats...with her address and her parent’s address...it sort of escalated this time. The threats were in her in-box: “I’m going to rape you, bitch, I know exactly where you live,” including her parent’s address and her home address. But also, she was set to give a talk at Utah State University and the day she came—so she’s already there to give the talk—the dean’s office received anonymous phone calls that said, “if Anita Sarkeesian speaks this evening, there will be a school shooting worse than Columbine and New Town combined.” Now Anita was already there, so she was like, “All right, I’ll give this talk so long as there’s some police presence and we’re checking people for guns.” But the response to her was, “Well, we have concealed carry laws, so we can’t help you. We can’t check people.” And isn’t that the most ridiculous thing you’re ever heard, right? So, if President Obama was coming to Utah State, or the CEO of Pepsi, you really, seriously think that’s going to be the response? “We can’t check people?” To make sure that they have no guns? So she didn’t speak, And it was, like, directly impeding what she does for a living, right, is speak and talk–it’s her work as a journalist and media critic.

Low profile people: Holly Jacobs

[57:04] But of course, it’s not just the high profile, whether it’s Kathy Sierra or Anita Sarkeesian. Really, more, I think, importantly, more prevalently, it’s the every day person, right, the grad student, the nurse, the dentist. I can’t tell you the countless people I talked to in research and writing my book. So I’ll just tell you about one, so we can get a full understanding of what this abuse feels like from the perspective of the victim. Holly Jacobs—she had just graduated from BC, and she was in grad school at FIU, and she had a long distance relationship with someone. And they, as young people do (I feel like I’m forty-six; I’m allowed to call her a young person), there were cameras everywhere, and they shared an intimate images with each other. It was a two-way street, it wasn’t, like, one way. They shared images and videos with each other. She was surreptitiously taped during sex too, which she did not consent to, but much of it was consensual sharing. But it was on the understanding, of course, that it was for their eyes only.

So, they break up, and about six months later, she starts getting emails from strangers, and texts, that say “I saw your ad, I’m interested in sex, like, where can we meet?” And at the time—so this is 2009 when it starts. Holly wasn’t in the habit of googling her name, like, it just wasn’t something that we preached and talked to our kids about, and discussed. [58:34] And so, what she found when she did google her name, was over three hundred sites—revenge porn sites, porn sites, and adult finder sites—with her nude images, some videos, and one video of her masturbating, which if you can only imagine the, sort of, sheer embarrassment and horror–but with all of her contact information.

So, she had a part-time job–her work address, her work telephone number, her cell phone number, all of her contact information. And some of the posts—because she was a grad student—said “hot for teacher, Holly and her former last name, she sleeps with students”. Other posts were like fake ads, that said she was interested in threesomes, in anonymous sex, you know, others looked like they were solicitations from her. And her dean’s office—dean of students—received anonymous phone calls that said she was sleeping with her students. And her part time employer—she did some consulting work—she got an email that said “if you don’t send me more nude photos, that I will send these photos that I have of you (nude photos) to your employer, because I know where you work”. And of course, she didn’t write back, as I recommended, because you know, to feed the beast and give more photos is, like, extortion—clearly we have a crime going on here.

And so of course the person lived up to the threat, sent the nude photos to her employer, her part-time employer, right. So she left that job, because it was clear from these postings where she worked, the address. She moved out of her apartment, into her parents’ house, She changed her name. She had a very unique former last name, and her dean of students said to her, “look, if you’re going to teach students, the only way we’re going to let you teach students is you really have to change your name. These young undergrads can’t be googling their teacher.”

Declaring “reputational bankruptcy”

Can you imagine, to declare bankruptcy, as my friend Jonathan Zittrain would say “reputational bankruptcy”–to foist that on someone. It seems like an untenable choice to say to her…. And she hadn’t yet gotten her PhD, so these are people who were in control of her doctorate. So she did. And there’s a wonderful—at the end of my talk I’ll tell you what Holly’s doing today. But she is an anti-harassment advocate. She’s running this wonderful organization called the Cyber Civil Rights Initiative, to raise awareness, change laws. They’ve been incredible, so brave. But when we first met, she could barely tell me her real name, and would call me under different phones. She was afraid she would get hacked. So going public was a huge step for her in 2014 and 2013. But she did it, and she’s trying to change the way we understand this kind of online abuse.

Cyber-harassment: the harm

[1:01:22] But if you can imagine, it’s a hard thing to swallow. And victims…so the damage–we can imagine what the damage is–of all this abuse. Victims change their lives. So Anita did–she couch surfed. Holly moved–she changed her name, right?

The harm – employers google you

[1:01:40] We know that the professional costs are incredibly steep. So people lose their jobs, or it’s impossible to get one. Why? Because online searches are our CV. So, we know that over ninety-five per cent of employers tell us that they are searching people in order to figure out if they are going to interview or hire people. And there are companies that are working on mining data online, to identify candidates. So sometimes you don’t have control over this, right. What a Microsoft study found, that over 80% of employers who are searching people, there’s a negative result. And why does that make sense?

From the employers perspective, it makes eminent sense. Do you want a client saying to you, “Why would you hire that person? Did you see in a google search, like, what this person did?” And it’s not that employers believe that individuals have posted their nude photos online by themselves. It’s not that they believe that Holly’s a prostitute or whatever, right? It’s so much safer, and cheaper and easier to hire someone who doesn’t come with baggage. Just that simple. And victims really struggle emotionally. So many told me, like, “it’s so hard not to wake up in the middle of the night and google yourself, because you think, ‘what next shoe is dropping'”.

The harm – silencing

[1:03:07] And it is silencing. This is a perfect community to talk about all the wonderful things the internet offers–all the knowledge creation, and access to knowledge that it provides. But for victims of online abuse, they feel like the more that they engage online, the more provocation it is for their harassers, so they totally disappear. It’s just safer, and it’s going to provoke less abuse. So they lose the opportunity to do all of the things that we’re all engaging in, politically, socially, economically — all the wonderful things that all these network spaces offer, but they can’t enjoy it.

So sometimes I’m often—this was in the beginning of my work, people sometimes say, “Danielle, can you relax, like, stop making a mountain out of a molehill. It’s one or two bad cases.” And the bottom line is, I wish I could say it was just one or two bad cases, but the Bureau of Justice statistics has made clear in a number of studies, that at least it was estimated in 2006–so you can imaging this was a while ago, it’s clearly gotten worse—that over 850,000 people experience stalking via network technologies every year. And the study defines stalking in the way that I initially...remember my first initial definition?...so it’s not a loosy-goosy definition. It was a clear definition of this sort of study. And we know that...so, Pew came out with a report last year that found this is especially so for women in their 20’s. That at least 20 per cent of women in their twenties will experience cyber-gender harassment in the way that I’ve described it.

Profile of abuse tactics

So, we know that the majority of victims of online abuse are women, but men experience it too. And the playbook for men and women is — it’s so strikingly the same. Every time I get another email or phone call, I’m like — it’s as if I press ‘rewind’ and ‘play’ each time. The abuse is sexually threatening and it’s sexually humiliating. So for both men and women, it’s not just any old threat, but it’s either threats of anal rape, impersonation, suggesting for men that they’re interested in rape, or anal rape – anonymous rape – and for women it’s rape threats. It’s the same thing. It’s sexually targeted threats. And it’s not just any old lie or defamation, it’s sexually humiliating lies. So, individuals are accused of having sexually transmitted diseases, right, they have herpes or AIDS or so alleged. They’re accused of being prostitutes and available for sex. And it’s not just any old privacy invasion, because golly, we know we can invade privacy in so many sundry ways, but it’s often either the theft and hacking and then posting of nude photos that were shared in confidence — the breach of that confidence, and then the sharing online, often alongside personal information.

What can the law do?

[1:06:06] So, it’s a bleak picture now. So, I’m a law professor, naturally I’m going to incline towards what law can do. And it can do something. It can’t solve all of our problems, but it can do something. So what can it do? What can law do right now, and then what are some reforms that I think we should … [phone rings in audience]… Don’t worry, it’s all good, right? (laughs) …What are some reforms that I think we really need and are working on already.

Civil law

[1:06:36] So, what can law do right now? So often victims are told, just sue your harasser, if, of course–if–you can figure out who they are. There are civil claims that victims can bring against their harassers. And it’s absolutely true there are well-tailored torts, like defamation, intentional infliction of emotional distress—[unintellible], privacy tort, public disclosure of private fact. But really, the problem is the practicality of it. It’s so expensive to bring a lawsuit. I always think it’s sort of fabulous and fiery, but not in practical fact. So, it’s so expensive to sue that the civil system, and the kind of corrective justice it could provide for us, is often just unavailable.

Criminal law

[1:07:24] So what about criminal law, right?

At the federal level, which has constrained resources, there are really well-designed federal cyber-stalking — threat, extortion — laws on the books. But the problem is, I think, as the FBI often tells victims, like, “look, this is not an issue of national security, it’s not terrorism, it’s not drugs, we’re not involved”. And they do have their own set of priorities. So, I think to overly rely on the FBI and federal law enforcement is, I think, too tall a task.

[1:08:02] The states have stepped in and long been the first movers, when it comes to sex-stalking and harassment. California was the first in the nation to pass a law against stalking in 1990. And then, within three years, everyone followed, that is, all 50 states. So, you know, we traditionally look to the states. So, about half of the states have really well designed– or they have laws that would reach online abuse. So they have stalking laws, anti-harassment criminal laws, threat laws, laws against solicitation and extortion. In 25 states, we now have laws that criminalize invasions of sexual privacy, when you go after the knowing privacy invader, like the knowing person who posts revenge porn on revenge porn sites, the initial poster.

Problems with enforcement

[1:08:49] But the key problem is enforcement of these laws. So when victims… in describing some of the kind of trivializing that we do to online harassment…. You know, a key problem is that when people go to law enforcement, they’re bedeviled by the same social attitudes that many are bedeviled by, which is “turn your computer off”, is what they’re told. “Oh, boys will be boys.” Like, “Ignore it, it will go away.” And some of that comes from—I have talked to so many law enforcement officers, so it’s not to say that they are, you know, uniquely have really poor attitudes. They’re also just not used to the technology. Like, it’s not a B crime, right? It’s not off-line assault where they say, like, “I’m good at this”. This involves technology, and often they just aren’t familiar with the idea of getting a warrant for an online service provider, to trace a IP address, and then to so produce the warrant to the ISP to figure out if it’s a static IP address–who is this person? They’re like, “whoa, we’ve no idea what we’re doing”. And they also are unfamiliar with the laws, right.

Training law enforcement

[1:10:01] So, we need a whole lot of training. So, it’s work I’m doing with the attorney general of California, Kamala Harris–is training law enforcement. On Wednesday I’m going out to LA to, sort of, announce and roll out this exciting project that we have in training law enforcement. So we have some amazing law enforcers on the beat. But it is really the beginning stages. It’s really the first of its kind in California, and we’re hoping it’s going to catch fire across the country, the training of law enforcement in the laws and the technology. And in getting assistance maybe from the FBI, in training officers, how to approach and investigate these cases.

Inadequate state laws

[1:10:37] But it’s also true that in about half of the states, they just do not have laws on the book, that reach this kind of abuse. So you’re got an aggravated harassment law in New York that only covers communications that are sent directly to victims. So, case from New York: Ian Barber posts his ex-girlfriend’s nude photos on Twitter, he sends them to her employer, and then to everyone in her in-box, including all of her friends, and work contacts. And he’s arrested for aggravated harassment. But rightly so, the court dismisses the indictment against him. Why? Because he never directly sent the harassing abusive communications directly to her. And that makes sense, just the way the legislators wrote that law. So we’re got some work to do at the state level. We’ve got to update harassment and stalking laws that were written in the 90s that are–and some of them are like “email harassment” laws–do we want to have a moment and laugh– or “telephone harassment” laws, which always makes me giggle, right. We’ve got to update those laws; they need to be technology-neutral. And it’s work we can do. We just have to engage some of our state law-makers, among the many issues that they have to worry about. You know, I think we can engage them with this.

Progress with state laws

[1:11:59] You know, we’re seen some progress, 25 states. So Holly Jacobs, the Cyber Civil Rights Initiative, and professor Mary Ann Franks, a colleague of mine at Miami who has been basically working with state legislators across the country. We went from 18 months ago, having one law, New Jersey, that criminalized an invasion of sexual privacy—and that was the law that was invoked to punish Tyler Clementi‘s harasser. Remember the roommate who tapes Tyler having sex with a man... views it...so, surreptitiously watches him having sex with someone and shares it with friends, and he’s then convicted of an invasion of sexual privacy. New Jersey had this law in the books since 2004. Now, 26 states, within 18 months, 26 states have criminalized invasions of sexual privacy. So we’re making some really important movement in the law.

The law: a civil rights approach

[1:12:48] The, kind of, court argument in my book for the way to understand the laws, not only that...look, what we’re seeing is torts, and we’re seeing crimes, but we should fundamentally understand the abuse as a civil rights violation. Why do I say that?

You know, we always think of civil rights violations, or have long thought of them, is when you interfere with people’s substantial and tangible life opportunities. And you do that, because of their membership in a protected group. So think about Anita Sarkeesian. Like, what did the cyber-mob know about Anita? But that she was a woman writing about sexism in video games, truly. And for that, the wrath of the rape threats, privacy invasions, defamation, intimidation, and attempts to basically stop her from doing her work. DDOS attacks aimed at Feminist Frequency, her website, shutting it down intermittently over the years, So, you know, Amanda (Hess) has said really eloquently in Slate—she was writing a piece for Pacific Standard—she writes for Slate. But she had this piece about online harassment in which she said, “Rape threats—what they say to all women, is that they can’t be at ease online.” And I think that that’s exactly right.

Online harassment is often designed to basically make people unemployable, right, and unable to express themselves, which is a fundamental liberty and right. And we have some work to do with our civil rights laws. But I think once we wrap our head around what this is, the law can do only so much. We can only enforce it only a little bit. We can’t [have] perfect-- and frankly don’t want perfect-–enforcement. But what the law does is teach us, right, it’s an educator. And it can help teach us that it’s the wrong thing to do, to help change both social attitudes and behavior.

What that means for free expression

The First Amendment, free speech, and unprotected speech

[1:14:48] So how is any of this, sort of, legal solution consistent with the First Amendment? So, I initially started by explaining–when I first stated talking about online harassment, I was told I was going to break the internet, that is, the internet as a forum for public discourse. And I think a crucially important lesson I think we’ve all now, with eight, nine, years behind us, is that we know that the First Amendment—as a doctrine—it doesn’t operate in absolutes, right? There are certain categories of speech that we know either receive no protection or less rigorous protection, because they give us so little and they cause so much harm.

Unprotected speech

[1:15:29] So much of online harassment is constituted by speech that has either no protection or very little protection. So, what kind of speech am I talking about? True threats, the defamation of private individuals about purely private matters, crime-facilitating speech like extortion and solicitation, and speech that enjoys less rigorous protection, are laws that protect the privacy of communications. So we think of wiretap laws and laws that intervene on behalf of privacy in our communications, we see that they get less — they’re not going to be subject to strict scrutiny. Why? Because they foster private speech. If we don’t think our communications are private, we’re not going to communicate, right. We’re not going to use our cell phones. We’re not going to share nude photos, right? If the assumption is that inherently there is the assumption of risk that they’re public, it will chill speech. And that’s why the Supreme Court has explained that we can uphold, and we have less rigorous protection of speech when what we’re doing is punishing knowing violations of privacy.

Why we protect free speech

Okay, so what about free speech values. There are so many reasons, wonderful reasons, why we protect free speech. We protect expression, and we think it’s incredibly important. Why? To help us figure out how to govern ourselves, right? Like, how to we know the kind of society we want to live in, if we don’t speak, if we don’t—I think for me more importantly—is to listen. If we don’t take in ideas, then we can’t figure out the kind of world we want to live in.

But I think it’s really important to realize, what is a rape threat, a nude photo of someone posted without their consent, and defamatory—like a iie—that contributes nothing to the set of ideas we need to figure out the kind of world we live in. Even offensive ideas, right? It’s not an idea that we’re punishing, right,

And really importantly, it drives people from self-governance, so from self-expression. As a victim said to me, “I can not be a digital citizen, when I’m under assault.” And that’s no surprise.

How to identify harassing speech

So another reason why we protect free speech is that we think of the marketplace of ideas, right, that’s like one of the more popular ways. We describe the kind of truth-seeking function of this, kind of, market place of discussions. And we think of, if there’s bad speech, or offensive speech, we counter it with other speech, and that’s precisely what we do and should do with hate speech.

But we’re not talking about annoying, offensive, degrading speech to groups. We’re talking about, like, what is there to say to a rape threat? “Don’t rape me?” What counter-speech is going to help have a conversation? What is there to say to defamation? “No, I don’t have herpes?” Right? I mean, right? I mean, is anyone going to believe it—I guess you could say it, but is anyone going to buy it? No. There isn’t the kind of conversation and truth-seeking function—like, it’s almost broken at that point. And it is true, of course, that harassers have expressive interests. I’m not going to say they don’t. They’re using ones and zeros, ultimately. A lot of this is words and images, so they are engaging in speech themselves. But their whole reason for doing what they’re doing is to silence other people.

And if that’s what is motivating you, your attempt to silence and destroy peoples' careers and lives, I think we should be less, frankly, worried about their speech. And the law is less worried.

Private companies: policing norms on their own sites

[1:19:15] But of course, we’re limited in what law can do, although there’s plenty that the First Amendment lets us do. But that leaves a huge gap. So often we rely on all of us. We rely on private companies that are not constrained by the First Amendment. Twitter and Google and Facebook, they’re not constrained by it. They’re private actors. They can do what they want. And federal law provides immunity. The choices that they make–and this is the Communications Decency Act, which I always find very ironically titled. It’s the law that explains the Good Samaritans [Section 230]. Good Samaritans, right, are online service providers. If they publish other peoples’ speech, we’re not going to treat them as speakers or publishers. Why? And it’s a great law. It’s incredibly important how it’s been interpreted, right?

But it also means that private companies can help police norms on their own sites. That’s the whole point of that law, right, is to provide the immunity, as the title calls it, for “Good Samaritan blocking” of offensive, and removal of offensive speech. It’s literally the title that these very conservative congressmen, senators and house of representatives–that’s what they wrote in the statute, right.

Reddit, Facebook, Google, Bing, and Twitter take action

[1:20:25] So, we are seeing private companies step into the act and ban online harassment, revenge porn. We’ve seen them do it. It’s been sort of revolutionary. In the last 18 months we’ve seen, one by one, after the fappening, and the iCloud hack. AG Harris had a convening with all of these companies, and I would say, about a month later, we had announcements from Reddit and Facebook, that they were banning revenge porn. And Google and Bing, that was the summer’s excitement, right, They announced that they will de-index nude photos of individuals who’d explain to them that it’s posted without their consent, right? And it was a struggle to get them there, right. Twitter has banned harassment–this is a big move for Twitter, right. It has long, in my conversations with them…Since its inception it always said it’s a speech platform. That is, “we do not ban anything, except for impersonations, spam, and of course copyright violations”, right. Which you can all have a moment and not like either. [laughter] No one is a fan of DMC in this room, including myself. “Credible threats” and chase-you-offline threats

[1:21:37] But, that said, in the last six months, they have banned specifically what they call targeted harassment, they have banned revenge porn, they have banned threats. And not just “credible threats”, the way which the law would strictly understand a true threat, but threats that are more amorphous, so Anita Sarkeesian explained to me is incredibly encouraging, right. Because so much of the threats that she faces on Twitter, they’re not “I’m going to kill you”, but they are “someone should kill you”. They’re designed to scare, and terrorize. But you’re doing it with just enough wiggle room, so that you’re not violating the law, but enough to make clear, that “I’m doing this to really chase you offline”. Due process: talking to the user base

[1:22:20] So, we see these companies moving in, but at the same time they’re doing this, part of my discussions with them, is that they’re got to be clear about what their terms of service and community guidelines really mean. How are they defining harassment, stalking, threats, revenge porn, or privacy invasions, right. And beyond just defining these terms for our user base, what do we do about it. Explain to your users, honestly and clearly, what happens when … what are the repercussions, right? And are there–and this will be very familiar to you guys, because I feel like you’re trying to do this really well–is this sense of due process, right. That when there has been speech that’s complained about, if you’re going to remove it or you’re going to ban someone, give them a chance to have something to say about it. Right? You know, most companies just “too bad, so sad. this is what’s happened to you”, and there’s really no conversation. And I think we’ve got to, sort of, adapt and move beyond that, because they so incredibly are speech platforms, are incredibly important for conversation, Digital citizenship

[1:23:25] So our broader, I think, system of free expression, I hope, and I think they’re listening, is going evolve to think about digital citizenship, that they often invoke that term, to justify their bans of harassment, stalking, threats and privacy invasions. But my advice is yes, digital citizenship is so important. It’s not only rights but responsibilities. But part of your responsibility as host of digital citizens, is to engage even the harassers, in the conversation about why you’re doing it to them, and being really clear. So there is some sense of we have some protections, right, for the people who are misusing platforms. Remember, shoving Anita offline, was reporting her site as hate speech, spam, and terrorism. So when you’re misusing abuse complaint systems, we got to get at that too, right?

Wikipedia and process

[1:24:15] So, I feel like this is such a wonderful audience, because Wikipedia–I always feel like… This is what happens: law professors always look at Wikipedia like you have so much process, [extended laughter–for a possible reason see WP:Wikilawyering and maybe WP:DRAMA] , you have this, really I know you’re laughing, but this is like some of my… I know, you can laugh louder. But we really admire you. Really. At least for your first principles, right, that you say “we’re going to have a dispute system. We’re going to…” I know you’re like, what are you saying lady, hold on…[laughter] no, “but we’re going to arbitrate, that we’re going to have some fair process, and engage in really hard conversations,” right?

So, I think it’s an exciting group to talk to, because you’re at the front lines of trying to work out what that means–due process and fairness, and bringing procedural regularity and fairness to online speech. That doesn’t mean you always get it right, so I’m not going to criticize what you’re doing, because I don’t know. [laughter] I have now many friends in the audience who are going to say, like, “we want to talk to you about this stuff”, so I think that’s great, [audience giggles] right, but at least it’s a really important model, I think, for others to follow. So I’m real excited for your questions.

Q&A

[1:25:30] Danielle Citron: Questions… I like that that I see folks going to the microphone. So I’m ready. [applause] Thank you.

Question #1: Official Wiki process and banning all the women

[1:25:34] Questioner #1 (male voice): Thank you for being here. I actually didn’t know you were going to be here. I’m using the [unintelligible–off mic] …vandalizing the Wikipedia article about revenge porn a few weeks ago and rewrote it using one of your law journal articles, or re-wrote at least a substantial portion of it, but a lot of my other comments are going to be directed actually towards the audience, just in contrast to your speech.

So, I get about twenty emails a week from women Wikipedians who don’t want to deal with any of the process on Wiki, because every arbitration committee case that has involved women in the last two years, has involved all of them being banned. The entire oversight team made a blanket statement that they were unwilling to oversight two words that were gender pronouns that had never been privately disclosed, or rather publicly disclosed. And in one of our breakout sessions about gender, the word I heard more than harassment, was Manchester.

Danielle Citron: Hmm, which, I may need a little clarification on that one.

Questioner #1: The audience will understand.

Danielle Citron: [crosstalk] Manchester. I’m with you all the way, until Manchester.

Questioner #1: [crosstalk] I’d be in trouble if I talked more about that.

Danielle Citron: [crosstalk] Really?…

Male voice: He’s teasing us.

Danielle Citron: Oh, is he teasing, I had a feeling…

Voice from audience: …user…[crosstalk in audience]

Another voice from audience: ….ohhh…[more crosstalk]

Danielle Citron: Okay, can I just say what I heard, and then maybe we can create like a question or conversation, right? So, what I think I heard–and thank you so much for your comments and question–because you said, really, you were saying it more as a dialogue with the audience, right, which maybe I can kind of spark, is that I think I heard you saying that women editors, female editors were being banned, right, as a result of disputes. Am a wrong? Is that what I heard?

Questioner #1: [off mic] That was what you heard.

Male voice: [audience crosstalk] Yeah, that’s correct.

Questioner #1: [off mic] Every major arbitration case, to the best of my knowledge, in the last two years, that has involved a woman editor has had her banned, even in situations where behavior on the other side would have also–I mean some of them deserve to be banned, but behavior on the other side also warranted sanctions, that were not put in place. For reference, I moderate our Gender Gap mailing list, I seriously regularly receive twenty to thirty emails a week related to Wikipedia-related problems from women who do not want to participate in any of our official processes because of what happens to them when they do. And I’m just going to leave “Manchester” at “Manchester”.

Danielle Citron: Okay. No, that’s fine. What’s really interesting is–what I’m hearing–and I just want to look back in history, right. So it’s true that, like, if we think about these, sort of, important turning points in the way that we change our attitudes. If we think about, sort of, women in the workplace in the 60s and 70s, right, the workplace that my mom, right, a lot of our parents grew up in, you know. If a woman objected to sexual harassment in the workplace, the response was like “beat it”. You know, “you don’t like it, leave”, right? And that kind of punishment is not–that’s just the way we said to people “you don’t like this culture, get out of here”, and so, I guess–and I’m not suggesting your community is doing anything consciously, right?

Questioner #1: I am. [laughter]

Danielle Citron: And that’s okay if you are. but…and these are important discussions to have. I so value being here to hear you have these conversations, because you know, so often at least as a consumer of–I love Wikipedia. So, I’m just–I eat it up. Like, I go…you know. I just observe. I don’t edit, right, and I admire people who do. But…do you know what I’m saying? I’m one of the people that admires fans using it. And you know, the conversation about how there’s so few female editors has always depressed me. So I think having this important conversation and you were saying “I want to be part of the solution”, right, “I want to make it easier and more welcoming for women to be part of this important endeavor” is really wonderful. So thank you, right, you know. So that’s sort of my response, I think.

Question #2: Silencing victims of harassment

[1:30:16] Questioner #2 (male voice): Hi. Thank you so much for your talk. On the subject of, sort of, norms and phasing people out of the community if they, sort of respond to sexual harassment, I really apologize, so hit your conference bingo cards: this is a comment not a question. So, recently one of the particular cases that Kevin was talking about was an editor who had faced pretty extreme sexual harassment and was involved in a dispute on Wikipedia. Someone had modified nude images of someone else and put them up on a site with their name, and among other things — not saying this editor was otherwise blameless — but among other things the arbitration committee banned them for seeking out the identify of the person who had placed those things up there, because, you know, they didn’t feel that they had any other alternative, And one of the, sort of, proposed decisions in this case basically said that the way editors should respond to harassment was “an editor who is harassed and attacked or who generally perceives themselves to be harassed or attacked whether on Wikipedia or off should not see that harassment as an excuse for fighting back or attacking those who are criticizing them”. Explicitly–explicitly there and implicitly on other pages–telling people that the way to respond to this is to lower your profile. And so, that’s really all I had to say, I think it’s a shame. [applause]

[1:31:39] Danielle Citron: And I think that’s the, you know, the devil in all of this, is that it’s so silencing, right, that it troubles me that if we were to say to people who have been targeted, that they shouldn’t at the very least be able to talk about the abuse that they’re facing–it strikes me as the wrong move. But I often think of, sort of, self help as inevitably troubling because the smash-back by harassers is always worse. You know, like, self help or vigilante justice, right, always comes with–it’s a gamble, right? So when victims and their supporters try to, sort of, talk back–and I’m not talking about Wikipedia–I’m just talking generally. You often see a cyber mob that then comes back so much harder at victims, right? So that it’s just perilous–it’s personally perilous–for victims and their supporters. But I hope that in thinking forward about your, kind of, policies and, sort of, norm-creation on your site, that you, you know, think hard about how hard it is for people who face, you know, their own photos have appeared on line, or doctored photos, and all that kind of abuse is so silencing. So the very fact that you’re able to in some respects defend yourself is huge, right, it’s a huge leap forward in many ways, right, and we ought to credit that, and think hard about what that means, and how we want to process that, So I’m not going to tell you how to think about it, right? But I think it’s something you should think hard about.

Question #3: Coordinated offsite harassment

[1:33:10] Questioner #3: (male voice) Hi, I’m Gamaliel, and I have some things to say about Gamergate. [laughter] Wikipedia, I think, is completely unequipped to deal with harassment, especially this new paradigm of coordinated offsite harassment. Two problems–I’ll make two points and then turn this over to a question, I promise–the first one is that we have this distributed volunteer method of policy enforcement, and so what I’ve seen is an effort to ramp up the harassment so hard that no one wants to get involved in the drama. I’ve had so many administrators tell me privately, I’m looking at this and you’re doing okay, but I don’t want to get involved, I don’t want to get harassed, I don’t want to get doxxed. I don’t want this to happen like it happened to you. And so, they’re working the refs, is what I call it, keeping people from the situation, keeping people who want to enforce policy away from the situation, so they can do what they want. And the second point is we have this paradigm on Wikipedia, we come out of this paradigm: we have rights and access. We worry about the rights and the access of the harassers: we need to offer them a path to rehabilitation, what about their right to edit, and all this; we don’t think about the people they are harassing off Wikipedia — what about their right to edit?

Danielle Citron: Right.

Questioner #3: And what about their right to access the encyclopedia. So we have that paradigm, and even without that paradigm, we have this problem of enforcing the rules that we do have. So my question is: what can we do, what suggestions can you offer to us as a community, that we can address these issues in a new way, because we’re not doing it–at all–correctly–now.

Danielle Citron: So, that’s a great question and comment, and, you know, I think your comment helps answer some of it. That is, we’ve got to prioritize, and think really hard, and I hope what I’m conveying to all of us today is that the cost expression on the person who is targeted is something we have to work hard to protect, right, their expression. So I think, prioritizing the person who’s being targeted, so that they don’t slink away, and you lose their voices. It’s a real social cost; it’s not just a cost to the individual, but it’s a cost to all of us, right, who could hear their voices. So I think, if we don’t–if the balance is off, and we’re only thinking about the harasser’s access and ability to speak, and write, and you know provide “knowledge” if you call it that, if they’re posting nude photos, or whatever it is, and rape and death threats, I think we could be more circumspect, right. I guess my call is for us to think really hard about what it is they’re expressing, and contributing, right. So if they’re contributing very little, and crucially, driving people from partaking and engaging–but I think you ought to think hard about if you want them in your community. I mean, this is your community to self-police, right, so I don’t want to tell you what to do, but I think it sounds like the balance is skewed, and you’ve got to bring the speech interests of the targeted person back into the picture. Right? And it may help you re-calibrate. It’s not to say it’s easy, right, so as your folks were telling, the administrators were saying like “hey, we don’t want to get doxxed”. I mean, honestly I was doxxed, it’s not fun–on Gamergate thing–platform, I was like “shoot, right, like this is not a good time”. My whole family lost their minds. But you know, imagine Anita Sarkeesian. So no one wants to be, right, no one wants to be targeted. It’s true the blowback on anyone who intervenes, is “how dare you”, right. The, sort of, mob is “nooo”. It’s amazing, right, it spirals way out of control. So I appreciate that no one wants skin in the game, right. So maybe there’s–I don’t know if it’s possible–but to kind of insulate the decision makers, make them less transparent so you can’t — I don’t know if that’s a possibility, right, but ways in which you can help intervene, so that the outside mob–a lot of this is what you’re talking about, isn’t destruction happening from within your community, but outside of it. But maybe there are ways in which you can, sort of, change how you present to the public about who’s doing some of this blocking or fixing, you know, like when you’re monkeying with Anita Sarkeesian’s site–if we shut it down, right? So we can’t go after the people making the decisions, the mob that’s outside of Wikipedia. We’re gonna “protect your own”, so to speak. I mean, maybe you gotta rethink some of those policies and transparency. Transparency is not an unalloyed good. I mean Lawrence Lessig did a sort of terrific work on this. We know it’s, it’s– we need to temper it, right, sometimes. So, it may be self-preservation that you don’t tell the public who is doing this sort of blocking, right? So that you can police your community, and not put your own hides on the line, right? And I think we may have to rethink hard about ways in which we need to start thinking about the targets of harassment, and putting them closer into the picture. So thank you

Questioner #3: Thank you.

Danielle Citron: Thank you.

Question #4: Conduct policy for technical spaces – “precision in recall”

[1:38:31] Questioner #4: (male voice) Thank you very much. This is a topic I’ve been very close to recently because we’re been working on a code of conduct policy for our technical spaces around Wikimedia things. So, I hope you’ll forgive me, I’m a computer scientist, and so I want to like, throw some jargon at you, but I’ll define it. Because I think it’s important for the question I want to ask.

So, with any rule that we put in place, like we like to talk about things as “precision in recall”. “Recall” is “how much of the bad stuff are we getting with this”, and then “precision” is “how much of that stuff that we get is actually bad”. And so like, precision isn’t very good, if we for example made a rule that was very easy for people who are harassed to report that harassment, but it also enabled people to use reporting harassment,..

Danielle Citron: “…to harass other people. Sure.”

Questioner #4: Exactly. And so what I want to ask about is, if you have some general advice, or rules of thumb, or maybe strategies that we can follow to keep “precision” really high, so that we can shut down the arguments, that, you know, “people are going to use this to harass me”, they’re going to use the harassment policy to harass me.

Danielle Citron: So I’m going to UCS with you, so, audit trails. To get at ..we need some traceability. That sounds like–I’ve always thought Wikipedia is like beautifully working with audit trails, right, and all sort of “recall”, right, to know who’s doing what. So that when the harassed person says, “I’m being harassed”, and the harasser says, “no, no, you’re harassing me”, right, that, sort of, like, nonsense cycle, which may or many not be true. Sometimes it’s true, let’s assume it’s not. Then you have some audit..there’s some ability to have that recall, and have precision, because you have very complex rich audit trails, that tell you what’s going on, right? So how do we know the code isn’t buggy? Right? We test it, right? We test it, we have audit trails and that’s ..i think you gotta bring that to this project. Right?…is what I would say, and that’s maybe, you’re already doing it and you want more, (laughs) [crosstalk] … we’ve been there and .. why isn’t that helping you, Why don’t audit trails get you there?

[1:40:37] Questioner #4: Yeah, so, I mean, I think the thing that we struggle with is [crosstalk]… I mean there’s a lot of rhetorical back and forth that we need to do in order to even put this thing in place. So even if the system will work, we need to convince people that they should sign on to it, so we can even try it in the first place. And so the thing that we’re struggling with now is, yes, we’re making, like, a place where you can safely report harassment, and somebody will turn around and well, now somebody is going to in a very hidden place, report that I’m harassing them, and you know, work against me, And it’s probably a boogy man, and definitely something that we can iterate on and improve, but getting past that threshold is the thing that we’re really struggling with. Getting it so we can try it so we can iterate on it.

Danielle Citron: Don’t we do that? With software, we experiment, we throw it out there, where it’s open source, let’s do it–or open code? So let’s experiment. So sorry, Yeah. [laughs]

Question #5: Arbcom’s gender problem

[1:41:27] Questioner #5: (male voice) Hello, this is Smallbones here. I know Wikipedians don’t like to politicize things. I’m going to continue on Kevin’s comment. Arbitration committee has been a major problem. There is an election, I believe it’s in December. I believe there are four seats up, and I would love to see four women. [applause] And I will also say, if my memory serves correctly, if everybody in this room voted for all four women, there would be I think now five women

Voice in audience: Canvassing. [laughter] [crosstalk]

Questioner #5: Yeah, ah. I think I’ll just leave it there, unless you want me to ask a question. [laughter]

Danielle Citron: No, it sounds like a great call to action. Right?

Voice from audience: Thank you.

Danielle Citron: And you have to feel like, so that, I mean i think this is going to be obvious, but we have to feel safe if we’re going to put ourselves out there. Right? So, I’ll leave it there too. Right?

Question #6: All-male arbitration committees

[1:42:40] Questioner #6: (female voice) Thank you. I would like to ask you as a legal scholar, if there are constitutional insights or legal insights that could help Wikipedia come up with better processes, so that it doesn’t have all male arbitration committees, given the fact that such a disproportionate number of the participants are men, even though of course there are many men in this audience who are supporting the anti-harassment, which I think we also need to bear in mind it’s not just women who can be supporting this–you know, the whole “He for She campaign“–but clearly, if Wikipedia is going to be “the encyclopedia that anyone can edit“, and as you mention, going to produce knowledge that reflects the experiences of more diverse people, it needs to think really hard about what are the processes that put people in place in Wikipedia’s power hierarchies, that lead to decisions like banning women who complain about harassment, and all the ways of silencing them. But I do think that Wikipedia often tries to think about these processes without looking to the insights that come from all these legal structures, and constitutions and so forth around the world. And if there’s a way to make that information more accessible to those who are trying to sort though these rules, it would be really helpful.

[1:44:12] Danielle Citron: Right, So, I love this question, of course, because it asks me to think about, sort of, due process, which I’ve done a bunch of work on. So I have an article called “Technological due process“ ^[1] (Berkman Center video) ^[2], and it’s really about automated decision making systems, but there’s so much to draw on–the lessons of procedural due process. Like “what is a fair hearing?”, right? And there’s a whole literature on, like the core components of a fair hearing and what that means and it includes an impartial decision maker. Now in our, sort of, doctrine of what that means, it basically means there‘s no conflict of interest, that is, you’re not invested in the company that you’re so adjudicating. So you, sort of, spoke about gender, right, how is it that we’re going to … how can a man adjudicate where another man is the harasser. It’s not always men who are the harassers, it could be women, of course, too, right? But I think we haven’t, at least in the law, that lesson, we’ve never said that, right, We think really narrowly about the way in which we understand a impartial decision maker, right? But the due process tradition also teaches us that we should give people a chance to bring evidence forward, and have the ability to present evidence and to have council if they so want it. That is, we provide, at least when we’re talking about having a hearing, right, if the government wants to take something away from you, a benefit of some sort, a license, then they owe you some fair hearing of some sort, some process, either before or after adjudication. And so there are some lessons I think we can take from it. And ref (?), so the impartiality thing is something that’s really narrowly understood. Like what do we mean by an impartial decision-maker, that maybe you can develop and think through in a way that’s more reflective of the Wikipedia experience.

Questioner #6: Thank you.

Question #7: The abusers

[1:46:02] Questioner #7: (woman’s voice) Hi. Okay, so you had a lot about the victims, but often the problem is a lot more two-sided–or no, multi-faceted than this, because the victims can often be abusers themselves, and a lot of the abusers are often victims in their own right, and then of course there’s also just abusers as trolls or bullies or whatever. So, what I want to ask is, what about the abusers, like what are their motivations, and how can this basically affect how we deal with them.

Danielle Citron: So, I mean, some of the answer about who these folks are is really hard to answer, because unlike in Wikipedia most other sites people are either pseudo-anonymous or anonymous and people can’t trace them, right? And if law enforcement never intervenes, we’ve no idea who they are. For the most part, law enforcement hasn’t. So I can’t–does that make sense?–like, even in Anita’s case, we’ve not seen any forward movement on the law enforcement piece. So it’s very hard to make a set of assessments about a set of people we have no idea who they are, and it’s just seriously understudied, the question, at least from a social science perspective, right? Who the abusers are , you don’t know. It’s true that in my book and in my work, I am focused more on the person who is truly targeted and isn’t striking back, And that is the better part of the story for harassment and stalking, outside of Wikipedia, right? That is, when you face–either it’s in a domestic circumstance, which you then recruit and you have, sort of, cyber-stalking by proxy, you’ve got people that help you, right? You get a mob going. It’s sort of one person who knows the victim and then starts a flame. And usually it’s someone who you know, right? But it’s true that strangers can engage in it, and we just don’t know who they are, right?

Questioner #7: So perhaps that *is* something to study, perhaps not the individuals themselves, but the psychology behind it?

Danielle Citron: Absolutely. No, no,.. so a lot of my–like when I talk at universities, is like “we need you social scientists” right, like I’m not writing about state AGs and privacy and norm entrepreneurship, so I’m kinda, not that I’m leaving the space, but we need more work “I wrote a book, I’m good”, like we need more work, we need to pass this on to another generation of people to think about. So I think that’s right We do need to think about that. But I think it’s also unfair to say that–it’s true that, I actually know a bunch of cases where a) victim became a harasser and a harasser has become a victim. Does that make sense? And there is a whole psychological story that–I am a lawyer, right, that I can’t answer–I don’t want to get into stuff I don’t really know. Does that make sense? But we do know that domestic violence perpetrators are victims too, from their childhood — like, this is not a surprising cycle or story, right, of abuse, it’s just one I’m not an expert on.

Questioner #7: All right, thank you.

Danielle Citron: …but it’s a valid one to think about.

Question #8: Harassment from Wikipedia criticism sites

[1:49:04] Questioner #8: (man’s voice) I want to take it from just a tad different angle. When I was fifteen years old I was raped, or nearly raped, rather, by a younger boy in my high school. And it took me eight years of traumatic work to get back to normality, but since then –in Wikipedia I have a lot of friends, a lot who are gay–I’m not–but, there has been harassment in getting people to these conferences, including Wikimania, the international conference here in 2012 , And it’s becoming a case of –unless there’s absolute work done and absolute pushing we don’t have any way of dealing with this, plus, for those who have been here on the site a long time, we have two websites–I will not advertise them for obvious reasons, that are ex-Wikipedians or Wikipedia critics who take it upon themselves to harass other Wikipedians in a stage where usually nobody looks. I mean, it was pointed out to me, I had very open on my user page, and people were harassing me on the website looking up my credit card information, and doing other things that really should be targeted. and yet I don’t exactly see any method, the Foundation won’t step in, to shutting these sites down, and getting rid, and having Google or something get rid of access to these, Because you google my user name, it’s going to show up, and who’s to say it’s fair.

Danielle Citron: Right, so let me take the–so let me just say thank you for sharing that with us, and I think it really important to note that my work, that of course it’s not just women but it’s so often sexual minorities, right, who face this kind of abuse. So I hope I was conveying that. If I didn’t, I apologize. And it’s true that, especially for women, the sort of darker their skin, or the more they are sort of the non-normal, non-traditional story of sexuality, the abuse is so grotesque, right. So I think you’re totally right, is like, Danielle, also widen your lens, it’s true that LGBT folks get like really harassed online, and it’s absolutely right. But your question is, okay, so just quick clarification, the two sites you’re talking about, is that within Wikipedia, so that it’s

Questioner #8: It’s…

Danielle Citron [crosstalk] …it’s outside…

Questioner #8 : They are not owned by us but they are sites basically dedicated about us.

Danielle Citron: Right, We can’t control them right. And so of course, you know, what’s interesting is, you said, if we’re going to publish peoples’ credit card information, though, this is one area which, normally we think of truthful fact. We don’t inhibit that, that is, we have this profound commitment to truths, so even a credit card number is a truthful fact, that even if betrayed in confidence, if it’s posted online, you know, we might say look are you going to punish someone, but in fact, one of the very small areas, where we say nude photos, credit card numbers, and SSNs–you can actually sue someone for public disclosure of private fact, like, so yes a commitment to the first amendment, but that can actually–a credit card number is like a key to your bank account, right, so that we don’t think it’s speech-producing or truth-producing, in that way that we think of truth, so I mean you could–google ‘s policy for search is for credit card information, social security numbers, and now nude photos, they’ll de-index those sites if your card data is there, just say.

Questioner: Its not that. the two sites in question also have done harassment of me on the website, and many other editors over the years, and yet we still somehow let Google link to them and we still condone their existence on the internet despite the very vulgar and very questionably…

Danielle Citron: Yeah, the same problem that victims face, does that make sense, like that’s where the rubber does hit the road, in our commitment to free expression.

Voice from audience: I think one of them is hosted overseas.

Danielle Citron: Right, and then we can’t even control those folks, so to the extend that they’re engaging in criminal activity that we can proscribe and regulate, we do often have jurisdictional struggles, right, we can’t get at the abuser because we don’t have the resources to extradite them and there’s no extradition treaty. So we have to have both the will, the resources, and the, sort of, legal grounds, to grab the person, and we may not have them. So thank you.

Question #9: Bias in all-male consensus process

[1:53:36] Questioner #9 (male voice): There was something left dangling earlier. You were asking why the audit trail doesn’t help in establishing whether there was harassment or not. There’s several reasons for that. One of them is that Wikipedia’s archives are absolutely voluminous, they’re huge, and to retrace all the instances where a person communicated with another person, and which article they were communicating about, and what this article looked like at the time, and what the issue was at the time can be incredibly complex, and quite apart from retracing everything that’s happened, maybe over a period of years between a couple of contributors. The other problem is that people will see it from different perspectives, so if you have a woman arguing and a man arguing like in the recent arbitration case with an editor called Lightbreather who was in many ways a very valuable voice in the Wikipedia which has now been silenced, because she’s gone. I really loved the way she spoke up for women’s perspectives. She was very outspoken. She got into many arguments about that as a result. What happens is, you have a community which is 90% male and who will tend to sympathize more with the male half of that conversation than with the female half of that conversation. So while–it’s difficult to get consensus that something is harassment, or that something shouldn’t be allowed, Someone mentioned Manchester earlier, the situation here a British contributor who uses English in a British way so he uses the c-word differently than people in the states would use it. And he insists on his right to use it when he feels like it. And he doesn’t really care how that affects women who feel that word should really not be used in conversation with them. And you know he’s a valuable…

Voice from audience: …a lot of Manchester people, and not just that…

Questioner #9: Yeah, okay, so basically he has got his defenders, and there are very many women who are simply flabbergasted that something like that can fly. Which is a similar situation where you’re dealing with a community that’s skewed gender-wise in the first place, And people’s judgement about what is harassment differs.

Danielle Citron: And that’s where I want to–that wonderful comment by — I don’t know your name– [crosstalk] yes, your question about due process and what that looks like. This is the community that can make its own choices, so when you have a case in which we have a woman who’s very outspoken, and who has been very outspoken for feminist opinions and is accused of harassment, I think that’s where you get three adjudicators, two are women. I mean, that is your choice, to decide who is on your review panel, right?

Questioner #9: Well, that is what the community is struggling with. because the arbitration committee is all–nearly all male, and …

Danielle Citron: And we might change that, no? Did we have like a political moment a second ago? No? [laughter] Right? We’re going to fix that. [applause]

Voice from audience: Isn’t that next July? [Crosstalk] November. Just look. [laughter] November – December. It’s already posted.

Danielle Citron: So I think we have some stuff we need to fix here. [crosstalk] We have work to do.

Questioner #9: And I think also some of this work needs to be done in public, you know, I think there need to be press articles about this situation, and if you can help with that, then that would be very..

Danielle Citron: Okay, yeah, I blog for Forbes, we can figure that out, right. [crosstalk] No but thank you.

Question #10: defining harassment, stalking, and threats

"Title 7 applies only to employers and employees, in spaces that employers can control. So while we can learn from civil rights laws–Title 7 and the Civil rights Act of 1964– we can’t directly import them.

[1:57:23] Questioner #10 (female voice): Hi, I’d like to speak up on behalf of women who live in the United States who have professional jobs, who are covered by the EEOC guidelines, on non-discrimination and harassment, who cannot bring their families, their agencies, their employers, etc. into disrepute. Here we are on this wild tumblin’ website, and it’s supposed to be open to everyone, and if you implement non-harassment for everyone, well, that includes people just coming out of prison or still in prison, it includes people in war zones, it includes people in fire stations and emergency response who have a legitimate reason to defuse a little tension now and then, and a morbid sense of humor. It includes places like a place where I used to live overseas where it was considered perfectly normal to require women to sleep with their boss, that was just part of life in that society. It includes regional differences, where somebody says something to you and it means you better clean their clock or feet don’t fail me now, and for them it’s just hey, you know, we’re hanging out with our friends doing our thing.

But, you know, here, you are, you don’t know if this person is local. You don’t know if this is some rap musician cuttin’ loose, or you don’t know if it’s a threat, and at the same time, you can’t say anything because you have the constraints of maintaining a professional job, the reputation of your family, your employer, and your agency. If there’s a certain level of social norms and a level of politeness in public, there’s a level of politeness in an international setting that’s pretty well determined here in Washington DC. In the meantime you’ve set up this, kind of, social group for people who are cuttin’ loose. And I have no idea–how do you work in the different cultures, the different nationalities, the different social norms, and the different requirements?

I mean, I am really tired of tiptoeing around a lot of total psycho-maniacs on this website. I’m tired of it frankly. I’m tired of it. [applause] What do we do.

Danielle Citron: I think like, you mentioned, let me just, because I think you have a couple of questions in there, so let me take at least two of them.

One question is how do we figure out what is harassment, stalking, and threats, when we have all sorts of… a) we don’t know what it means b) we have all sorts of different norms, depending on the countries we’re in, right. And we do have some instruction from the law, which is that how do we determine the facts? We look at all the facts and circumstances, we look at what’s said, we look at how it was said, we look at who said it, who is it directed at, right? We do have some cues, right, on Wikipedia, so it’s not like we’re helpless. I don’t think we are. I think we do have some measure of assessment.

As a community you can figure out how you want to take those cues and understand them, right, and work on helpfully providing examples to your community of what constitutes a threat, what constitutes harassment and stalking, like not just define it, but give examples. And give examples from within your own community, right, that is, editors vis–à–vis each other, what is not okay, right. And we have done it in the law, you can do it here. Right? You can make contextual assessments, so long as you know the question–the right questions to ask, right?

Now, I can’t solve the problem of like, different norms, international, the U.S., I guess I can only work with ours, right. And if that’s going to guide you, that is, our kind of U.S.-centric norms, I mean, with, …I mean what’s interesting is internationally you can’t say whatever you want. They have hate speech laws, I’m actually really shocked that you say that, right. They don’t have a first amendment, right, so in Canada, in France, you can’t…. Holocaust denial– to utter it is a crime, which in the United States, it’s not.

So it’s interesting to me that you say anything goes across the Atlantic, which I find shockingly not true, as a matter of the law. So let’s all ground ourselves in the reality of that, right, and I think we should take a U.S. approach, if that’s what you want to do, right, and we have lots of norms we can incorporate from stateside, from here, and I think we have the tools, right.

Your second question was about the workplace and what do we do when editors have like, full time jobs and like they’re being targeted and there’s a problem. And we know that Wikimedia isn’t–I mean, Wikipedia isn’t for most people their work place, so that–it isn’t in the way that we traditionally understand Title 7, right. There isn’t that kind of accountability. Title 7 applies only to employers and employees, in spaces that employers can control, So while we can learn from that civil rights, sort of, laws–Title 7 and the Civil rights Act of 1964– we can’t directly import them, right. But it doesn’t mean we can’t–and I guess that’s the point of my book–is for us to understand the economic consequences of online abuse, to realize that when we target someone, and it’s searchable, and it’s in the first page of a search of their name, that it truly interferes with their tangible life opportunities. And it’s something you as Wikipedians can take into consideration. So thank you. [extended applause] Thank you very much. Thank you.

I really appreciate that was a lot of fun. [laughter]

[1] ttp://papers.ssrn.com/sol3/papers.cfm?abstract_id=1012360

[2] ttp://blogs.harvard.edu/mediaberkman/2008/01/15/danielle-citron-on-technological-due-process-video/

[1]

[2]