Justin Cheng
Michael Bernstein
Jure Leskovec
ABSTRACT
In online communities, antisocial behavior such as trolling disrupts constructive discussion. While prior work suggests that trolling behavior is confined to a vocal and antisocial minority, we demonstrate that ordinary people can engage in such behavior as well. We propose two primary trigger mechanisms: the individual's mood, and the surrounding context of a discussion (e.g., exposure to prior trolling behavior). Through an experiment simulating an online discussion, we find that both negative mood and seeing troll posts by others significantly increases the probability of a user trolling, and together double this probability. To support and extend these results, we study how these same mechanisms play out in the wild via a data-driven, longitudinal analysis of a large online news discussion community. This analysis exposes temporal mood effects, and explores long range patterns of repeated exposure to trolling. A predictive model of trolling behavior reveals that mood and discussion context together can explain trolling behavior better than an individual's history of trolling. These results combine to suggest that ordinary people can, under the right circumstances, behave like trolls.
- Yavuz Akbulut, Yusuf Levent Sahin, and Bahadir Eristi. 2010. Cyberbullying Victimization among Turkish Online Social Utility Members. Educ. Technol. Soc. (2010).Google Scholar
- Jeff Atwood. 2013. Why is there a topic reply limit for new users? http://bit.ly/1XDLk8a. (2013).Google Scholar
- Lars Backstrom, Jon Kleinberg, Lillian Lee, and Cristian Danescu-Niculescu-Mizil. 2013. Characterizing and curating conversation threads: expansion, focus, volume, re-entry. In Proc. WSDM. Google ScholarDigital Library
- Paul Baker. 2001. Moral panic and alternative identity construction in Usenet. J. Comput. Mediat. Commun. (2001).Google Scholar
- Abhijit V Banerjee. 1992. A simple model of herd behavior. Q. J. Econ. (1992).Google Scholar
- Sigal G Barsade. 2002. The ripple effect: Emotional contagion and its influence on group behavior. Adm. Sci. Q. (2002).Google Scholar
- Roy F Baumeister, Ellen Bratslavsky, Catrin Finkenauer, and Kathleen D Vohs. 2001. Bad is stronger than good. Rev. Gen. Psychol. (2001).Google Scholar
- Leonard Berkowitz. 1993. Aggression: Its causes, consequences, and control.Google Scholar
- Amy Binns. 2012. DON'T FEED THE TROLLS! Managing troublemakers in magazines' online communities. Journalism Practice (2012).Google Scholar
- Galen V Bodenhausen, Geoffrey P Kramer, and Karin Susser. 1994. Happiness and stereotypic thinking in social judgment. J. Pers. Soc. Psychol. (1994).Google Scholar
- Erin E Buckels, Paul D Trapnell, and Delroy L Paulhus. 2014. Trolls just want to have fun. Pers. Individ. Dif. (2014).Google Scholar
- Irina Ceaparu, Jonathan Lazar, Katie Bessiere, John Robinson, and Ben Shneiderman. 2004. Determining causes and severity of end-user frustration. Int. J. Hum. Comput. Interact. (2004).Google ScholarCross Ref
- Damon Centola. 2010. The spread of behavior in an online social network experiment. Science (2010).Google Scholar
- Stevie Chancellor, Zhiyuan Jerry Lin, and Munmun De Choudhury. 2016. This Post Will Just Get Taken Down: Characterizing Removed Pro-Eating Disorder Social Media Content. In Proc. SIGCHI. Google ScholarDigital Library
- Daphne Chang, Erin L Krupka, Eytan Adar, and Alessandro Acquisti. 2016. Engineering Information Disclosure: Norm Shaping Designs. In Proc. SIGCHI. Google ScholarDigital Library
- Zhansheng Chen, Kipling D Williams, Julie Fitness, and Nicola C Newton. 2008. When Hurt Will Not Heal: Exploring the Capacity to Relive Social and Physical Pain. Psychol. Sci. (2008).Google Scholar
- Justin Cheng, Cristian Danescu-Niculescu-Mizil, and Jure Leskovec. 2014. How community feedback shapes user behavior. In Proc. ICWSM.Google Scholar
- Justin Cheng, Cristian Danescu-Niculescu-Mizil, and Jure Leskovec. 2015. Antisocial Behavior in Online Discussion Communities. In Proc. ICWSM.Google Scholar
- Daegon Cho and Alessandro Acquisti. 2013. The more social cues, the less trolling? An empirical study of online commenting behavior. In Proc. WEIS.Google Scholar
- Robert B Cialdini and Noah J Goldstein. 2004. Social influence: Compliance and conformity. Annu. Rev. Psychol. (2004).Google Scholar
- Robert B Cialdini, Raymond R Reno, and Carl A Kallgren. 1990. A focus theory of normative conduct: recycling the concept of norms to reduce littering in public places. J. Pers. Soc. Psychol. (1990).Google Scholar
- CNN. n.d. Community Guidelines. http://cnn.it/1WAhxh5. (n.d.).Google Scholar
- John P Davis, Shelly Farnham, and Carlos Jensen. 2002. Decreasing online 'bad' behavior. In Proc. SIGCHI Extended Abstracts. Google ScholarDigital Library
- Nicholas Diakopoulos and Mor Naaman. 2011. Towards quality discourse in online news comments. In Proc. CSCW. Google ScholarDigital Library
- Discourse. n.d. This is a Civilized Place for Public Discussion. http://bit.ly/1TM8K5x. (n.d.).Google Scholar
- Judith S Donath. 1999. Identity and deception in the virtual community. Communities in Cyberspace (1999).Google Scholar
- Maeve Duggan. 2014. Online Harassment. Pew Research Center (2014).Google Scholar
- Klint Finley. 2015. A brief history of the end of the comments. http://bit.ly/1Wz2OUg. (2015).Google Scholar
- Joseph P Forgas and Gordon H Bower. 1987. Mood effects on person-perception judgments. J. Pers. Soc. Psychol. (1987).Google Scholar
- Becky Gardiner, Mahana Mansfield, Ian Anderson, Josh Holder, Daan Louter, and Monica Ulmanu. 2016. The dark side of Guardian comments. Guardian (2016).Google Scholar
- Francesca Gino, Shahar Ayal, and Dan Ariely. 2009. Contagion and differentiation in unethical behavior: the effect of one bad apple on the barrel. Psychol. Sci (2009).Google ScholarCross Ref
- Scott A Golder and Michael W Macy. 2011. Diurnal and seasonal mood vary with work, sleep, and daylength across diverse cultures. Science (2011).Google Scholar
- Roberto González-Ibáínez and Chirag Shah. 2012. Investigating positive and negative affects in collaborative information seeking: A pilot study report. In Proc. ASIST.Google ScholarCross Ref
- Jeff Greenberg, Linda Simon, Tom Pyszczynski, Sheldon Solomon, and Dan Chatel. 1992. Terror management and tolerance: Does mortality salience always intensify negative reactions to others who threaten one's worldview? J. Pers. Soc. Psychol. (1992).Google Scholar
- The Guardian. n.d. Community Standards and Participation Guidelines. http://bit.ly/1YtIxfW. (n.d.).Google Scholar
- Noriko Hara, Curtis Jay Bonk, and Charoula Angeli. 2000. Content analysis of online discussion in an applied educational psychology course. Instr. Sci. (2000).Google Scholar
- Claire Hardaker. 2010. Trolling in asynchronous computer-mediated communication: From user discussions to academic definitions. J. Politeness Res. (2010).Google Scholar
- Susan Herring, Kirk Job-Sluder, Rebecca Scheckler, and Sasha Barab. 2002. Searching for safety online: Managing? trolling? in a feminist forum. The Information Society (2002).Google Scholar
- Chiao-Fang Hsu, Elham Khabiri, and James Caverlee. 2009. Ranking comments on the social web. In Proc. CSE. Google ScholarDigital Library
- Clayton J Hutto and Eric Gilbert. 2014. Vader: A parsimonious rule-based model for sentiment analysis of social media text. In Proc. ICWSM.Google Scholar
- John W Jones and G Anne Bogat. 1978. Air pollution and human aggression. Psychol. Rep. (1978).Google Scholar
- Laura June. 2015. I'm Voting for Hillary Because of My Daughter. http://thecut.io/1rNEeBJ. (2015).Google Scholar
- Joseph M Kayany. 1998. Contexts of uninhibited online behavior: flaming in social newsgroups on Usenet. J. Am. Soc. Inf. Sci. (1998). Google ScholarDigital Library
- Dacher Keltner, Phoebe C Ellsworth, and Kari Edwards. 1993. Beyond simple pessimism: effects of sadness and anger on social perception. J. Pers. Soc. Psychol. (1993).Google Scholar
- Philip C Kendall, W Robert Nay, and John Jeffers. 1975. Timeout duration and contrast effects: A systematic evaluation of a successive treatments design. Behavior Therapy (1975).Google Scholar
- Peter G Kilner and Christopher M Hoadley. 2005. Anonymity options and professional participation in an online community of practice. In Proc. CSCL. Google ScholarDigital Library
- Ben Kirman, Conor Lineham, and Shaun Lawson. 2012. Exploring mischief and mayhem in social computing or: how we learned to stop worrying and love the trolls. In Proc. SIGCHI Extended Abstracts. Google ScholarDigital Library
- Silvia Knobloch-Westerwick and Scott Alter. 2006. Mood adjustment to social situations through mass media use: How men ruminate and women dissipate angry moods. Hum. Commun. Res. (2006).Google Scholar
- Agata Kołakowska. 2013. A review of emotion recognition methods based on keystroke dynamics and mouse movements. In Proc. HSI.Google ScholarCross Ref
- Adam DI Kramer, Jamie E Guillory, and Jeffrey T Hancock. 2014. Experimental evidence of massive-scale emotional contagion through social networks. PNAS (2014).Google Scholar
- Travis Kriplean, Jonathan Morgan, Deen Freelon, Alan Borning, and Lance Bennett. 2012a. Supporting reflective public thought with considerit. In Proc. CSCW. Google ScholarDigital Library
- Travis Kriplean, Michael Toomim, Jonathan Morgan, Alan Borning, and Andrew Ko. 2012b. Is this what you meant?: promoting listening on the web with reflect. In Proc. SIGCHI. Google ScholarDigital Library
- Cliff Lampe and Paul Resnick. 2004. Slash (dot) and burn: distributed moderation in a large online conversation space. In Proc. SIGCHI. Google ScholarDigital Library
- Hangwoo Lee. 2005. Behavioral strategies for dealing with flaming in an online forum. Sociol Q (2005).Google Scholar
- So-Hyun Lee and Hee-Woong Kim. 2015. Why people post benevolent and malicious comments online. Commun. ACM (2015). Google ScholarDigital Library
- Karen Pezza Leith and Roy F Baumeister. 1996. Why do bad moods increase self-defeating behavior? Emotion, risk tasking, and self-regulation. In J. Pers. Soc. Psychol.Google Scholar
- Jerome M Levine and Gardner Murphy. 1943. The learning and forgetting of controversial material. J. Abnorm. Soc. Psychol. (1943).Google Scholar
- Tanya Beran Qing Li. 2005. Cyber-harassment: A study of a new method for an old behavior. In J. Educ. Comput. Res.Google Scholar
- Nina Mazar, On Amir, and Dan Ariely. 2008. The dishonesty of honest people: A theory of self-concept maintenance. J. Mark. Res. (2008).Google Scholar
- Brian James McInnis, Elizabeth Lindley Murnane, Dmitry Epstein, Dan Cosley, and Gilly Leshed. 2016. One and Done: Factors affecting one-time contributors to ad-hoc online communities. In Proc. CSCW. Google ScholarDigital Library
- DM McNair. 1971. Manual profile of mood states.Google Scholar
- Stanley Milgram, Leonard Bickman, and Lawrence Berkowitz. 1969. Note on the drawing power of crowds of different size. J. Pers. Soc. Psychol. (1969).Google Scholar
- Stanley Milgram and Ernest Van den Haag. 1978. Obedience to authority. (1978).Google Scholar
- Lev Muchnik, Sinan Aral, and Sean J Taylor. 2013. Social influence bias: A randomized experiment. Science (2013).Google Scholar
- J. Nathan Matias. 2016. Posting Rules in Online Discussions Prevents Problems & Increases Participation. http://bit.ly/2foNQ3K. (2016).Google Scholar
- Vlad Niculae and Cristian Danescu-Niculescu-Mizil. 2016. Conversational markers of constructive discussions. In Proc. NAACL.Google ScholarCross Ref
- Deokgun Park, Simranjit Sachar, Nicholas Diakopoulos, and Niklas Elmqvist. 2016. Supporting Comment Moderators in Identifying High Quality Online News Comments. In Proc. SIGCHI. Google ScholarDigital Library
- James W Pennebaker, Martha E Francis, and Roger J Booth. 2001. Linguistic inquiry and word count: LIWC 2001.Google Scholar
- Adrian Raine. 2002. Annotation: The role of prefrontal deficits, low autonomic arousal, and early health factors in the development of antisocial and aggressive behavior in children. J. Child Psychol. Psychiatry (2002).Google Scholar
- Juliana Raskauskas and Ann D Stoltz. 2007. Involvement in traditional and electronic bullying among adolescents. Dev. Psychol. (2007).Google Scholar
- Emmett Rensin. 2014. Confessions of a former internet troll. Vox (2014).Google Scholar
- Paul R Rosenbaum and Donald B Rubin. 1983. The central role of the propensity score in observational studies for causal effects. Biometrika (1983).Google Scholar
- Lee Ross. 1977. The intuitive psychologist and his shortcomings: Distortions in the attribution process. Adv. Exp. Soc. Psychol. (1977).Google Scholar
- James Rotton and James Frey. 1985. Air pollution, weather, and violent crimes: concomitant time-series analysis of archival data. J. Pers. Soc. Psychol. (1985).Google Scholar
- Paul Rozin and Edward B Royzman. 2001. Negativity bias, negativity dominance, and contagion. Pers. Soc. Psychol. Rev. (2001).Google Scholar
- Matthew J Salganik, Peter Sheridan Dodds, and Duncan J Watts. 2006. Experimental study of inequality and unpredictability in an artificial cultural market. Science (2006).Google Scholar
- Mattathias Schwartz. 2008. The trolls among us. NY Times Magazine (2008).Google Scholar
- Norbert Schwarz and Herbert Bless. 1991. Happy and mindless, but sad and smart? The impact of affective states on analytic reasoning. Emotion and Social Judgments (1991).Google Scholar
- Norbert Schwarz and Gerald L Clore. 1983. Mood, misattribution, and judgments of well-being: informative and directive functions of affective states. J. Pers. Soc. Psychol. (1983).Google Scholar
- Pnina Shachaf and Noriko Hara. 2010. Beyond vandalism: Wikipedia trolls. J. Inf. Sci. (2010). Google ScholarDigital Library
- Muzafer Sherif, Oliver J Harvey, B Jack White, William R Hood, Carolyn W Sherif, and others. 1961. Intergroup conflict and cooperation: The Robbers Cave experiment.Google Scholar
- Robert Slonje, Peter K Smith, and Ann Frisé N. 2013. The nature of cyberbullying, and strategies for prevention. Comput. Hum. Behav. (2013). Google ScholarDigital Library
- Abhay Sukumaran, Stephanie Vezich, Melanie McHugh, and Clifford Nass. 2011. Normative influences on thoughtful online participation. In Proc. SIGCHI. Google ScholarDigital Library
- John Suler. 2004. The online disinhibition effect. Cyberpsychol. Behav. (2004).Google Scholar
- Peter C Terry and Andrew M Lane. 2000. Normative values for the Profile of Mood States for use with athletic samples. J. Appl. Sport Psychol. (2000).Google Scholar
- Kris Varjas, Jasmaine Talley, Joel Meyers, Leandra Parris, and Hayley Cutts. 2010. High school students perceptions of motivations for cyberbullying: An exploratory study. West. J. Emerg. Med. (2010).Google Scholar
- Lu Wang and Claire Cardie. 2014. A Piece of My Mind: A Sentiment Analysis Approach for Online Dispute Detection. In Proc. ACL.Google ScholarCross Ref
- Yang Wang, Gregory Norcie, Saranga Komanduri, Alessandro Acquisti, Pedro Giovanni Leon, and Lorrie Faith Cranor. 2011. I regretted the minute I pressed share: A qualitative study of regrets on Facebook. In Proc. SUPS. Google ScholarDigital Library
- Ladd Wheeler. 1966. Toward a theory of behavioral contagion. Psychol. Rev. (1966).Google Scholar
- David Wiener. 1998. Negligent publication of statements posted on electronic bulletin boards: Is there any liability left after Zeran? Santa Clara L. Rev. (1998).Google Scholar
- Robb Willer, Ko Kuwabara, and Michael W Macy. 2009. The False Enforcement of Unpopular Norms. Am. J. Sociol. (2009).Google Scholar
- James Q Wilson and George L Kelling. 1982. Broken Windows. Atlantic Monthly (1982).Google Scholar
- Carrie L Wyland and Joseph P Forgas. 2007. On bad mood and white bears: The effects of mood state on ability to suppress unwanted thoughts. Cognition and Emotion (2007).Google Scholar
- Justine Zhang, Ravi Kumar, Sujith Ravi, and Cristian Danescu-Niculescu-Mizil. 2016. Conversational flow in Oxford-style debates. In Proc. NAACL.Google ScholarCross Ref
- Philip G Zimbardo. 1969. The human choice: Individuation, reason, and order versus deindividuation, impulse, and chaos. Nebr. Symp. Motiv. (1969).Google Scholar
Index Terms
- Anyone Can Become a Troll: Causes of Trolling Behavior in Online Discussions
Recommendations
The Bad Boys and Girls of Cyberspace: How Gender and Context Impact Perception of and Reaction to Trolling
A significant feature of virtual interactions involve online deviance and trolling; these include behaviors that range from mild mischief, to offensive language, to hacking and trolling, and to expression of complex social problem, such as by ...
From Royals to Vegans: Characterizing Question Trolling on a Community Question Answering Website
SIGIR '18: The 41st International ACM SIGIR Conference on Research & Development in Information RetrievalThe phenomenon of trolling has emerged as a widespread form of abuse on news sites, online social networks, and other types of social media. In this paper, we study a particular type of trolling, performed by asking a provocative question on a community ...
Identity and User Behavior in Online Communities
GROUP '20: Companion Proceedings of the 2020 ACM International Conference on Supporting Group WorkIn online communities, people share and discuss information at all levels of topic sensitivity. Identity policies within these communities range from real names to anonymity. The amount of user engagement, the quality of the information, disinformation ...
Comments