When the Filter Bubble Bursts: Collective Evaluation Dynamics in Online Communities

A diy a Abishev a, Da vid Garcia, F rank Sc h w eitzer: When the Filter Bubble Bursts: Collectiv e Ev aluation Dynamics in Online Communities Submitted to the 8th In ternational ACM W eb Science Conference 2016 When the Filter Bubble Bursts: Collectiv e Ev aluation Dynamics in Online Comm unities A diya Abishev a, David Garcia, F rank Sch w eitzer Abstract W e analyze online collectiv e ev aluation processes through p ositiv e and negative votes in v arious social media. W e find tw o mo des of collective ev aluations that stem from the existence of filter bubbles. Ab ov e a threshold of collective attention, negativity grows faster with positivity , as a sign of the burst of a filter bubble when information reaches b eyond the lo cal so cial context of a user. W e analyze how collectively ev aluated conten t can reach large so cial con texts and create p olarization, sho wing that e motions expressed through text pla y a key role in collective ev aluation processes. Categories and Sub ject Descriptors: Human-centered computing, Collab orative and so cial comput- ing, Empirical studies in collab orativ e and social computing Keyw ords: Social filtering, emotions, collective dynamics 1 In tro duction When the filter bubble bursts Reb ecca Black, an amateur teenage singer, p osted a music video 1 on YouTube on F ebruary 10, 2011. The song originally circulated mostly among the F aceb o ok friends of its 13-year old singer and was lov ed and positively commented. Reb ecca Black’s song received the "all the usual friends things" [36] and w as enough to please her, but it suddenly wen t viral in the wr ong dir e ction . F rom initial 4,000 views on YouTube her song skyrock eted to 13 Million views. This sudden p opularit y brought mostly negativ e atten tion, up to the p oin t of b ecoming officially the most dislik ed YouTube video 2 , and by June 15, 2011 the song received 3.2 Million dislikes in YouTube against less than half a million lik es. F rom lo c al fame her song soared to the heigh ts of glob al shame. The anecdotal example of Reb ecca Blac k’s song is paradigmatic of some asp ects of the collective dynamics of ev aluations in online media. A video can b ecome relativ ely p opular within a small comm unity and receiv e initial p ositive ev aluations, but when larger audiences are reached, negativity rises faster than in early moments. Figure 1 shows this phenomenon through an example of the relativ e daily volume of lik es and dislikes of a YouTube video. Initially , the video is positively ev aluated, but the volume of lik es decreases quickly . While initial dislik es also decrease, they start rising after the fourth da y , reac hing a p eak at the ninth day . The early viewers of a YouTube video are prone to lik e it, either due to a so cial connection with the uploader, or giv en the similarit y of the video with their past liked con tent. This is a consequence of the 1 The original video w as deleted and reuploaded again at: https://www.youtube.com/watch?v=kfVsfOSbJY0 2 http://knowyourmeme.com/memes/rebecca- black- friday 1/20 A diy a Abishev a, Da vid Garcia, F rank Sc h w eitzer: When the Filter Bubble Bursts: Collectiv e Ev aluation Dynamics in Online Communities Submitted to the 8th In ternational ACM W eb Science Conference 2016 5 10 15 20 Daily v olume Lik es Dislik es da y 0.15 0 0.05 0.10 0.20 0.25 Figure 1: Example of ev aluation dynamics in Y outub e. Normalized daily volume of lik es and dislikes for a video in our YouTube dataset. Likes appear so on after the video is uploaded, while dislikes tend to app ear later. purp ose of so cial filtering mechanisms and recommender systems, whic h is to p ersonalize conten t selection suc h that users find con tent that they consider relev ant and of go o d quality . In contrast, the video can also spread through other media to wards more general users, and even tually reach a global audience with users more critical or negativ e tow ards the video. Bey ond YouTube videos, this phenomenon can b e seen as another aspect of the filter bubble [51]: The reinforcemen t of opinions caused b y filtering mec hanisms creates an initial p o ck et of p ositivity , but when the filter bubble bursts , collective negativity can bac klash. Our study sets out to understand collective ev aluation processes in v arious social media through likes and dislik es, as manifestations of opinions tow ards the ev aluated conten t. W e test the duality of collective ev aluations in the lo cal v ersus global b ehavior illustrated ab o ve, lo oking for the existence of a threshold of p ositivity after whic h negativ e ev aluations rise faster and p olarization emerges. Emotions in p olarization T echnological filters are not the only factor that shapes collective ev alua- tions; emotions influence cognitive information pro cessing, shaping opinions and attitudes to wards online con tent. A num b er of studies in so cial psychology show how emotions influence individual ev aluations, judgemen ts, and opinions [35, 24, 25], based on the theory of c or e affe ct [57]. Within this theoretical frame- w ork, emotions are comp osed of tw o dimensions: i) valenc e , which characterizes the feeling of pleasure or displeasure, and ii) ar ousal , which encompasses a feeling of activ ation or deactiv ation, and quan tifies mo- bilization and energy [57]. Additional dimensions can improv e the representation of emotional exp erience, suc h as p otency or surprise [15], but their consistent inclusion in psychological research ab out opinions is still to be explored. Researc h in psyc hology on the role of emotions in ev aluations show that arousal can lead to extreme reactions and p olarized resp onses [55]. The theory of misattribution explains this effect [76, 55] as a transfer of residual emotions b et ween even ts that intensifies the reaction to the second even t. F or instance, men in a state of high emotional arousal (for example from physical exercises) give more extreme ratings of attractiveness to w omen in comparison to the situation in whic h raters are in a calm emotional state. Similarly , v alence can b e misattributed and bias ev aluations [58], in particular when individuals insp ecting their current feelings, whic h migh t b e caused b y an inciden tal source rather than the ev aluated con tent. F urther theoretical explanations for the role of emotions in ev aluations p ose the reduction of cognitive complexit y induced by emotional states, whic h bias the form ulation of ev aluations tow ards fast rather than informed resp onses. The theory of affect priming explains this through the attribution of an individual’s 2/20 A diy a Abishev a, Da vid Garcia, F rank Sc h w eitzer: When the Filter Bubble Bursts: Collectiv e Ev aluation Dynamics in Online Communities Submitted to the 8th In ternational ACM W eb Science Conference 2016 mo od to similarly v alenced signals in memory , which helps reducing the effort of ev aluation tasks [29]. Empirical evidence sho ws that the sub jective exp erience of arousal motiv ates ev aluation on the extremes [52]. F or example, the ratings of famous figures by studen ts are found to b e more p olarized righ t b efore taking an exam, in comparison to weeks b efore or after. This kind of reactions are esp ecially salient when arousal is exp erienced along with negative v alence (suc h as the stress b efore an exam), and th us w e can exp ect the expression of negative and aroused emotions to motiv ate more p olarized collective ev aluations in so cial media. The digital traces of collective ev aluations allow us to analyze further the role of emotions in online ev aluation pro cesses. Con tributions of this article In this work we analyze collective ev aluations across different so cial media to rev eal statistical regularities related to information filters and emotions. First, w e test if the distributions of likes and dislikes of ev aluated conten t shows signs of the existence of multiplicativ e growth pro cesses of social in teraction. Second, we test if the relationship betw een lik es and dislikes is non-linear with a division in to tw o differen t mo des, corresp onding to local and global collective ev aluations. Third, w e test if the emotions expressed in the ev aluated conten t lead to global and p olarized collective responses. Our w ork pro vides insights into the prop erties of collectiv e ev aluations and tests established psychology theories on the role of emotions in opinion formation. 2 Bac kground Collectiv e dynamics In the last y ears, lots of researc h focused on the topic of online comm unities, i.e. large groups of individuals that in teract through an online medium. Collective phenomena suc h as dynamics of trends [75, 70], or viral mark eting [38] can b e assessed with data from online comm unities. Examples of studies on online user b ehaviour are understanding dynamics of replying activity and website engagemen t of users [56], buyer activity in online shopping websites [37] and communication dynamics in forums [30]. Another example is researc h on so cial influence, which was shown to exist in YouTube [12], in Facebook [50], and in Twitter [69]. F urthermore, so cial influence on p opularity of Facebook applications has b een shown to arise from a mixture of lo cal and global signals [50]. While the former notion indicates ho w friends and local communit y influence an individual’s behaviour, the latter suggests the effect of the aggregate popularity of pro ducts or b eha viours on an individual. A dditionally , previous results for p opularit y distributions show that the amount of votes for Digg stories [67] and tw eets in trending topics [6] follow log-normal distributions that are explained by so cial coupling. Collectiv e ev aluations Online voting dynamics and dynamics of human appraisal were studied in a n umber of previous researc h. Studies on collective ev aluations mostly in terrelates and finds explanations in research on collectiv e popularity of the online conten t, with the assump tion that more likes lead to item’s p opularity . Despite the differences in the w ays of measuring p opularit y , as a num b er of views in YouTube [60], or as a num b er of likes and dislikes in Reddit [44], or as a n umber of votes in Digg [67] or as a time span of trending topics in Twitter [6], these measures sho wed the existence of statistical regularities of con ten t p opularit y , and fit to the log-normal distribution. 3/20 A diy a Abishev a, Da vid Garcia, F rank Sc h w eitzer: When the Filter Bubble Bursts: Collectiv e Ev aluation Dynamics in Online Communities Submitted to the 8th In ternational ACM W eb Science Conference 2016 Studies on collective dynamics of negativ e ev aluations are scarcer, but some recen t w orks illustrate that so cial influence effects are present in movie ratings from imdb.com [41], and that con tro versialit y expressed through mo vie ratings evolv es with time [5]. Additionally , herding effects hav e been observ ed in random manipulations of votes in Reddit [73], which shows that the w ay users vote dep ends on the votes of other users. F urther research on Reddit [44] sho wed a non trivial dep endency b etw een likes and dislikes at the collectiv e lev el, in line with the questions we address in this article. Online p olarization The proliferation of online participatory media, such as so cial net work sites, blogs and online fora, increases users engagement in discussions on p olitical and so cietal issues, which in its turn may - under certain conditions - split individuals apart in their opinion space. Opinion p olarization is c haracterzied by a division of the p opulation into a small num b er of fractions with high internal consensus and sharp disagreemen t b etw een them [14]. Agent-based mo dels [40, 42] and exp erimental studies [68] explain some asp ects of opinion formation and its role in consensus and p olarization. Based on data from digital traces, previous researc h inv estigated p olarization from the netw ork p ersp ective in p olitical blogs [3], in follow er and mention links in Twitter [11], and in Swiss p oliticians profiles [19], as w ell as in a non-p olitical domains lik e friendship net works [26], and cultural expression [22]. Additionally , exprerimen tal evidence shows that group pro cesses like p olarization function differently in computer- mediated comm unication than in a face-to-face in teraction [62], for example as the relative annon ymit y of online media dampens inhibiting effects lik e the spiral of silence [48]. Online emotions Emotional expression through online text has b een analyzed in earlier research on data from MySpace [66], Y ahoo answers [34], IRC channels [18], Wikip edia [28], BBC, Digg, Y ouT ube and T witter [64]. A v ailability of large-scale quantitativ e datasets allo ws us to understand emotions and their role in v arious domains. Studies in the field of sub jec tiv e well-being leverage extensiv ely on quantifying emotions through text. F or instance, sub jectiv e well-being is manifested in Facebook status up dates [71], and sho ws a pattern of assortativity in so cial netw orks [54] in relation to feelings of loneliness [7]. This is in a close relation to the quantification of moo d in Twitter whic h has b een used to v alidate theories of p eriodic mo od oscillations [23]. Twitter mo od measured in terms of v alence and arousal reveals asp ects of the relation b etw een mo o d states and online in teraction and participation [9], and the psycholinguistic analysis of emotions reveal the traces of men tal health issues [10]. F urthermore, segregation patterns in geographical space [39] and gender-based patterns [32, 66] can b e partially attributed to differences in emotional expression. In online in teraction, for example in real-time chat con versations [18] and pro duct reviews [21], emotions are not a phenomenon c haracteristic to just an individual, but exhibit collectiv e properties [59]. Lastly , information-cen tric role of online emotions has b een studied through blogs [45], in Twitter [53], and in Yahoo answers [34]. Emotions are the building blo cks for a creation of so cial netw ork structures [74, 61] through empath y [31] that lead to correlations b et ween emotional expression and p opularit y [33, 63]. Negative emotional p osts w ere shown to b e drivers of communication among users and resp onsible for extension of the lifetime of online discussions in forums [8]. F urthermore, the digital traces of emotions sync hronize with p olitical outcomes [24], which go es inline with the findings that p olitical discussions are emotionally charged [27], in particular during election p erio ds [20]. 4/20 A diy a Abishev a, Da vid Garcia, F rank Sc h w eitzer: When the Filter Bubble Bursts: Collectiv e Ev aluation Dynamics in Online Communities Submitted to the 8th In ternational ACM W eb Science Conference 2016 3 Data and Metho ds 3.1 Data on collectiv e ev aluations Datasets The data used in this research is the result of our cra wl of four publicly accessible online comm unities. YouTube ( http://www.youtube.com/ ) is a video sharing w ebsite on whic h registered users can upload and view videos, as w ell as p ost commen ts and rate videos with likes and dislikes. Our crawl 3 w as launched in June 2011 to daily collect a combination of top videos in v arious categories and to iterativ ely explore the channels of general users [2], including 6.3 Million videos b y F ebruary 2015. Reddit ( http://www.reddit.com ) is a message board in which registered users submit p osts with links and text, and vote up and down for p osts to app ear on a fron tpage. Conv ersations b etw een users appear in one of the man y thematic b oards, called subreddits, co vering diverse topics from p olitics to science fiction and adult conten t. F rom 2012 to 2014 our daily Reddit crawl 4 collected 338,000 submissions from 1,972 subreddits. While the user interface of Reddit provides fuzzed amounts of votes, it is p ossible to construct the total amoun t of up and down votes to a submission based on the JSON fields of reddit score and like ratio. This wa y , w e count with the text and the final amount of up and down vo tes for each submission i n our dataset. Imgur ( http://www.imgur.org/ ) is an image hosting and sharing website where registered users upload, rate, and discuss uploaded images. Image sharing traffic of Imgur has a large presence in Reddit such that every 6th successful Reddit p ost has a link to an image on Imgur [49]. Our daily crawl 5 collected 200,000 images and their user activit y statistics b etw een December 2015 and January 2016. Finally , Urban Dictionary ( http://www.urbandictionary.com/ ) is an online crowdsourced platform consisting of non-standard lexicon of slang words and idioms. Registered users can submit new terms and pro vide definitions, and all users of the w ebsite, registered and anonymous, can vote up and down for the b est definitions. Betw een April and May 2013 our python-based cra wl c ollected 220,000 definitions and their votes. All platforms provide functionality for users to ev aluate uploaded conten t p ositively and negativ ely by clic king an up vote/ like or do wn vote/ dislike button resp ectively . F or simplicity , from now on we refer to ev aluated videos, submissions, images and definitions as items and we denote as lik es and dislikes to p ositiv e and negative ev aluations, including up and down votes resp ectively . Sen timent Analysis T o quan tify emotional expression, we applied sentimen t analysis to headers or titles of eac h item, lea ving for a future researc h the analysis of longer descriptions, transcripts, and commen ts. W e applied sen timent analysis tec hniques to video descriptions in YouTube , image titles in Imgur , submission headers in Reddit and term definitions in Urban Dictionary . Headers and titles are a 3 Y ouT ub e Data API Jav a wrapp er ( https://developers.google.com/youtube/v3/ ) 4 PRA W ( https://pypi.python.org/pypi/praw ) 5 PyImgur Python API wrapp er ( https://github.com/Damgaard/PyImgur ). Seed images were selected from Imgur ’s gallery sitemap ( http://imgur.com/gallery/sitemap.xml ) 5/20 A diy a Abishev a, Da vid Garcia, F rank Sc h w eitzer: When the Filter Bubble Bursts: Collectiv e Ev aluation Dynamics in Online Communities Submitted to the 8th In ternational ACM W eb Science Conference 2016 Dataset Num b er of items, N Num. of likes Num. of dislikes N crawled N year > 1 N L,D > 1 Urban Dict. definitions 220 , 270 213 , 512 208 , 441 61 , 100 , 699 26 , 508 , 869 YouTube video descriptions 6 , 279 , 461 3 , 864 , 480 2 , 750 , 554 763 , 291 , 676 41 , 214 , 035 Reddit submissions 338 , 845 174 , 444 142 , 662 5 , 078 , 242 947 , 519 Imgur image titles 201 , 181 147 , 752 125 , 230 54 , 786 , 629 1 , 931 , 918 T able 1: Number of items in each dataset . N crawled coun ts the num b er of crawled items, and N year > 1 the num b er of items in English and that existed for more than a year. N L,D > 1 coun ts items that received at least 1 like and 1 dislike. go od proxy of the emotional tone of a discus sion, in line with earlier researc h on forum-lik e conv ersations [24]. W e measured emotional con tent of items by applying tw o complementary sentimen t analysis methods. First, we apply a lexicon of affectiv e norms of v alence V , arousal A and dominance D of nearly 14,000 English words [72]. In line with previous findings [72], the scores of v alence and dominance in our dataset are highly correlated, in comparison with the weak er correlation b etw een v alence and arousal as explained more in detail in the Results section. This motiv ates our focus to only v alence and arousal as suggested b y the theory of core affect. Second, we apply the SentiStrength classifier [65, 64] a state-of-the-art lexicon-based metho d [34, 1] that has b een used in earlier researc h on the online data from MySpace [66], Y ahoo! [34], IRC c hannels [18], BBC, Digg, Y ouT ub e [64], T witter [64, 53] and Wikip edia [28]. The core of the Sen tiStrength method is to predict the sentimen t of a text, based upon the o ccurrences of the w ords from a lexical corp ora, whic h con tains the set of terms with known sen timent of a text. The classifier incorp orates v arious rules, whic h strengthen or w eaken sen timents of the lexicon words detected in the short text. Among the rules are syn tactic rules, e.g. exclamation marks and punctuation, language mo difiers and intensifiers, suc h as negation and bo oster w ords, and spelling correction rules. The final sen timent score is comp osed of a p ositiv e P and a negative N score for each text as tw o discrete v alues in the range of [+1 , +5] and [ − 5 , − 1] resp ectiv ely . In our analysis, w e normalize all emotions v ariables to [0 .. 1] mapping P from [+1 , +5] to [0 , 1] and reversing and rescaling N from [ − 1 , − 5] to [0 , 1] . T o ensure a v alid measurement of sentimen t and collective ev aluations, we apply tw o filters to our datasets. First, since both sen timent analysis tec hniques are designed only for English texts, we apply language classification [47] and filter out all non-English texts. Second, we remov e all items with less than a like and a dislike, and that existed for less than a year in all platforms, to ensure that positive and negativ e ev aluations are stable. Detailed statistics on the n um b er of p osts in each dataset are shown in T able 1, sho wing that they are still sufficien t for large scale analyses. W e will mak e these datasets av ailable for researc h purp oses. 3.2 Statistical analysis metho ds Distribution fits W e apply a Maxim um Lik eliho od criterion to fit the distributions of likes and dislik es [4], to confirm early findings of the fits of the p opularity distribution to the log-normal distribution 6/20 A diy a Abishev a, Da vid Garcia, F rank Sc h w eitzer: When the Filter Bubble Bursts: Collectiv e Ev aluation Dynamics in Online Communities Submitted to the 8th In ternational ACM W eb Science Conference 2016 [67, 6]. W e use the powerlaw python pack age to fit four statistical distributions related to complex gro wth phenomena [46]: p o wer law, log-normal, truncated p ow er law and exp onential distributions. W e compare the likelihoo d of each distribution using the log-likelihoo d ratio R = l n ( L 1 /L 2 ) b etw een the tw o candidate distributions and its significance v alue p . Positiv e ratios indicate evidence for the first distribution, and negativ e ratios for the second one. Instead of testing the h yp othesis of the data follo wing a certain distribution, this comparative test answers the question of which parametric distribution provides the b est fit a v ailable, follo wing the principle of Maximum Lik eliho o d estimation [4]. T o finally assess the qualit y of the best fit, we measure the Kolmogoro v-Smirnov distance b etw een the b est fitting distribution and the emprical data. Dual regime analysis W e test the existence of a dual lo cal versus global regime in collectiv e ev alua- tions by analyzing the non-linear properties of the relationship betw een the amounts of lik es and dislikes for each item. W e use an extension of a traditional linear mo delling, multiv ariate adaptiv e regression splines (MARS) [17, 16] implemented in the R programming language pack age e arth . MARS fits a contin uous piecewise regression function with knots that join locally linear pieces. In our analysis, we are in terested to test a dual pattern in the relationship b etw een the n um b er of likes L and the num b er of dislikes D, therefore w e set the num b er of knots to one and fit a mo del of the form D(L) = I + α 1 ∗ max (0 , L − L c ) + α 2 ∗ max (0 , L c − L ) The v alues of likes ab ov e L c and the v alues of dislikes ab o ve D( L c ) corresp ond to observ ations in the global regime, after the bubble bursts, and the v alues in which any is below map to the local regime. T o ev aluate the quality of the MARS mo del, w e compare it to the Ordinary Least Squares (OLS) regression using the Generalized Cross-V alidation prediction error (GCV) defined as GC V = RS S N ∗ (1 − ENP N ) 2 where N is the n um b er of observ ations, R S S is the residual sum of squares, and E N P is the effective n umber of parameters to a void o v erfitting [16]. W e use the implementation pro vided b y the pac k age b o ot in R as well as the co efficient of determination R 2 of b oth OLS and MARS fits. Emotion and p olarization analysis Ha ving identified the tw o regimes and their thresholds in the relationship b etw een the num b er of dislikes and the num b er of likes, we can mark items either in the global or the lo cal regime as a binary class. W e test how emotions influence the chances of items reac hing the global regime through tw o logistic regression mo dels, one for each sentimen t analysis technique. Similarly , w e com bine the v alues of lik es and dislikes through their geometric mean to measure p olarization, as manifested by simultaneous large amounts of p ositive and negativ e ev aluations. W e regress this measure of p olarization through tw o linear mo dels depending on the emotions expressed on the items. Prior to modelling, w e examine the normalized emotional dimensions for m ulticollinearity by computing the Sp earman’s rank correlation coefficients, to a void singularities. W e assess the quality of fits in com- parison to n ull mo dels, b y measuring the χ 2 statistic of model lik eliho od ratio tests implemen ted in the lmtest R pac k age. 7/20 A diy a Abishev a, Da vid Garcia, F rank Sc h w eitzer: When the Filter Bubble Bursts: Collectiv e Ev aluation Dynamics in Online Communities Submitted to the 8th In ternational ACM W eb Science Conference 2016 4 Results 4.1 St ylized facts of ev aluation distributions Figure 2 shows the probabilit y density functions of the distributions of the amoun t of likes and dislik es for items in eac h of the four datasets. T o understand the pro cess that generates these distributions, we fit a set of parametric distributions that provide insigh ts in to how lik es and dislikes are given to items. F ollo wing the categorization of [46], generative mec hanisms pro duce stylized size distributions that can b e traced bac k to the properties of gro wth processes. If the appearance of lik es and dislik es follo ws an uncorrelated process and new ev aluations are independent of previous ones, likes and dislik es should follo w exp onential distributions. On the other hand, the presence of lik es and dislikes can motiv ate further ev aluations through so cial effects, creating m ultiplicative gro wth (also kno wn as preferential attachmen t in the con text of netw orks). In the presence of multiplicativ e growth, if items ha ve similar lifespans, lik es and dislikes follow lo g-normal distributions. On the other hand if multiplicativ e gro wth is combined with heterogeneous lifespans, likes and dislikes follo w a p ower law distribution. This p ow er law can b e corrected by adding an exp onen tial cutoff if finite size effects limit the growth of lik es and dislikes, a case in which the distributions would b e better fitted by a trunc ate d p ower law . ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 10 − 8 10 − 5 10 − 2 10 1 10 3 10 5 ● ● ● ● ● ● ● ● ● ● ● ● ● 10 − 8 10 − 5 10 − 2 10 1 10 3 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 10 − 10 10 − 7 10 − 4 10 − 1 10 1 10 3 10 5 10 7 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 10 − 10 10 − 7 10 − 4 10 1 10 3 10 5 10 7 ● ● ● ● ● ● ● ● ● ● ● 10 − 8 10 − 5 10 − 2 10 1 10 3 ● ● ● ● ● ● ● ● ● ● ● 10 − 8 10 − 5 10 − 2 10 1 10 3 ● ● ● ● ● ● ● ● ● ● ● ● ● 10 − 8 10 − 6 10 − 4 10 − 2 10 1 10 3 ● ● ● ● ● ● ● ● ● ● ● 10 − 7 10 − 5 10 − 3 10 − 1 10 1 10 3 P(Lik es) P(Disli k es) Urban Di c � onar y Lik es Dislik es Y ouT ube P(Lik es) P(Disli k es) Lik es Dislik es R eddit P(Lik es) P(Disli k es) Lik es Dislik es Imgur P(Lik es) P(Disli k es) Lik es Dislik es Figure 2: Probability density function of collectiv e ev aluations . Probability densit y function of the num b er of lik es (top) and the num b er of dislik es (b ottom) with exp onential binning and fits to log-normal distribution l n N ( µ, σ ) (red dashed lines). F or all datasets, the results of the log-likelihoo d pairwise comparisons of the four distributions (see text) identified the log-normal distribution as the b est fit. F or all datasets, the results of pairwise comparisons of the four proposed distributions identified the lo g-normal distribution as the b est fit, with significant and positive log-likelihoo d ratios as shown in T able 2 along with the b est fitting parameter estimates. The dashed lines in Figure 2 show the fitted distributions, revealing the qualit y of the fit. The cases of YouTube and Urban Dictionary pro vide very 8/20 A diy a Abishev a, Da vid Garcia, F rank Sc h w eitzer: When the Filter Bubble Bursts: Collectiv e Ev aluation Dynamics in Online Communities Submitted to the 8th In ternational ACM W eb Science Conference 2016 Urban Dictionary YouTube Reddit Imgur P(Likes) P(Disl ikes) P(Lik es) P(Dislikes) P(Likes) P(Dislikes) P(Likes) P(Dislikes) µ 4.092 3.657 5.492 1.405 2.197 0.492 4.668 1.821 σ 1.705 1.435 2.28 2.528 1.332 1.35 2.46 1.447 D 0.008 0.008 0.009 0.002 0.029 0.01 0.098 0.031 ln  L LN L PL  123115 . 6 ∗∗∗ 129462 . 1 ∗∗∗ 174835 . 6 ∗∗∗ 42309 . 5 ∗∗∗ 45355 . 3 ∗∗∗ 13898 . 9 ∗∗∗ 65314 . 8 ∗∗∗ 34103 . 6 ∗∗∗ ln  L LN L TPL  55538 . 6 ∗∗∗ 61135 . 1 ∗∗∗ 252646 . 1 ∗∗∗ 42592 . 3 ∗∗∗ 17182 . 5 ∗∗∗ 10084 ∗∗∗ 20108 . 5 ∗∗∗ 3662 . 4 ∗∗∗ ln  L LN L EXP  260098 . 8 ∗∗∗ 126865 ∗∗∗ 953075 ∗∗∗ 1270237 . 6 ∗∗∗ 149405 ∗∗∗ 70261 . 5 ∗∗∗ 75817 . 7 ∗∗∗ 26681 . 4 ∗∗∗ T able 2: Log-normal fit parameters of collective ev aluations and comparison to other distributions . Estimated parameters of the fitted log-normal distribution l n N ( µ, σ ) and K olmogorov-Smirno v distances D . The bottom ro w sho ws the log-likelihoo d ratios of pairwise comparison b et ween the log-normal distribution fit (n umerator) and the other three distribu- tions: p ow er la w, truncated p ow er law and exp onen tial. All three ratios are p ositiv e, large and significan t ( p < 0 . 05 ) which confirms that among the four candidate distributions the log-normal distribution is the b est fit. go od fits with extremely low Kolmogoro v-Smirno v D statistics. The fits are not so go o d at the tails of Reddit and Imgur , but the the Kolmogoro v-Smirnov D statistic pro vide go o d v alues below 0.05 and the lo g-normal distribution clearly outperforms all others. The worst fit is for the n umber of lik es in Imgur , for which Figure 2 suggests a bimo dal pattern. Identifying the p ossible mechanisms that can pro duce such bimo dalit y go es b eyond the scope of this research. W e can conclude that the amoun ts of lik es and dislikes displa y a general heavy tailed behavior of lo g-normal distributions, lending evidence for the pro duction of ev aluations follo wing so cially coupled gro wth pro cesses with homogeneous life spans. 4.2 The dual pattern of collective ev aluations W e explore the existence of a dual relationship betw een likes and dislikes through non-linear MARS fits, testing if the relationship can b e divided in a local and a global regime. W e restrict the num b er of mo del terms to ha ve a single knot, measuring if a dual model outperforms a linear pattern. Figure 3 shows the results of MARS fits b etw een the logarithms of likes and dislikes. V ertical and horizontal lines mark the lik es cutoff v alue L c and its corresp onding v alue of dislikes in the fit D( L c ). These cutoff v alues divide the system in a lo cal v ersus a global regime, with the fitted functions of the form D ∝ L λ and D ∝ L γ resp ectiv ely . In all datasets, the exp onent of the global regime is larger than exp onen t of the lo cal one, for example in YouTube γ = 0 . 93 > λ = 0 . 29 . While b oth exp onents are b elo w 1 and indicate sublinear s caling, the m uch higher v alue of the second one shows that, beyond a threshold v alue of likes, the dislik es given to items grow faster than b elo w the threshold as a sign of the burst of a filter bubble. The presence of scaling in Reddit votes was previously rep orted in a smaller data subsample [44], concluding the existence of sup erlinear scaling of dislik es with lik es. Our analysis sho ws that the relationship b etw een likes and dislik es in Reddit is b etter approximated b y a dual regime mo del, in line with the results of the other three datasets. 9/20 A diy a Abishev a, Da vid Garcia, F rank Sc h w eitzer: When the Filter Bubble Bursts: Collectiv e Ev aluation Dynamics in Online Communities Submitted to the 8th In ternational ACM W eb Science Conference 2016 D ∝ L 0.67 D ∝ L -0.37 10 1 10 3 10 5 10 1 10 3 10 5 1 10 100 1000 count D ∝ L 0.75 D ∝ L 0.42 10 1 10 3 10 1 10 3 1 100 10000 D ∝ L 0.9 D ∝ L 0.61 10 1 10 3 10 5 10 1 10 3 10 5 1 10 100 1000 count D ∝ L 0.93 D ∝ L 0.29 10 1 10 3 10 5 10 1 10 3 10 5 1 10 100 1000 count Urban Di c � onar y Y ouT ube R eddit Imgur count Lik es Dislik es Lik es Dislik es Lik es Dislik es Lik es Dislik es Figure 3: Relationship b et ween the num b er of dislikes and lik es . T wo-dimensional join t distributions with 50 bins, bin colors indicate the count of observ ations within the bin. Pur- ple and red lines sho w the lo cal and global regimes of the non-linear relationship b etw een the n umber of dislikes and the n umber of likes. Threshold estimates are lo cated at L c , estimated as L c = 155 in Urban Dictionary ; L c = 131 in YouTube ; L c = 7 in Reddit ; and L c = 27 in Imgur . W e ev aluate the go o dness of the dual mo del against a single regime mo del in T able 3. The dual mo del outp erforms in R 2 and GCV to the single regime mo del, lending strong evidence to the existence of t wo regimes. W e further tested if additional knots could improv e the fits, and found that a dual regime is the optimal mo del for Urban Dictionary , YouTube , and Reddit , and only a 4 knot mo del could im pro ve the Imgur fit b y a marginal GCV of less than 0.01. mo del Urban Dict. YouTube Reddit Imgur R 2 ( lm ) 0 . 646 0 . 634 0 . 727 0 . 505 R 2 ( MARS ) 0 . 654 0 . 683 0 . 741 0 . 597 GC V ( lm ) 0 . 785 1 . 283 0 . 301 0 . 804 GC V ( MARS) 0 . 767 1 . 111 0 . 286 0 . 654 T able 3: The go o dness of the dual and the linear model . Comparison of the linear and the MARS mo dels of the relationship b etw een the num b er of dislikes and the num b er of likes. T op ro w shows the coefficient of determination R 2 (higher is b etter). Bottom ro w sho ws the generalized 10-fold cross-v alidation prediction error (GCV) (low er is b etter). The dual mo del outp erforms in R 2 and in GCV compared to the linear mo del. 4.3 Emotions in the global regime Figure 4 illustrates the rank correlations b et ween emotional dimensions. In all datasets v alence and dominance are highly c orr elate d with ρ > 0 . 7 ∗∗∗ , and therefore we discard the dominance v ariable from regression analysis as it is difficult to distinguish from v alence. V alence and p ositivity P hav e a minor p os- itiv e significant correlation ρ ∈ [0 . 2 , 0 . 3] , and v alence and negativity N ha ve a slightly negative correlation ρ ≈ − 0 . 3 , illustrating the relation of emotion v ariables accross b oth v alence/arousal and p ositiv e/negative mo dels. 10/20 A diy a Abishev a, Da vid Garcia, F rank Sc h w eitzer: When the Filter Bubble Bursts: Collectiv e Ev aluation Dynamics in Online Communities Submitted to the 8th In ternational ACM W eb Science Conference 2016 A D P N V A D P -0.01 0.7 -0.08 0.33 0.11 0.25 -0.34 0.16 -0.27 0.14 A D P N V A D P 0.12 0.69 0.02 0.2 0.06 0.18 -0.34 0.16 -0.28 0.24 A D P N V A D P -0.05 0.71 -0.12 0.29 0 0.27 -0.33 0.19 -0.24 0.16 -1 -0.5 0 0.5 1 A D P N V A D P 0.26 0.71 0.15 0.31 0.17 0.25 -0.31 0.23 -0.25 -0.05 Figure 4: Correlations of emotions . Sp earman’s correlation matrix of emotional dimensions, in an order from left to right: A) Urban Dictionary , B) Y ouT ub e, C) Reddit, D) Imgur. Significance lev el p < 0 . 05 . Insignificant correlations are crossed out. Predictors are normalized to [0 .. 1] . Dominance is highly correlated with v alence, and therefore the dominance v ariable is discarded from the further analysis. W e fit t wo regression models in which the probability of the even t of an item reaching the global regime G depends on the emotions expressed in the ev aluated item. The first model uses V and A as explana- tory v ariables, and fo cuses on the role of emotions as quan tified through their pleasant/unpleasan t and activ e/calm dimensions. The second mo del takes P and N as predictors, and measures significance of p ositiv e and negativ e sen timen ts in bringing an item to global regime. T able 4 rep orts the results of logistic regression of the form log it ( G ) ∼ V + A and log it ( G ) ∼ P + N resp ectiv ely . The role of arousal is heterogeneous, having a significan t p ositiv e effect in Urban Dictionary and Imgur , but a weak nega- tiv e effect in YouTube and a non-significan t one in Reddit . The effect of v alence is also mixed, in Urban Dictionary and YouTube the chances of reac hing the global regime gro w with v alence, while in Reddit and Imgur is the opp osite case. The second mo del sheds more light to this: the pattern is the same for p ositiv e sentimen t, but negative sentimen t increases the chance of reaching the global regime in all datasets but Reddit , where the effect is not significant. Urban Dict. YouTube Reddit Imgur In tercept − 2 . 071 ∗∗∗ − 0 . 305 ∗∗∗ − 0 . 111 ∗∗∗ 0 . 228 ∗∗∗ V 0 . 976 ∗∗∗ 0 . 618 ∗∗∗ − 0 . 262 ∗∗∗ − 0 . 209 ∗∗∗ A 0 . 584 ∗∗∗ − 0 . 049 ∗∗ − 0 . 006( n ) 0 . 300 ∗∗∗ χ 2 547 . 3 ∗∗∗ 2791 . 4 ∗∗∗ 44 . 1 ∗∗∗ 35 . 7 ∗∗∗ Urban Dict. YouTube Reddit Imgur In tercept − 1 . 369 ∗∗∗ − 0 . 115 ∗∗∗ − 0 . 259 ∗∗∗ 0 . 261 ∗∗∗ P 1 . 019 ∗∗∗ 0 . 581 ∗∗∗ − 0 . 166 ∗∗∗ − 0 . 296 ∗∗∗ N 0 . 170 ∗∗∗ 0 . 218 ∗∗∗ − 0 . 006( n ) 0 . 191 ∗∗∗ χ 2 2552 . 6 ∗∗∗ 17150 . 9 ∗∗∗ 41 . 9 ∗∗∗ 120 . 3 ∗∗∗ ∗∗∗ p < 0 . 001 , ∗∗ p < 0 . 01 , ∗ p < 0 . 05 , ( n ) not significant. T able 4: The role of emotions in the global regime . Logistic regression mo dels, log it ( G ) ∼ V + A and l og it ( G ) ∼ P + N , results for probability of an item to b e in a global regime. The effect of arousal and v alence is heterogeneous, and dep ends on the dataset. 11/20 A diy a Abishev a, Da vid Garcia, F rank Sc h w eitzer: When the Filter Bubble Bursts: Collectiv e Ev aluation Dynamics in Online Communities Submitted to the 8th In ternational ACM W eb Science Conference 2016 4.4 Analysis of emotions in p olarization Since the distributions of lik es and dislikes are approximately log-normal, w e can treat the logarithms of lik es ln ( L ) and dislikes l n ( D ) as cen trally distributed around their means h ln ( D ) i and h l n ( L ) i . W e standardize the logarithmic coun ts ln ( D ) and ln ( L ) as: Z L = ln ( L ) − h l n ( L ) i sd ( ln ( L )) Z D = ln ( D ) − h ln ( D ) i sd ( ln ( D )) where sd ( l n ( L )) and sd ( ln ( D )) are the standard deviations. Then, we compute a measure of p olariza- tion as the geometric mean of b oth v alues: P ol = √ Z L ∗ Z D . This measure captures the principle that p olarization is high under simultaneous large amoun ts of positive and negativ e ev aluations, and that p olarization is lo w when only one of the v alues is dominant. T o understand whic h kind of emotional conten t creates p olarization, we fit t wo regression mo dels as in the previous section, one of polarization as a function of v alence and arousal in the ev aluated item, and another as a function of p ositive and negative sentimen t scores. The results of the fits are shown in T able 5. In line with the theory that links arousal to more extreme opinions, we find a general pattern in three datasets where arousal leads to higher lev els of polarization. While there is no significan t effect in Reddit , all the other datasets show that items that contain words that transmit higher arousal also create a stronger polarized response. Urban Dict. YouTube Reddit Imgur In t. 2 . 0508 ∗∗∗ 1 . 4543 ∗∗∗ 1 . 3511 ∗∗∗ 1 . 7623 ∗∗∗ V 0 . 3132 ∗∗∗ 0 . 2980 ∗∗∗ − 0 . 1954 ∗∗∗ − 0 . 1908 ∗∗∗ A 0 . 2662 ∗∗∗ 0 . 1005 ∗∗∗ − 0 . 0327( n ) 0 . 2399 ∗∗∗ χ 2 480 . 64 ∗∗∗ 3420 . 4 ∗∗∗ 97 . 315 ∗∗∗ 88 . 669 ∗∗∗ Urban Dict. YouTube Reddit Imgur In t. 2 . 2744 ∗∗∗ 1 . 5896 ∗∗∗ 1 . 2220 ∗∗∗ 1 . 7625 ∗∗∗ P 0 . 4889 ∗∗∗ 0 . 3059 ∗∗∗ − 0 . 1107 ∗∗∗ − 0 . 1484 ∗∗∗ N 0 . 1194 ∗∗∗ 0 . 1698 ∗∗∗ 0 . 0077( n ) 0 . 1672 ∗∗∗ χ 2 3271 . 2 ∗∗∗ 19830 . 0 ∗∗∗ 63 . 073 ∗∗∗ 170 . 81 ∗∗∗ ∗∗∗ p < 0 . 001 , ∗∗ p < 0 . 01 , ∗ p < 0 . 05 , ( n ) not significan t T able 5: The role of emotions in the p olarization . Linear regression mo dels, P ol ∼ V + A and Pol ∼ P + N , results for p olarization of the ev aluation of an item as a function of emotions expressed on its text. Arousal and negativity driv e p olarization in all datasets except Reddit . The effect of v alence and p ositivit y is dataset-dep endent. This also manifests in the mo del using positive and negativ e scores, where negativ e conten t predicts higher p olarization in the same three cases as for arousal. The results of these tw o metrics are consistent with the hypothesis that the expression of activ ating and negative feelings, suc h as anger or outrage, 12/20 A diy a Abishev a, Da vid Garcia, F rank Sc h w eitzer: When the Filter Bubble Bursts: Collectiv e Ev aluation Dynamics in Online Communities Submitted to the 8th In ternational ACM W eb Science Conference 2016 tend to create more p olarized resp onses, in line with the theoretical argumen t that p oses emotions as mec hanisms to speed up ev aluation processes at the expense of more extreme reactions. V alence in ev aluated items creates different resp onses. T wo communities, Imgur and Reddit , show a negativ e relation of polarization with v alence and positive sentimen t. The other tw o, Urban Dictionary and YouTube , show the opposite, where polarization increases with v alence. This suggests a con text dep enden t interpretation of p ositiv e expression, whic h do es not necessarily motiv ate p ositiv e empathy but can also fuel p olarized resp onses. The p ositiv e and negative scores mo del works b etter than the v alence and arousal mo del in all cases but Reddit , where th e v alence and arousal mo del was more explanatory for p olarization, as evidenced b y χ 2 tests comparing b oth mo dels. 5 Discussion Our study of emotions fo cuses on understanding the role of emotions expressed in the text of items with relation to the c hances that the items reac h the global regime and pro duce polarized ev aluations. While w e used tw o established and v alidated sentimen t analysis metho ds based on metrics from psyc hology , future adv anced techniques can reveal new patterns and p otentially falsify the conclusions of our analysis with curren t tec hniques. F urthermore, deep er analyses on individual data can correlate the expression of individual emotions in the comments of a user and the ev aluations given b y the user, bridging closer this wa y the measurement of emotional states and ev aluations and pro viding a b etter understanding of in terp ersonal emotions. F ollo wing an observ ational approach to collective ev aluations has the adv antage of having high ecological v alidity , but lacks the level of con trol that can b e induced in exp erimental scenarios. W e can deduce insigh ts on the factual prop erties of collective ev aluations, such as the dual regime b et ween lik es and dislik es, but testing the conditions that pro duce them requires a controlled set up. Our motiv ation and explanation for the dual regime stems from the phenomenon of filter bubbles [51], but to fully understand ho w these filters affect our behavior w e need to experiment on how individual ev aluations resp ond to filtering mechanisms. While these exp eriments can b e carried out it in t ypical psyc hological settings and surv eys, large platforms like Facebook can also exp eriment with the b ehavior of their users in this resp ect (under the appropriate ethical considerations). A complete understanding of online ev aluations can only b e achiev ed when our results are complemen ted b y exp erimen tal approac hes. The use of observ ational data has the adv antage of taking a natur al exp osur e approach: w e analyze the ev aluations of what p eople actually see, rather than the for c e d exp osur e to con tent in exp eriments [43]. In contrast, using digital traces of ev aluations con tains a selection bias by which some users might b e resp onsible for m uch larger amoun ts of likes and dislik es than other users. While this selection bias is natural at the collective level, inferring conclusions ab out the b ehavior of individuals needs to consider corrections and use richer datasets [13], or apply agent-based mo delling approaches to connect the micro and macro lev els [59]. W e explain the dual pattern b et ween lik es and dislik es as the result of filter bubbles, but other p ossible explanations might also b e plausible. Some unko wn deleting mec hanism migh t downsample videos with a lot of dislikes in the lo cal regime, or some external factor like audience size might explain the v alues of the thresholds. The results of our statistical analyses of distributions of likes and dislikes fit to h yp othetical 13/20 A diy a Abishev a, Da vid Garcia, F rank Sc h w eitzer: When the Filter Bubble Bursts: Collectiv e Ev aluation Dynamics in Online Communities Submitted to the 8th In ternational ACM W eb Science Conference 2016 mec hanisms of multiplicativ e gro wth, in line with previous findings on p opularit y metrics rather than ev aluations [67, 6]. Our in depth statistics also provide a clear view on the limits of our results, for exam- ple in the worse fits of log-normal distributions in Imgur . F uture researc h can conjecture on the p ossible alternativ e explanations of our findings, in particular with resp ect to which filtering mechanisms are in place. Our results do not allow us to distinguish so cial filtering, based on friends and follo wer links, from recommender systems, whic h are based on previous ev aluations of a user. F urther research with informa- tion on individual b eha vior can shed light on these different processes, for example measuring ev aluation tendencies to con tent pro duced by friends versus strangers, or across assortative and disassortative links with resp ect to opinions. Our analysis of the relation b etw een likes and dislikes is based on the amoun ts given to items after a long time has passed. This wa y , we ev aluate items after they do not attract lots of atten tion and their counts are stable. In a figurative wa y , we study the fossils of brok en filter bubbles, but we do not study them in a live setting. T o fully understand the dynamics of collectiv e ev aluations, w e need d ata with temp oral resolution on the coun ts of lik es and dislik es. In general, such data is not publicly a v ailable on the sites, whic h requires a m uch more p ow erful cra wling approac h to monitor items on a frequen t basis, or access to proprietary data. 6 Conclusions Our analysis of collective ev aluations across v arious online media shows statistical regularities in the distributions of ev aluations and their relationships. Our contribution is threefold: First we rep ort that the distributions of the amoun ts of likes and dislikes p er item are well fitted by log-normal distributions, a result that gives insights in to the prop erties of the pro cess that creates ev aluations. Second, we test the existence of a dual pattern in the relation b etw een likes and dislikes, finding robust evidence of the existence of a lo cal and a global regime that is consistent with our h yp otheses ab out the burst of filter bubbles. Third, we found evidence for the role of emotions in the creation of p olarization and the access to the global regime, lending supp ort for psyc hology theories ab out the role of affect, in particular arousal, in the p olarization of opinions. Our results hav e implications for the design of online platforms and filtering mechanisms. Recommender systems and filtering mec hanisms allo w users to disco ver con tent of relev ance and quality , but can hav e unin tended consequences in the large scale. Our results suggest that the increasing p olarization levels of discussions migh t b e created by these filtering mechanisms, and that users are at risk of receiving a negativ e backlash to their con tent when it go es b eyond their lo cal so cial con text. Such abrupt behavior with resp ect to negative ev aluations can hav e imp ortant consequences to user motiv ation and engagement, whic h migh t only b e visible on the long run. Our findings shed light on fundamen tal p olarization pro cesses, in particular with resp ect to the role of emotions. Increasing levels of p olarization p ose a risk of so cial conflict and hinder collab oration and common go o ds, but a healthy so ciety needs certain level of disagreement to b e able to delib erate, discuss, and take decisions ab out imp ortant topics. Calibrating the design of web and so cial media offers this w ay the chance to find a balance b etw een stagnation and p olarization, leading to pro ductiv e in teraction in our current online so ciet y . 14/20 A diy a Abishev a, Da vid Garcia, F rank Sc h w eitzer: When the Filter Bubble Bursts: Collectiv e Ev aluation Dynamics in Online Communities Submitted to the 8th In ternational ACM W eb Science Conference 2016 7 A c kno wledgments: This research was funded b y the Swiss National Science F oundation (CR21I1_146499/1). References [1] Abbasi, A., Hassan, A., and Dhar, M. Benc hmarking t witter sen timent analysis tools. In Pr o- c e e dings of the Ninth International Confer enc e on L anguage R esour c es and Evaluation (LREC’14) (Reykja vik, Iceland, ma y 2014), Europ ean Language Resources Association (ELRA). [2] Abishev a, A., Garimella, V. R. K., Garcia, D., and Weber, I. Who watc hes (and shares) what on y outub e? and when? using t witter to understand youtube view ership. In In Pr o c e e dings of the 7th ACM international c onfer enc e on W eb se ar ch and data mining, p ages: 593-602 (F ebruary 2014). [3] Adamic, L. A., and Glance, N. The p olitical blogosphere and the 2004 us election: divided they blog. In Pr o c e e dings of the 3r d international workshop on Link disc overy (2005), A CM, pp. 36–43. [4] Alstott, J., Bullmore, E., and Plenz, D. pow erlaw: A p ython pac k age for analysis of heavy- tailed distributions. PL oS ONE 9 , 1 (01 2014), e85777. [5] Amendola, L., Marra, V., and Quar tin, M. The ev olving perception of contro versial movies. Palgr ave Communic ations 1 (2015). [6] Asur, S., Huberman, B. A., Szabo, G., and W ang, C. T rends in so cial media: Persistence and decay . Available at SSRN 1755748 (2011). [7] Burke, M., Marlow, C., and Lento, T. So cial net work activity and social w ell-b eing. In Pr o c e e dings of the SIGCHI Confer enc e on Human F actors in Computing Systems (2010), A CM, pp. 1909–1912. [8] Chmiel, A., Sobko wicz, P., Sienkiewicz, J., P al toglou, G., Buckley, K., Thel w all, M., and Hołyst, J. A. Negativ e emotions b o ost user activit y at {BBC} forum. Physic a A: Statistic al Me chanics and its Applic ations 390 , 16 (2011), 2936 – 2944. [9] Choudhur y, M. D., Counts, S., and Gamon, M. Not all mo o ds are created equal! exploring h uman emotional states in social media. In ICWSM (2012), J. G. Breslin, N. B. Ellison, J. G. Shanahan, and Z. T ufekci, Eds., The AAAI Press. [10] Choudhur y, M. D., and De, S. Mental health discourse on reddit: Self-disclosure, social sup- p ort, and anon ymity . In Pr o c e e dings of the Eighth International Confer enc e on W eblo gs and So cial Me dia, ICWSM 2014, Ann A rb or, Michigan, USA, June 1-4, 2014. (2014). [11] Conover, M., Ra tkiewicz, J., Francisco, M. R., Gonçal ves, B., Menczer, F., and Flammini, A. Political p olarization on t witter. ICWSM 133 (2011), 89–96. 15/20 A diy a Abishev a, Da vid Garcia, F rank Sc h w eitzer: When the Filter Bubble Bursts: Collectiv e Ev aluation Dynamics in Online Communities Submitted to the 8th In ternational ACM W eb Science Conference 2016 [12] Crane, R., and Sornette, D. Robust dynamic classes revealed by measuring the resp onse function of a social system. Pr o c e e dings of the National A c ademy of Scienc es 105 , 41 (2008), 15649–15653. [13] Cuddeback, G., Wilson, E., Orme, J. G., and Combs-Orme, T. Detecting and statistically correcting sample selection bias. Journal of So cial Servic e R ese ar ch 30 , 3 (2004), 19–33. [14] Flache, A., and Macy, M. W. Small w orlds and cultural polarization. The Journal of Mathe- matic al So ciolo gy 35 , 1-3 (2011), 146–176. [15] Font aine, J. R., Scherer, K. R., Roesch, E. B., and Ellswor th, P. C. The world of emotions is not t wo-dimensional. Psycholo gic al scienc e 18 , 12 (2007), 1050–1057. [16] for Comput a tional St a tistics, S. U. L., and Friedman, J. H. F ast MARS. 1993. [17] Friedman, J. H. Multiv ariate adaptiv e regression splines. The annals of statistics (1991), 1–67. [18] Garas, A., Garcia, D., Sk owr on, M., and Schweitzer, F. Emotional p ersistence in online c hatting comm unities. Scientific R ep orts 2 , arXiv:1205.2466 (May 2012), 402. 34 p. Comments: 34 pages, 4 main and 12 supplemen tary figures. [19] Garcia, D., Abishev a, A., Schweighofer, S., Serdül t, U., and Schweitzer, F. Ideo- logical and temp oral comp onents of net work p olarization in online political participatory media. Policy & Internet 7 , 1 (2015), 46–79. [20] Garcia, D., Mendez, F., Serdul t, U., and Schweitzer, F. Political p olarization and p op- ularit y in online participatory media : An integrated approach. In Pr o c e e dings of the first e dition workshop on Politics, ele ctions and data - PLEAD ’12 (2012), pp. 3–10. [21] Garcia, D., and Schweitzer, F. Emotions in pro duct reviews âĂŞ empirics and models. Pr o- c e e dings of 2011 IEEE International Confer enc e on Privacy, Se curity, Risk, and T rust, and IEEE International Confer enc e on So cial Computing, P ASSA T/So cialCom (2011), 483– 488. [22] García, D., and T anase, D. Measuring cultural dynamics through the euro vision song contest. A dvanc es in Complex Systems 16 , 08 (2013), 1350037. [23] Golder, S. A., and Ma cy, M. W. Diurnal and seasonal mo o d v ary with work, sleep, and da ylength across div erse cultures. Scienc e 333 , 6051 (2011), 1878–1881. [24] Gonzalez-Bailon, S., Banchs, R. E., and Kal tenbrunner, A. Emotional reactions and the pulse of public opinion: Measuring the impact of political ev ents on the sentimen t of online discussions. CoRR abs/1009.4019 (2010). [25] Gorn, G., Pham, M. T., and Sin, L. Y. When arousal influences ad ev aluation and v alence do es not (and vice v ersa). Journal of c onsumer Psycholo gy 11 , 1 (2001), 43–55. [26] Guerra, P. H. C., Meira Jr, W., Cardie, C., and Kleinberg, R. A measure of polariza- tion on so cial media net works based on communit y boundaries. In ICWSM (2013). [27] Hoang, T.-A., Cohen, W. W., Lim, E.-P., Pierce, D., and Redla wsk, D. P. Politics, shar- ing and emotion in microblogs. In Pr o c e e dings of the 2013 IEEE/ACM International Confer enc e on A dvanc es in So cial Networks Analysis and Mining (2013), ACM, pp. 282–289. 16/20 A diy a Abishev a, Da vid Garcia, F rank Sc h w eitzer: When the Filter Bubble Bursts: Collectiv e Ev aluation Dynamics in Online Communities Submitted to the 8th In ternational ACM W eb Science Conference 2016 [28] Iosub, D., Laniado, D., Castillo, C., Fuster Morell, M., and Kal tenbr unner, A. Emotions under discussion: Gender, status and communication in online collab oration. PL oS ONE 9 , 8 (2014), e104880. [29] Isen, A. M., Shalker, T. E., Clark, M., and Karp, L. Affect, accessibilit y of material in memory , and b ehavior: A cognitiv e loop? Journal of p ersonality and so cial psycholo gy 36 , 1 (1978), 1. [30] Kan, A., Chan, J., Ha yes, C., and Hogan, B. Studying forum dynamics from a user engage- men t p erspective. [31] Kim, S., Bak, J., and Oh, A. H. Do y ou feel what i feel? social asp ects of emotions in t witter con versations. In ICWSM (2012). [32] Kivran-Sw aine, F., Brod y, S., Diakopoulos, N., and Naaman, M. Of joy and gender: Emotional expression in online so cial netw orks. In Pr o c e e dings of the ACM 2012 Confer enc e on Computer Supp orte d Co op er ative W ork Comp anion (2012), CSCW ’12, pp. 139–142. [33] Kivran-Sw aine, F., and Naaman, M. Net work prop erties and so cial sharing of emotions in so cial aw areness streams. In Pr o c e e dings of the ACM 2011 c onfer enc e on Computer supp orte d c o op er ative work (2011), A CM, pp. 379–382. [34] Kucuktunc, O., Cambazoglu, B. B., Weber, I., and Ferha tosmanoglu, H. A large-scale sen timent analysis for yahoo! answ ers. In Pr o c e e dings of the Fifth ACM International Confer enc e on W eb Se ar ch and Data Mining (New Y ork, NY, USA, 2012), WSDM ’12, ACM, pp. 633–642. [35] Kühne, R., et al. Political news, emotions, and opinion formation: T o ward a model of emo- tional framing effects. In A nnual Confer enc e of the International Communic ation Asso ciation (ICA), Pho enix, AZ (2012). [36] Larsen, P. O.c.’s reb ecca black talks about ’frida y’. http://www.ocregister.com/articles/ - 292662- - .html . A ccessed: 2016-02-02. [37] Lee, M., Ha, T., Han, J., and Rha, J.-Y. Online footsteps to purchase: Exploring consumer b eha viors on online shopping sites. [38] Lesko vec, J., Adamic, L. A., and Huberman, B. A. The dynamics of viral marketing. ACM T r ansactions on the W eb (TWEB) 1 , 1 (2007), 5. [39] Lin, Y.-R. Assessing sen timent segregation in urban comm unities. In Pr o c e e dings of the 2014 International Confer enc e on So cial Computing (2014), pp. 9:1–9:8. [40] Lorenz, J. Contin uous opinion dynamics under b ounded confidence: A surv ey . International Journal of Mo dern Physics C 18 , 12 (2007), 1819–1838. [41] Lorenz, J. Universalit y in movie rating distributions. The Eur op e an Physic al Journal B 71 , 2 (2009), 251–258. [42] Mäs, M., and Flache, A. Differentiation without distancing. explaining bi-p olarization of opin- ions without negativ e influence. PloS one 8 , 11 (2013), e74516. 17/20 A diy a Abishev a, Da vid Garcia, F rank Sc h w eitzer: When the Filter Bubble Bursts: Collectiv e Ev aluation Dynamics in Online Communities Submitted to the 8th In ternational ACM W eb Science Conference 2016 [43] McPhee, W. N. F ormal the ories of mass b ehavior . The F ree Press of Glenco e, Collier- Macmillian, London, 1963. [44] Mieghem, P. V. Human psyc hology of common appraisal: The reddit score. IEEE T r ansactions on Multime dia 13 , 6 (2011), 1404–1406. [45] Miller, M., Sa thi, C., Wiesenthal, D., Lesko vec, J., and Potts, C. Sentimen t flo w through hyperlink net w orks. In ICWSM (2011). [46] Mitzenmacher, M. A brief history of generative mo dels for p ow er la w and lognormal distribu- tions. Internet mathematics 1 , 2 (2004), 226–251. [47] Naka t ani, S. Language detection library for ja v a. https://github.com/shuyo/ language- detection/blob/wiki/ProjectHome.md , 2010–curren t. [48] Noelle-Neumann, E., and Noelle-Neumann, E. The spir al of silenc e: Public opinion, our so cial skin . Universit y of Chicago Press Chicago, 1993. [49] Olson, R. A data-driven guide to creating successful reddit p osts, redux. Accessed: 2016-02-02. [50] Onnela, J.-P., and Reed-Tsochas, F. Sp ontaneous emergence of so cial influence in online systems. Pr o c e e dings of the National A c ademy of Scienc es 107 , 43 (2010), 18375–18380. [51] P ariser, E. The filter bubble: What the Internet is hiding fr om you . P enguin UK, 2011. [52] P aulhus, D. L., and Lim, D. T. Arousal and ev aluative extremity in social judgmen ts: A dy- namic complexity mo del. Eur op e an Journal of So cial Psycholo gy 24 , 1 (1994), 89–99. [53] Pfitzner, R., Garas, A., and Schweitzer, F. Emotional div ergence influences informa- tion spreading in t witter. In ICWSM (2012), J. G. Breslin, N. B. Ellison, J. G. Shanahan, and Z. T ufekci, Eds., The AAAI Press. [54] Quercia, D., Capra, L., and Cro wcr oft, J. The social w orld of twitter: T opics, geograph y , and emotions. In ICWSM (2012), J. G. Breslin, N. B. Ellison, J. G. Shanahan, and Z. T ufekci, Eds., The AAAI Press. [55] Reisenzein, R. The sc hach ter theory of emotion: tw o decades later. Psycholo gic al bul letin 94 , 2 (1983), 239. [56] Ro we, M., and Alani, H. Mining and comparing engagemen t dynamics across m u ltiple so- cial media platforms. In Pr o c e e dings of the 2014 ACM c onfer enc e on W eb scienc e (2014), ACM, pp. 229–238. [57] Russell, J. A., and Barrett, L. F. Core affect, prototypical emotional episo des, and other things called emotion: Dissecting the elephan t. Journal of Personality and So cial Psycholo gy 76 , 5 (1999), 805–819. [58] Schw arz, N., and Clore, G. Ho w do i feel ab out it? the informativ e function of affectiv e states. Affe ct, c o gnition, and so cial b ehavior (1988), 44–62. [59] Schweitzer, F., and Garcia, D. An agen t-based model of collective emotions in online com- m unities. The Eur op e an Physic al Journal B 77 , 4 (2010), 533– 545. 18/20 A diy a Abishev a, Da vid Garcia, F rank Sc h w eitzer: When the Filter Bubble Bursts: Collectiv e Ev aluation Dynamics in Online Communities Submitted to the 8th In ternational ACM W eb Science Conference 2016 [60] Szabo, G., and Huberman, B. A. Predicting the p opularity of online conten t. Communic ations of the ACM 53 , 8 (2010), 80–88. [61] T an, C., Lee, L., T ang, J., Jiang, L., Zhou, M., and Li, P. User-lev el sen timen t analysis incorp orating so cial net works. In Pr o c e e dings of the 17th ACM SIGKDD International Confer enc e on Know le dge Disc overy and Data Mining (2011), KDD ’11, pp. 1397–1405. [62] T a ylor, J., and Macdonald, J. The effects of async hronous computer-mediated group in terac- tion on group processes. So cial Scienc e Computer R eview 20 , 3 (2002), 260–274. [63] Tchokni, S., Ó Séaghdha, D., and Quercia, D. Emoticons and phrases: Status symbols in so cial media. In Pr o c e e dings of the 8th International AAAI Confer enc e on W eblo gs and So cial Me dia (ICWSM 2014) (Ann Arbor, MI, 2014). [64] Thel w all, M., Buckley, K., and P al toglou, G. Sentimen t strength detection for the social w eb. J. Am. So c. Inf. Sci. T e chnol. 63 , 1 (Jan. 2012), 163–173. [65] Thel w all, M., Buckley, K., P al toglou, G., Cai, D., and Kapp as, A. Sentimen t in short strength detection informal text. J. A m. So c. Inf. Sci. T e chnol. 61 , 12 (Dec. 2010), 2544–2558. [66] Thel w all, M., Wilkinson, D., and Upp al, S. Data mining emotion in so cial net work commu- nication: Gender differences in myspace. Journal of the A meric an So ciety for Information Scienc e and T e chnolo gy 61 , 1 (2010), 190–199. [67] V an Mieghem, P., Blenn, N., and Doerr, C. Lognormal distribution in the digg online so cial net work. The Eur op e an Physic al Journal B 83 , 2 (2011), 251–261. [68] V an Swol, L. M. Extreme members and group p olarization. So cial Influenc e 4 , 3 (2009), 185– 199. [69] V arol, O., Ferrara, E., Ogan, C. L., Menczer, F., and Flammini, A. Evolution of on- line user b eha vior during a so cial upheav al. In Pr o c e e dings of the 2014 ACM c onfer enc e on W eb scienc e (2014), ACM, pp. 81–90. [70] W ang, C., and Huberman, B. A. Long trend dynamics in social media. EPJ Data Scienc e 1 , 1 (2012), 1–8. [71] W ang, N., Kosinski, M., Still well, D., and Rust, J. Can w ell-b eing be measured using faceb ook status updates? v alidation of faceb o ok âĂŹs gross national happiness index. So cial Indi- c ators R ese ar ch 115 , 1 (2014), 483–491. [72] W arriner, A., Kuperman, V., and Br ysbaer t, M. Norms of v alence, arousal, and dominance for 13,915 english lemmas. Behavior R ese ar ch Metho ds 45 , 4 (2013), 1191–1207. [73] Weninger, T., Johnston, T. J., and Glenski, M. Random v oting effects in so cial-digital spaces: A case study of reddit p ost submissions. In Pr o c e e dings of the 26th ACM Confer enc e on Hyp ertext & So cial Me dia (2015), ACM, pp. 293–297. [74] West, R., P ask ov, H., Lesko vec, J., and Potts, C. Exploiting social net work structure for p erson-to-person sen timent analysis. T r ansactions of the Asso ciation for Computational Linguis- tics 2 (2014), 297–310. 19/20 A diy a Abishev a, Da vid Garcia, F rank Sc h w eitzer: When the Filter Bubble Bursts: Collectiv e Ev aluation Dynamics in Online Communities Submitted to the 8th In ternational ACM W eb Science Conference 2016 [75] Wu, F., and Huberman, B. A. No v elty and collectiv e atten tion. Pr o c e e dings of the National A c ademy of Scienc es 104 , 45 (2007), 17599–17601. [76] Zillmann, D. Excitation transfer in comm unication-mediated aggressiv e b eha vior. Journal of exp erimental so cial psycholo gy 7 , 4 (1971), 419–434. 20/20