Authors: (1) Anh V. Vu, University of Cambridge, Cambridge Cybercrime Centre (anh.vu@cl.cam.ac.uk); (2) Alice Hutchings, University of Cambridge, Cambridge Cybercrime Centre (alice.hutchings@cl.cam.ac.uk); (3) Ross Anderson, University of Cambridge, and University of Edinburgh (ross.anderson@cl.cam.ac.uk). Table of Links Abstract and 1 Introduction 2. Deplatforming and the Impacts 2.1. Related Work 2.2. The Kiwi Farms Disruption 3. Methods, Datasets, and Ethics, and 3.1. Forum and Imageboard Discussions 3.2. Telegram Chats and 3.3. Web Traffic and Search Trends Analytics 3.4. Tweets Made by the Online Community and 3.5. Data Licensing 3.6. Ethical Considerations 4. The Impact on Forum Activity and Traffic, and 4.1. The Impact of Major Disruptions 4.2. Platform Displacement 4.3. Traffic Fragmentation 5. The Impacts on Relevant Stakeholders and 5.1. The Community that Started the Campaign 5.2. The Industry Responses 5.3. The Forum Operators 5.4. The Forum Members 6. Tensions, Challenges, and Implications and 6.1. The Efficacy of the Disruption 6.2. Censorship versus Free Speech 6.3. The Role of Industry in Content Moderation 6.4. Policy Implications 6.5. Limitations and Future Work 7. Conclusion, Acknowledgments, and References Appendix A. 5.4. The Forum Members People sharing the same passion naturally coalesce into communities, in which some key actors may play a crucial role in influencing the ecosystem [81], [82], [83]. We separate the pre-disruption and post-disruption by 3 September 2022, when Cloudflare took action. KIWI FARMS activity is highly skewed, with around 80%[12] of pre-disruption posts made by 8.96% most active users (5 159), while the remaining 20% posts were made by the 91.04% less active (52 430). There was around a 30% drop in the number of users after the disruption, as seen in Figure 4. There were 1 571 new usernames after the disruption, which could be either newcomers or old members creating new accounts after losing access to old ones. Multi-platform users tend to pick similar pseudonyms on different platforms [85]; we believe returning users are also likely do that to preserve their reputation, so can be detected if their usernames are very similar and rare enough, although common handles are often picked up by multiple individuals [86]. While the similarity of two usernames can be determined by the Levenshtein distance, we use a n-gram model trained by the Reuters corpus [87] to estimate the rarity of usernames, considering one is rare if the highest probability observed is not greater than 1%. We found 5.31% such users among 1 571 new pseudonyms: 11 returning core actors (0.21% of core users), 72 returning casual actors (0.14% of casual users), while the rest 1 488 are newcomers. The estimation may overlook all-new usernames, yet we believe this number is relatively small as a mass password reset was mandated after the breach instead of account replacement. We analyse the behaviour of those active after the disruption, namely the ‘core survivors’ (2 529, returning actors included), ‘casual survivors’ (6 915, returning actors included), and ‘newcomers’ (1 488). Around half of key users (49.02%) remained engaged, while only 13.19% of casual users stayed (86.95% has left). On average, before the disruption, each ‘core survivor’ posted 22.3 times more than each ‘casual survivor’ (1800.03 vs 80.82 posts), while their active period – between their first post and last post – was around 2.5 times longer (1306.94 vs 516.84 days). Posting Activity. Before the takedown, each core survivor made about 3.5 posts per day on average, while it was around 3 afterwards – see Figure 8. The activity of the other survivors appears consistent with the pre-disruption period; their average posts were at around 2 per day before the incident and almost unchanged afterwards. These figures suggest that the decreasing posting volume seen in Figure 4 was mainly due to users leaving the forum, instead of surviving ones largely losing interest – they engaged back quickly after the forum recovered. Newcomers posted slightly less than casual survivors before the forum was completely down on 18 September 2022 (less than 2 posts per day), yet their average posting volume then increased quickly. This suggests that the disruption, besides removing a very large proportion of old casual users, drew in many new users who then became roughly as active as the core survivors. Toxicity Levels. We further examine the toxicity of posts made by the surviving actors and newcomers, before and after the disruption. Figure 9 shows the average levels of toxicity, identity attack and threat of core survivors, casual survivors, and newcomers by days. In general, the toxicity, identity attack, and threat scores were rather low as most postings are non-toxic (despite some having very high scores). There were small changes in the average scores of surviving actors, notably the peaks occurred 2 days after the campaign sparked on Twitter, with the average scores increasing significantly to around 30–50%, especially toxicity and identity attack. However, these dropped quickly a couple of days after and retreated to normal levels. Newcomers, on the other hand, expressed a significant increase of toxicity and identity attack during the first two weeks after the disruption took place (about 2–2.5 times higher), largely surpassing surviving actors. Their scores for threat did not increase at that time but largely peaked after the forum first recovered on 27 September 2022, with around 2 times higher. These activities suggest that while the surviving members were becoming more toxic when their community was under attack, new users became much more toxic for a few weeks after they engaged in the discussion before declining gradually to the same levels as old users. This is in line with the recent finding that users moving to other platforms can become more toxic than before [33]. Social Interactions. To measure how these survivors interact with each other, we build a social interaction network among KIWI FARMS members over time. We consider each active user as a node, with an edge between two users if they posted in the same thread (weighted by the number of such interactions) [88]. We then explore changes of that sharedinterest engagement in the network structure with a focus on Degree Centrality, which indicates how well-connected a user is over the entire network [89]. In a healthy community, such engagement should grow steadily. The network had developed stably before the disruption, with around 55.3k nodes and 131.3M edges on 1 July 2022, reaching to around 57.2k nodes and 137.6M edges just before the Twitter campaign started (see Figure 10). There was a rapid increase in both nodes and edges shortly after the Twitter campaign, suggesting that the campaign drew more actors involved in interacting with others. The Cloudflare and DDoS-Guard actions paused the network for a few weeks, yet it resumed shortly after the forum’s recovery. As of 31 December 2022, the network size has reached 59.1k nodes and 149.3M edges. Core users are better connected than casual users, see Figure 11. The Twitter campaign largely boosted the centrality of both core and casual survivors. Before that, while core survivors were getting more centralised over time, casual survivors were becoming less centralised. But after the campaign on Twitter, the centralisation of both steadily increased. Newcomers came into play quickly afterwards and the forum recovery also made them more centralised. Discussion of the Incident. We examine how users mention the two major involved parties (KIWI FARMS and Cloudflare) during the period by extracting posts containing caseinsensitive keywords ‘kiwifarm’, ‘kiwi farm’, ‘cloudflare’, and ‘cloud flare’ from KIWI FARMS, its Telegram channel, and LOLCOW FARM. Table 4 shows that discussions about the two parties were highly skewed and significantly dependent on the platforms. Telegram users tended to discuss things relevant to KIWI FARMS more than Cloudflare (13.3 times higher), while the ratios were less skewed for KIWI FARMS and LOLCOW FARM, with 6.7 and 7.6, respectively. These discussions are centralised around a small number of conversations, for example, over 50% of posts mentioning Cloudflare on KIWI FARMS are just from 4 threads. Although these posts accounted for a trivial contribution to the total posting volume on all three platforms as shown in Figure 4, most happened after the Twitter campaign, with almost no discussion before. The topic was popular for a short period, as shown in Figure 12. Users on both forums started discussing the incident shortly after the campaign started on 22 August 2022. The topic was energised on both forums after Cloudflare’s action on 3 September 2022, peaking on 4 September 2022 on KIWI FARMS with over 400 and 600 posts about KIWI FARMS and Cloudflare (around 5% and 7.5% of all posts on that day), respectively. After KIWI FARMS activity was significantly reduced due to DDoSGuard’s action on 5 September 2022, posts mentioning KIWI FARMS and Cloudflare on LOLCOW FARM peaked at around 80 and 20, respectively.[1]3 Telegram activity regarding the incident was a bit different, as comments were only allowed after the forum was completely down; it followed the same trends as overall activity, with a peak of discussion about KIWI FARMS happening largely when the forum was inaccessible, as part of the discussion had moved here. Discussion mentioning KIWI FARMS greatly exceeded those mentioning Cloudflare until the day Cloudflare took action (see the first graph in Figure 12). The pattern seen on LOLCOW FARM suggests that the attention toward the incident was reflected there, although the peak did not correlate with the overall volume observed in Figure 4 as this contribution is trivial compared to the total. There were almost no posts about Cloudflare after KIWI FARMS became completely inaccessible, but there were still around 20 posts about KIWI FARMS seen on LOLCOW FARM during that week. While nothing changed on KIWI FARMS during the second recovery, there was an increase in posts on LOLCOW FARM about the incident, presumably as people got the news. Overall, attention on KIWI FARMS, its Telegram channels, and LOLCOW FARM was directed to the incident by the Twitter campaign, with posting volume peaking after the industry action. We believe it shows a genuine effect as none of the users there discussed Cloudflare and KIWI FARMS before. However, the effect was temporary and almost dropped to the pre-disruption level after the second recovery: they lasted for a few days on KIWI FARMS, around one week on LOLCOW FARM (partly due to many domains of KIWI FARMS being down while LOLCOW FARM was still active), and a few weeks on Telegram. Users’ interest was fleeting; they largely stopped talking about the incident after a few weeks. This paper is available on arxiv under CC BY 4.0 DEED license. [12] We make use of the 80/20 rule – the Pareto principle [84]. [13] The numbers for LOLCOW FARM are typically lower than KIWI FARMS as LOLCOW FARM is smaller and centred on images instead of text. We do not collect images for safety and ethical reasons, but we believe the trends observed are likely indicative if not reliable. Authors: (1) Anh V. Vu, University of Cambridge, Cambridge Cybercrime Centre (anh.vu@cl.cam.ac.uk); (2) Alice Hutchings, University of Cambridge, Cambridge Cybercrime Centre (alice.hutchings@cl.cam.ac.uk); (3) Ross Anderson, University of Cambridge, and University of Edinburgh (ross.anderson@cl.cam.ac.uk). Authors: Authors: (1) Anh V. Vu, University of Cambridge, Cambridge Cybercrime Centre (anh.vu@cl.cam.ac.uk); (2) Alice Hutchings, University of Cambridge, Cambridge Cybercrime Centre (alice.hutchings@cl.cam.ac.uk); (3) Ross Anderson, University of Cambridge, and University of Edinburgh (ross.anderson@cl.cam.ac.uk). Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2. Deplatforming and the Impacts 2. Deplatforming and the Impacts 2.1. Related Work 2.1. Related Work 2.2. The Kiwi Farms Disruption 2.2. The Kiwi Farms Disruption 3. Methods, Datasets, and Ethics, and 3.1. Forum and Imageboard Discussions 3. Methods, Datasets, and Ethics, and 3.1. Forum and Imageboard Discussions 3.2. Telegram Chats and 3.3. Web Traffic and Search Trends Analytics 3.2. Telegram Chats and 3.3. Web Traffic and Search Trends Analytics 3.4. Tweets Made by the Online Community and 3.5. Data Licensing 3.4. Tweets Made by the Online Community and 3.5. Data Licensing 3.6. Ethical Considerations 3.6. Ethical Considerations 4. The Impact on Forum Activity and Traffic, and 4.1. The Impact of Major Disruptions 4. The Impact on Forum Activity and Traffic, and 4.1. The Impact of Major Disruptions 4.2. Platform Displacement 4.2. Platform Displacement 4.3. Traffic Fragmentation 4.3. Traffic Fragmentation 5. The Impacts on Relevant Stakeholders and 5.1. The Community that Started the Campaign 5. The Impacts on Relevant Stakeholders and 5.1. The Community that Started the Campaign 5.2. The Industry Responses 5.2. The Industry Responses 5.3. The Forum Operators 5.3. The Forum Operators 5.4. The Forum Members 5.4. The Forum Members 6. Tensions, Challenges, and Implications and 6.1. The Efficacy of the Disruption 6. Tensions, Challenges, and Implications and 6.1. The Efficacy of the Disruption 6.2. Censorship versus Free Speech 6.2. Censorship versus Free Speech 6.3. The Role of Industry in Content Moderation 6.3. The Role of Industry in Content Moderation 6.4. Policy Implications 6.4. Policy Implications 6.5. Limitations and Future Work 6.5. Limitations and Future Work 7. Conclusion, Acknowledgments, and References 7. Conclusion, Acknowledgments, and References Appendix A. Appendix A. 5.4. The Forum Members People sharing the same passion naturally coalesce into communities, in which some key actors may play a crucial role in influencing the ecosystem [81], [82], [83]. We separate the pre-disruption and post-disruption by 3 September 2022, when Cloudflare took action. KIWI FARMS activity is highly skewed, with around 80%[12] of pre-disruption posts made by 8.96% most active users (5 159), while the remaining 20% posts were made by the 91.04% less active (52 430). There was around a 30% drop in the number of users after the disruption, as seen in Figure 4. There were 1 571 new usernames after the disruption, which could be either newcomers or old members creating new accounts after losing access to old ones. Multi-platform users tend to pick similar pseudonyms on different platforms [85]; we believe returning users are also likely do that to preserve their reputation, so can be detected if their usernames are very similar and rare enough, although common handles are often picked up by multiple individuals [86]. While the similarity of two usernames can be determined by the Levenshtein distance, we use a n-gram model trained by the Reuters corpus [87] to estimate the rarity of usernames, considering one is rare if the highest probability observed is not greater than 1%. We found 5.31% such users among 1 571 new pseudonyms: 11 returning core actors (0.21% of core users), 72 returning casual actors (0.14% of casual users), while the rest 1 488 are newcomers. The estimation may overlook all-new usernames, yet we believe this number is relatively small as a mass password reset was mandated after the breach instead of account replacement. We analyse the behaviour of those active after the disruption, namely the ‘core survivors’ (2 529, returning actors included), ‘casual survivors’ (6 915, returning actors included), and ‘newcomers’ (1 488). Around half of key users (49.02%) remained engaged, while only 13.19% of casual users stayed (86.95% has left). On average, before the disruption, each ‘core survivor’ posted 22.3 times more than each ‘casual survivor’ (1800.03 vs 80.82 posts), while their active period – between their first post and last post – was around 2.5 times longer (1306.94 vs 516.84 days). Posting Activity. Before the takedown, each core survivor made about 3.5 posts per day on average, while it was around 3 afterwards – see Figure 8. The activity of the other survivors appears consistent with the pre-disruption period; their average posts were at around 2 per day before the incident and almost unchanged afterwards. These figures suggest that the decreasing posting volume seen in Figure 4 was mainly due to users leaving the forum, instead of surviving ones largely losing interest – they engaged back quickly after the forum recovered. Newcomers posted slightly less than casual survivors before the forum was completely down on 18 September 2022 (less than 2 posts per day), yet their average posting volume then increased quickly. This suggests that the disruption, besides removing a very large proportion of old casual users, drew in many new users who then became roughly as active as the core survivors. Posting Activity. Toxicity Levels. We further examine the toxicity of posts made by the surviving actors and newcomers, before and Toxicity Levels. after the disruption. Figure 9 shows the average levels of toxicity, identity attack and threat of core survivors, casual survivors, and newcomers by days. In general, the toxicity, identity attack, and threat scores were rather low as most postings are non-toxic (despite some having very high scores). There were small changes in the average scores of surviving actors, notably the peaks occurred 2 days after the campaign sparked on Twitter, with the average scores increasing significantly to around 30–50%, especially toxicity and identity attack. However, these dropped quickly a couple of days after and retreated to normal levels. Newcomers, on the other hand, expressed a significant increase of toxicity and identity attack during the first two weeks after the disruption took place (about 2–2.5 times higher), largely surpassing surviving actors. Their scores for threat did not increase at that time but largely peaked after the forum first recovered on 27 September 2022, with around 2 times higher. These activities suggest that while the surviving members were becoming more toxic when their community was under attack, new users became much more toxic for a few weeks after they engaged in the discussion before declining gradually to the same levels as old users. This is in line with the recent finding that users moving to other platforms can become more toxic than before [33]. Social Interactions. To measure how these survivors interact with each other, we build a social interaction network among KIWI FARMS members over time. We consider each active user as a node, with an edge between two users if they posted in the same thread (weighted by the number of such interactions) [88]. We then explore changes of that sharedinterest engagement in the network structure with a focus on Degree Centrality, which indicates how well-connected a user is over the entire network [89]. In a healthy community, such engagement should grow steadily. Social Interactions. The network had developed stably before the disruption, with around 55.3k nodes and 131.3M edges on 1 July 2022, reaching to around 57.2k nodes and 137.6M edges just before the Twitter campaign started (see Figure 10). There was a rapid increase in both nodes and edges shortly after the Twitter campaign, suggesting that the campaign drew more actors involved in interacting with others. The Cloudflare and DDoS-Guard actions paused the network for a few weeks, yet it resumed shortly after the forum’s recovery. As of 31 December 2022, the network size has reached 59.1k nodes and 149.3M edges. Core users are better connected than casual users, see Figure 11. The Twitter campaign largely boosted the centrality of both core and casual survivors. Before that, while core survivors were getting more centralised over time, casual survivors were becoming less centralised. But after the campaign on Twitter, the centralisation of both steadily increased. Newcomers came into play quickly afterwards and the forum recovery also made them more centralised. Discussion of the Incident. We examine how users mention the two major involved parties (KIWI FARMS and Cloudflare) during the period by extracting posts containing caseinsensitive keywords ‘kiwifarm’, ‘kiwi farm’, ‘cloudflare’, and ‘cloud flare’ from KIWI FARMS, its Telegram channel, and LOLCOW FARM. Table 4 shows that discussions about the two parties were highly skewed and significantly dependent on the platforms. Telegram users tended to discuss things relevant to KIWI FARMS more than Cloudflare (13.3 times higher), while the ratios were less skewed for KIWI FARMS and LOLCOW FARM, with 6.7 and 7.6, respectively. These discussions are centralised around a small number of conversations, for example, over 50% of posts mentioning Cloudflare on KIWI FARMS are just from 4 threads. Discussion of the Incident. Although these posts accounted for a trivial contribution to the total posting volume on all three platforms as shown in Figure 4, most happened after the Twitter campaign, with almost no discussion before. The topic was popular for a short period, as shown in Figure 12. Users on both forums started discussing the incident shortly after the campaign started on 22 August 2022. The topic was energised on both forums after Cloudflare’s action on 3 September 2022, peaking on 4 September 2022 on KIWI FARMS with over 400 and 600 posts about KIWI FARMS and Cloudflare (around 5% and 7.5% of all posts on that day), respectively. After KIWI FARMS activity was significantly reduced due to DDoSGuard’s action on 5 September 2022, posts mentioning KIWI FARMS and Cloudflare on LOLCOW FARM peaked at around 80 and 20, respectively.[1]3 Telegram activity regarding the incident was a bit different, as comments were only allowed after the forum was completely down; it followed the same trends as overall activity, with a peak of discussion about KIWI FARMS happening largely when the forum was inaccessible, as part of the discussion had moved here. Discussion mentioning KIWI FARMS greatly exceeded those mentioning Cloudflare until the day Cloudflare took action (see the first graph in Figure 12). The pattern seen on LOLCOW FARM suggests that the attention toward the incident was reflected there, although the peak did not correlate with the overall volume observed in Figure 4 as this contribution is trivial compared to the total. There were almost no posts about Cloudflare after KIWI FARMS became completely inaccessible, but there were still around 20 posts about KIWI FARMS seen on LOLCOW FARM during that week. While nothing changed on KIWI FARMS during the second recovery, there was an increase in posts on LOLCOW FARM about the incident, presumably as people got the news. Overall, attention on KIWI FARMS, its Telegram channels, and LOLCOW FARM was directed to the incident by the Twitter campaign, with posting volume peaking after the industry action. We believe it shows a genuine effect as none of the users there discussed Cloudflare and KIWI FARMS before. However, the effect was temporary and almost dropped to the pre-disruption level after the second recovery: they lasted for a few days on KIWI FARMS, around one week on LOLCOW FARM (partly due to many domains of KIWI FARMS being down while LOLCOW FARM was still active), and a few weeks on Telegram. Users’ interest was fleeting; they largely stopped talking about the incident after a few weeks. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv [12] We make use of the 80/20 rule – the Pareto principle [84]. [13] The numbers for LOLCOW FARM are typically lower than KIWI FARMS as LOLCOW FARM is smaller and centred on images instead of text. We do not collect images for safety and ethical reasons, but we believe the trends observed are likely indicative if not reliable.
