(b) any abusive comment or abusive pictorial representation that, when taken in context, tends to or is likely to expose an individual or group or class of individuals to hatred or contempt on the basis of race, national or ethnic origin, colour, religion, sex, sexual orientation, age or mental or physical disability

In principle, the distribution of such content by a licensee can lead to fines and even to the removal of the license by the CRTC. Although there is currently a bill before Parliament to amend the *Broadcasting Act* that would extend the definition of "broadcasting undertaking" to include "an online undertaking," the CRTC would not be empowered to establish a class of licences for such online undertakings.19 Thus, the current *Broadcasting Distribution Regulations* would not give the CRTC regulatory authority over hate speech distributed by online undertakings. Nonetheless, the bill would grant discretion to the CRTC to develop conditions applicable to all broadcasting undertakings, including online undertakings, "that the Commission considers appropriate for the implementation of the broadcasting policy set out in subsection 3(1)."20 In principle the CRTC could

17 Canadian Radio-television and Telecommunications Commission (CRTC), "Frequently asked questions," April 1, 2015, https://crtc.gc.ca/eng/faqs.htm.

18 *Broadcasting Distribution Regulations*, SOR/97-555, https://laws.justice.gc.ca/eng/re gulations/SOR-97-555/page-3.html#h-1010707.

19 Bill C-10, *An Act to amend the Broadcasting Act and to make related and consequential amendments to other Acts*, 2d sess., 43d Parliament, November 3, 2020, ss. 1(1) and 6(1)(a), https://parl.ca/DocumentViewer/en/43-2/bill/C-10/first-reading#ID0 E02B0AA. For a discussion of this proposed legislation see the article by Michael Geist in this collection.

20 Bill C-10, s. 9.1.

therefore develop a code of conduct parallel s. 8 of the *Broadcasting Distribution Regulations.*

The Canadian Human Rights Commission, for its part, until 2013 oversaw section 13 of the *Canadian Human Rights Act* which then provided (emphasis added):21

13 (1) It is a discriminatory practice for a person or a group of persons acting in concert to communicate or to cause to be so communicated, repeatedly, in whole or in part by means of the facilities of a telecommunication undertaking within the legislative authority of Parliament, *any matter that is likely to expose a person or persons to hatred or contempt by reason of the fact that that person or those persons are identifiable on the basis of a prohibited ground of discrimination.*

Interpretation

(2) For greater certainty, subsection (1) applies in respect of a matter that is communicated by means of a computer or a group of interconnected or related computers, including the Internet, or any similar means of communication, but does not apply in respect of a matter that is communicated in whole or in part by means of the facilities of a broadcasting undertaking.

Interpretation

(3) For the purposes of this section, no owner or operator of a telecommunication

undertaking communicates or causes to be communicated any matter described in subsection (1) by reason only that the facilities of a telecommunication undertaking owned or operated by that person are used by other persons for the transmission of that matter.

This provision was repealed by a private member's bill that notably had the support of the Canadian Civil Liberties' Association largely on the grounds that it had infringed upon free speech.22 The repeal arose under the previous Conservative government despite the fact that the Federal Court of Appeal had upheld the constitutional validity of the provision.23

21 *Canadian Human Rights Act*, *Revised Statutes of Canada,* 1985, c. H-6, archived version, https://laws-lois.justice.gc.ca/eng/acts/h-6/section-13-20021231.html#:~:tex t=13%20(1)%20It%20is%20a,Parliament%2C%20any%20matter%20that%20is.

22 See Joel Webe, "Hate speech no longer part of Canada's Human Rights Act" *National Post*, June 27, 2013, https://nationalpost.com/news/politics/hate-speech-n o-longer-part-of-canadas-human-rights-act.

23 *Lemire v. Canada (Human Rights Commission),* 2014 FCA 18, https://canlii.ca/t/g2x 2d.

Since the repeal of s. 13, the Canadian Human Rights Commission has periodically made public statements about the need to address online hate but has not engaged in any enforcement strategy.24

Provincial human rights commissions do not have the equivalent to the former s. 13 and have thus had to rely upon general anti-discrimination protections to pursue online hate speech, something they have seldom ever done. In 2015, the then Minister of Justice of Québec proposed legislation that would have added a hate speech regime to the Québec *Charter of Human Rights and Freedoms* giving rise to remedies and enforcement by the Quebec Human Rights Commission.25 However, the government eventually backed away from the proposal, with the Commission for its part recommending the introduction of such a regime but cautioning against potential overreach.26 Since that time, and in the wake of a horrific mass shooting at a mosque in Quebec City, the Commission was tasked with conducting a study on hateful acts, notably those motivated by islamophobia, in which it underscored the proliferation of hate speech on the internet.27 However, the Commission also noted that most cases are not reported to authorities, and for those that are it was typically very difficult to trace the origin of the message to an individual.28

For a comment on the consistency of s. 13 with human rights protection, see Pearl Eliadis, "The Controversy Entrepreneurs," *Maisonneuve*, August 20, 2009, https://maisonneuve.org/article/2009/08/20/controversy-entrepreneurs/.

24 See for example, Canadian Human Rights Commission, "Statement – We must do more to curb online hate," January 21, 2021, https://www.chrc-ccdp.gc.ca/en/r esources/statement-we-must-do-more-curb-online-hate.

25 Projet de loi 59, *Loi édictant la Loi concernant la prévention et la lutte contre les discours haineux et les discours incitant à la violence et apportant diverses modifications législatives pour renforcer la protection des personnes*, 1er sess., 42e législature, June 10, 2015, http://m.assnat.qc.ca/fr/travaux-parlementaires/projets-loi/projet-loi-59-41-1. html.

26 See Commission des droits de la personne et des droits de la jeunesse, *Mémoire à la Commission des institutions de l'Assemblée nationale*, August 2015, https://www.c dpdj.qc.ca/storage/app/media/publications/memoire\_PL59\_discours-haineux.pdf.

27 Commission des droits de la personne et des droits de la jeunesse, *Les actes haineux à caractère xénophobe, notamment islamophobe : résultats d'une recherche menée à travers le Québec,* August, 2019, 111, https://www.cdpdj.qc.ca/storage/app/ media/publications/etude\_actes\_haineux.pdf.

28 Commission des droits de la personne et des droits de la jeunesse, *Les actes haineux,* 186 ff. and 249.

To pursue the Québec example further, section 11 of the Québec *Charter of Human Rights and Freedoms* specifies that:

No one may distribute, publish or publicly exhibit a notice, symbol or sign involving discrimination, or authorize anyone to do so.

The Québec Human Rights Tribunal has found that a message reading "Landlords Go Home" addressed to persons of Haitian origin violated s. 11 and gave rise to damages.29 In principle this kind of case suggests a pathway toward civil remedies against online hate if the person at the origin of the message can be identified.

Although Canada's Privacy Commissioner has commented publicly on the issue of online hate speech, the role of the Commissioner's office is currently all but non-existent.30 However, Lex Gill has noted that to the degree platforms adopt algorithmic approaches to screening online hate:31

content filtering and censorship technology is almost always surveillance technology

as well. It is therefore rare that the adoption of such measures will not involve at least indirect impacts on users' privacy rights.

Furthermore, the business model of platforms is focused on keeping users engaged (addicted) and collecting as much data from them as possible so as to increase advertising revenue.32 This business model tends to amplify the spread of extreme content.33 Under proposed new Canadian legislation, modelled on the *California Consumer Privacy Act*, 34 the Privacy Commissioner and a new Data Protection Tribunal would have significantly augmented enforcement powers, including to impose fines of up to 3% of the organization's yearly gross global revenue. These powers would en-

29 *Commission des droits de la personne et des droits de la jeunesse (Coffy et une autre) c. Brisson*, 2009 QCTDP 3, https://canlii.ca/t/22qhm.

30 See Office of the Privacy Commissioner of Canada, "A Data Privacy Day Conversation with Canada's Privacy Commissioner," January 28, 2020, https://www.priv. gc.ca/en/opc-news/speeches/2020/sp-d\_20200128/.

31 Gill, "Legal Aspects of Hate Speech," 15.

32 Andrew Burt, "Can Facebook Ever be Fixed?," *Harvard Business Review*, April 8, 2019, https://hbr.org/2019/04/can-facebook-ever-be-fixed.

33 Gilad Edelman, "Social Media CEOs Can't Defend Their Business Model," *Wired*, March 25, 2021, https://www.wired.com/story/social-media-ceo-hearing-cant-defe nd-business-model/.

34 *California Code*, Title 1.81.5. *California Consumer Privacy Act of 2018*, https://leginf o.legislature.ca.gov/faces/codes\_displayText.xhtml?division=3.&part=4.&lawCode =CIV&title=1.81.5.

force a range of requirements touching digital platforms directly, notably that companies must:

The bolstered privacy regime has the potential to disrupt the business model of digital platforms and thus, indirectly, to have an impact on the spread of online hate.36 This point will be contextualized somewhat further in the fourth part of this article.

#### *Criminal law provisions c.*

Canada's *Criminal Code* contains a number of prohibitions touching upon hate speech: advocating genocide (s. 318); publicly inciting hatred (s. 319(1)); and promoting hatred (s. 319(2)).37 These provisions have been found consistent with constitutional protections of free speech.38

Although the dataset is incomplete, Statistics Canada has reported that there were only some 50 police-reported online cases per year across Cana-

35 Bill C-11, *Digital Charter Implementation Act, 2020*, 2d sess., 43d Parliament, November 17, 2020, https://parl.ca/DocumentViewer/en/43-2/bill/C-11/first-r eading. See also the parallel proposed Quebec legislation, which with respect to the "right to de-indexation" and the "right to be forgotten" goes further than the federal legislation: Projet de loi 64, *Loi modernisant des dispositions législatives en matière de protection des renseignements personnels*, 1er sess., 42e legislature, June 12, 2020, http://m.assnat.qc.ca/fr/travaux-parlementaires/projets-loi/projet-loi-64-42-1. html.

36 See Jon Swartz, "California's landmark privacy law is Facebook's next 'nightmare'," *Market* Watch, August 22, 2020, https://www.marketwatch.com/story/calif ornias-landmark-privacy-law-is-facebooks-next-nightmare-2020-08-18.

37 *Criminal Code, Revised Statutes of Canada*, 1985, c. C-46, https://laws-lois.justice.gc. ca/eng/acts/c-46/.

38 See notably *R v Keegstra*, [1990] 3 SCR 697, https://scc-csc.lexum.com/scc-csc/scc-c sc/en/item/695/index.do.

da, with no records kept on successful prosecutions.39 Police forces have noted that they face major resource constraints investigating and prosecuting these crimes, which confront them with significant technical obstacles involving encryption and the generalized use of virtual private networks.40

There are two other areas of Canadian criminal law relating to online hate speech. The *Protecting Canadians from Online Crime Act* of 2014 added a new offence prohibiting non-consensual distribution of intimate images including (s. 162.1) as well as complementary amendments to authorize the removal of such images, including child pornography, from the Internet (s. 164.1(5)) and the restriction of the use of a computer or the Internet by a convicted offender (s. 162.2).41 The *Anti-Terrorism Act of 2015* amended the *Criminal Code* to add a prohibition against counselling another person to commit a terrorism offence (s. 83.221.).42 Furthermore, a judge may order that "terrorist propaganda" available to the public be deleted from a computer system (s. 83.223).

#### *The Canada-U.S.-Mexico Trade Agreement d.*

Article 19.17 of the *Canada-U.S.-Mexico Trade Agreement* (in force 2020) all but extends Section 230 of the *U.S. Communications Decency Act* to Canada by providing:43

39 House of Commons, Standing Committee on Justice and Human Rights, *Taking Action to End Online Hate*, June, 2019, 21, https://www.ourcommons.ca/Content/ Committee/421/JUST/Reports/RP10581008/justrp29/justrp29-e.pdf. A review of the caselaw on www.canlii.org reveals 5 successful reported prosecutions over the last 5 years.

40 See Canadian Commission of Democratic Expression, *Harms Reduction: A Six-Step Program to Protect Democratic Expression Online*, Public Policy Forum, January, 2021, 21, https://ppforum.ca/wp-content/uploads/2021/01/CanadianCommission OnDemocraticExpression-PPF-JAN2021-EN.pdf.

41 *Protecting Canadians from Online Crime Act*, *Statutes of Canada*, 2014, c. 31, https:// laws-lois.justice.gc.ca/eng/annualstatutes/2014\_31/FullText.html.

42 *Anti-Terrorism Act of 2015*, *Statutes of Canada*, 2015, c. 20, https://laws-lois.justice.g c.ca/eng/annualstatutes/2015\_20/page-3.html#h-20.

43 *Canada-U.S.-Mexico Trade Agreement*, Article 19.17.1, https://www.international.gc .ca/trade-commerce/trade-agreements-accords-commerciaux/agr-acc/cusma-aceum /text-texte/19.aspx?lang=eng. By contrast, *U.S. Communications Decency Act*, *U.S. Code* 47 (2018) § 230(c) provides: "No provider or user of an interactive computer service shall be treated as the publisher or speaker of any information provided by another information content provider."

no Party shall adopt or maintain measures that treat a supplier or user of an interactive computer service as an information content provider in determining liability for harms related to information stored, processed, transmitted, distributed, or made available by the service, except to the extent the supplier or user has, in whole or in part, created, or developed the information.

This entails that Canada cannot treat online platforms as providing user-generated content and treat them as liable for harms caused by that content.44

There is some question as to how much this provision will constrain any new Canadian online hate speech regime. Krishnamurthy and Fjeld have argued that by contrast with Section 230, which bars all causes of action against a platform that treat it as "publisher or speaker" of hosted information, Article 19.17 only excludes it from being held "liable," thus according to them opening the possibility of "equitable" remedies including restraining orders and injunctions.45 Notice and takedown would thus be possible.

It should be added that a regulatory framework applying to online platforms, including, for example a duty to act responsibly in overseeing its own community standards, which might be accompanied by enforcement powers including fines, should not in principle run afoul of Article 19.17 as long as it does not impose liability on platforms (i) in the same way as it does to content providers and (ii) for steps taken by platforms themselves to control "harmful or objectional" content.

The former point suggests that the CRTC should be cautious before using any new powers acquired pursuant to Bill C-10 simply to extend to online platforms the regime applicable to broadcasting licensees under the *Broadcasting Distribution Regulations*. 46 Even were it to do so, however, it is arguable that those regulations give rise to equitable remedies rather than to a liability regime.

44 Article 19.17.4 makes clear that the Article does not apply to the protection of intellectual property rights or to the enforcement of criminal law.

45 For a detailed discussion, see Vivek Krishnamurthy and Jessica Fjeld, "CDA 230 Goes North American? Examining the Impacts of the USMCA's Intermediary Liability Provisions in Canada and the United States," *SSRN*, July 7, 2020, https:// ssrn.com/abstract=3645462.

46 See *supra* notes 17 to 20 and accompanying discussion.

This latter point is underscored by Article 19.17.3:

No Party shall impose liability on a supplier or user of an interactive computer service on account of:

(a) any action voluntarily taken in good faith by the supplier or user to restrict access to or availability of material that is accessible or available through its supply or use of the interactive computer services and that the supplier or user considers to be harmful or objectionable; or

(b) any action taken to enable or make available the technical means that enable an information content provider or other persons to restrict access to material that it considers to be harmful or objectionable.

This provision emphasizes immunity from liability for platforms on what might be called a "Good Samaritan" basis: where platforms make good faith efforts to control harmful or objectional material, they should not be held liable for those actions. However, if for example a regulatory requirement is imposed on platforms to take steps to restrict access to harmful or objectional material, and the platform fails to comply, it could not invoke Article 19.17.3 as a shield. It would no longer be operating in the realm of voluntary corporate social responsibility: it would be subject to legal constraints.

#### *Reports on directions for law reform II.*

 See also Daniel Kilvington, "The virtual stages of hate: Using Goffman's work to conceptualise the motivations for online hate," *Media, Culture & Society* 43, no. 2 (2020): 256-272, https://journals.sagepub.com/doi/10.1177/016<PHONE_NUMBER>318.

lems."48 Furthermore, "the majority of Canadians—at least 60 percent believe that the federal government has an obligation to put forward regulation to prevent the spread of hateful and racist rhetoric and behaviour online," and "nearly 80 percent of Canadians said they would support regulation that would require social media companies to remove hateful or racist content from their platforms within 24 hours of it being posted."

Not surprisingly, therefore, there have been a number of recent prominent reports and consultation papers paving the way for legislative reform. Six of them are singled out here for review: a) *Taking Action to End Online Hate*, the 2019 Report of the House of Commons Standing Committee on Justice and Human Rights; b) C*anada's communications future: Time to act*, the 2020 Report of the Broadcasting and Telecommunications Legislative Review Panel; c) *Defamation Law in the Internet Age*, the 2020 Report of the Law Commission of Ontario; d) *Recommendations to Strengthen Canada's Response to New Digital Technologies and Reduce the Harm Caused by their Misuse*, the 2021 Report of the Canadian Citizens' Assembly on Democratic Expression, and e) the *Harms Reduction: A Six-Step Program to Protect Democratic Expression*, the 2021 companion Report of the Canadian Commission on Democratic Expression.49

49 Two additional relevant reports are not treated in detail here. The wide-ranging 2018 Report of the House of Commons Standing Committee on Access to Information, Privacy and Ethics entitled *Democracy under Threat: Risks and Solutions in the Era of Disinformation and Data Monopoly*, Our Commons https://www.ourcom mons.ca/Content/Committee/421/ETHI/Reports/RP10242267/ethirp17/ethirp17-e .pdf touched upon the issue of notice and takedown and recommended (at 3) that "to remove manifestly illegal content in a timely fashion, including hate speech, harassment and disinformation, or risk monetary

sanctions commensurate with the dominance and significance of the social platform, and allowing for judicial oversight of takedown decisions and a right of appeal." See also Jacob Davey, Mackenzie Hart, Cécile Guerin, ed. Jonathan Birdwell, *Interim Report: An Online Environmental Scan of Right-wing Extremism in Canada, The Institute for Strategic Dialogue*, June 19, 2020, https://www.isdglobal.o rg/wp-content/uploads/2020/06/An-Online-Environmental-Scan-of-Right-wing-Ex tremism-in-Canada-ISD.pdf. The Report "identified 6,660 right-wing extremists channels, pages, groups and accounts across 7 social media platforms" operating in Canada (at 5).

48 Canadian Race Relations Foundation, "Poll demonstrates support for strong social media regulations to prevent online hate and racism," January 25, 2021, https://www.crrf-fcrr.ca/en/news-a-events/media-releases/item/27349-poll-demons trates-support-for-strong-social-media-regulations-to-prevent-online-hate-and-racis m.

#### *Taking Action to End Online Hate a.*

The Standing Committee on Justice and Human Rights ("Committee"), which was composed of six voting members of the Liberal government, three from the Conservative opposition and one from the NDP opposition, chose to frame its Report with a pointed quotation from the reasons of Rothstein J. in the Supreme Court of Canada's *Walcott* decision:50

Hate speech is not only used to justify restrictions or attacks on the rights of protected groups on prohibited grounds … hate propaganda opposes the targeted group's ability to find self-fulfillment by articulating their thoughts and ideas. It impacts on that group's ability to respond to the substantive ideas under debate, thereby placing a serious barrier to their full participation in our democracy. Indeed, a particularly insidious aspect of hate speech is that it acts to cut off any path of reply by the group under attack. It does this not only by attempting to marginalize the group so that their reply will be ignored: it also forces the group to argue for their basic humanity or social standing, as a precondition to participating in the deliberative aspects of our democracy.

A concern with the rise in hate crimes reported by the police as well as with the connection between online hate and acts of violence led the Committee to initiate a study in March of 2019. The Committee, chaired by Anthony Housefather of the Liberal Party, heard from forty groups and organizations, including Facebook, Twitter and Google, as well as nine individuals. The result was a report culminating in nine recommendations. The Conservative Party members of the Committee dissented from the Report51 and the New Democratic Party ("NDP") issued a supplementary report essentially endorsing the general direction taken but proposing some further detail.52 Five of these recommendations concerned ways to improve existing mechanisms for combatting online hate, placing emphasis upon improved funding for training of police, crown attorneys and judges, better collection of data on hate crimes including via a national database on hate crimes and hate incidents, facilitation of reporting and

50 See *Saskatchewan (Human Rights Commission) v. Whatcott*, [2013] 1 SCR 467 at 507, https://scc-csc.lexum.com/scc-csc/scc-csc/en/item/12876/index.do?q=whatcott. Cited by House of Commons, *Taking Action to End Online Hate*, 5.

51 House of Commons, *Taking Action to End Online Hate*, 55-6.

52 House of Commons, *Taking Action to End Online Hate*, 57-61.

public education. Without recommending specific language, the Committee urged:53

That the Government of Canada formulate a definition of what constitutes 'hate' or 'hatred' that is consistent with Supreme Court of Canada jurisprudence. It is critical that this definition acknowledges persons who are disproportionately targeted by hate speech including but not limited to racial, Indigenous, ethnic, linguistic, sexual orientation, gender identity, and religious groups.

The Committee also recommended that a civil remedy be established, perhaps by reinstating s. 13 of the *Canadian Human Rights Act* or some analogous measure.54 The Committee favoured the implementation of a timely notice and takedown regime, with platforms required to "make it simple for users to flag problematic content," but did not specify applicable standards.55 The NDP for its part favoured a "manifestly illegal" standard like that adopted in Germany's NetzDG, as well as "monetary sanctions commensurate with the dominance and significance of the social platform, and allowing for judicial oversight of takedown decisions and a right of appeal."56 The Report placed considerable emphasis upon transparency, recommending common standards for platform reporting mechanisms and a duty to report regularly to users concerning incidents reported, actions taken, and the speed of response, which significant monetary penalties for failure to report.57 Finally, the Report signalled support for an effort to enhance the authentication of online content by recommending:58

That online platforms be encouraged to provide optional mechanisms to authenticate

contributors and digitally sign content, and couple this with visual indicators signifying

that given user or content is authenticated, and provide users options for filtering nonsigned or non-authenticated content.

53 House of Commons, *Taking Action to End Online Hate*, 41.

54 House of Commons, *Taking Action to End Online Hate*, 41.

55 House of Commons, *Taking Action to End Online Hate*, 42.

56 House of Commons, *Taking Action to End Online Hate*, 61. Note that this recommendation tracked the language House of Commons, *Democracy under Threat Report*, 349.

57 House of Commons, *Taking Action to End Online Hate*, 42.

58 House of Commons, *Taking Action to End Online Hate*, 42.

### *Richard Janda*

It is worth noting, finally, as the Report itself underscored, that in its presentation to the Committee, Facebook supported "the establishment of clear baseline standards applicable to all platforms would help to counter online hate" since "people use many different online platforms to communicate".59

#### *Canada's communications future: Time to act b.*

Although the Report of the Broadcasting and Telecommunications Legislative Review Panel was mainly of significance for the preparation of Bill C-10,60 the Report did address the fact that "[o]nline platforms have … created forums that enable the dissemination of harmful content, fake news and disinformation, and violent and extremist content."61 One of the core recommendations, taken up in Bill C-10, was to ensure that the CRTC can impose codes of conduct "regarding all media content undertakings" – including online platforms.62 The Report also formulated a specific recommendation about liability for harmful content:63

We recommend that the federal government introduce legislation with respect to liability of digital providers for harmful content and conduct using digital technologies, separate and apart from any responsibilities that may be imposed by communication legislation. Given that the challenges in this area are global in nature, we also encourage the federal government to continue to participate actively in international fora and activities to develop international cooperative regulatory practices on harmful content.

59 House of Commons, *Taking Action to End Online Hate*, 27.

60 Bill C-10.

61 Broadcasting and Telecommunications Legislative Review Panel, Final Report, *Canada's communications future: Time to act*, January, 2020, https://www.ic.gc.ca /eic/site/110.nsf/vwapj/BTLR\_Eng-V3.pdf/\$file/BTLR\_Eng-V3.pdf. The Panel, appointed by the federal government, was chaired by Janet Yale, former CEO of the Canadian Cable television Association and a former Director General of the CRTC. Other panelists included lawyers Peter Grant, Hank Intven, and Monica Song, academics Marina Pavlović and Pierre Trudel, and Monique Simard, who had been CEO of the Société de développement des entreprises culturelles.

62 Broadcasting and Telecommunications Legislative Review Panel, 10 and 34.

63 Broadcasting and Telecommunications Legislative Review Panel, 37 as well as discussion at 190-193.

Finally, it made a parallel recommendation with respect to illegal content and conduct:64

We recommend that the federal government regularly review the efficiency of enforcement mechanisms for monitoring and removing illegal content and conduct found online. Given the diverse range of governing frameworks for these matters in Canada, we encourage the federal government to coordinate with provincial and territorial governments.

The Report has thus set the stage for a greater role for the CRTC in applying codes of conduct to online platforms but has at the same time envisaged a separate "liability" regime for online platforms (Article 19.17 of Canada-U.S.-Mexico Trade Agreement was not referenced) and placed considerable emphasis upon inter-governmental coordination.

#### *Defamation Law in the Internet Age c.*

The Law Commission of Ontario spent four years studying "how best to reform defamation law in response to the social and technological revolution in written communications brought about by the internet."65 There is, of course, some significant overlap between defamatory speech and hate speech. The Commission made clear that it had "explored the role of defamation law in relation to an array of legal tools for regulating online speech in the 21st century" including "myriad laws directed at particular types of harmful speech, such as child pornography and hate speech."66 Nonetheless, the Commission excluded "direct examination of these related areas of law."67

Taken as a whole, the Report contains three chapters with significant implications for the regulation of online hate speech and the protection

64 Broadcasting and Telecommunications Legislative Review Panel, 193.

65 Law Commission of Ontario, *Defamation Law in the Internet Age*, March 2020 at 1, https://www.lco-cdo.org/wp-content/uploads/2020/03/Defamation-Final-Repo rt-Eng-FINAL-1.pdf. The Law Commission of Ontario was originally created by the Ontario Ministry of the Attorney General, the Law Foundation of Ontario, the Law Society of Ontario, Osgoode Hall Law School and the Law Deans of Ontario and is now funded by the Law Foundation of Ontario, the Law Society of Ontario, Osgoode Hall Law School, and York University: "Learn about us," https://www.lco-cdo.org/en/learn-about-us/.

66 Law Commission of Ontario, *Defamation Law in the Internet Age*, 15.

67 Law Commission of Ontario, *Defamation Law in the Internet Age*, 15.

of democratic expression: chapters on new legal responsibilities for intermediary platforms, notice and takedown, and online dispute resolution. As to legal responsibilities, the Commission came to the conclusion that the existing common law framework imposing liability for defamation upon publishers is "clumsy and overinclusive when applied to the unique functioning of the internet."68 The Commission produced an interesting table summarizing the reasons why platforms should not be liable for defamation:69

Arguably each of these rationales applies as well to other forms of harmful content including hate speech. It is striking that the Commission was unimpressed by existing and evolving quasi-judicial processes established by platforms such as the Facebook Oversight Board.70 In the end, it rec-

68 Law Commission of Ontario, *Defamation Law in the Internet Age*, 74.

69 Law Commission of Ontario, *Defamation Law in the Internet Age*, 77.

70 See Facebook Oversight Board, https://oversightboard.com/. The Commission observed that "Although Facebook's Oversight Board contemplates some adjudicative elements, it is probably not a promising model of [online dispute resolution] ODR in the absence of a more direct focus on the interests of the parties. Furthermore, it does not contemplate any supervisory role for government." Law Commission of Ontario, *Defamation Law in the Internet Age*, 103.

ommended that online platforms should be excluded from the category "publisher" by defining that term to "to require an intentional act of communicating a specific expression."71

### *The Commission also recommended that there be "a takedown obligation on intermediary*

platforms hosting third party content available to users in Ontario."72 Notice of complaint would be forwarded by the platform to the publisher of the allegedly defamatory material. The publisher would then have two days to respond in writing. Where a response was received by the platform within the prescribed period, that response would be forwarded to the complainant and no further action would be taken, since the platform would be given no role in assessing the merits of the complaint. On the other hand, if no response was forthcoming, the platform would be required to take down the specific language that is alleged to be defamatory. Notice of takedown would be provided to the publisher, who could require put-back if "there is evidence that the publisher failed to receive the notice or unintentionally missed the deadline and where it is technologically reasonable to do so."73 Regulations would specify an administrative fee that platforms could charge to complainants. Failure by the platform to comply with takedown requirements would entitle the complainant to statutory damages. The Ontario Superior Court of Justice, the court of general jurisdiction, would enforce the notice and takedown regime.

Interestingly, the Commission specified that the regime would only apply to platforms hosting content available in Ontario and recommended excluding search engines from its ambit. Since the publication of the Report, Google has come under considerable pressure to intervene to prevent websites from running a successful business involving the publication of defamatory material that appears high in Google searches, in turn allowing these websites to charge thousands of dollars to victims to take the posts down.74 Google has announced that it will change its search algorithm to prevent predatory websites from appearing in the list of results when

71 Law Commission of Ontario, *Defamation Law in the Internet Age*, 80 and 109.

72 Law Commission of Ontario, *Defamation Law in the Internet Age*, 96 and 109.

73 Law Commission of Ontario, *Defamation Law in the Internet Age*, 96 and 109.

74 See Adam Krolik and Kashmir Gill, "The Slander Industry," *New York Times*, April 24, 2021, https://www.nytimes.com/interactive/2021/04/24/technology/onli ne-slander-websites.html.

someone searches for a person's name, and that it has created a "known victims" service for those who report having been attacked by sites that charge for the removal of posts.75 After a report to the "known victims" service, Google will suppress similar content when someone searches for a victim's name. This represents an important departure for Google, since it had heretofore taken the position, parallel to that of the Commission, that "[w]e never touch search, no way, nohow."76

The Commission was aware that its recommendations on notice and takedown arose in a context where there was increasing pressure on the federal government to enact a takedown regime for "manifestly illegal content," which could include defamation.77 While the Commission eschewed taking any position on the merits of proposals such as those contained in the *Taking Action to End Online Hate* Report, it did signal the relevance to the debate of proposals to create a statutory duty of care, underscoring in particular the importance of the 2019 UK *White Paper on Online Harms*. 78

Finally, while the Commission expressed deep skepticism about the development of online dispute resolution by platforms themselves, characterizing them as having "few of the hallmarks of procedural fairness and none of the authoritativeness of a judicial decision,"79 it did consider a "coregulatory approach"80 as well as the possibility of creating social media councils to be a "multi-stakeholder accountability mechanism for platform

/eur-lex.europa.eu/eli/dir/2019/790/oj.

75 See Kashmir Hill and Daisuke Wakabayashi, "Google Seeks to Break Vicious Cycle of Online Slander," *New York Times*, June 10, 2021, https://www.nytimes.co m/2021/06/10/technology/google-algorithm-known-victims.html.

76 Kashmir Hill and Daisuke Wakabayashi.

77 Law Commission of Ontario, *Defamation Law in the Internet Age*, 95.

78 Law Commission of Ontario, *Defamation Law in the Internet Age*, 95*.* See United Kingdom, *White Paper on Online Harms*, April 6, 2019, https://assets.publishing.ser vice.gov.uk/government/uploads/system/uploads/attachment\_data/file/793360/On line\_Harms\_White\_Paper.pdf.

79 Law Commission of Ontario, *Defamation Law in the Internet Age*, 102.

80 Law Commission of Ontario, *Defamation Law in the Internet Age*, 103. The Commission referenced the "right to be forgotten" regime of the European Union, Article 29 Data Protection Working Party, "Guidelines on the Implementation of the Court of Justice of the European Union Judgment on '*Google Spain and Inc v Agencia Española de Protección de Datos* (*AEPD) and Mario Costeja González*'," November 26, 2014, C-131/12, https://ec.europa.eu/newsroom/ article29/document.cfm?action=display&doc\_id=64437. It also referenced the EU *Directive on Copyright in the Digital Single Market*, April 17, 2019), 2019/790, https:/

content decisions."81 In the end the Commission took a favourable view of social media councils because:82

Nonetheless, because it judged that the subject of social media councils went beyond the scope of its mandate, Commission chose to make a recommendation only calling for the future exploration by the Ontario government of online dispute resolution, including by means of social media councils or other regulatory models.83

#### *Report of the Citizens' Assembly on Democratic Expression d.*

In 2020, the Public Policy Forum with Funding from the McConnell Foundation and the Government of Canada launched an ambitious threeyear initiative to study how to strengthen Canadian democracy in response to the ubiquitous presence of online technologies. In its first year, the goal was to develop a plan on how to mitigate the negative effects on Canadian democracy of online hate, disinformation and other forms of harmful content while encouraging the broadest possible application of the freedom of expression in Canada's *Charter of Rights and Freedoms*. The first year of the initiative involved two parallel and innovative processes: creating a blue-ribbon Canadian Commission on Democratic Expression, the Report of which is discussed in the next section of this article; and convening a Canadian Citizen's Assembly on Democratic Expression, made up of

81 Law Commission of Ontario, *Defamation Law in the Internet Age*, 103. The Commission referenced in particular the work of Article 19, *The Social Media Councils: Consultation Paper*, June, 2019, 7, https://www.article19.org/wp-content/uploads/2 019/06/A19-SMC-Consultation-paper-2019-v05.pdf.

82 Law Commission of Ontario, *Defamation Law in the Internet Age*, 104.

83 Law Commission of Ontario, *Defamation Law in the Internet Age*, 104 and 110.

representative body of 42 Canadians, which authored its own Report that came out almost contemporaneously with that of the Commission.84

Over 12,000 Canadians were invited to serve in the Assembly, nearly 400 volunteered and in the end 42 were selected at random to "represent the widest possible range of voices and perspectives."85 The Assembly was convened in March 2020 and its members met eighteen times, hearing from over a dozen experts and from senior representative of Google and Facebook.86

The Report of the Assembly contains notably a set of guiding values,87 a set of key concerns,88 and 33 recommendations grouped largely around the key concerns.

The recommendations were far-reaching and served to demonstrate perhaps that informed non-experts can generate fresh proposals that at the very least provided a valuable stress test for the proposals developed in parallel by the Canadian Commission on Democratic Expression.89 Key among them were creating a new digital platforms regulator, becoming more savvy at international cooperation, establishing user ownership of personal data, introducing user-friendly standardized descriptions of terms of service across platforms, and making anonymous users accountable for their actions.90

84 Canadian Citizens' Assembly on Democratic Expression, *Recommendations to Strengthen Canada's Response to New Digital Technologies and Reduce the Harm Caused by their Misuse*, Public Policy Forum, January, 2021, https://ppforum.ca/wp -content/uploads/2021/01/CanadianCitizens%E2%80%99AssemblyOnDemocratic Expression-PPF-JAN2021-EN.pdf.

85 Canadian Citizens' Assembly on Democratic Expression, 5. An overview of the representativeness of the Assembly is provided at 9 and the assembly process is described in detail at 22-28.

86 Canadian Citizens' Assembly on Democratic Expression, 7.

87 Canadian Citizens' Assembly on Democratic Expression, 32-33. The Assembly sought an Internet that is 1) accessible; 2) accountable; 3) reliable; 4) safe and secure; and 5) amplifies diverse voices.

88 Canadian Citizens' Assembly on Democratic Expression, 34-37. The Assembly focussed on 1) lack of oversight, transparency and accountability of digital platforms; 2) the spread of misinformation; 3) protecting digital rights and user control; and 4) harms to vulnerable persons and minority groups.

89 The Assembly presented its recommendations to the Commission in advance of the Commission issuing its Report. See Canadian Citizens' Assembly on Democratic Expression, at 28.

90 Canadian Citizens' Assembly on Democratic Expression, 37-42.

As regards the digital platforms regulator, the Assembly wanted it to:

As regards international cooperation, the Assembly urged strategic cooperation with democratic countries to establish common practices, adoption by Canada of certain existing frameworks, such as the EU GDPR, and the enhancement of collaborative competition law enforcement.

As regards user ownership of personal data, the Assembly challenged platforms *inter alia*:

It also sought to enshrine user ownership of data in law, policies and regulations.

As regards standardized interface and descriptions of terms of service across platforms, the Assembly sought that these include clear descriptions of i) user rights, ii) information being collected and iii) how it is used and stored, as well as iv) data controls and permissions.

Finally, as regards anonymous users, the Assembly affirmed that anonymity is not a right. The Assembly sought the development of policies, laws and regulations to ensure that anonymity cannot be used to shield individuals from the consequences of producing harmful, hateful, or defamatory speech.

#### *Canadian Commission on Democratic Expression e.*

The seven distinguished members including of the Canadian Commission on Democratic Expression included Beverley McLachlin, former Chief Justice of the Supreme Court of Canada.91 The Commission at a number of points drew explicitly upon the work of the Assembly and indeed met twice with the members of the Assembly during the course of its own deliberations.92

The Commission considered and, in the end, recommended against following either a hands-off self-regulation approach or the German NetzDG notice and takedown approach. It opted instead for a regulatory regime that would include six interconnected elements.

The **first element** was a new legal duty placed upon platforms to act responsibly. The virtue of this was said to be that it provides a "regulatory focus on systemic issues" rather than having regulatory intervention into thousands of content disputes.93 The idea was drawn in significant degree from the UK *White Paper on Online Harms*, which has proposed imposing a statutory duty of care on platforms.94 The idea was to impose on companies the onus to fulfil this legal duty, with the regulator positioned to set out how to do this in codes of practice.

However, the UK White Paper also recommended that "If companies want to fulfil these duties in a manner not set out in the codes, they will have to explain and justify to the regulator how their alternative approach will effectively deliver the same or greater level of impact."95 It was not

91 The other members were Rick Anderson, Principal, Earnscliffe Strategy, Julie Caron-Malenfant, Director General, Institut du Nouveau Monde, Adam Dodek, Dean, Faculty of Law (Common Law Section), University of Ottawa, Amira Elghawaby, Journalist and Human Rights Advocate, Jameel Jaffer, Executive Director, Knight First Amendment Institute at Columbia University, and Jean La Rose, Former CEO, Aboriginal Peoples Television Network. See Canadian Commission of Democratic Expression, 46-47.

92 Canadian Commission of Democratic Expression, 51.

93 Canadian Commission of Democratic Expression, 31-2.

94 See United Kingdom, *White Paper on Online Harms*, 7. The Commission noted that "[t]he United Kingdom, within its own legal traditions, is currently advancing a similar type of duty of care for online platforms": Canadian Commission of Democratic Expression, 31. Since Québec is not a common law jurisdiction and does not include duty of care analysis as part of the law of extra-contractual obligations, it is understandable that the Commission adopted the idea of a "duty to act responsibly," which arguably can be applied within both common law and civil law contexts.

95 See United Kingdom, *White Paper on Online Harms*, 7.

made clear in Commission's Report whether platforms would be allowed to depart from codes of conduct.

It should be noted that the proposed new statutory duty gave rise to a partially dissenting minority report. Commission member Jameel Jaffer wrote:96

I find it difficult to endorse the proposed Duty to Act Responsibly when the content of that duty is left almost entirely to Parliament and the new regulator to decide. Defining the duty will require difficult tradeoffs, not only between free speech and other values—for example, privacy, equality, and due process—but also between different conceptions of free speech.

This point is particularly striking in light of Heritage Minister Steven Guilbeault's perhaps incautious public statement that "hurtful" speech could be included within the scope of what is to be regulated in an eventual bill.97

The **second element** in the Commission's proposed regime was a new regulator to oversee and enforce the duty to act responsibly. The goal of creating such a regulator would be to move content moderation and platform governance beyond the exclusive preserve of the platforms. "The regulator would oversee a code of conduct to guide the actions of parties under its supervision, while recognizing that not all platforms can be treated in the same manner."98 The Commission also sought to ensure that the regulator would be able to impose significant fines and even pursue imprisonment for platform executives.99

The **third element** was a social media council to serve as an accessible forum in reducing harms and improving democratic expression on the internet. The social media council would be conceived as "an independent, stakeholder-based body with dedicated professional support that is attached to the regulator."100 It would serve as a consultative body for the regulator on codes of conduct and on how changing technology, business models and user experience affect policy.

96 Canadian Commission of Democratic Expression, 48.

97 Michael Geist, "The real consequences of Steven Guilbeault's battle with the web giants," *Maclean's*, May 3, 2021, https://www.macleans.ca/opinion/the-real-c onsequences-of-steven-guilbeaults-battle-with-the-web-giants/.

98 Canadian Commission of Democratic Expression, 9.

99 Canadian Commission of Democratic Expression, 33.

100 Canadian Commission of Democratic Expression, 33.

The idea of a social media council had been raised by the Law Commission of Ontario101 and could perhaps interact with online dispute resolution already being established by the platforms themselves and in some cases across platforms (such as the Global Internet Forum to Counteract Terrorism).102 A difficult question concerns how the social fractures that are evident on social platforms would be represented on the social media council. Surely it would damage the credibility of such a council to have its membership swing radically according to political winds.

The **fourth element** was a "world-leading transparency regime" to provide the flow of necessary information to the regulator and social media council.103 The Commission envisaged i) periodic public risk assessment reports from the platforms, ii) power granted to the regulator to compel access to information, notably to the black box of platform algorithms, iii) disclosure rules on data sharing, iv) rules on advertising transparency, v) public labelling and registration of bots, and vi) disclosure of the ownership structure behind those disseminating user-generated and other third-party content.104

The **fifth element** was an e-tribunal to facilitate and expedite dispute resolution and a process for addressing complaints swiftly and lightly before they become disputes. The key idea would be to: 105

allow for the resolution of Canadian disputes within Canada. Currently, with content moderation under the control of platform companies, the training and domicile of the content moderators is a black box. Meanwhile, Facebook's new Oversight Board hears only a handful of global cases and has no Canadian member.

This was another point on which Jameel Jaffer dissented, writing:106

I am not persuaded, though, that establishing a new tribunal system with a broad mandate would be preferable to requiring large platforms themselves to establish, at their own expense, review and appeals processes that are more efficient and transparent than the ones some of them have already established. Before endorsing the proposed e-tribunals, I would want to know more about their mandate, and also

101 See *supra* notes 81 to 83 and accompanying discussion.

102 See Global Internet Forum to Counteract Terrorism, https://gifct.org/.

103 Canadian Commission of Democratic Expression, 9.

104 Canadian Commission of Democratic Expression, 35-6.

105 Canadian Commission of Democratic Expression, 38.

106 Canadian Commission of Democratic Expression, 48.

about what relationship the proposed tribunals would have to the processes that some of the platforms have already established.

The **sixth element** was a mechanism for quick removal of content that presents an imminent threat to a person. This would constitute an exception to the general avoidance of a notice and takedown mechanism. It would involve a "quick-response system" within 24 hours under the authority of the regulator to ensure the rapid removal– even temporarily – of content that creates a reasonable apprehension of an imminent threat to the safety of the targeted party. The Commission insisted that such decisions should be subject to judicial sanction before either the e-tribunal or the courts.107

In addition to these six main recommendations, Commission also flagged a number of further issues including legal liability and fines, law enforcement resources, the interaction of the proposed regime with the Canada-US Mexico Agreement, and the need for periodic review of any legislation eventually adopted.

#### *Bill C-36 III.*

Although Bill C-36 was not adopted by Parliament before the 2021 election, the fact that the government revealed the proposed language for amendments to the *Criminal Code* and to the *Canadian Human Rights Act* merits discussion. Should the government be re-elected, there is a strong likelihood that the bill will be reintroduced.

The *Taking Action to End Online Hate* Report had called on the government to define hate or hatred so as to acknowledge "persons who are disproportionately targeted by hate speech including but not limited to racial, Indigenous, ethnic, linguistic, sexual orientation, gender identity, and religious groups."108 This indeed is the approach adopted as far as defining "hate speech" for the purposes of the *Canadian Human Rights Act* is concerned. The bill provides that "*hate speech* means the content of a

107 Canadian Commission of Democratic Expression, 39.

108 House of Commons, *Taking Action to End Online Hate*, 41.

communication that expresses detestation or vilification of an individual or group of individuals on the basis of a prohibited ground of discrimination."109 It also provides that "communication does not express detestation or vilification… solely because it expresses mere dislike or disdain or it discredits, humiliates, hurts or offends."

The bill takes a somewhat different though parallel approach for the purposes of the *Criminal Code*, defining "hatred" as "the emotion that involves detestation or vilification and that is stronger than dislike or disdain" and specifying that "the communication of a statement does not incite or promote hatred … solely because it discredits, humiliates, hurts or offends."110 Since the *Criminal Code* already includes sanctions against inciting and promoting hatred "against any identifiable group," the Minister presumably concluded that the only purpose of a *Criminal Code* definition was to follow the guidance of the Supreme Court of Canada and orient the courts as to the intensity of the emotion communicated through hate propaganda.111

#### *Peace bond b.*

Bill C-36 introduces a new peace bond to help forestall hate crimes.112 Someone who reasonably fears that they could be a target of hate propaganda or criminal mischief could apply for a peace bond to be imposed on an individual to deter that person from committing the crime. Such a peace bond could involve the imposition of conditions including wearing an electronic monitoring device, a curfew, prohibition against consuming drugs or alcohol together with a requirement to provide samples for testing, a prohibition against communicating with any person, and a prohibition against possession of firearms. Surprisingly enough, no specific mention is made of prohibiting visits to or participation in online fora known to convey hate propaganda.

A breach of the proposed peace bond would carry a maximum penalty of four years' imprisonment, the same penalty that exists for breaches of other peace bonds. Consent by the appropriate Attorney General would

109 Bill C-36, s. 13.

110 Bill C-36, s. 2. This in effect codifies the approach taken by the Supreme Court of Canada in *Saskatchewan (Human Rights Commission) v. Whatcott* (para. 41).

111 *Criminal Code*, s 319.

112 Bill C-36, s. 3, adding a new s. 810.012 to the *Criminal Code.*

be required before the peace bond could be used, as is the case with some existing peace bonds.

#### *Canadian Human Rights Act c.*

Rather than simply restoring the former s. 13 of the *Canadian Human Rights Act*, Bill C-36 includes a revised version of it.113 In addition to the new definition of "hate speech" already signalled, under the proposed legislation the Canadian Human Rights Tribunal would gain new powers 1) to order the party complained against to cease the hate speech and provide redress in consultation with the Canadian Human Rights Commission, 2) to pay damages of up to \$20,000 to each victim personally identified in the communication, and 3) to pay a fine of up to \$50,000.114 The bill excludes "private communication" such as private emails or direct messages from the scope of hate speech, and does not apply to online communication service providers. Indeed, in the materials accompanying the release of the bill, the Department of Justice underscored that online platforms "are the focus of upcoming engagement by Canadian Heritage, which will outline a proposed approach to regulating social media and harmful content, including hate speech, online."115

The bill would empower the Commission to prevent the disclosure of the identity of the complainant to the person against whom the complaint is filed,116 and gives further scope to the Tribunal to conduct confidential inquiries where there is a real and substantial risk that a complainant or witness "will be subjected to threats, intimidation or discrimination."117 Violation of such confidentiality orders made by the Commission or Tribunal would be subject to a fine of up to \$50,000.118

Since it had been objected that the former s. 13 caused the Commission to flood the Tribunal with an unmanageable caseload, the bill gives stricter

113 Bill C-36, s. 13.

114 Bill C-36, s. 19.

 14.

117 Bill C-36, s. 17.

118 Bill C-36, s. 20. Unlike other penal offences envisaged under the *Canadian Human Rights Act*, prosecution for these offences would not require prior approval of the Attorney-General of Canada, implying that prosecution should be undertaken as a matter of course.

### *Richard Janda*

guidance to the Commission not to pursue cases where "the complaint indicates no *hate speech*."119 The bill creates an additional caseload management tool through the possibility of awarding costs for abuse of process.120 It would also expand the Tribunal so as to address the anticipated increase in workload, adding two to five new members for a maximum of seventeen and eventually twenty members.121

#### *Ideas not retained d.*

Two ideas raised in the Minister of Justice's *Consultation Paper* were not retained.122 Individuals will not be empowered to pursue complaints themselves before the Tribunal and thus will rely upon the Commission to initiate complaints. Nor does Bill C-36 remove the requirement that the appropriate Attorney-General provide express consent to prosecutions for alleged wilful promotion of hatred pursuant to s. 319(2) of the *Criminal Code*.

#### *The Digital Citizen Initiative Consultation Papers IV.*

The Discussion Guide and Technical Paper issued by the Digital Citizen Initiative of the Ministry of Canadian Heritage on July 26, 2021 have the appearance of internal documents drafted in preparation of legislation. It seems clear that the Minister was in the end reluctant to put forward legislation without a prior public consultation. Thus, the public now has access to the road map offered to the Minister by his civil service together with a narrow set of options presented to him. It remains to be seen whether the public consultation will raise issues and concerns going beyond the scope of the consultation documents, especially given the broader range of options canvassed in the various reports discussed above.

The Discussion Guide is meant to background and justification for the Technical Paper and uses lay language. The Technical Paper has the same structure as the Discussion Paper but uses formulations that could

119 Bill C-36, s. 15.

120 Bill, C-36, s.19.

121 Bill C-36, s. 16.

122 Minister of Justice and Attorney General of Canada, *Consultation Paper: Online Hate*, July 14, 2020. https://ocla.ca/wp-content/uploads/2020/07/2020-07-14-Cons ultation-paper-Online-hate.pdf.

find their way into legislation. This summary focuses primarily on the Technical Paper.

The Technical Paper is divided into two "Modules" that could presumably become two parts of the legislative framework: 1) a new legislative and regulatory framework for social media platforms; and 2) modifications to existing legislation. Not surprisingly, Module 1 is the more elaborate and is itself divided into four parts: A) a general framework of purposes, interpretation and application; B) new rules and obligations; C) new regulators; and D) new regulatory powers and enforcement mechanisms.

Module 1(A) puts forward a set of "premises" about the benefits and potentially harmful impacts of "Online Communication Services" (OCSs).123 An OCS is defined as "a service that is accessible to persons in Canada, the primary purpose of which is to enable users of the service to communicate with other users of the service, over the internet," but would "exclude services that enable persons to engage only in private communications."124 The Discussion Paper explains that the definition "is intended to capture major platforms, (e.g., Facebook, Instagram, Twitter, YouTube, TikTok, Pornhub), and exclude products and services that would not qualify as online communication services, such as fitness applications or travel review websites."125 Nonetheless, it would appear that "private" communication services owned by major platforms, such as Facebook's WhatsApp, and which can be used to spread harmful content, are *prima facie* to be excluded from the application of the legislation. The proposed legislation would give power to the cabinet, upon consultation with the new Digital Safety Commissioner described below, to narrow or extend the definition of OCS by regulation. The legislation would apply to OCSs and to "the closest legal entity to a regulated OCS", called an Online Communication Service Provider (OCSP).126

123 Digital Citizen Initiative, "Technical Paper," Module 1, para. 1.

124 Digital Citizen Initiative, "Technical Paper," Module 1, para. 2.

125 Digital Citizen Initiative, "Discussion Guide," "Who and what would be regulated".

126 Digital Citizen Initiative, "Technical Paper," Module 1, para. 6.

### *Richard Janda*

The legislation would concern "harmful content" falling into five defined categories:127

It would thus not extend to defamatory speech. Nor would it extend to misinformation or other "awful but lawful" content.

Module 1(B) proposes creating a new obligation that "an OCSP must take all reasonable measures, which can include the use of automated systems, to identify harmful content that is communicated on its OCS and that is accessible to persons in Canada."128 It thus comes close to the duty to act responsibly proposed by the Canadian Commission on Democratic Expression. The obligation would extend to abiding by regulations prescribed by the Digital Safety Commissioner and would also require that OCSPs ensure that the measures they take not give rise to differential treatment of any group based on a prohibited ground of discrimination.

Furthermore, "an OCSP must address all content that is flagged by any person in Canada as harmful content" within 24 hours.129 This means

127 Digital Citizen Initiative, "Technical Paper," Module 1, para. 8.

128 Digital Citizen Initiative, "Technical Paper," Module 1, para. 10.

129 Digital Citizen Initiative, "Technical Paper," Module 1, para. 11.

either responding to that person that the content does not meet the definition of harmful content or taking down the content. Thus, in this respect the proposed approach follows the NetzDG model despite the recommendation of the Canadian Commission on Democratic Expression not to do so.

Module 1(B) also includes significant transparency and procedural requirements for OCSPs.130 The flagging mechanism must be "accessible and easy-to-use," as must be the opportunity to make representations and compel prompt review and reconsideration by the OCSP. Upon reconsideration, notice must be given of the recourse to the new Digital Recourse Council of Canada, discussed below. The OCSP must publish "clear content-moderation guidelines," and "must generate and provide reports on a scheduled basis to the Digital Safety Commissioner [also discussed below] on Canada-specific data." The content of these latter reports is to be prescribed in significant detail, including, for example, information from the OCSPs about "how they monetize harmful content". Regulations are envisaged to determine what records OCSPs must keep.

The Technical Paper leaves open two options as to how OCSPs should meet a mandatory notification requirement for law enforcement agencies. The options involve differing thresholds of potential harm. Option (a) would require notification to the Royal Canadian Mounted Police where the OCSP has reasonable grounds to believe that defined "harmful content reflects an imminent risk of serious harm to any person or to property." Option (b) would set the notification requirements by regulation. As the Discussion guide notes: "The legal thresholds (reasonable suspicion, reasonable grounds to believe) for reporting this content… could differ based on the category. For example, the threshold for reporting potentially terrorist and violent extremist content could be lower than that for potentially criminal hate speech."131 Option (b) would include mandatory reporting of potential terrorist activity to the Canadian Security Intelligence Service. At stake is that police forces are seeking to require platforms to inform them when they take down illegal content and to provide the deleted content to the police as evidence for possible further criminal investigation. The RCMP seeks to hold these materials in a database. Other government partners are resisting this approach, although apparently all stakeholders agree with requiring the platforms to keep the content they

130 Digital Citizen Initiative, "Technical Paper," Module 1, paras. 12-15.

131 Digital Citizen Initiative, "Discussion Guide," "Engaging law enforcement and CSIS".

remove for a year. Option (a) is a compromise position involving narrower disclosure.

Module 1(C) proposes the establishment of four new bodies: the Digital Safety Commissioner, the Digital Recourse Council of Canada, an Advisory Board and the Digital Safety Commission. The Commissioner would handle the basic administration of the new legislation, including giving general advice to OCSPs (though not about specific content-moderation decisions),132 establishing an Incident Response Protocol for potential terrorist activity,133 receive complaints from the public about OCSP non-compliance,134 and have the power to issue regulations,135 subject to binding directions from cabinet.136

The Digital Recourse Council of Canada would be designed to provide an independent recourse in response to OCSP decisions and would arise only upon exhaustion of remedies available with the OCSP. It is not entirely clear how this would interact, say, with the Facebook Oversight Board, which does not necessarily provide a timely remedy. It is not made entirely clear whether this body would operate as an eCourt. Compliance orders issued by the Digital Safety Commissioner would be appealed to the Personal Information and Data Protection Tribunal created under the proposed overhaul of data privacy legislation.137

The Commissioner and Council would be counselled by a new, external Advisory Board having the characteristics of the recommended social media council.138 Its members would be drawn from civil society, academia, and cultural groups. Its role would be to inform the Council and the Commissioner, both of which would be appointed by the government but independent from ministerial oversight.

The Commissioner, Council and Advisory Board would all operate supported by an umbrella Digital Safety Commission of Canada, which would have a Chief Executive Officer who is not the Commissioner.139 The entire

139 See Digital Citizen Initiative, "Technical Paper," Module 1, paras. 60-65.

132 Digital Citizen Initiative, "Technical Paper," Module 1, para. 16.

133 Digital Citizen Initiative, "Technical Paper," Module 1, paras. 18-19.

134 Digital Citizen Initiative, "Technical Paper," Module 1, paras. 40-44.

135 Digital Citizen Initiative, "Technical Paper," Module 1, para. 17.

136 Digital Citizen Initiative, "Technical Paper," Module 1, para. 39.

137 See Bill C-11. See also Digital Citizen Initiative, "Technical Paper," Module 1, para. 81.

138 See Digital Citizen Initiative, "Technical Paper," Module 1, paras. 71-75.

apparatus would operate on a cost-recovery basis through charges imposed on the OCSPs.140

Module 1(D) proposes a new set of powers and enforcement remedies, including compliance orders against OCSPs and broad inspection powers for the Commissioner, and Administrative Monetary Penalties for non-compliance issued by the Personal Information and Data Protection Tribunal.141 It would also be an offence for an OCSP to fail to comply with a compliance agreement, adhere to an order issued by the Council or Commissioner, resist or obstruct and inspection, or knowingly make false or misleading statements to the Commissioner or Council.142 For the most serious offences, the maximum penalty would be a fine not exceeding five percent of gross global revenues in the financial year that precedes the date of sentencing or \$25,000,000, whichever is higher. If an OCSP "repeatedly demonstrates persistent non-compliance" with respect to orders for removing content relating to child sexual exploitation or terrorism, the Commissioner could apply to the Federal Court for an order requiring telecommunications service providers to block access to the offending OCS in Canada.143

Module 2 proposes certain amendments to existing Canadian legislation. As regards child pornography, the *Mandatory Reporting Act*144 would be amended so as to extend its application to OCSPs and other internet services, centralize reporting with the National Child Exploitation Crime Centre of the Royal Canadian Mounted Police (RCMP) and generally strengthen its administration.145 An unresolved issue concerning the reporting of clear child pornography offences has to do with transmission data (i.e., Internet protocol address, date, time, type, origin, destination of the material) or basic subscriber information (BSI) (i.e., customer's name, address, phone number, billing information associated with the IP address). One option is simply to require that such information be shared with the police. The other, stricter, option would require the police to seek

140 See Digital Citizen Initiative, "Technical Paper," Module 1, paras. 66-70.

141 See Digital Citizen Initiative, "Technical Paper," Module 1, paras. 81-82, 88-114.

142 See Digital Citizen Initiative, "Technical Paper," Module 1, paras. 119.

143 See Digital Citizen Initiative, "Technical Paper," Module 1, paras. 120.

144 An Act respecting the mandatory reporting of Internet child pornography by persons who provide an Internet service 2011, S.C. c. 4.

145 See Digital Citizen Initiative, "Technical Paper," Module 2, Mandatory Reporting Act, paras. 1-11.

### *Richard Janda*

a production order from the court to obtain BSI.146 A parallel option being considered is to allow the Canadian Security Information Service easier access to BSI in the case of terrorist content, since currently CSIS must seek a warrant, a process that can take four to six months.147

#### *Final critical observations V.*

In my concluding remarks, I would like to signal what I believe are two important conceptual challenges raised by the emerging Canadian approach. The first of these was signalled by Jameel Jaffe in his dissenting report to the Commission.148 How should a national regulatory approach properly interconnect with the emerging online dispute resolution regimes developed by platforms themselves, notably the Facebook Oversight Board? The second challenge was signalled by the Commission when quoting Prof. Taylor Owen to the effect that the negative effects of social media companies are "baked into their business models."149 How can regulators retool business models that rely on algorithms that amplify the propagation of extreme content as well on the sweeping collection of personal data that allows platforms to target users with recommended, sometimes extreme, content?

Let me make some affirmations designed to provoke debate. The establishment of the Facebook Oversight Board gives rise to the counter-intuitive conclusion that national regulatory responses should seek to strengthen and widen the reach of online dispute resolution offered by platforms regime rather than simply to substitute for it.150 I take the point that as of August 19, 2021 the Oversight Board had only rendered fifteen decisions in its inaugural year. But in addition to the much-discussed decision on Donald Trump,151 the Zwarte Piet decision handed down on April 13, 2021 by the Oversight Board illustrates that a transnational body perhaps

146 See Digital Citizen Initiative, "Technical Paper," Module 2, Mandatory Reporting Act, paras. 7-8.

147 See Digital Citizen Initiative, "Technical Paper," Module 2, Canadian Security Intelligence Service Act, paras. 1-6.

148 Canadian Commission of Democratic Expression, 48.

149 Canadian Commission of Democratic Expression, 12.

150 See a detailed discussion of the Facebook Oversight Board in chapter 1.6. of this volume (Schultz, Mårten: Six Problems with Facebook's Oversight Board).

151 Facebook Oversight Board, Case decision 2021-001-FB-FBR, https://oversightboa rd.com/decision/FB-691QAMHJ/.

can accomplish something important that a national regulator might be less capable of achieving.152 That case concerned the removal of a 17 second video showing a child meeting three adults, one dressed to portray "Sinterklaas" (the Dutch version of Santa Claus) and two portraying "Zwarte Piet," also referred to as "Black Pete," who in the Dutch Christmas tradition accompanies Sinterklaas during the Feast of Saint Nicholas, distributing sweets. The video in question was posted to document this event. The two adults portraying Zwarte Piet had their faces painted black, wore Afro wigs under hats and colourful renaissance-style clothes. All the adults and the child in the video appeared to be white, including those with their faces painted black. The Board conducted a sophisticated analysis grounded notably in the UN Guiding Principles on Business and Human Rights and in human rights standards including Article 19 of the International Covenant on Civil and Political Rights. It considered the Zwarte Piet "tradition" in a comparative perspective and noted that it could be practiced without blackface or racial stereotypes. The majority upheld the removal of the content but concluded that Facebook had not sufficiently notified users about its community standard.

The point is that a purely Dutch body may or may not have been willing to put the Dutch tradition in comparative context and to consider the matter from the vantage point of how the stereotypes depicted were to be perceived in the context of global communications. Consequently, I would suggest that Canada consider the following points in addition to the direction being apparently being considered:

152 Facebook Oversight Board, Case decision 2021-002-FB-UA, https://oversightboar d.com/decision/FB-S6NRTDAJ/.

153 See Freedom Online Coalition, https://freedomonlinecoalition.com/.

### *Richard Janda*

4) Promulgate a standard promoting whistleblower access to "appellate" online dispute settlement bodies such as the Oversight Board. Facebook moderators have often been highly dissatisfied with the internal implementation of Facebook community standards and have encountered heavy-handed control from senior management.154 One thinks as well of the controversy around the Google AI Ethics Unit.155 Internal dissent at the platforms on these issues should be harnessed and subject to independent oversight.

The largest challenge to producing an online speech environment conducive to democratic expression is surely the need to adjust the underlying business models of the platforms. How does regulation get to the algorithmic ghost inside the machine? Here I have only more speculative ideas to offer, but ones that show some signs of promise. First, a duty to "take all reasonable measures to identify harmful content," conjoined with the fiduciary duty that companies owe to shareholders and stakeholders, could give rise to pressure on business models, notably as advertisers also attract scrutiny for having their messages accompany hateful content. Second, as discussed earlier,156 the adoption of a more robust privacy regime with powers given to the Privacy Commissioner to constrain the collection of personal information, if acted upon together with other like-minded jurisdictions, could have significant impact on the business case of some platforms. Indeed, one is already seeing the impact of Apple's new privacy drive on Facebook's advertising relationships.157 Third, and somewhat more ambitiously, perhaps a group of like-minded states could adopt legislation akin to the proposed *Protecting Americans from Dangerous Algorithms Act* designed to hold platforms liable for algorithms designed to amplify extreme content.158 Finally, and most ambitiously of all, momentum could

156 See *supra* note 32 to 36 and accompanying discussion.

154 Andrew Marantz, "Why Facebook Can't Fix Itself," *New Yorker*, October 12, 2020, https://www.newyorker.com/magazine/2020/10/19/why-facebook-cant-fix-i tself.

155 Shirin Ghaffary, "Google says it's committed to ethical AI research. Its ethical AI team isn't so sure," *Vox*, June 2, 2021, https://www.vox.com/recode/22465301/go ogle-ethical-ai-timnit-gebru-research-alex-hanna-jeff-dean-marian-croak.

157 Laura Forman, "Facebook and Its Advertisers Feel Pinch of Apple's Privacy Drive," *Wall Street Journal*, June 12, 2021, https://www.wsj.com/articles/facebook -and-its-advertisers-feel-pinch-of-apples-privacy-drive-11623502980.

158 U.S. Congress, House, *Protecting Americans from Dangerous Algorithms Act*, H.R. 8636 116th Congress 2d Session, introduced in House October 20, 2020, https:/ /www.congress.gov/bill/116th-congress/house-bill/8636/text. See also Tom Malinowski, "Reps. Malinowski and Eshoo Reintroduce Bill to Hold Tech Platforms

gather among leading jurisdictions to overhaul competition law so as to make it easier to break up platforms and reshape the way they operate.159

### *Bibliography*

Accountable for Algorithmic Promotion of Extremism" March 24, 2021, https:// malinowski.house.gov/media/press-releases/reps-malinowski-and-eshoo-reintrod uce-bill-hold-tech-platforms-accountable.

159 Cecilia Kang, "Lawmakers, Taking Aim at Big Tech, Push Sweeping Overhaul of Antitrust*,*" *New York Times*, June 11, 2020, https://www.nytimes.com/2021/06/11 /technology/big-tech-antitrust-bills.html.

### *Richard Janda*

Webe, Joel. "Hate speech no longer part of Canada's Human Rights Act." *National Post*, June 27, 2013. https://nationalpost.com/news/politics/hate-speech-no-longe r-part-of-canadas-human-rights-act.

https://doi.org/10.5771/9783748929789, am 27.04.2022, 15:24:38 Open Access – - http://www.nomos-elibrary.de/agb

## Lessons learned from the first years with the NetzDG

*Maximilian Hemmert-Halswick*

**Abstract**: German lawmakers have done pioneering work with the Network Enforcement Act. This law makes specific compliance requirements for social networks in order for them to remove illegal content more quickly and reliably. Criticism was particularly strong at the beginning, but the law now seems to have taken its place in the field of platform regulation. Three years after its enactment, the law underwent an amendment process. The amendment is based primarily on the experience gained up to that point. In essence, the aim is to eliminate identified deficiencies, which is mainly achieved by extending user rights and tighter regulatory control.

**Keywords**: Content Regulation; Hate Speech; Social Networks; Overblocking; Compliance approach; Censorship

#### *Introduction - Balance between State Sovereignty and Economic Freedom I.*

"[A]s Nostradamus said: across the sea they will come like locusts, but they will not be animals... how right the man was...". This comment refers to the refugee crisis in Germany and reveals the author's displeasure or contempt for the policy and the people who have fled. Are the confines of civilized, permissible discourse being left behind here? This is certainly not a simple question. The court that had to rule on this also had a hard time with the decision, but ruled that the deletion of the content was lawful.1

When it comes to the legally mandated deletion of content on social networks, such cases inevitably take center stage. There will hardly be any discussion about extreme, blatant cases, i.e. where there is broad consensus that such social interaction is unacceptable. Think, for example, of a call to commit murder; or even dishonorable and false allegations about other people that can seriously damage the person. In contrast, borderline cases show the difficulty of the matter. One must be aware of this, and it is

1 OLG Stuttgart, 6.9.2018 – 4 W 63/18.

against this background that the German "Law to Improve Law Enforcement in Social Networks" – in short: NetzDG2 – must be seen. The law was passed just under 4 years ago. The aim is to protect public safety, in particular the general right to protection of one' s personality, and to ensure rational discourse. To this end, it requires providers of social networks to delete unlawful comments more quickly and reliably.3 In 2020, the legislator began the process for amending the NetzDG. With three years of experience applying the law, it is hoped to address some shortcomings and provide improved enforcement. A year later, the amendments were approved by all legislative bodies.

This chapter will present the beginnings and the experiences that have led to the amendments of the NetzDG. The lines of development clearly speak for a learning effect on the part of the German legislator, which is also likely to be of international interest, since parallels can also be found in other countries, even if they have not rushed ahead with a law like the NetzDG. However, a great deal has happened in the industry during this time: content regulation has often been in the media spotlight; Twitter has blocked accounts of highly public figures4 , and Facebook has set up an "Oversight Board" which serves as a quasi-court. Against the backdrop of these events, the tendency can probably be discerned that it has now become clear that states must become involved in content regulation in one way or another.

#### *The Approach of the NetzDG II.*

The law was the first of its kind worldwide. It was subject to harsh criticism, especially during the legislative process.5 Yet, a modus vivendi seems

2 Netzwerkdurchsetzungsgesetz, September 1, 2017 (BGBl. I S. 3352).

3 Cf. BT-Drs. 18/12356, 11 (German Parliament Document).

4 The most noted case was certainly the blocking of the account of then-President Donald Trump in response to the storming of the Capitol, and later, for example, of Mike Lindell ("MyPillow Guy") for spreading disinformation concerning the results of the 2020 presidential election.

5 Spindler, Gerald, "Der Regierungsentwurf zum Netzwerkdurchsetzungsgesetz – europarechtswidrig?", *Zeitschrift für Urheber- und Medienrecht ZUM* (2017): 473; Gersdorf, Hubertus, "Hate Speech in sozialen Netzwerken," *MMR Zeitschrift für IT-Recht und Recht der Digitalisierung* (2017): 439. Guggenberger, Nikolas, "Das Netzwerkdurchsetzungsgesetz – schön gedacht, schlecht gemacht," *Zeitschrift für Rechtspolitik ZRP* (2017): 98.

to have been found now. While the law was described as "innovative"6 on the one hand, criticism was also voiced that the law constituted a violation of freedom of expression. In the following, the approach of the law will be described, and the points of criticism will be explained and put into context.

#### *Compliance approach 1.*

The law has a compliance approach, which means that it just wants to make sure that the social networks act according to the law, and it does not – at least directly – address the users in any way.

The regulatory approach of the NetzDG is to make social network providers comply to a greater extent with their deletion obligations, which existed already without the NetzDG. In this respect, the NetzDG sets compliance requirements for how social network providers are to set up their complaint management, i.e. how they are supposed to deal with user complaints. Among other things, it is supposed to guarantee the deletion of illegal content within seven days or – if the illegality is "obvious" – within 24 hours. This focus on time constraints resulted from previous experience, as high-ranking politicians, among others, had been victims of online aggression and the content was not deleted for several days despite complaints. Subsequently, the German government determined in surveys that the social networks in fact only very rarely comply with their deletion obligations.7 This deficiency is therefore to be remedied by setting specific time limits.

In this approach, it is of course noticeable that the network providers take on the active role; ultimately, they are the ones who decide which content will be deleted. The state retreats to a position of an observer. Criticism was voiced during the legislative process that law enforcement was being "privatized".8 However, this criticism does not really hold up, since the NetzDG also emphasizes that network operators do not have any active monitoring obligations. One alternative, of course, is for the state

6 Holznagel, Bernd, "Das Compliance-System des Entwurfs des Netzwerkdurchsetzungsgesetzes," *Zeitschrift für Urheber- und Medienrecht ZUM* (2017): 615; Bundestag Protokoll-Nr. 18/153, 21.

7 BT-Drs. 18/12356, 11.

8 Wimmers, Jörg and Heymann, Britta, "Zum Referentenentwurf eines Netzwerkdurchsetzungsgesetzes (NetzDG) – eine kritische Stellungnahme," *AfP Zeitschrift für das gesamte Medienrecht* (2017): 98.

### *Maximilian Hemmert-Halswick*

to take over monitoring. Conversely, there is the option of not imposing strict requirements on network operators. With the compliance approach, the German legislator wants to strike a balance between a too far-reaching encroachment on corporate freedom and an ineffective regulatory regime.

#### *Terminology 2.*

Just as in any law, it is of particular importance to define the essential terms in order to ensure its applicability in the first place. In many areas of law, reference can be made to familiar terms (that are already in use (in other laws). In the case of the NetzDG, this applies only to a limited extent, since content regulation on social networks is practically a new phenomenon. First and foremost, the definitions must clarify what social networks are and what content is to be covered by the NetzDG.

#### *Addressees: social networks a)*

Regarding the term "social network", perhaps the "you know it when you see it"-approach would be a viable option, especially for younger Internet users who have a more or less clear idea of what is meant by a social network without ever having taken a closer look at the individual criteria. Ultimately, the question of definition is related to what the NetzDG is intended to achieve. The legislator was primarily concerned with the perpetuation effect associated with publication on a network used by many people. The fact that only the big ones are to be covered means that various parameters have to be set in a way to exclude the smaller ones. The number of members plays a role – but does it only depend on registered users? What about messenger services, where the groups can sometimes be so large that they resemble "classic" social networks? What about comment functions on ,for example, newspaper sites? In order not to cover these aspects, the German legislator has chosen the following definition:

"Telemedia service providers who operate platforms on the Internet with the intention of making a profit, which are intended for users to share any content with other users or make it accessible to the public."9 Thematically limited social networks such as Linkedin are excluded by the arbitrariness criterion ("to share *any* content"). Furthermore, only

9 § 1 sec. 1 NetzDG.

networks with a user base of more than 2 million users are included. Journalistic content is excluded. Gated communities such as messenger services are also explicitly excluded, as this is not directly clear from the definition.10

#### *The most important term: illegal content b)*

One of the most important points to consider when passing a law that targets content on social networks is what content is covered. What content is at issue in the first place? What content is so important or harmful that it should be removed from the platform as quickly as possible. If the guidelines are too far-reaching, the accusation of censorship can legitimately be made. However, if too few specifications are made, the effectiveness can be questioned and the scope of application will be relatively small. In addition, the delimitation of content must be as precise as possible so that it is clear which content is covered - so that, on the one hand, network providers know which content is to be deleted. On the other hand, it is just as important for users to know what they are allowed to post.

According to the NetzDG, illegal content is content that fulfills one of the criminal offenses listed in § 1 of the NetzDG, which are primarily those that roughly cover the phenomenon of "hate speech," i.e., the crimes of defamation and the crimes against public order such as incitement to hatred. The fact that the NetzDG is limited to criminal offenses makes it clear that only relatively vile statements are covered by it, although it goes without saying that difficulties of demarcation cannot be avoided here either. However, by this the legislator avoids defining the term hate speech or disinformation, as this is hardly possible.11 The criminal offense of "insult" is included in the NetzDG. This is, of course, also an offense that is open to interpretation, and in this respect is similarly vague as the term hate speech. Social networks, on the other hand, define hate speech and disinformation in their guidelines, but can do so comparatively easily because, unlike democratically constituted states, they – generally speaking – do not have to meet balanced legal requirements. Here, however, it also

10 BT-Drs. 18/12356, 12 (German Parliament Document).

11 Hoven, Elisa and Krause, Melena, "Die Strafbarkeit der Verbreitung von 'Fake News'," *Juristische Schulung (JuS)* (2017): 1167.

*Maximilian Hemmert-Halswick*

becomes obvious that the question of definitional sovereignty is of utmost importance.12

#### *Complaints management 3.*

#### *Establishment of a complaint management system a)*

The most important element is that the social networks must implement a proper complaints management system as outlined in § 3 NetzDG: a functional system of how they handle the complaints against hate speech content. The submission of complaints must be as simple as possible. Ideally, the complaint option is set up directly next to the post in the news feed. It must then be ensured that the network provider takes note of the complaint quickly. This is then followed by an assessment within the appropriate deletion period. Setting up a complaints management system also involves providing regular training for employees to ensure that they are up to the task; it has been found ot that the task of deleting illegal content is very stressful.

12 The subject of a current debate is whether network providers may also delete content if the content is not constitutionally objectionable. This has to do with the indirect third-party effect of fundamental rights that exists in Germany. This means that under certain circumstances, private parties such as big companies must also comply with fundamental rights requirements in their actions. In relation to the case of content regulation on social networks, this means that Facebook and Co. must respect users' freedom of expression. However, the extent to which they must respect users' fundamental rights has not yet been conclusively decided. The answer is particularly relevant to the question of whether network providers may also delete lawful content. If there is a strong connection to fundamental rights, this is likely to be negated.

*Illustrative diagram of the complaints management system, according to § 3 NetzDG:*

#### *The Danger of Overblocking b)*

The main goal of the NetzDG is that the networks get illegal content quickly off their platforms. However, since a fine can be imposed for violations of the NetzDG, there is a risk that the networks will, in case of doubt, delete content rather than leave it on the platform. The consequence of this situation might be an overblocking. That was the main critique by the experts during the legislative hearing.13 The hope was that an assessment of whether there really was overblocking could be made on the basis of the transparency reports that providers were required to produce under § 2 of the NetzDG. Unsurprisingly, however, the transparency reports hardly allowed any conclusions to be drawn in this regard. The bare figures on complaints received and deletions made say practically nothing about the

13 Cf. protocol of the parliamentary expert hearing protocol Nr. 18/153, 16, 30, 38; Guggenberger, Nikolas, "Das Netzwerkdurchsetzungsgesetz in der Anwendung, "TRENNUNG *Neue Juristische Wochenschrift (NJW)* (2017): 2577; Schwartmann, Rolf, "Verantwortlichkeit Sozialer Netzwerke nach dem Netzwerkdurchsetzungsgesetz,"TRENNUNG *GRUR-Prax Praxis im Immaterialgüter- und Wettbewerbsrecht* (2017): 317.

existance of overblocking.14 However, a recent study has come to the conclusion that overblocking is taking place.15

Closely related to this is the accusation of insufficient consideration of the interests of the authors of deleted posts. In principle, the NetzDG does not provide for the author to be heard at any point in the proceedings or to obtain a reversal of the deletion. There may be various reasons for not involving the author, for example because this would prolong the proceedings. After all, expeditiousness in cancellation is the key purpose of the law. A possible starting point is the establishment of an option for the author to complain after the deletion.

#### *Establishment of regulated self-regulation c)*

In response to this criticism, which was voiced during the legislative process, another mechanism was introduced to ensure the accuracy of the decisions on the one hand and to take the pressure off the social network providers on the other. To this end, in § 3 sec. 6-9 NetzDG, the option was created for the social networks themselves to establish independent bodies to review complaints on behalf of the social networks: establishment of regulated self-regulation.16 The first and most prominent of its kind is the *Freiwillige Selbstkontrolle Multimedia-Dienste* (FSM17). The social networks can therefore then always decide whether to forward a complaint to this body. They must then accept the decision of this body and make a deletion accordingly. This regulation has existed from the beginning, but the first trials were not carried out until 2020 , as the establishment of the body had taken some time.18

14 Löber, Lena Isabell and Roßnagel, Alexander, "Das Netzwerkdurchsetzungsgesetz in der Umsetzung," *MMR* (2019): 73; Ladeur, Karl-Heinz, "Ist der Regierungsentwurf eines NetzDG 2.0 vom 19.2.2020 netzgerecht?", *Kommunikation und Recht K&R* (2020): 250; Heindorf, Manon, "Das Netzwerkdurchsetzungsgesetz in der Umsetzung: Zwei Jahre NetzDG – eine Bilanz," *Verwaltungsrundschau VR* (2020): 117.

15 Liesching, Marc, et al., *Das NetzDG in der praktischen Anwendung* (Berlin: Carl Grossmann Verlag, 2021), 143, doi:10.24921/2021.94115953.

16 This is modeled after voluntary monitoring in the film industry (*Freiwillige Selbstkontrolle der Filmwirtschaft*).

17 In English: Association for Voluntary Self-Regulation of Digital Media Service Providers.

18 See chapter in this book *Holznagel/Kalbhenn*.

*Illustrative diagram of the process with the FSM19:*

#### *Transparency obligations: Conflict between NetzDG and community standards – Facebook case study 4.*

As already mentioned, the obligations of the NetzDG apply primarily to the network providers. They have to set up complaint management, they have to check the content, and they have to carry out deletions if necessary. State authorities stay out of this practice. However, since the state does not want to leave the network providers to act idly and unsuspectingly, the NetzDG provides for comprehensive transparency obligations. Network providers must report on how they implement the requirements of the NetzDG. In terms of the role of the state in the structure of the NetzDG, this is the most important regulation, so that the state does not completely relinquish responsibility. The transparency obligations provide the state and the public with information about how many deletions are made, how many complaints are received by the networks, and what type of content the complaints focus on. It should be mentioned that the transparency reports essentially only provide quantitative information on how complaints are handled.  Examples of deletions are – unfortunately – not to be included in the transparency reports.

19 On the basis of the diagram at "NetzDG," FSM, accessed July 6, 2021, https://ww w.fsm.de/de/netzdg.

It is particularly noteworthy that Facebook – i.e. the largest social network – is attempting to circumvent the transparency obligation by setting up two complaints procedures. One is the NetzDG complaint procedure, which is very hidden and can only be reached via several clicks. The other is Facebook's own complaint procedure, which is based on Facebook's community standards and is located directly next to a post. Facebook does not list any information about the latter complaint procedure in the transparency report, although the vast majority of complaints are made via this procedure: approximately 2,000 complaints are listed in the NetzDG Transparency Report20, whereas several million complaints are counted with regard to the Community Standards, albeit globally.21 The Federal Office of Justice has imposed a fine of two million euros on Facebook for this. The courts are currently reviewing whether this was lawful. It is not clear what the answer will be, because the NetzDG only stipulates transparency obligations in relation to the NetzDG complaints procedure. Rather, the legislator had probably not even considered this as an option, and had simply assumed that the network providers would report on *any* deletions. However, it is also clear that Facebook's approach is a deliberate circumvention of the law. On the other hand, it may not always be so easy for the social networks to determine exactly whether a complaint is applicable to German law.

#### *Conclusion 5.*

So far, the basic requirements of the NetzDG have been presented. It became clear that there was potential for improvement. For example, the danger of overblocking could be further mitigated with a counter-appeal procedure, in which the person affected by the deletion also presents his or her point of view. Also, the users had little say in the matter. However, the lack of inclusion of network-internal complaints was an obvious shortcoming. It was also stated that the legislator had simply not thought of this and subsequently realized that the NetzDG should also include these com-

20 "NetzDG Transparenzbericht Januar 2021," Facebook, accessed July 6, 2021, https://about.fb.com/de/wp-content/uploads/sites/10/2021/01/Facebook-Netz DG-Transparenzbericht-Januar-2021.pdf (July-December 2020: 4.211 complaints, 29% deleted).

21 "Community Standards Enforcement Report, Third Quarter 2020," Facebook, accessed July 6, 2021, https://about.fb.com/news/2020/11/community-standards-enf orcement-report-nov-2020/ (22 Mio. regarding hate speech, 95% deleted).

plaints. The fact that there is a fundamental misunderstanding is also clear from the fact that in the Summer of 2019, the Federal Minister of Justice, *Christine Lambrecht*, had to make clear that the community standards of the networks are "not above the law".22

#### *Amending the NetzDG III.*

Three things can be identified that fueled the amendment process: First, the lawmaker still want to make sure that there is no overblocking ocurring. Secondly, they want to know about the complaints handling in more detail, as in enhance the transparency of deletion decisions. And thirdly, the power struggle in which the state wants to be in control of the rules that govern online speech. The new provisions that address these findings are presented below.

#### *Countercomplaints procedure 1.*

The countercomplaints procedure, which is introduced in the new § 3b of the NetzDG, obliges the providers of social networks to have a procedure in place that allows users to state their opinion on a decision made by the provider and to seek a re-examination. Both the author of a content and the complaining user can take the initiative. According to the new provision, the provider must "promptly subject its initial decision to reconsideration by a person not involved with the initial decision." Only if the provider wishes to remedy the objection the other party must be given the opportunity to state its position; in this way the new law takes into account that objections can also be raised abusively. The procedure ends with the review decision, which must be forwarded to the two users concerned.

With this, the legislator wants to address the criticism regarding overblocking and the lack of procedural participation of the author. Too much deletion is to be prevented by allowing the authors of content to take action against deletions in an internal network procedure. Conversely,

22 "Bundesamt für Justiz (BfJ) erlässt Bußgeldbescheid gegen Facebook," Federal Ministry of Justice and Consumer Protection, Press release from July 3, 2019, accessed July 6, 2021, https://www.bmjv.de/SharedDocs/Artikel/DE/2019/070319\_ Facebook.html.

it should be easier to have a review of content that has been objected to by users but not removed.

The countercomplaints procedure is in particular a response to the widespread called for a "put-back procedure".23 However, the law precisely does not stipulate the right to reinstatement of the unlawfully deleted content. Indeed, it does not say anything about what is to be done after the provider's repeated decision. Presumably, the lawmaker implicitly assumes that this will lead to a put-back, in case of the lawfulness of the previously deleted content. For example, at one point the explanatory memorandum talks about the provider reporting "in how many cases the counterproposal was remedied."24 It is difficult to imagine a remedy that does not involve a put-back.

It has been argued in the literature that the legislator is reluctant to establish an explicit put-back procedure, as this would entail a right to publication. Whether such a right could be upheld constitutionally is still being debated. In this respect, the legislative reluctance can be explained by the legal situation, which is still developing.

Another criticism goes against the timing of the countercomplaints procedure, or rather that the author of a deleted content must first accept the deletion; he or she is not heard before the deletion. It is argued here that the protection of freedom of expression requires that the author be heard beforehand and given the opportunity to respond to the complaint. The "Stadium Ban Decision" of the Federal Constitutional Court is being referenced for this argument.25 In this case, the court suggests that a hearing be held prior to the exclusion of a socially significant event (attending a football match).26 Facebook and other social networks are – in a way – comparable to visiting a football stadium in terms of social relevance... yet it does not seem necessary to require a pre-removal cross-appeal on constitutional grounds: Deleting a piece of content is not comparable to an exclusion - blocking the account would be – furthermore, not removing an unlawful piece of content weighs heavier than removing a lawful piece of content (that is, especially, if the countercomplaints procedure can ensure a speedy restoration).

23 Löber, and Roßnagel, "Das Netzwerkdurchsetzungsgesetz," 75; Peukert, Alexander, "Gewährleistung der Meinungs- und Informationsfreiheit in sozialen Netzwerken," *MMR* (2018): 572; Schwartmann,"Verantwortlichkeit sozialer Netzwerke," 318.

24 BT-Drs. 19/18792, 8 (§ 2 (2) No. 11).

25 Niggemann, Sandra, "Die NetzDG-Novelle," *Computer und Recht CR* (2020): 329.

26 BVerfGE 148, 267.

#### *Transparency rules 2.*

Since shortcomings in the transparency regulations also became apparent relatively quickly, the legislator had to take action here as well. In particular, the circumvention by Facebook was a deficiency with an easy remedy. Here, the bill on combating right-wing extremism and hate crime brings about the most significant innovation. Therein it is regulated that, for example, complaints and deletions in connection with the Community Directives are also listed in the transparency report. This is achieved by broadening the definition of "complaints" to now include complaints under the Community Standards. This certainly makes sense, as do the stricter requirements for user-friendliness of reporting channels included in the amendment. A dichotomy of reporting channels, as explained at the beginning with regard to Facebook, should then no longer exist.

The amendment expands the reporting obligations, requiring reporting on the use of procedures for automated detection of illegal content. And also on whether and to what extent academia has been given access to information from the network provider to enable evaluation.

These additions are all reasonable, but the decision-making practice of the network providers should be even more transparent. *All* decisions about content should be published, as some have already pointed out.27 As a response to this criticism, the legislature added another amendment "at the last minute".28 § 5a now gives "researchers" a right to access information from social network providers about "the use and specific mode of operation of procedures for the automated recognition of content" as well as "the circulation of content which has been the subject of complaints about illegal content or which has been removed or blocked by the provider". The scope of those entitled to make a claim extends to "any ... person conducting scientific research" (paragraph 1). An excessive level of claims will be prevented by the fact that a concept for the protection of the data received must be submitted to both the network provider and the supervisory authority along with the claim. In addition, the right to information will be mitigated by giving the provider grounds for refusing to provide information ("if his interests worthy of protection significantly outweigh the public interest in the research"). One obstacle to asserting the claim may

27 Eifert, Martin, "Rechenschaftspflichten für soziale Netzwerke und Suchmaschinen," *Neue Juristische Wochenschrift* (NJW) 2017: 1453; Löber, and Roßnagel, "Das Netzwerkdurchsetzungsgesetz," 75.

28 BT-Drs. 19/29392 (German Parliament Document).

also be that the network provider is entitled to reimbursement for the costs incurred in providing the information; as a general rule, the law limits the reimbursement costs to EUR 5,000.

#### *Regulatory supervision 3.*

#### *Powers of intervention a)*

§ 4a introduces a new supervisory regime. According to this, the Federal Office of Justice monitors compliance with the provisions of the NetzDG. From now on, this agency can take the necessary measures against providers in the event of violations of the NetzDG. Previously, the agency could only take repressive action in the form of fines. With its supervisory authority, the Federal Office of Justice can impose a forward-looking obligation on providers to put an end to a violation of the NetzDG without having to initiate fine proceedings.

The more flexible and constructive form of regulatory supervision now impleemented is to be welcomed. The threat of fines - one of the main points of criticism of the NetzDG - will thus lose its intimidating effect.

#### *Duty to cooperate – Duty to report b)*

A provision contained in the amendment that obliges network providers to report certain content to the law enforcement authorities has been met with much skepticism. This obligation applies to certain criminal offenses enlisted in § 1 NetzDG; this involves relatively serious offenses in whose prosecution there is a great interest, such as incitement to hatred, the formation of terrorist organizations, or the preparation of a serious act of violence that endangers the state. The idea is that law enforcement agencies will no longer be able to keep track of all the crimes committed on the Internet. Legislators hope that this will not only improve law enforcement, but also have a deterrent effect on users.

In the first draft, the procedure was designed in such a way that the network providers report such content to the Federal Criminal Police Office (*Bundeskriminalamt*); the IP address and port number are also reported; the Federal Criminal Police Office then assesses whether the content is actually relevant to criminal law. If the answer is affirmative, the full data identifying the person is requested from the social network. The data is then forwarded to the law enforcement authorities by the Federal Criminal Police Office. There has been much criticism of this regulation. An expert opinion from the Bundestag's academic service pointed to data protection concerns.29 Another expert opinion commissioned by the Green Party pointed to problems with regard to the timing of the data transfer. According to the opinion, the Federal Constitutional Court had only shortly before ruled that content should not be transmitted at the same time as the traffic data.30 The legislative package was therefore not signed by the President (as the final required legislative act). In February 2021, the bill was updated31 and approved a few month later32 .

Furthermore, this new regulation is also criticized for promoting the "privatization" of law enforcement.33 It would practically make the social networks responsible for law enforcement. Of course, it cannot be denied that the social networks decide which content they forward to the Federal Criminal Police Office and which they do not. However, it must be seen that ultimately only the social networks are in a position to comprehensively monitor the content and install a comprehensive complaints management system, which makes them aware of the damaging content. In this respect, the entire problem of content regulation can be illustrated with the following: *The state wants to, but cannot - the social networks can, but do not want to*. Of course, the latter is only true as long as it is not financially profitable, but state paternalism is probably always an evil for businesses such as social networks.

29 Scientific service of the Bundestag, "Die Vereinbarkeit der Meldepflicht nach § 3a Abs. 4 NetzDG n.F. mit dem Recht der Sitzländer der Anbieter von sozialen Netzwerken und das Verhältnis der verschiedenen Einrichtungen der Entscheidungskontrolle nach NetzDG und JMStV," September 11, 2020 Document No. WD 10 - 3000 - 043/20.

30 Bäcker, Matthias, "Folgerungen aus dem zweiten Bestandsdatenbeschluss des BVerfG für die durch das Gesetz zur Bekämpfung des Rechtsextremismus und der Hasskriminalität geschaffenen Datenverarbeitungsregelungen," Opinion for the Green Party from September 16, 2020: 3.

31 Bundestag Drucksache 19/25294 (German Parliament Document).

32 Bundestag Drucksache 19/29392 (German Parliament Document).

33 Stefan Krempl, "NetzDG-Reform: Gesetzgeber verstrickt sich in unauflösbare Widersprüche," *heise online*, last modified June 17, 2020, https://www.heise.de/ne ws/NetzDG-Reform-Gesetzgeber-verstrickt-sich-in-unaufloesbare-Widersprueche-4 786964.html.

#### *Out-of-court conciliation 4.*

The Amending Act also contains a provision for the introduction of an out-of-court dispute resolution. This is intended to enable alternative dispute resolution on a low-key basis. The dispute resolution bodies are not required to have any special qualifications; the goal is merely to make a serious attempt to resolve disputes possible. The parties involved are to be made aware of this possibility after the complaint procedure and also the counter-proposal procedure have been carried out; it thus serves to settle a dispute at a rather late stage. Together with the regulated self-regulation, this represents an attempt by the legislator to also involve other actors from civil society in the issues of content management on social networks. It is unclear how such models will evolve. The state is using these approaches primarily for the purpose of dispelling the impression of state censorship, but also to limit the power of social networks.

#### *Outlook IV.*

The NetzDG has already done pioneering work and paved the way for similar laws on content regulation. The amendment makes meaningful additions and will likely allow for better law enforcement on social networks. Of course, the NetzDG has the insurmountable shortcoming that it only applies to Germany, which generally puts obstacles in the way of law enforcement. The European Digital Services Act is intended to put a lid on this under EU law and set standards throughout the Union.34 Until then, the NetzDG fulfills its role as an experimental field; since Germany is also big enough to stand up to Facebook and Co. in some respects. So, here the famous saying from American constitutional law comes into play, that the states are the "laboratories" for the entire federal system.35 The NetzDG has made clear what kind of rules are to be found in such regulations. Complaint management is at the heart of this, and transparency regulations enable control by the state and the public. One lesson that Germany had to learn only in the course of application is the inclusion of the network's own deletion guidelines.

34 See chapter in this book *Holznagel/Kalbhenn*.

35 New State Ice Co. v. Liebmann, 285 U.S. 262, 311 (1932) (Brandeis, J., dissenting).

### *Bibliography*

# Platform Governance at the Periphery: Moderation, Shutdowns and Intervention

*Giovanni De Gregorio, Nicole Stremlau*

**Abstract:** After illustrating how the spread of dangerous content has led to troubling consequences beyond digital boundaries, this chapter describes how online hate speech has become criminalised in the global south. It analyses Internet shutdowns to understand their socio-legal consequences, and explores the applicability of public international law and the humanitarian doctrine to information interventions.

The spread of online hate and disinformation is increasingly provoking dramatic and troubling consequences beyond digital boundaries. False information about health treatments during the Covid-19 pandemic,1 or the use of social media in mobilizing actors for the attack on Capitol Hill,2 are some prominent examples of how online speech can affect the general public. But offline harms are far broader and often less explicitly tied to online speech. Our focus here is on areas of the world that have not been considered 'priorities' by social media companies.3 For example, in

1 Julie Posetti and Kalina Bontcheva, *Disinfodemic: deciphering COVID-19 disinformation. Policy brief 1*. (2020), https://en.unesco.org/covid19/disinfodemic.

2 Joan Donovan, Brian Friedberg and Emily Dreyfuss, "The Capitol siege was the biggest media spectacle of the Trump era," *The Guardian*, January 11 (2021) https:// www.theguardian.com/commentisfree/2021/jan/11/the-capitol-siege-was-the-bigges t-media-spectacle-of-the-trump-era.

3 In April 2021 the Guardian published an except from an email by a top Facebook executive explaining that the company should address concerns of abuse online by focusing on "top countries, top priority areas… and try to somewhat work our way down [to peripheral countries, or those that are seen as less strategic and driving

the Central African Republic, online hate speech has contributed to mass atrocities between Christians and Muslims,4 and in Sri Lanka, rumours on social media have led to a number of religious attacks, including the 2019 Easter Sunday church and hotel bombings,5 while the use of Facebook in inciting violence against Myanmar's minority Muslim population has elevated concerns about the role of online platforms in perpetrating genocides.6

The *fil rouge* connecting these examples is the role of social media companies7 in governing speech on a global scale.8 Online platforms that process content rely on a mix of human moderators and artificial intelligence systems that define which content must be removed according to non-transparent standards and without explanation, providing very few opportunities for remedies.9 As the global pandemic has altered working arrangements for human coders (along with many office workers) it has also made the implementation of artificial intelligence systems in content moderation more urgent for companies.10 But this has brought to the fore

news]". *See:* https://www.theguardian.com/technology/2021/apr/12/facebook-looph ole-state-backed-manipulation.

4 Office of the High Commissioner for Human Rights, *Preventing incitement to hatred and violence in the Central African Republic* (2019) https://www.ohchr.org/E N/NewsEvents/Pages/PeacekeepersDay2019.aspx.

5 Newley Purnell, "Sri Lankan Islamist Called for Violence on Facebook Before Easter Attacks," *Wall Street Journal,* April 30, 2019 https://www.wsj.com/articles/sr i-lankan-islamist-called-for-violence-on-facebook-before-easter-attacks-1155665095 4.

6 Fanny Potkin and Poppy McPherson, "Spreading like Wildfire: Facebook Fights Hate Speech before Myanmar Poll," *Reuters*, November 5, 2020, https://www.reut ers.com/article/myanmar-election-facebook-idUSL4N2HQ3QU.

7 When referring to 'social media' we are primarily speaking of user-generated content on large platforms such as Facebook, Twitter, YouTube or TikTok.

just how problematic AI can be when it comes to effectiveness; during the pandemic there have been significant cases when the involvement of human moderators was restricted and an over-reliance on the automated system led to the spread of disinformation and blocking of accounts that were actually countering disinformation.11

Against this opaque framework of governance and fragmented responses by social media companies, a variety of actors, from non-governmental organizations to various public authorities around the world have tried to tackle the harm produced by the spread of hate and disinformation online.12 Governments have reacted in different ways, particularly in poorer and less geopolitically influential countries. They have accused online platforms of disseminating hate and disinformation online, criminalised the spread of hate and disinformation,13 have used platforms for surveillance, worked to push alternative narratives (sometimes flooding platforms with disinformation), and have attempted to censor content.14 The spread of hate on social media has also been one of the primary reasons why governments have increasingly justified the use of Internet shutdowns, which can involve a range of tools from slowing down the internet (making it practically unusable) to completely switching it off.15 Whereas only a few years ago such forms of censorship would have been seen as a grave violation of freedom of expression, increasingly they being seen to be one of the few mechanisms available for addressing online speech and offline harms in a moment of crisis.

The role of media in contributing to disseminate hate and violence is not new.16 In some cases of violence and mass atrocities, international actors, including the United Nations (UN), have relied on "information

11 Statt Nick, "How Facebook Is Using AI to Combat COVID Misinformation and Detect 'Hateful Memes,'" The Verge, May 12, 2020, https://www.theverge.com/20 20/5/12/21254960/facebook-ai-moderation-covid-19-coronavirus-hateful-memes-ha te-speech.

12 Roxana Radu, Fighting the 'Infodemic': Legal Responses to COVID-19 Disinformation. *Social Media+ Society*, *6*(3), 2020.

13 Dickens Olewe, "Kenya, Uganda and Tanzania in "anti-fake news campaign," *BBC News,* 16 May 2018.

14 Adrian Shahbaz, "The Rise of Digital Authoritarianism: Freedom on the Net 2018," *Freedom House,* October (2018) https://freedomhouse.org/report/freedom-n et/2018/rise-digital-authoritarianism.

15 De Gregorio, Stremlau, " Internet Shutdowns and the Limits of Law".

16 Robert Edwin Herzstein. *The war that Hitler won: The most infamous propaganda campaign in history* (Putnam Publishing Group, 1978). Nicole Stremlau. *Media, Conflict and the State in Africa* (Cambridge University Press, 2018).

interventions," an expression developed in the 1990s in response to the conflict in Rwanda and the Balkans.17 While information interventions have been applied to traditional media outlets, we ask whether such a response could be relevant for social media, particularly when online platforms have a leading role in disseminating content directly associated with mass atrocities.

Within this framework, this chapter explores the challenges raised by online hate and disinformation in areas of the world that are less of a business priority for large social media companies. By focusing on content moderation as an expression of platform governance, we underline how the spread of online hate and disinformation have led to troubling consequences beyond digital boundaries. In the first part, we focus on the criminalisation of online hate and disinformation as a response to the consequences this content produces in the online and offline world. The second part explores how the spread of online hate speech and disinformation has provided governments with further justifications, that are increasingly becoming internationally acceptable (or at least understood), to censor the Internet for protecting national security or other public interests. The third part focuses on the role of international actors in addressing the spread of online hate and disinformation by looking at the applicability of the doctrine of information intervention to social media.

Our focus in this chapter is on the variety of legal and censorship responses to online hate. We recognize that there are considerable efforts on the part of governments to address online speech with different techniques ranging from attempting to shift narratives through flooding social media with specific content (as seen with the role of Cambridge Analytica), or using surveillance and both online and offline coercion or harassment to silence certain voices. In this chapter, however, our emphasis is on the intersection of concerns around content moderation and the use of law or force to address these concerns.

17 Monroe E Price and Mark Thompson, eds. *Forging peace: intervention, human rights, and the management of media space* (Indiana University Press, 2002); Jamie F. Metzl, "Information intervention: When switching channels isn't enough," *Foreign Affairs* (1997): 15-20.

#### *An initial response: Criminalising online hate and disinformation 2.*