Notes

• [1]
  Cathy O’Neil, computer scientist and author of the book, Weapons of Math Destruction (2016).
• [2]
  I would like to extend my thanks to the lnternational Institute of the Sociology of Law in Oñati, the Basque Country, for my research stay in June and July 2018, and for allowing me to use their well-stocked library while preparing an early draft of this article.
• [3]
  Stefan Larsson, “Sociology of Law in a Digital Society—A Tweet from Global Bukowina”, Societas/Communitas, 15 (1), 2013, p. 281-295; cf. Danièle Bourcier, “De l’intelligence artificielle à la personne virtuelle : émergence d’une entité juridique ?”, Droit et Société, 49, 2001, p. 847-871.
• [4]
  Or Biran and Courtenay Cotton, “Explanation and Justification in Machine Learning: A Survey”, IJCAI-17 Workshop on Explainable AI (XAI), 2017.
• [5]
  Cf. Iyad Rahwan, “Society-in-the-Loop: Programming the Algorithmic Social Contract”, Ethics and Information Technology, 20 (1), 2018, p. 5-14.
• [6]
  Susan Leigh Anderson, “Asimov’s ‘Three Laws of Robotics’ and Machine Metaethics”, AI & Society, 22 (4), 2008, p. 477-493.
• [7]
  Cf. Nick Bostrom, Superintelligence: Paths, Dangers, Strategies, Oxford: Oxford University Press, 2014.
• [8]
  Arthur Samuel, “Some Studies in Machine Learning Using the Game of Checkers”, IBM Journal of Research and Development, 3 (3). 1959, p. 210-229.
• [9]
  AI HLEG, “Draft Ethics Guidelines for Trustworthy AI,” 18 December 2018, <https://ec.europa.eu/
  digital-single-market/en/news/draft-ethics-guidelines-trustworthy-ai>.
• [10]
  Id., Ethics Guidelines for Trustworthy AI, Brussels: The European Commission. 2019.
• [11]
  Regeringskansliet, Nationell inriktning för artificiell intelligens. Näringsdepartementet, 2018, p. 10.
• [12]
  E.g., see <https://www.fatml.org>; For an overview of research on ethical, social and legal consequences of AI, see Stefan Larsson, Mikael Anneroth, Anna Felländeret al., Sustainable AI: An Inventory of the State of Knowledge of Ethical, Social, and Legal Challenges Related to Artificial Intelligence, Stockholm: AI Sustainability Center, 2019.
• [13]
  For an analysis on the conceptual origins and background of “transparency” with regards to AI, see Stefan Larsson and Fredrik Heintz, “AI Transparency”, Internet Policy Review, 2019 (forthcoming).
• [14]
  As reported in Wired, “Machines taught by photos learn a sexist view of women”, by Tom Simonite, 21 August 2017: <https://www.wired.com/story/machines-taught-by-photos-learn-a-sexist-view-of-women/amp>; for a study, see Jieyo Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez and Kai-Wei Chang. “Men also like shopping: Reducing gender bias amplification using corpus-level constraints”, arXiv preprint, 2017, arXiv:1707.09457.
• [15]
  The study was carried out and published by civil rights-motivated investigative journalists at ProPublica, “Machine Bias”, by Julia Angwin, Jeff Larson, Surya Mattu and Lauren Kirchner, 23 May 2016, <https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing>.
• [16]
  Correctional Offender Management Profiling for Alternative Sanctions.
• [17]
  This case is discussed in a growing body of literature from several angles, and is particularly interesting from a socio-legal perspective, not the least from the fact that it is explicitly dealing with the automation of court decisions; cf. Robyn Caplan, Joan Donovan, Lauren Hanson and Jeanna Matthews,Algorithmic Accountability: A Primer, NYC: Data & Society, 2018. For a critique of the judicial use of automated risk assessment tools in ways that undermine the fundamental values of due process, equal protection and transparency, see Han-Wei Liu, Ching-Fu Lin and Yu-Jie Chen, “Beyond State v Loomis: Artificial Intelligence, Government Algorithmization and Accountability”, International Journal of Law and Information Technology, 27 (2), 2019, p. 122-141.
• [18]
  Amit Datta, Michael Carl Tschantz and Anupam Datta, “Automated Experiments on Ad Privacy Settings—A Tale of Opacity, Choice, and Discrimination”, Proceedings on Privacy Enhancing Technologies, 1, 2015, p. 92-112, DOI: 10.1515/popets-2015-0007.
• [19]
  Joy Buolamwini and Timnit Gebru, Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification, in Conference on Fairness, Accountability and Transparency, 2018, p. 77-91.
• [20]
  As noted by, among others, Arvind Narayanan, “21 Fairness Definitions and Their Politics”, presented at the conference on Fairness, Accountability, and Transparency, 2018, <http://fairmlbook.org/tutorial2.html>.
• [21]
  The IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems, Ethically Aligned Design: A Vision for Prioritizing Human Well-being with Autonomous and Intelligent Systems, IEEE, 2019.
• [22]
  Émile Durkheim, Les règles de la méthode sociologique, Paris: PUF, 1982 [1895]. Steven Lukes (ed.), The Rules of Sociological Method and Selected Texts on Sociology and its Method, W. D. Halls (translator), New York: Free Press, 2014; cf. Roger Cotterrell. Emile Durkheim: Law in a Moral Domain, Edinburgh: Edinburgh University Press, 1999.
• [23]
  Eugen Ehrlich, Fundamental Principles of the Sociology of Law, New Brunswick, NJ: Transaction Publishers, 2002. For a modern application, see for example Rustamjon Urinboyev and Måns Svensson, “Living Law, Legal Pluralism, and Corruption in Post-Soviet Uzbekistan”, The Journal of Legal Pluralism and Unofficial Law, 45 (3), 2013, p. 372-390.
• [24]
  Roscoe Pound, “Law in Books and Law in Action”, American Law Review, 44, 1910, p. 12.
• [25]
  E.g. Håkan Hydén and Måns Svensson, “The Concept of Norms in Sociology of Law”, in Peter Wahlgren (ed.), Scandinavian Studies in Law, Stockholm: Law and Society, 2008, p. 15-33; Måns Svensson and Stefan Larsson, “Intellectual Property Law Compliance in Europe: Illegal File sharing and the Role of Social Norms”, New Media & Society, 14 (7), 2012, p. 1147-1163.
• [26]
  Cf. Batya Friedman and Helen Nissenbaum, “Bias in Computer Systems”, ACM Transactions on Information Systems, 14 (3), 1996, p. 330-347.
• [27]
  Cf. Stefan Larsson and Fredrik Heintz, “AI Transparency”, op. cit.; Meredith Whittaker, Kate Crawford, Roel Dobb et al., AI Now Report 2018, New York: AI Now Institute, 2018.
• [28]
  Shreya Shankar, Yoni Halpern, Eric Brecket al., “No Classification Without Representation: Assessing Geodiversity Issues in Open Data Sets for the Developing World”, arXiv preprint, 2017, arXiv:1711.08536.
• [29]
  Cf. Eszter Hargittai, “The Social, Political, Economic, and Cultural Dimensions of Search Engines: An Introduction”, Journal of Computer-Mediated Communication, 12 (3), 2007, p. 769-777.
• [30]
  Rex L. Troumbley, Taboo Language and the Politics of American Cultural Governance, Doctoral dissertation, University of Hawai’i at Manoa, 2015.
• [31]
  Safiya Noble, Algorithms of Oppression: How Search Engines Reinforce Racism, New York: New York University Press, 2018.
• [32]
  It is sometimes attributed to American sociologist John McKnight, cf. William Norton, Cultural Geography: Environments, Landscapes, Identities, Inequalities, Oxford: Oxford University Press, 2013. A number of studies suggest a long‐standing relationship between geography, race and contemporary housing and credit markets; cf. Jesus Hernandez, “Redlining Revisited: Mortgage Lending Patterns in Sacramento 1930-2004”, International Journal of Urban and Regional Research, 33 (2), 2009, p. 291-313.
• [33]
  Safiya Noble in Robyn Caplan, Joan Donovan, Lauren Hanson and Jeanna Matthews,Algorithmic Accountability: A Primer, op. cit., p. 4.
• [34]
  Robyn Caplan, Joan Donovan, Lauren Hanson and Jeanna Matthews,Algorithmic Accountability: A Primer, op. cit.
• [35]
  Alex Campolo, Madelyn Sanfilippo, Meredith Whittaker and Kate Crawford, AI Now 2017 Report, AI Now Institute at New York University, 2017, p. 18.
• [36]
  Mireille Hildebrandt, Smart Technologies and the Ends of Law, Cheltenham: Edward Elgar Publishing, 2015.
• [37]
  Susan Leigh Anderson, “Asimov’s ‘Three Laws of Robotics’ and Machine Metaethics”, op. cit., p. 477-493.
• [38]
  Sundar Pichai, “AI at Google: Our Principles”, Google blog, 7 June, 2018. <https://www.blog.google/
  topics/ai/ai-principles/>.
• [39]
  The Verge, “Google Reportedly Leaving Project Maven Military AI Program After 2019”, by Nick Statt, June 1, 2018, <https://www.theverge.com/2018/6/1/17418406/google-maven-drone-imagery-ai-contract-expire> (last visited 10 June 2019).
• [40]
  Miles Brundageet al., The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation, 2018, <https://maliciousaireport.com>.
• [41]
  Marco T. Bastos and Dan Mercea, “The Brexit Botnet and User-Generated Hyperpartisan News”, Social Science Computer Review, 2017, <https://doi.org/10.1177/0894439317734157>.
• [42]
  E.g., David A. Broniatowski, Amelia M. Jamison, SiHua Qiet al., “Weaponized Health Communication: Twitter Bots and Russian Trolls Amplify the Vaccine Debate”, American Journal of Public Health, 2018. DOI: 10.2105/AJPH.2018.304567; for more on the social impact of platforms, see Stefan Larsson and Jonas Andersson Schwarz, Developing Platform Economies. A European Policy Landscape, Brussels: European Liberal Forum asbl, Stockholm: Fores, 2018.
• [43]
  Miles Brundageet al., The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation, op. cit., p. 7.
• [44]
  Cf. Engin Bozdag, “Bias in Algorithmic Filtering and Personalization”, Ethics and Information Technology, 15 (3), 2013, p. 209-227.
• [45]
  Cf. Nicholas Diakopoulos, “Algorithmic Accountability: Journalistic Investigation of Computational Power Structures”, Digital Journalism, 3 (3), 2015, p. 398-415.
• [46]
  Robyn Caplan, Joan Donovan, Lauren Hanson and Jeanna Matthews,Algorithmic Accountability: A Primer, op. cit., p. 12.
• [47]
  Lawrence Lessig, “Code is Law”, The Industry Standard, 18, 1999; Lawrence Lessig, Code: Version 2.0, 2006; Cf. Stefan Larsson, “Sociology of Law in a Digital Society—A Tweet from Global Bukowina”, op. cit.
• [48]
  Cf. Jonas Andersson Schwarz, “Platform Logic: An Interdisciplinary Approach to the Platform-Based Economy”, Policy & Internet, 9 (4), 2017, p. 374-394; Tarleton Gillespie, Custodians of the Internet: Platforms, Content Moderation, and the Hidden Decisions that Shape Social Media, New Haven: Yale University Press, 2018.
• [49]
  When the persons running The Pirate Bay file-sharing site were prosecuted in 2009 for complicity in violation of the Copyright Act, a similar conceptual challenge emerged when the court was forced to assess this “platform’s” liability; Stefan Larsson, “Metaphors, Law and Digital Phenomena: The Swedish Pirate Bay Court Case”, International Journal of Law and Information Technology, 21 (4), 2013, p. 329-353; Id., Conceptions in the Code. How Metaphors Explain Legal Challenges in Digital Times, Oxford: Oxford University Press, 2017.
• [50]
  Cf. Tarleton Gillespie, Custodians of the Internet: Platforms, Content Moderation, and the Hidden Decisions that Shape Social Media, op. cit.
• [51]
  Ulrich Dolata, Apple, Amazon, Google, Facebook, Microsoft: Market concentration-competition-innovation strategies, 2017-01, Stuttgarter Beiträge zur Organisations-und Innovationsforschung, SOI Discussion Paper, 2017.
• [52]
  A news story that received much attention when journalist Carole Cadwalladr published an article about a whistle-blower in The Guardian, 18 March 2018, <https://www.theguardian.com/news/2018/mar/17/
  data-war-whistleblower-christopher-wylie-faceook-nix-bannon-trump>.
• [53]
  Evgeny Morozov, To Save Everything, Click Here: The Folly of Technological Solutionism, New York: Public Affairs, 2013.
• [54]
  Kirsten Gollatz, Felix Beer and Christian Katzenbach, “The Turn to Artificial Intelligence in Governing Communication Online”, Social Science Open Access Repository, 21, 2018. Cf. BuzzFeed News, “Why Facebook Will Never Fully Solve Its Problems With AI”, by Davey Alba, 11 April 2018, <https://www.buzzfeednews.com/
  article/daveyalba/mark-zuckerberg-artificial-intelligence-facebook-content-pro>.
• [55]
  Cf. SOU 2018:16, Vägen till självkörande fordon–introduktion, in which delegation of responsibility and data protection issues is a key component.
• [56]
  Cf. Alexander Hevelke and Julian Nida-Rümelin, “Responsibility for Crashes of Autonomous Vehicles: An Ethical Analysis”, Science and Engineering Ethics, 21 (3), 2015, p. 619-630.
• [57]
  Riccardo Guidotti, Anna Monreale, Salvatore Ruggieriet al., “A Survey of Methods for Explaining Black Box Models”, ACM Computing Surveys (CSUR), 51 (5), 2018, p. 1-45; cf. Frank Pasquale, The Black Box Society. The Secret Algorithms That Control Money and Information, Cambridge: Harvard University Press, 2015.
• [58]
  Mike Ananny and Kate Crawford, “Seeing Without Knowing: Limitations of the Transparency Ideal and its Application to Algorithmic Accountability”, New Media & Society, 20 (3), 2018, p. 973-989.
• [59]
  Communication from the commission to the European Parliament, the European Council, the European Economic and Social Committee and the Committee of the Regions, Artificial Intelligence for Europe, SWD (2018) 137 final.
• [60]
  EU Commission,Algorithmic Awareness-Building, 25 April 2018, <https://ec.europa.eu/digital-single-market/en/algorithmic-awareness-building>.
• [61]
  Sarah Spiekermann and Jana Korunovska, “Towards a Value Theory for Personal Data”, Journal of Information Technology, 23 (1), 2016, p. 62-84, doi:10.1057/jit.2016.4.
• [62]
  Cf. Frank Pasquale, The Black Box Society. The Secret Algorithms That Control Money and Information, op. cit.
• [63]
  Rashida Richardson, “Optimizing for Engagement: Understanding the Use of Persuasive Technology on Internet Platforms”, AI Now Institute: statement before the United States Senate Committee on Commerce, Science, and Transportation. Subcommittee on Communications, Technology, Innovation and the Internet, June 25, 2019, p. 6.
• [64]
  Cf. Cathy O’Neil, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy, Londres: Allen Lane, 2016.
• [65]
  Robyn Caplan, Joan Donovan, Lauren Hanson and Jeanna Matthews,Algorithmic Accountability: A Primer, op. cit., point out that only the slightest disclosure of how Twitter’s trending method works has made it possible to manipulate parts of their environment and fill selected topics with automated bots or bot-networks in order to influence, manipulate or simply ruin discussions.
• [66]
  Derived from media and information literacy, cf. Jutta Haider and Olof Sundin, Invisible Search and Online Search Engines: The Ubiquity of Search in Everyday Life, Chicago: Routledge Studies in Library and Information Science, 2019.
• [67]
  Stefan Larsson, “Algorithmic Governance and the Need for Consumer Empowerment in Data-Driven Markets”, Internet Policy Review, 7 (2), 2018.
• [68]
  Finale Doshi-Velez, Mason Kortz, Ryan Budishet al., “Accountability of AI Under the Law: The Role Of Explanation”, arXiv preprint, 2017, arXiv:1711.01134.
• [69]
  Stefan Larsson, Conceptions in the Code. How Metaphors Explain Legal Challenges in Digital Times, op. cit.
• [70]
  Wolfie Christl, Corporate Surveillance in Everyday Life: How Companies Collect, Combine, Analyze, Trade, and Use Personal Data on Billions, Vienna: Cracked Labs, 2017.
• [71]
  Frank Pasquale, “Exploring the Fintech Landscape”, Written Testimony of Frank Pasquale Before the United States Senate Committee on the Banking, Housing, and Urban Affairs, 2017, September 12; Stefan Larsson, “Algorithmic Governance and the Need for Consumer Empowerment in Data-driven Markets”, Internet Policy Review, 7 (2), 2018, p. 1-12.
• [72]
  Information Commissioner’s Office (ICO), UK, Update Report into Adtech and Real Time Bidding, 20 June 2019.
• [73]
  Stefan Larsson, “Algorithmic Governance and the Need for Consumer Empowerment in Data-driven Markets”, op. cit.
• [74]
  Riccardo Guidotti, Anna Monreale, Salvatore Ruggieriet al., “A Survey of Methods for Explaining Black Box Models”, op. cit.
• [75]
  Or Biran and Courtenay Cotton, “Explanation and Justification in Machine Learning: A Survey”, op. cit.
• [76]
  Tim Miller, “Explanation in Artificial Intelligence: Insights from the Social Sciences”, Artificial Intelligence, 267, 2019, p. 1-38, <https://doi.org/10.1016/j.artint.2018.07.007>.
• [77]
  Mireille Hildebrandt, Smart Technologies and the Ends of Law, op. cit., p. 133 sq.
• [78]
  Eugen Ehrlich, Fundamental Principles of the Sociology of Law, op. cit.
• [79]
  Cf. The IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems, Ethically Aligned Design: A Vision for Prioritizing Human Well-being with Autonomous and Intelligent Systems, op. cit., p. 36.
• [80]
  Cf. Måns Svensson and Stefan Larsson, “Intellectual Property Law Compliance in Europe: Illegal File Sharing and the Role of Social Norms”, op. cit.
• [81]
  Cf. Tarleton Gillespie, Custodians of the Internet: Platforms, Content Moderation, and the Hidden Decisions that Shape Social Media, op. cit.
• [82]
  E.g., as noted by researchers and published in Nature; James Zou and Londa Schiebinger, “AI Can Be Sexist and Racist—It’s Time to Make It Fair”, Nature, comment, 18 July 2018.
• [83]
  Cf. Meredith Whittaker, Kate Crawford, Roel Dobb et al., AI Now Report 2018, op. cit.
• [84]
  Cf. ibid., p. 6, point 10.
• [85]
  Karen Yeung, “‘Hypernudge’: Big Data as a Mode of Regulation by Design”, Information, Communication & Society, 20 (1), 2017, p. 118-136.
• [86]
  Iyad Rahwan, “Society-in-the-Loop: Programming the Algorithmic Social Contract”, op. cit.
• [87]
  This is in line with for example AI HLEG’s Ethics guidelines for trustworthy AI (2019); the IEEE’s Ethically Aligned Design, 2019; and Luciano Floridi, Josh Cowls, Monica Beltramettiand al., “AI4People—An Ethical Framework for a Good AI Society: Opportunities, Risks, Principles, and Recommendations”, Minds and Machines, 28, 2018, p. 689-707.