Meanwhile, US researchers have demonstrated that AI's entry into society can also boost employment. The number of available AI-related positions went up from 0.3% of all US vacancies in 2012 to 0.8% in 2019. Having stood at 0.26% in 2010, the proportion of jobs accounted for by AI-related roles reached 1.32% in 2019.25 Moreover, AI has become one of the most popular felds of study for postgraduate researchers in computer science in North America. In 2010 the proportion of PhD graduates in AI taking jobs in industry was about the same as the percentage going into academia. Since then, though, the balance has shifted: in 2019 more than half went on to take industry jobs, while fewer than a quarter followed academic careers.26 According to technology expert Tim O'Reilly, 'data scientist' is now the most coveted job title in Silicon Valley. The McKinsey Global Institute estimates that, in 2018, the US already had between 140,000 and 190,000 fewer machine learning experts than it needed.27

# *3.1.5 Governments Are Also Focusing on AI*

It is not only through commercial activities and private-sector applications that AI is entering society; a wide variety of public organizations are also contributing towards the transition. Police services use the technology to investigate and fght crime, social security agencies use it for fraud detection and various AI-based control initiatives were launched during the COVID-19 pandemic. Although no global historical overview is available, the European Commission estimates that roughly 230 public-sector AI applications were in use in 2019.28 It seems very likely that the actual number was higher; in the Netherlands alone, 74 public-service projects were making use of AI that year.29

Further evidence of AI's societal traction is provided by the growing number of national AI strategies being produced. Once it became clear that AI had reached the point where various practical applications were in the offng and the business community was investing heavily, many governments began developing strategies to reap the associated benefts. First came the Pan-Canadian Artifcial Intelligence Strategy in March 2017, in which the Ottawa government announced plans to invest C\$125 million in AI. Singapore, Japan and the United Arab Emirates followed suit later that year. China then published the New Generation Artifcial Intelligence

24McKinsey & Company, 2020.

25Perrault et al., 2019.

26Zhang et al., 2021: 118.

27Domingos, 2017: 9.

28Misuraca & Van Noordt, 2020.

29Van Veenstra et al., 2019.

**Fig. 3.1** AI gathers momentum outside the lab

Development Plan, setting out its ambition to be the absolute global leader in AI by 2030. Soon afterwards strategies were presented by Finland, the US, France, the UK, Germany and other countries. As part of its commitment to 'a Europe that is ready for the digital age', the EC also began a number of AI-related programmes accompanied by a European Action Plan for AI30 and a data strategy.31 Since then, dozens of nations have produced action plans for utilizing AI, including less obvious countries such as Kenya, India and Mexico.32 The fow of publications hit a peak in 2019 when twenty national AI strategies appeared; a total of about sixty are now in circulation. There is also one international AI strategy: the EU's Co-ordinated Plan on Artifcial Intelligence (2018).33

Following the acceleration of AI development from around 2000 onwards, it is apparent from the increasing number of patent grants, the growing level of private investment, the appearance of new business models, the growth of AI-related employment and the publication of national strategies that we have reached a new chapter in the history of AI: the technology is entering society. Figure 3.1 illustrates

30European Commission, 2021b [2018].

31European Commission, 2020.

32Holoniq, 9 April 2020; Future of Life Institute, undated (a); Van Roy et al., 2021.

33European Commission, 2021b [2018].

this progress using the indicators referred to above. It is therefore pertinent to ask what mechanisms are at work here and what forms is AI taking in society. We address those questions in the next section.

## **Key Points – Momentum from Lab to Society**

# **3.2 The Practical Application of AI**

AI has thus made the transition from the lab to society. As a result, we nowadays encounter all kinds of applications of the technology in our everyday lives: chatbots, smart cameras, translation apps, recommendation systems, risk analyses, driving systems and so on. In practice, AI takes many different forms which may be divided into several broad groups based on the type of task performed. Within the discipline, various classifcation systems are used. For the purpose of this overview, we distinguish fve types of AI: applications for predictive analysis (machine learning), for image processing (computer vision), for language (natural language processing) and speech (speech recognition) and for the performance of physical tasks (robotics). All of these are already visible around us. Figure 3.2 provides an overview of the fve types, which are considered individually below.

# *3.2.1 Machine Learning*

The most common type of AI is machine learning. That can be slightly confusing, because the same term is also used for the form of technology currently dominant within AI. In this case, however, 'machine learning' refers to a particular type of

application for predictive or advanced analytics, which is used to identify patterns in datasets as a basis for making predictions. Although machine learning technology can be used in other types of AI as well, this form is characterized by prediction being the primary task. It could thus also be referred to as 'predictive systems'.

The ability to use data to make better-informed estimates about the future has huge potential value in many different contexts. The organization of energy supplies is a good example. Google's DeepMind has developed an AI system that uses weather forecasts and turbine data to predict the infow of energy from wind farms 36 h in advance.34 Optimum use can then be made of wind power, despite the variability of the elements.

Because risk forecasting has always played an important role in fnancial services, machine learning is now widely used in that sector. Examples include AI-based credit rating, where a person's creditworthiness is predicted based on their credit history and personal data.

Machine learning is also used for fraud prevention. For instance, Mastercard uses a system called Decision Intelligence to detect abnormal, potentially fraudulent activity by analysing transaction patterns.35 There are also AI applications for customers, including systems that predict fnancial trends to help inform investment decisions.36 Like banks and insurers, local authorities and police forces are looking into machine learning in the fght against fraud and other forms of crime. The UK's Department of Work and Pensions uses AI to assess beneft claims and estimate the probability of fraud, for example.37 In the Netherlands, the Ministry of Social Affairs and Employment and the country's local authorities introduced System Risk

34DeepMind, 26 February 2019.

35Mastercard, undated.

36 ING's Smart Working Capital Assistant and Katana Lens.

37Gov.uk, undated; Marr, 29 October 2018.

Indication (SyRI) to tackle beneft fraud. This approach proved controversial, however, and was ultimately deemed unlawful by the courts. Meanwhile, some Dutch local authorities use AI to predict which of their residents are liable to fall into debt and may therefore require assistance.

AI is also deployed in police work, in the form of prediction systems. Many examples of 'predictive policing' can be found in the US, where AI is used, for example, to assess the risk of reoffending. Forces in other countries are investigating the scope for using machine learning to provide intelligence. For instance, the Dutch police have a Crime Anticipation System (CAS) that detects patterns of criminality and predicts where and when robberies are most likely to occur. Based on this output, surveillance and preventive activities can be tailored to the anticipated risk. Almost all of the 168 police districts in the Netherlands are currently using a version of CAS.38 Furthermore, the Dutch police experiment with machine learning to predict which cold cases have the highest chance of a breakthrough and are therefore worthy of further investigation.39

AI's accurate predictive capabilities can be valuable in other sectors as well. Some supermarket chains have announced plans to experiment with dynamic pricing as a tool for minimizing waste and maximizing income. They could also use machine learning for product-range optimization or automated discounting. This would involve an algorithm analysing data on product shelf-life, outlet location, weather conditions and historical sales patterns to make predictions.40

In the media industry, machine learning helps tailor products and services to consumers' wishes. The most familiar examples are platform services like Netfix, YouTube and Spotify, which use AI to make relevant recommendations based on users' previous choices. Predictive technologies of this kind are known as 'recommender systems'. Machine learning-aided personalization has become an important pillar of e-commerce as well. Online retailers like Amazon, Alibaba and Zalando use AI to compile user profles and adapt their marketing accordingly.

Similarly, advertising can be aligned with the interests and sensibilities of individual users. Known as microtargeting, this technique lends itself not only to commercial applications but also to political ends. Political microtargeting made waves around the world when it became known that the company Cambridge Analytica had used Facebook data to disseminate personalised advertising during the US presidential election and the UK's Brexit referendum in 2016 (see Box 3.1). However, microtargeting is a widespread phenomenon that has been occurring in many countries for quite some time.41 Investigative journalists have found that almost all political parties in the Netherlands engage in bespoke online messaging.42

38Waardenburg et al., 2020: 70.

39Politie, 23 May 2018.

40Albert Heijn, 20 May 2019.

41Prins, 2017; Zuiderveen Borgesius et al., 2018.

42Davidson and Delhaas, 22 April 2020. The planned Political Parties Act will regulate such activities (Dutch national government, 26 June 2019).

## **Box 3.1: Cambridge Analytica and Microtargeting**

Microtargeting is directing particular messages at particular people. AI is used for 'psychographic profling', so that the content shown to an individual is tailored to their personal profle, thus (supposedly) maximizing its effectiveness. The best-known examples of the dark side of this technique involve the data-mining frm Cambridge Analytica, which was closely associated with the 2016 Trump and pro-Brexit campaigns. Machine learning was applied to huge volumes of data on people's online behaviour to build an understanding of public thinking, thus enabling targeted messaging on Facebook and other platforms to infuence the way individuals voted. Cambridge Analytica has since ceased trading, but predictive systems of a new type are now being developed: 'multi-agent artifcial intelligence' (MAAI). It is claimed that these can predict behaviour even more accurately, opening the way for more precise infuence by putting targeting strategies to the test in simulated communities.43

# *3.2.2 Computer Vision*

Our second main type of AI relates to image recognition, also known as computer vision. This is about automating the observation, analysis and interpretation of visual information. That may be in the form of photographs, videos or live input from the physical world. Its development has been accelerated by the increasing availability of digital imagery. Social media and smartphones have facilitated a veritable explosion of images, some publicly available, which can be used to train computer vision algorithms. Indeed, we now communicate increasingly through images – 'If there isn't a pic, it didn't happen!' Since Instagram was launched, users have uploaded about 50 billion photos, while 350 million photos a day are posted on Facebook and 500 h of video material are added to YouTube every minute.44

One of the best-known applications of computer vision is facial recognition. Moving beyond the mere detection of a face in an image by a computer, this entails the computer actually identifying whose it is. Camera input is analysed and features such as chin proportions, eye separation and cheek roundness are measured with millimetre-level accuracy. The computer the translates this data into a code representing the unique characteristics of a face, enabling it to be recognized when next encountered.

Facial recognition software is built into some smartphones, enabling users to unlock their phones simply by looking into the camera – in other words, to use their face like a password. Various apps use the technique in a similar way so that, for example, a PIN is not needed to authorize a payment.

43Hern, 30 July 2019; Lewis & Hilder, 23 March 2018; Lawton, 2 October 2019.

44Apple, 24 June 2020; Smith, 18 September 2013; Wojciki, 14 February 2020.

China leads the world in the state use of facial recognition. The technology is widely deployed by the police there and for the surveillance of urban public spaces.45 Many US government organizations, including the police, investigative agencies and border forces, use facial recognition as well.46 Although currently controversial in Europe, most countries here are experimenting with the technology for use in airports, stadiums, schools and casinos as well as law enforcement.47 According to AlgorithmWatch, a research and lobby organization concerned with algorithms and AI, facial recognition is used by police forces in at least eleven European nations.48 However, the EU plans to introduce strict controls on its deployment in public places; the recently proposed Artifcial Intelligence Act would prohibit such use except where strong grounds exist in its favour.49

Facial recognition is by no means the only application of computer vision. It is also crucial for self-driving vehicles. Autonomous and semi-autonomous cars currently under development by Tesla, BMW, Volvo, Audi and Uber are equipped with multiple cameras that scan the surrounding space and recognize objects, road markings, traffc signs and traffc lights. Other applications of computer vision are intended primarily for monitoring of the physical environment. Examples include the detailed inspection of roads, bridges and machines with a view to facilitating prompt maintenance and the automated detection of vehicles and objects. In Amsterdam, for instance, cameras read the number plates of vehicles entering the city's low emissions zone and the details of any not entitled to be there are sent to the agency responsible for issuing and collecting traffc fnes. During the COVID-19 pandemic, computer vision has been utilized in various countries to scan public spaces for people who might not be respecting the rules on social distancing.50

As well as lending itself to applications in public space, computer vision has great potential for the agricultural and livestock sectors and the food industry. It can be used to monitor and harvest crops, for example, and also play an important role in so-called 'precision agriculture'.51 Computer vision is suitable for animal-welfare applications, too, with cameras used to monitor behaviour.52 Dutch start-up OneThird has developed a fruit-and-vegetable scanner that can accurately estimate their remaining shelf life by means of image recognition.53 Such information

45Chen, 12 October 2017; Simonite, 3 September 2019, 3. It was previously revealed that the Chinese government also uses facial recognition to trace and monitor Uyghurs, an Islamic minority group (Mozur, 14 April 2019).

46The digital rights lobby organization Fight for the Future maintains a map of all the places where the US government uses facial recognition technology (Fight for the Future, undated).

47Chiusi et al., 2020.

48Kayser-Bril, 11 December 2019.

49European Commission, 2021c.

50A 'one-and-a-half-metre monitor' was deployed in Amsterdam, for example (Amsterdam Algorithm Register, undated).

51Tian et al., 2020.

52Serket, undated.

53OneThird, undated.

facilitates better decision-making and thus helps minimize waste. When a consignment of tomatoes, say, arrives at a distribution centre, the decision might be taken to send them for immediate processing because it is possible to see that they will be unsaleable by the time they reach the shops.

Although progress is being made in this feld, the applications of computer vision are still often limited to specifc tasks in specifc domains. In clinical medicine it has proven relatively successful in the form of 'image-based diagnostics'54: images are scanned for particular irregularities that could indicate a disorder, helping radiologists, dermatologists and pathologists to detect and diagnose illness.55 Such successes have been aided by healthcare being a data-rich sector, and much of that data being visual, so there is ample material to train the algorithms.

Computer vision also has the potential to improve the quality of medical imagery and help surgeons perform operations. The US Food and Drug Administration (FDA, the agency responsible for regulating medical devices) has recently approved ten diagnostic tools based on the technology for use in hospitals.56 Computer visionenabled apps have also been developed so that people can check themselves for health issues, in most cases skin conditions; one is the Dutch SkinVision utility, another Google's recently unveiled Derm Assist. Users scan their own skin and are then given advice on any follow-up that may be appropriate.57 Although the medical world has been quite critical of such apps,58 the examples we give do illustrate how computer vision can be utilized in practice.

# *3.2.3 Natural Language Processing*

Our third general type of AI application automates the reading, analysis and generation of human language. The 'holy grail' of natural language processing is algorithms that can understand human language well enough to perform tasks requiring the interpretation of text. Language processing algorithms dissect sentences in various ways; for example, by distinguishing letters and words, labelling text elements and reading both left-to-right and right-to-left. This enables inferences to be made regarding the meaning of the text. Like computer vision, natural language processing has undergone a period of accelerated development in recent years, driven by

54A meta-analysis has revealed that the diagnostic performance of deep learning systems is similar to that of human medical professionals (Liu et al., 2019). However, the authors qualify that conclusion by saying that most of the studies included in the meta-analysis were not externally validated, meaning that the results did not support the general conclusion that AI was as good at image-based diagnosis as human doctors.

55Yu et al., 2018.

56Topol, 2019: 46.

57SkinVision, undated; Bui & Liu, 18 May 2021.

58Freeman et al., 2020.

advances in deep learning. Supported by sophisticated learning technology, the models can now be trained to understand human language more quickly and easily.

Because language is central to the way we communicate and how we gather, record and transfer knowledge, the potential applications of sophisticated natural language processing are enormous. In another parallel with computer vision, though, current systems are limited to specifc tasks that require relatively little actual understanding of text input. Examples include tools that auto-correct, autocomplete or check text as it is typed, as well as automated translation systems like Google Translate.59 Spam flters and search engines also make use of natural language processing. Google's search algorithm, for instance, applies two techniques when processing each query. First it links the words entered to relevant words in documents. The algorithm then ranks the various documents containing the words in question on the basis of assumed quality and relevance, as determined from the number of previous clicks on the page – a process known as 'page ranking'. This application of natural language processing has revolutionized the way we fnd information online. But it does not involve any true understanding of human language.

Another example is 'messenger bots', the automated chat systems that many organizations use for website-based customer support. Here AI helps provide customers with prompt, effcient assistance. In such applications, language processing is actually combined with expert systems: the algorithm analyses a question and, using a decision tree, selects the most appropriate reply or follow-up question. The Dutch police use such a chatbot to help people report internet fraud online; it checks that the report is complete, makes a preliminary appraisal of the case and advises the victim as to their best course of follow-up action.

# *3.2.4 Speech Recognition*

Speech recognition is the AI domain concerned with the detection, analysis and interpretation of spoken human language. It involves the use of algorithms to distinguish words and sentences in spoken language and convert them to text – speech-totext translation. One feld in which this kind of application is being tested is healthcare, where AI systems transcribe discussions between doctors and patients.60 A natural language processing tool then analyses the result, identifes important clinical information and produces a summary of the consultation – the aim being to reduce doctors' administrative workload and thus ultimately yield better consultation reports.61 The same technology can also work in reverse, converting text into speech. As, for example, when a device reads an e-book out loud, or a speech

59Lewis-Kraus, 14 December 2016.

60Van Buchem et al., 2021. The study shows that such digital scribe systems are very promising. However, they are not currently used in clinical practice anywhere, making it impossible to draw conclusions regarding their value in clinical settings. Ajami, 2016.

61Wouda and Hutink 2019.

computer acts as a voice for someone who cannot speak or has diffculty doing so (such as a patient with motor neurone disease).62

Voice-controlled smart assistants like Apple's Siri, Google Assistant, Microsoft's Cortana and Amazon's Alexa combine the two technologies described above to enable spoken communication between human and computer. After responding to 'wake words' such as 'Siri' or 'Alexa', the tools are able to perform all sorts of tasks: searching the internet, compiling to-do lists, playing music, making restaurant reservations and so on.  Speech recognition technology converts their speech into text, then natural language processing interprets the written information and determines what action is required.

Unlike people, who could speak and listen before they invented writing, computers fnd written language easier to process than the spoken word. Speech recognition is considerably more diffcult because of the variability of spoken language and the noise in audio streams; picking out the words, identifying them and converting them into a type of text the computer can process is extremely challenging. Nor is interpretation of the speech signals themselves straightforward. When we speak, the sounds we make are not separated into distinct words. What a computer hears is very like what a person hears when listening to a language they are totally unfamiliar with: a continuous stream of sound, with individual words very hard to distinguish. Yet telling them apart is essential if we are ever to translate those words into a language we understand.

The problem posed by speech recognition thus differs fundamentally from the interpretation of written language or images. Unlike computer vision and natural language processing, speech recognition involves the processing of a single input variable – sound waves – that changes dynamically over time. The great challenge is distinguishing words and sentences within this input, so that they can be translated into a language the algorithm is able to process.

A further challenge is that some of the meaning of speech is conveyed by changes in volume, cadence and tone – the characteristics of spoken language. Effective interpretation therefore depends on more than simply distinguishing words from one another. The phonetic aspects need to be detected and interpreted as well, in order to determine the meaning of what is being said. Another stumbling block is homophones: words that sound the same but mean different things, such as 'hour' and 'our' or 'air' and 'heir'. Their interpretation depends on the context: both the narrow context of the sentence and the wider context of the situation, the speaker and so on.

As in other domains, advances in machine learning have led to progress in the feld of speech recognition since it has become possible to process much greater volumes of speech data to train the algorithms. Relatively successful practical applications of speech-to-text and text-to-speech conversion are now viable, providing

62 In collaboration with Google, DeepMind is currently working on a project to develop a text-tospeech program that would enable someone with a speech impediment to speak with their own voice via a computer (Chen et al., 18 December 2019).

that the speech is clear in both auditory and content terms. However, much spoken communication is unclear in one or both of these respects. Consequently, speech recognition technology has not yet reached the stage where it can be used reliably on a wide scale and for a range of purposes. To a large extent this is attributable to the limitations of natural language processing and AI's ability to truly understand language. Although it has made the transition from lab to society, AI is still far from being a mature technology – a point we return to in the fnal section of this chapter.

# *3.2.5 Robotics*

In this report the term 'robotics' is applied to the type of AI used in combination with robots. Robotics brings together all types of AI: the ability to reason and learn, to see and hear, to communicate and to understand. However, it differs from other AI disciplines in that it additionally involves physical processing: the ability to manipulate objects.

A robot needs to be able to move and undertake physical actions to perform tasks. Those may be so-called 'dull, dirty, dangerous and dear jobs' or activities in which robots can outperform people. Examples include space exploration, the clean-up operations following the nuclear accident at Fukushima and defusing bombs.63 However, robotics are important as well in the context of innovations in healthcare, retailing, manufacturing, livestock husbandry, agriculture and horticulture. Autonomous vehicles can also be regarded as a form of robotics. Robots thus come in countless shapes and sizes, making a precise defnition of the word very diffcult. Joseph Engelberger, a pioneer in the feld of industrial robotics, addressed that challenge with a variation on the classic one-liner often used in respect of familiar but undefnable things: "I can't defne a robot, but I know one when I see one."

In classic robotics, expert systems play an important role. They are particularly suitable for standardized tasks in situations where a choice needs to be made from a number of predefned courses of action. For that reason, robots are currently used mainly in controlled manufacturing and port environments. Deploying them in highly dynamic and often chaotic everyday human settings, such as on the roads, involves far more complex challenges. Coping with the variety and spontaneity of such situations requires a degree of understanding of how the world works; the robot needs to be able to observe its surroundings, assess situations, predict plausible future scenarios and decide, in a dynamic setting, which of all the possible courses of action is most appropriate for the circumstances.64 A system fexible enough to operate in the world outside the lab must therefore be underpinned by such understanding.

63One example being the PackBot developed by Endeavor Robotics (previously iRobot), part of fir ugs (unmanned ground systems), which supplies robot systems to the US military (fir ugs, undated).

64Marcus & Davis, 2019: 113.

Because a robot of this kind would have to cope with the open-ended nature of our world, in which the possibilities are endless, it is crucial that it incorporate a wide range of capabilities. At present, though, the limitations of the other forms of AI effectively restrict the practical potential of this kind of robotics. For example, robots currently fnd it diffcult to pick up a dark handkerchief from a dark table of their own accord, because computer vision is not yet suffciently sensitive to light. Progress in the other branches of AI and advances in machine learning are therefore vital to the further development of robotics. Although the hardware and human control aspects are already quite impressive, everyday tasks require extremely refned motor control, planning and perceptual capability. While a commercial glasshouse may seem an orderly environment, for instance, it is still extremely diffcult for a robot to pick a tomato without squashing it.

Three big players in the feld of robotics are Boston Dynamics, which specializes in the simulation of human movement by robots ('humanoids'), DJI, a specialist manufacturer of drones for consumer use, and Amazon Robotics, with a focus on automated logistics. Amazon develops and deploys robots capable of effciently navigating large warehouses and thus optimizing sorting processes. To do that the machines have to make allowances for and co-operate with one another, which they manage very successfully in facilities they are deployed in. Amazon's sorting robots perform specifc tasks and operate in environments that are predictable and surveyable – for robots, at least.

By contrast, Boston Dynamics is aiming to develop robots that are far more fexible both physically and 'mentally', enabling them to be used for a variety of purposes. Previously owned by Alphabet (Google) but sold to Japanese technology giant SoftBank in 2017, Boston Dynamics is well known from its impressive video footage of two and four-legged robots such as Atlas and BigDog. They can stand and move around in ways that closely resemble the locomotion of people and animals. However, the company has yet to develop any commercial products. But Chinese technology frm DJI does already operate commercially; in fact, it is the global market leader in unmanned aircraft (drones) for aerial photography and video applications.

As the real-world examples above illustrate, AI is making its presence felt in society through robotics, speech recognition, natural language processing, computer vision and machine learning. Indeed, its practical applications are now so numerous and varied that it is impossible to compile a comprehensive overview. Nevertheless, the appendix to this report lists examples of AI applications in various sectors of the economy, primarily to provide an impression of their huge breadth and diversity. The simple observation that AI is today utilized in many different ways and contexts emphasizes the extent to which it is now becoming established within society. That process is not inconsequential. In the next section we consider the societal dynamics set in motion by AI's transition from the lab to society.

## **Key Points – The Practical Application of AI**

# **3.3 AI as a Phenomenon in Society**

AI's transformation from something researchers investigate into something used in everyday life has clear repercussions for society as a whole. Moving out of the lab inevitably implies moving into the public arena. The world AI has now entered is one of divergent interests and forces, and its arrival there has triggered investment, experimentation, discussion and alarm. Visions and strategies to utilize AI to maximum effect have been published, but also open letters and reports calling for its regulation. In short, the appearance of this new technology is making waves within society. As well as evolving technologically, AI is developing as a societal phenomenon (see Fig. 3.3). Although that process is still very much in progress, we can already discern a number of trends. To aid understanding of the current situation, in this section we consider society's various responses to the arrival of AI and the shifting emphasis in them.

# *3.3.1 Interest in AI as a Revolutionary Technology*

The scientifc blossoming of artifcial intelligence after the most recent 'AI winter' focused attention on the technology's countless potential applications. That led to the appearance of several iconic books on its future, which often present the latest advances as the beginning of a new era. Visionary Ray Kurweil speculates on an imminent 'singularity' in which human and computer intelligence merge to form a

**Fig. 3.3** AI's development as a societal phenomenon, per selected indicators

single superintelligent entity.65 Scientists Erik Brynjolfsson and Andrew McAfee place AI at the heart of 'the second machine age', in which machines relieve humans not only of physical tasks but also cognitive ones.66 Philosopher Nick Bostrom views such developments as a serious threat, however: as AI becomes cleverer and faster than us, it becomes hard for humanity to maintain control over it.67 Another philosopher, Luciano Floridi, refers to a 'fourth revolution' in which digital technologies like AI fundamentally change our world view and our understanding of ourselves.68 Klaus Schwab, fnancial backer and chair of the World Economic Forum, talks of a 'fourth industrial revolution' when the application of smart technology transforms the way we work and live, just as the steam engine, electricity and digitalization did previously.69

A future in which human life is closely intertwined with AI is also a popular theme for the flm industry. In parallel with the recent 'AI spring', movies such as *Her* (2013), *Ex Machina* (2014) and *Transcendence* (2014) depict a future where AI reaches a critical threshold of intellectual ability. As such, these flms serve as a form of 'scenario thinking': they portray imagined situations in which humans have emotional relationships with AI and can even fall in love with it (*Her*), in which AI can pass the ultimate Turing test and become so like us that it is no longer possible to distinguish between human and machine (*Ex Machina*) or in which AI becomes a dangerous, barely controllable source of power (*Transcendence*). Although the

65Kurzweil, 2005.

66Brynjolfsson & McAfee, 2014.

67Bostrom, 2016.

68Floridi, 2014.

69Schwab, 2016.

idea of the superintelligent computer has long been a source of inspiration for screen writers, these recent productions have made the specifc term 'AI' familiar to the general public.

# *3.3.2 Applied Research and the Run on Talent*

Besides compelling screen depictions, what has mainly stimulated public interest in new AI technologies is their practical potential. That has also made AI economically attractive for the business community and governments. We have already described how private investment in AI has increased considerably all around the world. In addition, businesses and governments have teamed up with research institutes to set up special 'AI labs' where links are forged between fundamental science and practical requirements. In many parts of the world laboratories of this kind have been established to cater for particular sectors of the economy, ranging from agriculture and mobility to retail and manufacturing, healthcare and education to public administration. Others are addressing the societal aspects of AI; these are often known as ELSI (ethical, legal and social implications) or ELSA (ethical, legal and societal aspects) labs.

The frst applied research facility in the Netherlands devoted to AI began life in 2015. That was the QUVA Deep Vision Lab, a joint initiative by the University of Amsterdam and Qualcomm dedicated to translating computer vision research into industrial applications. Similar projects proliferated in the years that followed, with the Innovation Center for Artifcial Intelligence (ICAI) founded in 2018 by the University of Amsterdam and VU Amsterdam playing an important co-ordinating and supporting role. The Netherlands now has twenty ICAI labs, where companies including Bosch, TomTom, KPN, ING, Ahold-Delhaize and DSM, as well as hospitals, the national police and government bodies, collaborate with universities and research centres to develop innovative AI solutions.

With businesses also exploring AI's potential in many different felds, and developing applications for them, an enormous demand has arisen for talent in this domain. That in turn has sparked debate in various countries as to how best to nurture and retain people with the necessary skills.70 At the beginning of this chapter we pointed out that most AI-related PhD graduates in the US are now choosing careers in industry. Other countries are experiencing a similar 'brain drain' from academia to the business community, but in many cases with their trained specialists moving abroad to boot.71 In Europe, prominent scientists from more than twenty countries have written an open letter on the subject to sound the alarm and call on policymakers to invest in the European research climate.72

70Hoeks, 12 April 2019; Delcker, 27 June 2018; Boland, 2 September 2018.

71Elsevier, 2018.

72ELLIS, 2018.

# *3.3.3 AI Action Plans*

The focus on the potential of AI is also refected in a proliferation of national and international AI strategies. Most such documents deal primarily with the economic opportunities, often within those sectors already important for the countries in question.73 The OECD observes that the goal of national strategies is usually to boost national productivity and competitiveness by harnessing AI.74 They are therefore concerned primarily with the development and utilization of AI through mechanisms like research funding, enhanced support infrastructures and encouraging business interest. For the same reason the development and retention of talent is an important feature of many strategies.75

Although the main thrust of an AI strategy is typically the defnition of an innovation agenda, many additionally address societal and ethical aspects. However, the passages devoted to these points are often subordinate to the economic plans and usually less substantive and action-oriented. In Europe the rationale for discrepancies of that kind tends to be that, in order to align AI with our values, we need to be in the technological vanguard.

The Dutch think tank DenkWerk produced a report entitled *AI in Nederland* (AI in the Netherlands) in 2018. This stressed the urgent need for the country to commit seriously to artifcial intelligence, arguing that it was being left behind in terms of investment in the private sector and other forms of government support. DenkWerk pointed to the 'enormous societal potential' of AI and called on the government to formulate a national agenda for its development and application. The report urged immediate action, saying, "This is not a matter that should frst be considered for two years".76 That same year DenkWerk helped to initiate work on a national AI agenda. AiNed, a coalition of corporate, academic and government partners, then published a report substantiating the earlier call for urgent action. That argued that AI should be made a national priority to protect and enhance the nation's prosperity and international status. With a view to accelerating the development of AI and differentiating the Netherlands on the global stage, AiNed formulated several objectives as the basis for a national strategy.

The strategy was eventually published in autumn 2019. Following the release of the AiNed report, a task force was formed. Led by employers' confederation VNO-NCW, this also involved the Ministry of Economic Affairs and Climate Policy, which assumed responsibility for realizing the objectives formulated. The frst major practical step was to create a Dutch AI Coalition, a platform for collaboration between businesses, government bodies, non-governmental organizations and research institutes to catalyse AI development. The coalition quickly announced its intention to promote the formation of AI labs and to work with the government to

73Mols, 2019.

74OECD, 2019: 121.

75Mols, 2019.

76DenkWerk, 2018.

develop an AI strategy. Another outcome is the Strategic Action Plan for AI (SAPAI), presented by the Ministry of Economic Affairs and Climate Policy with the support of the ministries of Justice and Security, the Interior and Kingdom Relations, Social Affairs and Employment and Education, Culture and Science.

In this plan the government sets out a pathway for the period ahead and describes the frst practical initiatives to accelerate AI development and raise the Netherlands' profle in this feld. The SAPAI defnes a three-track policy. Track 1 relates to utilization of the societal and economic opportunities offered by AI, track 2 to creation of a conducive ecosystem and track 3 to safeguards. Inclusion of the third track refects how the debate has shifted, with attention focusing not only on the economic opportunities afforded by AI but also increasingly on the impact of its applications. The SAPAI was presented together with documents devoted to 'AI, civic values and human rights'77 and 'Safeguards against risks associated with government data analyses'.78

The private investment, the formation of AI labs and the launch of national AI strategies are indicative of the increased interest in AI outside the scientifc community. Much of that focuses on the potential of AI. There is growing awareness that it has now reached a certain level of maturity and so the time has come for the appropriate actors to realize its potential. AI is on the agenda, particularly the economic agenda. Stories about new applications and the doors they can open appear regularly in the media. As a result, the general public has become aware of the technology. Many may not understand quite what AI is, but they know of its existence.

# *3.3.4 Interest in the Practical Effects of AI*

As AI has become the focus of increased attention, questions about its impact have arisen. The technology's introduction to the real world has led people to consider its implications for everyday life. In some recent books the emphasis has shifted from the revolutionary nature of AI to the possible consequences of its use in real life. Various authors have highlighted potential problems associated with its transition from the lab to the mainstream. Moral, societal, political, legal and economic issues have all been raised. AI's effect on society and its core values has become a matter of public debate.

In her book *Weapons of Math Destruction* (2017), Cathy O'Neil warns of the harmful effects that the careless and short-sighted use of algorithms can have on people's lives.79 Meredith Broussard has a similar message, coining the term 'technochauvinism' to describe how mankind can be insidiously degraded by the idea

77*Kamerstukken II* 2019/20, 26 643, no. 642.

78*Kamerstukken II* 2019/20, 26 643, no. 641.

79O'Neil, 2016.

that technology is capable of meeting every human need.80 Shoshana Zuboff has also expressed concerns, but about the actors controlling the technology rather than the technology itself. She warns of 'surveillance capitalism', an economic philosophy based on excessive data gathering and use of predictive algorithms, allowing big tech companies to exercise unprecedented infuence over our behaviour. In *The Algorithmic Society* (2020), various authors highlight the association between data, algorithms and power and describe how that association can distort the relationship between citizen and state. In his most recent work even Stuart Russell, author of the defnitive AI handbook, expresses concern about AI's effect on the real world. A system that works well in a technical sense may nonetheless have undesirable effects, Russell writes. He argues that this makes it important to keep AI permanently under control: "What's worse than a society-destroying AI? A societydestroying AI that won't switch off."81

Living with AI has become an important theme. In 2016 the World Economic Forum was devoted to designing a world of smart technologies such as AI. In the same year G7 IT ministers agreed with the OECD that international talks should be held regarding the development of AI and its economic and societal implications. The OECD has since been increasingly active in this feld, organizing conferences and encouraging international policy discussions. In 2019 the organization presented its principles for AI, identifying the technology's effect on people and society as an important theme and setting out a framework for responsible further development. All OECD member states, the G20 and ten other countries have endorsed the principles, which have thus become the frst intergovernmental guidelines on AI.

# *3.3.5 Social Organizations Become Involved*

Attention is shifting from AI's economic impact to its societal impact. At the international level, UNESCO has also started taking an interest. In 2018 an entire edition of the organization's magazine, *The UNESCO Courier*, was devoted to the opportunities and threats society associates with AI. In her contribution, Director-General Audrey Azoulay stressed the importance of an ethical debate on AI. Unsurprisingly, she saw a role here for UNESCO: "It is our responsibility … to enter this new era with our eyes wide open."82 Her organization is seeking to discharge that responsibility by working on a global ethical standard for AI, to serve as a basis for the development of national policy.83 The European Commission has meanwhile established the High-Level Expert Group on AI (AI HLEG), a body charged with

80Broussard, 2019.

81Sample, 24 October 2019.

82Šopova, 2018.

83The draft of this document was presented in 2020 (Ad Hoc Expert Group, 7 September 2020).

providing European governments with an ethical framework for the technology. Following the publication of the European strategy for the development of AI, responsible development in which the effects of AI are taken into account is now also on the agenda. The AI HLEG's guidelines and recommendations for 'trustworthy AI' are intended to promote awareness among policymakers of the ethical and societal aspects of AI and to provide a framework for managing them.84

Concepts like 'human-centric' AI and ethical, humane and responsible AI are being mentioned with increasing frequency, indicating growing interest in the relationship between AI applications and human society and values. That trend is also refected in the proliferation of publications about AI and ethics.85 Research institutes devoted specifcally to the societal implications of AI have been established too. Even in Silicon Valley, the crucible of AI's technological development, the Stanford Institute for Human-Centered AI has been set up specifcally to investigate the technology's human and societal impact.

The AI Now Institute, also in the US, is perhaps the most prominent example of a research centre concerned with the societal effects of AI. Its annual reports serve as important catalysts of worldwide debate. Since the frst appeared in 2016, AI Now's messaging has become increasingly clear: having initially called for research into the effects of AI, the institute has moved on to arguing that certain applications should be prohibited, sometimes at least provisionally, and to setting out specifc requirements for the responsible use of the technology.

The changing tone of these recommendations illustrates how the public debate on AI has developed in recent years: from promoting awareness of its effects to a substantive discourse about how certain values can be impacted and protected. This trend is apparent in the Netherlands too. The Rathenau Institute – the Netherlands Organization for Technology Assessment – began by advancing the cause of public debate regarding the impact of digital technologies such as AI, but more recently has become an active contributor to the discussion on how that impact should be managed. As experience and research have made the practical effects of new technologies clearer, so frmer ideas have emerged as to how undesirable effects should be countered. The debate many commentators were advocating a few years ago is actually happening today and become increasingly substantive.

# *3.3.6 Sectoral Interest in AI*

Growing recognition of AI's practical potential has attracted attention from research centres and consultancies concerned primarily with non-technological felds. Having previously thought of AI as part of the general issue of digitalization, organizations active in such domains as education, healthcare, security, infrastructure

84High-Level Expert Group on Artifcial Intelligence, 2019a, c.

85Zhang et al., 2021: 130.

and the law have in recent years turned their attention to the specifc question of AI's implications for their disciplines.

Since 2018, various sectoral bodies in the Netherlands have published studies and advisory reports addressing the signifcance of AI for their particular domains. The Advisory Council on International Affairs (Adviesraad Internationale Vraagstukken, AIV) and the Advisory Committee Public International Law (Commissie van Advies inzake Volkenrechtelijke Vraagstukken, CAVV) have reported on the military applications of AI,86 while the Netherlands Environmental Assessment Agency (Planbureau voor de Leefomgeving, PBL) has considered what smart algorithms could mean for mobility,87 the Netherlands Centre for Ethics and Health (Centrum voor Ethiek en Gezondheid, CEG)88 and the Council for Public Health and Society (Raad voor Volksgezondheid en Samenleving, RVS)89 have explored the implications of AI in the healthcare sector and Dialogic was commissioned by the Ministry of Education, Culture and Science to investigate AI's impact on education.90

These explorations and recommendations add texture to the AI-debate: the different contexts in which AI can be applied, demonstrate the breadth and diversity of its prospective impact. Meanwhile, more experience is being gathered with the deployment of AI and this reveals the diffculties and risks in the step towards its practical application. Examples of discrimination by algorithms, accidents with self-driving cars and the disappearance of the human dimension through excessive 'algorithmization' give food for thought for the ways in which we want to integrate AI into society.

# *3.3.7 The Dark Side of AI*

Paralleling the growing sector-specifc interest in AI, more attention has been paid to the dark side of the technology. Examples of algorithmic discrimination, accidents involving self-driving cars and dehumanization associated with excessive reliance on technology have prompted people to refect on how they want AI integrated into society. In Europe, the EU's Agency for Fundamental Rights (FRA) is investigating potential implications in its feld. The role of AI in the development of autonomous weapons, the use of facial recognition by local authorities and police forces and the status of 'big tech' have all become topics of public concern. The potentially harmful side of AI is starting to dominate debate.

86AIV and CAVV, 2015.

87PBL, 2017.

88CEG, 2018.

89RVS, 2019.

90Van der Vorst et al., 2019.

Moreover, the risks posed by AI and its dual-use potential, combined with the speed at which the technology is currently evolving, are adding a degree of urgency to the debate: if its increasing use is not to have undesirable consequences, not only do we need a clear picture of the risks but we also have to respond accordingly. Various US states and cities have now prohibited the use of facial recognition by the police and in public places.91 The European Commission's draft Artifcial Intelligence Act seeks to do the same, except in special circumstances such as where compelling security considerations exist. Both the US and Europe have for some time been looking at possible ways to curb the burgeoning power of big tech corporations through competition law.

Campaign groups and lobby organizations are taking up AI-related causes as well. They include, frstly, groups dedicated to addressing problems of a particular kind – privacy or digitalization issues, for example – that have developed an interest in potential abuses associated with the use of AI. In Europe, EDRi (European Digital Rights) and other groups dedicated to protecting rights and freedoms in the digital environment are now concerning themselves with AI. Secondly, we are now seeing groups dedicated specifcally to AI-related issues. They include AlgorithmWatch in Germany, which systematically surveys and critically evaluates the international use of algorithmic systems. Another development is that some major human-rights organizations, such as Amnesty International, Hivos, Human Rights Watch and UNICEF, have also started to take an interest.92 In short, there is a growing movement within civil society concerned with the negative effects of AI.

# *3.3.8 On the Policy Agenda*

Within government too, AI-related issues are commanding greater attention. In part as one aspect of the wider debate about digitalization and privacy, as recognized by the research institutes and advisory councils. In the early 2010s discussion of digitalization was dominated by questions relating to big data and privacy. At that time the EU was working on the General Data Protection Regulation (GDPR) with a view to providing a legal framework to enhance the protection of personal data, particularly in the digital domain.

The focus on privacy gave rise to interest in transparency as well, another principle prominent in the GDPR, and both fgured in the political debate regarding AI from the outset. Alongside the more general debate regarding digitalization, the discourse around big data is gradually transforming into a discussion about how that is processed using ever more intelligent algorithms. In its advisory report *Big Data*

91 In the states of California, Oregon and New Hampshire, the use of biometric surveillance technology by the police is against the law, while all use of facial recognition in public places is prohibited in the cities of Portland, San Francisco and Oakland.

92Since 2016 Amnesty has had a separate department, Amnesty Tech, dedicated to digital technology matters including AI.

*in a Free and Secure Society*, the WRR has already highlighted the crucial role that algorithms play in big-data processes.93

Since 2018, however, the public debate regarding AI has broadened discernibly. At the European level that was the year in which the AI HLEG was set up. It went on to publish a set of *Ethical Guidelines for Trustworthy AI* (2019), which introduced such concepts as unfair bias, accountability and welfare to the discussion. Its effect has been to raise the profle of issues of discrimination and human control. The latter became an important principle in the European White Paper on AI (2020) and the subsequent draft Artifcial Intelligence Act (2021), which is considered in more detail in Chap. 7.

To begin with, however, it is the immediate challenges associated with AI applications that command most attention. In response, efforts are being made to fnd practical means to address those challenges. One idea that is regularly foated is the establishment of an 'AI authority' or 'algorithm watchdog' to supervise the use of artifcial intelligence. Other efforts to manage its direct effects include the development of standards for AI applications by organizations such as CEN-CENELEC at the European level and the ISO at the global level, as well as the ongoing legislative initiatives.

Meanwhile, a second broadening of the public debate is now discernible. There is interest not only in AI's implications for public values in particular contexts, but also increasingly in its impact on society as a whole. The work of the Council of Europe's Ad Hoc Committee on Artifcial Intelligence (CAHAI) illustrates this trend; it takes a broad interest in AI's relationship with human rights, democracy and the rule of law. The number of governments commissioning research into its societal impact and convening advisory committees to provide wide-ranging policysupport information is further evidence of a growing recognition that AI has potential implications for all aspects of society and therefore requires structural attention.

# *3.3.9 Ethics*

As awareness of the effects of AI has grown, ethics have become an important feature of the debate in recent years. Governments and private-sector actors have developed ethical codes and guidelines on the responsible use of AI, and university technology programmes have been adding ethics modules to their curriculums.94 In both technical and social studies, increasing interest in the wider relationship

93WRR, 2016.

94Ethics is now one of the research domains within Delft University of Technology's AI programme, while Eindhoven University of Technology has made Ethics a compulsory module. The University of Amsterdam includes a module entitled Fairness, Accountability, Confdentiality & Transparency in its programme.

between AI and society has become evident.95 Alongside their technology-based AI professorships, several Dutch universities have recently created chairs covering its societal and community aspects.

More systemic study of the implications of AI is coinciding with the emergence of a degree of 'ethics fatigue'. Although in practice many things covered by that term have little to do with ethics, there is growing dissatisfaction with the plethora of codes and guidelines that AI is expected to comply with. These often fail to refect the complexity of the feld in practice and provide an inadequate framework to prevent abuses and undesirable developments. It seems that more structural safeguards are required to ensure that AI is aligned with our common values, and that is shifting attention beyond its actual applications to the broader dynamics of its integration into society.

# *3.3.10 Interest in the Societal Integration of AI*

We are currently at a stage where there is widespread interest in AI as a multifunctional technology with great economic potential. It has also become clear that its use will have a transformative effect on established practices and could lead to undesirable situations. Until recently most attention focused on the short term and much of it on specifc values. However, the scope of the debate has now broadened to encompass AI's effects in a variety of domains and its impact on a wider range of values. Whereas the debate initially related mainly to matters of privacy, transparency and human control, there is now also interest in how AI affects other values, such as sustainability (see Box 3.2).

The Dutch government's request to the WRR for advice on AI, which led to this report, was signed by nine different ministers, indicating how its impact is relevant to all areas of policy and has the potential to affect all their core values. Interest in the effect of AI in particular contexts has effectively coalesced into an interest in its impact on society as a whole.

Since AI frst appeared on the public agenda as a revolutionary technology, its effects and especially its risks have gradually become more important topics of debate. Now that it is being put to practical use in more and more spheres, and is set to fnd even wider applications in the future, interest in its impact is becoming more structural: as we develop AI, how can we safeguard the things that we value as a society, our civic values? To answer this question, we must look beyond AI's immediate effects and consider the longer term. Safeguarding civic values depends not only on the robustness of our technical systems, but also on the structure of society itself.

95The AI programmes at Utrecht University and VU Amsterdam include modules entitled, respectively, Philosophy of AI and AI & Society. The Faculty of Social Sciences at Maastricht University now offers a Bachelor's degree programme entitled Digital Society, which explores the societal impact of digital technologies like AI.

## **Box 3.2: AI and Sustainability**

As AI enters everyday life, there is increasing interest in its impact on society. Issues concerning privacy, equal treatment, autonomy and security have become the focus of growing debate. Another pertinent subject – with a societal and political profle that has so far been quite low, but now attracting more and more attention from researchers – is the effect on sustainability.

There is an optimistic school of thought that AI can make a substantial contribution to enhancing sustainability. The UN's annual AI for Good conference is devoted to topics like ecological objectives in relation to AI. Furthermore, we are now seeing numerous initiatives through which AI is indeed adding substantively to sustainability. The best known are projects to make more effcient use of energy and improve wind and solar energy forecasting. But AI is also being used for smart farming. Amsterdam-based startup Connecterra uses Google algorithms in livestock husbandry, for example.96 There have also been interesting initiatives in nature conservation. One is eBird's use of machine learning algorithms in ornithology and utilization of the output data for bird protection. Another example is the use of AI by Global Fishing Watch for population monitoring. Finally, the EU's Destination Earth initiative (DestinE), for which the use of AI is also envisaged, should not be overlooked.97

On the other hand, there is a growing body of evidence that AI can have negative impacts on sustainability. The CO2 footprint of the global computing infrastructure is already greater than that of the aviation industry at its zenith. Running a single natural language processing algorithm is associated with emissions equivalent to 125 return fights between New York and Beijing.98 Furthermore, AI is being used to maximize fossil energy production and to promote non-sustainable consumption.

Peter Dauvergne has claimed that for every example of AI having a positive impact on sustainability, there are multiple cases of negative impacts. He attributes that to the wider political economy and the power structures associated with the technology. As long as the landscape is characterized by a commercial logic focused on exploitation, AI will not have a positive net effect on sustainability.99 Achieving its potential in this area, he argues, will require changes to power structures, to the actors using AI and to the purposes for which the technology is used.

96Dauverge, 2020.

97European Commission, undated (n.d.).

98Crawford, 2021.

99Dauverge, 2020.

At this point in the development of AI we face the challenge of determining what is needed to achieve the technology's structural integration within society. Before considering the various aspects of that issue, it is important frst to consider the role the lab will continue to play in the process.

## **Key Points – AI as a Phenomenon in Society**

# **3.4 The Future of the Lab**

We have seen how AI has moved out of the lab and become a feature of society in many different respects, in the form of numerous applications and wide-ranging public debate. What implications do such developments have for the future of the lab? We should not expect that now AI has established itself within society, the lab's signifcance or dynamism will decline – that would be a mistake. The transition from lab to society does not imply that AI is a perfected technology requiring no further development, or that from now on attention should focus solely on its applications.100 Rather, the breadth of the lab-to-society transition implies that a wider range of issues related to AI's societal integration will warrant attention, as we outline in the next chapter.

Despite all the activity in the application sphere, lab development remains vital for at least two reasons. The frst is that, despite all the advances made in recent years and the innovations they have enabled, AI still has signifcant limitations. The

100That was suggested by computer scientist Kai-Fu Lee (Lee, 2018: 143). Of course, he recognizes that AI can develop further, but he believes that the discoveries made in recent years are so signifcant that it is unlikely that equally important breakthroughs will emerge in the near future. For example, he suggests that, since Geoffrey Hinton's important paper, a decade has passed without any equally revolutionary advances in machine learning. He argues that applications therefore warrant more attention than fundamental research. That is not the message of this report, however.

current methods offer no answer to a variety of questions. People are already talking about the limits to the capabilities of deep learning, for example. While it is impossible to say whether this will lead to a third 'AI winter', it is clear that the technology still has a long way to go, and that signifcant progress will require further fundamental research. The second reason for the lab's continuing importance relates to the particular nature of AI. It is a form of technology in whose application the lab must remain involved. Strictly speaking, then, AI has not left the lab but extended beyond it.

# *3.4.1 The Need for Fundamental Research*

Various experts have made the point that access to more and better data is the key to overcoming many of the current limitations to machine learning. Furthermore, interesting developments are taking place in this particular feld, like the use of generative adversarial networks (GANs) in which multiple algorithms are used to improve one another. One algorithm generates something new, such as an image of a bird, and in response the other algorithm indicates whether it recognizes it as a bird or not. If not, the frst algorithm continues refning the image until the second one is 'convinced'.

Ray Kurzweil believes that such simulation methods can resolve many of the problems associated with data shortages. For example, rather than self-driving cars having to learn in real traffc, with all the attendant dangers, they could travel millions of kilometres in simulated worlds without putting anyone at risk.101 Similarly, defence robots could be trained within a simulation so that they are more advanced prior to deployment in the real world. Another promising approach is federated learning, where data to train a machine learning algorithm is not loaded onto a central server but algorithms are refned by adjusting their parameters with those from other datasets, without combining the actual data. This approach is particularly suited for use with privacy-sensitive data such as hospital records.

Despite such developments, scientists believe that innovation remains necessary, partly because machine learning appears to have inherent limitations. For example, progress is needed in the feld of computer vision if it is ultimately to be used for autonomous vehicles or security applications. Now its algorithms are relatively easy to fool, as demonstrated by experiments showing that tiny traffc signs too small for the human eye to see were treated like the real thing by self-driving cars. Current algorithms look for patterns, so if a minute sign closely matches the pattern sought then it will be interpreted with confdence as a real one. Its abnormal dimensions are not noticed by the existing algorithms.

101From an interview with Ray Kurzweil (Ford, 2018: 230).

This makes such algorithms vulnerable to adversarial attacks – with potentially disastrous consequences in the case of a self-driving car. A recent study demonstrated that changing a single pixel in an image can confuse an AI algorithm. The military use of AI is another feld where vulnerabilities like these can have serious repercussions; it has been shown, for instance, that an image classifer can be fooled into identifying a machine gun as a helicopter.102 The same attack strategy could be deployed for other purposes too. Google uses an algorithm to classify videos for the protection of intellectual property rights and so on. Researchers at the University of Washington showed that this could be tricked by inserting random images into a video for fractions of a second.103 In an incident in the US, a police offcer who was being flmed started playing music, presumably in the belief that YouTube's algorithms would prevent the video being shared on intellectual property rights grounds.104

# *3.4.2 Superfcial and Ineffcient*

Numerous other shortcomings with machine learning illustrate that a great deal of lab work is still required, as AI pioneers have themselves acknowledged. Yoshua Bengio has argued that deep neural networks can learn superfcial statistical regularities from datasets, but not higher abstract concepts. They therefore lack the type of understanding needed for certain tasks and forms of communication. Geoffrey Hinton and Demis Hassabis, founder of DeepMind, have both stated that general artifcial intelligence is currently nowhere near being a workable reality.105

Pioneer Hinton is critical of current methods and has highlighted various shortcomings.106 One is ineffciency. Machine learning is more like human learning than earlier technologies were. For example, images are recognized by identifying patterns rather than by following fxed rules. In that respect the machine and human learning processes are alike. Nevertheless, major differences also exist, and humans are still able to learn far more effciently. A small child only needs to see a few apples to acquire the ability to recognize apples in the future. By contrast, machine learning algorithms need to be shown thousands of images of apples before they are trained to identify the fruit. Furthermore, while the volume of data available globally to train algorithms is increasing, the situation varies from domain to domain; many still have a shortage. Another problem is that, for certain applications, high error rates during the training phase entail serious dangers.

102Tonin, 2019: 6.

103Agrawal et al., 2018: 200.

104Thomas, 9 February 2021.

105Marcus & Davis, 2019: 62.

106Dickson 2 March 2020.

# *3.4.3 Common Sense*

A related problem is AI's lack of common sense. The earlier example of the tiny road signs perfectly illustrates this. Current algorithms are designed to be capable of processing all possible images and therefore cannot distinguish between plausible and implausible contexts. Although they recognize patterns, they are not good at ascribing signifcance to them. CAPTCHAs – completely automated public Turing tests to tell computers and humans apart – are a good example. They are the tests you come across on the internet that ask you to prove you are not a robot by, for example, selecting all the photos in a group that include trees. Passing requires common sense. Even if a picture includes only a small part of a tree, a human can usually see what it is by drawing conclusions from the surrounding objects, such as bushes.

For an algorithm, however, the limited number of data points available as a basis for recognition is usually problematic. CAPTCHAs therefore reveal the current limitations of machine learning in situations that call for common sense. Machine learning algorithms have no access to the collective knowledge acquired elsewhere by other programs. They therefore have trouble answering questions that humans can answer without hesitation, like "Who is taller, Prince William or his young son Prince George?" or "If you stick a pin into a carrot, will you make a hole in the pin or the carrot?"107

Humans answer such questions by drawing on a large pool of implicit knowledge. When we speak, we do not provide all the relevant information because we assume that the listener will make deductions based on the context. If someone instructs a taxi driver to take them to the airport as quickly as possible, the driver knows that they are not expected to drive without regard for the rules of the road or the safety of other road users. An algorithm, however, lacks that kind of implicit background knowledge. In other words, the language is underspecifed, and no fact exists in isolation.108 Stuart Russell cites the example of progress in the feld of physics. By analysing data from telescopes, an algorithm can develop new knowledge. However, progress depends on more than merely studying additional data. The formulation of hypotheses and the selection of factors for inclusion from the universal data pool rely on prior knowledge of physics, which does not exist in a form an algorithm can process.109

107Marcus, 2018: 12.

108Marcus & Davis, 2019: 136–139.

109Russell, 2019: 83.

# *3.4.4 Lack of Transparency*

Another shortcoming is that current machine learning algorithms lack transparency, which often makes it extremely diffcult to ascertain how they come to a given conclusion. In many cases the decision-making process can be uncovered, but that does not necessarily imply that it is explainable: knowledge is not the same as understanding. A decision to classify something based on a pixel-level detail is unfathomable for humans, for example. In many cases this is not a great problem. If the algorithm's decision relates to something like a security risk, a mortgage application or a medical diagnosis, however, opacity has serious implications. In such contexts, explainability is therefore a requirement.

Some experts believe that the complexity of the algorithms needed for applications of this kind does not present an insurmountable problem. Hassabis, for example, takes the view that we are currently building the systems and that the construction phase will be followed by a process of reverse engineering aimed at understanding how they actually work. He therefore believes that, within a decade, most systems will no longer be black boxes.110 Yann LeCun suggests that it is as if we are still in the process of inventing the internal combustion engine but are already worrying about brakes and seatbelts. Such problems can be addressed at a later stage, he argues.111

Other experts see poor explainability as inherent to the technology, making a different approach necessary. According to Judea Pearl, human knowledge is expanded not by a blind process but by building and testing models of reality. Current machine learning approaches are limited because they focus on correlation, not causality.112 He draws an analogy with the difference between Babylonian and Greek astronomy. While the Babylonians were able to make very accurate predictions, better than the later Greeks, the process they used was unreproducible – a black box – and the mechanisms underpinning the predictions were not understood. The Greek approach was based on understanding those mechanisms, and the emphasis on causality proved central to the subsequent development of science. Pearl thus regards the current non-model-based approach to machine learning as inadequate.113

# *3.4.5 Old and New Approaches*

With a view to addressing shortcomings of this kind, alternative approaches are now being developed. Several build on 'good old-fashioned AI', the symbolic technology with which the discipline began. Such rule-based systems are used, for

110From an interview with Demis Hassabis (Ford, 2018: 178).

111From an interview with Yann LeCun (Ford, 2018: 136).

112From an interview with Judea Pearl (Ford, 2018: 363). 211 Pearl, 2019: 18.

113Pearl, 2019: 18.

example, where the amount of available data is limited. Siemens has a system built on that principle to control gas turbine processes in its factories. Without predefned rules the turbines would have to run for a century to train an algorithm to do the job effectively by means of machine learning.114 It is also diffcult to apply machine learning in situations where that would imply using large volumes of privacysensitive data and generating results with an opaque basis. In such cases, top-down logical systems may offer a solution. Another, related suggestion is to use rulebased systems in combination with machine learning to predict outcomes and thus deduce what rules are being followed, so making the results more transparent. People like Yann LeCun and Nick Bostrom believe that the future lies in adding structure and modelling to existing machine learning techniques.115

In a variation on the hybrid approach, efforts are being made to code common sense into algorithms. For example, DARPA has a Machine Common Sense programme. This is creating models that distinguish between various categories, such as objects, locations and actors, as happens in human cognition. There are also related approaches that involve building certain principles into algorithms so that they do not have to learn everything from scratch; these work like the inductive biases that infuence learning in children. At an early age children learn the basic physics of objects, how they move through space and, for example, that they cannot pass through each other. Principles of that kind guide and accelerate the learning process so that a child does not have to see thousands of examples of something before it can recognize the item in question. One approach that works that way is the graph network, in which objects are represented by circles and relationships by lines.116 Geoffrey Hinton is working on 'thought vectors' to better capture the meaning of language,117 while the Allen Institute for AI's Project Mosaic is endeavouring to program common sense into computers.

Another set of approaches has close links to neuroscience. Just as neural networks were inspired by the workings of the brain, so there are now initiatives to create neuromorphic chips. If successful, future computers could be ftted with chips modelled on the workings of neurons. The European Union's Human Brain Project is aimed at building a brain made up of computers,118 as is the BRAIN Initiative in the US.119

Besides the two familiar approaches of symbolic AI and artifcial neural networks, Margaret Boden distinguishes another three. They are evolutionary programming, cellular automata and dynamical systems.120 In his quest to fnd the ultimate algorithm, Pedro Domingos has identifed three new techniques that can

114Wilson et al., 14 January 2019.

115Ford, 2018: 78, 108, 126.

116Waldrop, 2019.

117Marcus & Davis, 2019: 128.

118Marsh, 10 January 2019.

119Domingos, 2017: 118.

120Boden, 2018: 5–6.

contribute to the search alongside the symbolic and neural approaches. Genetic programming is an approach used in the design of electronics and the optimization of factories.121 There are also Bayesian methods, such as naive Bayes classifers and hidden Markov models, which are used for spam flters, speech recognition systems, cleaning up data series and so on.122 Finally, there are analogy-based systems like the nearest-neighbour algorithm used by support vector machines. The analogybased approach has been used for modelling the solar system and atoms, and to produce music in the style of particular composers.123

Although machine learning has taken off in recent years and has rapidly been adopted in a wide variety of domains, it has its limitations and alternatives are being investigated. Each of these has its own strengths and weaknesses, meaning that the most suitable approach differs from one application to another. It may be that in the future the emphasis will be placed on selecting the right approach for each application, with no particular one regarded as universally preferable. Many experts see hybrid approaches as the future, arguing that human intelligence works in a similar way. For example, the unconscious recognition of familiar patterns is attributable to neural networks whereas unfamiliar situations are addressed using conscious reasoning, which is more akin to symbolic AI. From all this we can safely conclude that AI is far from perfected and so fundamental research is going to remain very important.

# *3.4.6 The Lab Belongs with AI*

As indicated earlier, the second reason why the lab will continue to play an important role relates to the nature of AI itself and of digital technology more generally. Whereas the traditional pattern is for something to be invented in a lab, then developed at a factory into a fnished product for sale to the customer, digital products are characterized by a different dynamic. It is normal for their developers to remain involved in their application. Consider, for example, the difference between traditional television and a streaming service like Netfix. In the former a broadcaster airs a programme and then receives feedback from viewers that can be used to create new, improved output. With a streaming service the user remains connected to the provider's platform, and it learns from their behaviour in real time, enabling immediate adaptation. A digital product can therefore be regarded not as a fnished product but as a semi-fnished one. Streaming platforms, smart thermostats, healthcare apps and all other digital products are continuously adapted and improved in use.

This is refected in the structure of the technology industry. A 'lean start-up' will quickly develop a 'minimal viable product' (MVP). In many cases this does not

121Domingos, 2017: 133.

122Domingos, 2017: 151–155.

123Domingos, 2017: 199.

work very well and is marred by numerous 'bugs', but is at least useable. Once in use it can learn and improve. Because of this 'the lab' in a sense remain present within the product and continues to play a major role in its further refnement. This is why collaborative initiatives like the AI labs referred to earlier, where scientists ('the lab') are in contact with businesses and/or government agencies, are common in the industry. We could even go so far as to say that with AI's transition from lab to society, the lab itself has entered mainstream society. Another way of looking at it is that society has been absorbed by the lab to become a 'living lab'. Facebook, for example, develops new services by continually running experiments involving the platform's users. It should be acknowledged, however, that this practice has sometimes proven highly controversial, as with Facebook's experiments aimed at infuencing users' emotions.124

The particular dynamics of digital product development gives rise to a range of issues, which we consider in later chapters. One is the 'technical debt' problem, whereby it can be diffcult to rectify a shortcoming in an MVP.125 Furthermore, the development process is liable to entail a variety of risks. Because rollouts are not initially developed to end-product status, users may be exposed to something with undesirable or even harmful effects. By the time those are detected, the damage has already been done.

The semi-fnished nature of AI products also presents challenges for regulators. For example, vehicles are normally tested by the responsible authorities before they can be used on public roads. This approach is workable in a world where the vehicles are end products, but not when they are semi-fnished and liable to change once in use – as with a Tesla that receives a software update. The possibility of a continuous testing regime is therefore being investigated. In the healthcare sector too, the functionality and safety of devices has traditionally been tested prior to licensing on the assumption that their basic functions will not subsequently change. An AI product is constantly evolving, however, and changes can be implemented remotely. Such 'lab dynamics' therefore require a more dynamic approach to testing. We return to this topic in Part 2. For now, it is important to recognize how the lab remains involved with and indistinguishable from an AI product after its practical rollout.

The lab is thus slated to continue to play a major role in the future of AI. The limitations of the current approaches are such that further fundamental research is required, while the very nature of AI implies that the lab will always be associated with the technology's practical application. In the interests of further technical advances and AI's successful integration within society, it is therefore important to keep sight of the lab's role, to involve it in practical implementation and to ensure that it has adequate resources and talent.

124See, for example, NOS, 29 June 2016, and Rutkin, 25 June 2014.

125Marcus & Davis, 2019: 188.

## **Key Points – The Future of the Lab**

# **References**

**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Chapter 4 AI as a System Technology**

Now we know what AI is and have seen how the technology has made the transition from the lab to society in recent years, we turn our attention to the process of embedding AI into society. What is required to incorporate AI into our society? To answer that question, this chapter presents a framework within which AI can be viewed as a particular type of technology, namely a system technology, with a number of historical precedents. By viewing AI in this way, we can draw various conclusions from the history of other system technologies. That in turn provides a basis for refecting on what we need to do with AI and how we can address the many issues associated with it. It is not our intention to imply that history always repeats itself or that technological development has deterministic characteristics. We do not set out a rigid framework but identify general patterns that shed light on the present, while recognizing that the past and the present differ. By adopting this approach, we seek to look beyond the current situation and thus beyond the whims of the day.

Various prominent commentators have drawn parallels between AI and other technologies. According to researcher Andrew Ng, the impact of AI is "comparable to that of electricity a century ago."1 Google's CEO Sundar Pichai and his predecessor Eric Schmidt have also compared AI with electricity. Indeed, Pichai went so far as to liken AI to fre.2

In the policy world too, similar comparisons are often made. In a paper on the strategic implications of AI, Michael Horowitz wrote that it is not an isolated technology but similar to general-purpose technologies such as electricity and the internal combustion engine.3 The breadth of AI's potential applications has also been highlighted by the European Commission's European Political Strategy Centre: "It

H. Sheikh et al., *Mission AI*, Research for Policy, https://doi.org/10.1007/978-3-031-21448-6\_4

1Lynch, 4 May 2021.

2Goode, 19 January 2018; Morozov, 2013: 1.

3Horowitz et al., 2018.

is hard to think of any sector of society that will not be transformed by AI in the years ahead."4 The EU accordingly regards AI as a 'key enabling technology' (KET).

In short, many authors and organizations have hinted at similarities with earlier technologies. But few have gone on to make a detailed comparative analysis. We therefore seek to do that in this chapter. To that end we consider the implications of placing AI in the same bracket as technologies such as electricity. We discuss the literature on various types of technology, focusing particularly on the concept of general-purpose technologies, and we introduce the term 'system technology'. By tracing the historical development of system technologies, we identify a number of general patterns in the way they are embedded in society. From there we defne fve overarching tasks associated with the process of societal integration. In Part 2 of this report, we consider how each of the fve overarching tasks applies to the embedding of AI within society.

# **4.1 Classifcation of Technologies**

Academics have long been interested in how different types of technology exert a general infuence over the economy and society. An early example is the Kondratiev wave theory, in particular as elaborated by Joseph Schumpeter. He observed that periods of high economic growth alternate with periods of lower growth, a pattern he attributed to the effect of new technologies; sets of new technologies periodically boosted growth, after which the effect gradually waned over time. According to Schumpeter, such dynamism was inherent to a capitalist market economy: "it is essential to understand that capitalism is an evolutionary process … 'industrial mutation' … is constantly bringing about revolutionary change to the structure of the economy from the inside, constantly destroying the old structure and creating a new one."5 Such reasoning forms the basis of the familiar concept of creative destruction.

In his acceptance speech when presented with the Nobel Prize for Economics in 1971, Simon Kuznets introduced the idea of 'epochal innovations' driving periods of great economic development. Innovation scientists Carlota Perez and Chris Freeman have written about a similar phenomenon, which they refer to as 'new technology systems' and 'technological revolutions'.6 A new technology system is a powerful and conspicuous cluster of new and dynamic technologies, products and industries that lead to major change throughout the economy and ultimately to economic growth. Perez has identifed fve such clusters since the Industrial Revolution, including the eras of steam power and railways, of steel and electricity, of oil, cars and mass production and of information and telecommunications. She argues that

4European Political Strategy Centre, 2018.

5Quotation from Joseph Schumpeter in Juma, 2016: 17.

6Freeman & Louçã, 2001: 144.

each of these brought its own 'techno-economic paradigm': a way of thinking and acting, leading to the relevant technologies becoming integral to the fabric of society.7 Alessandro Nuvolari has made a signifcant addition to Perez's theories by emphasizing that the observed effects are attributable not so much to individual technologies as to blocks of radical innovations that together bring about revolution.8 Some researchers accordingly take the view that innovation consists not of the development of major new things but of the combination of things that already exist.9

# *4.1.1 General-Purpose Technologies*

AI can be classifed based on its general, transformative impact on society. It is useful to view the technology in relation to the concept of the general-purpose technology (GPT). GPTs are technologies whose potential applications are not specifc, like those of a lawnmower, toaster or microscope, but generic insofar as they lend themselves to countless, highly diverse purposes. GPTs can therefore have a major infuence on the economy and society. Timothy F. Bresnahan and Manuel Trajtenberg introduced the concept in an article published in 1992,10 which cited three criteria for the classifcation of a technology as a GPT. First, a GPT is highly pervasive, being utilized in numerous sectors, production processes and products. Second, there is great scope for its technical improvement, meaning that the cost of the technology keeps falling and its effciency increasing. Third, a GPT spawns numerous 'innovational complementarities', leading to generalized economic productivity improvements.

A large body of literature on the concept of the GPT is now available. However, that has not led to the adoption of a uniform defnition11 or consistent use of the term. Some authors recognize only a small number of historical GPTs, while others argue that there have been many throughout human history, going back as far as the domestication of livestock and the forging of bronze. One author suggests that the literature identifes twenty-eight technologies as GPTs.12

Another topic of debate amongst academics is the existence of technologies that have great societal impact but are not particularly generic. Examples include the printing press and the steamship: technologies whose applications are limited but

7Perez, 2003: 8–11. Belgian economist Luc Soete is also active in this feld.

8Nuvolari, 2019.

9Brynjolfsson & McAfee, 2014: 78.

10Bresnahan & Trajtenberg, 1995.

11For example, Gavin Wright (2000) defnes GPTs as "deep new ideas or technologies that have the potential to signifcantly infuence numerous sectors of the economy".

12See the Working Paper Artifcial intelligence as a general-purpose technology – Strategic interests in responsible use in a historical perspective (Bakker & Korsten, 2021) produced by Freedomlab for the WRR.

have radically changed society. The technologies most widely recognized and cited as GPTs are the steam engine, electricity, the internal combustion engine and IT.13

Notwithstanding the qualifcations made above, a number of interesting studies in recent years have related AI explicitly to the concept of the GPT. Following a conference organized by the US National Bureau of Economic Research (NBER) in 2017, for instance, a collection of papers entitled The Economics of Artifcial Intelligence was published in 2019. The frst part, entitled 'AI as a GPT', includes contributions by renowned technology researchers and economists and contains various interesting analyses that we draw on in this report, although – in keeping with the nature of the original conference, but in contrast to our own focus – they are concerned primarily with the macroeconomic effects of AI.

Also of interest in this context is the thesis by Jade Leung of Oxford University, entitled Who will govern artifcial intelligence? Learning from the history of strategic politics in emerging technologies. In this she places AI alongside aerospace technology, biotechnology and cryptography as an example of what she calls 'strategic GPTs', and in that context emphasizes the relationship between governments and new technologies, particularly in the defence sector. Leung identifes three key actors here, the government, business and the research community, and demonstrates that each has different aims, instruments and limitations, which may converge in certain phases but are at odds in others.

Various researchers have recently sought to place AI in a broad historical perspective without making explicit use of the term 'GPT'. In a polemic on Andrew MacAfee and Erik Brynjolfsson's famous book The Second Machine Age, Carlota Perez wrote a nine-part series of articles entitled Second Machine Age or Fifth Technological Revolution? In these she explores how today's digital technologies – including AI – compare with previous technologies.14

All of these studies, and particularly the perspective developed in them, are relevant to the theoretical framework we use to view AI in this report. We additionally draw on a number of more empirical studies of the effects of specifc technologies. Sarah A. Seo has written about the best-known application of the internal combustion engine, namely the motor car. She also demonstrates how this symbol of freedom has simultaneously led to an enormous increase in the power that the state – particularly the police – have over citizens' private lives.15 In a general survey of a series of technologies ranging from tractors and margarine to electricity and GMOs, Calestous Juma investigates the dynamics of social resistance to new technologies.16 This report thus draws not only on research into GPTs but also analogies

13For a critical analysis of the various uses of the term 'GPT', see Field, 2008.

14Perez, 2017–2020.

15Seo, 2019.

16 Juma, 2016.

with recent technologies such as genetically modifed organisms (GMOs) and nanotechnology, which have interesting parallels with AI.17

# *4.1.2 AI as a GPT*

The question we need to ask at this point is whether AI can actually be regarded as a GPT. But the answer seems quite clear: although its global impact is currently in its early stages, it already appears that AI is indeed a GPT. If we consider Bresnahan and Trajtenberg's three criteria for classifcation as a GPT, a strong case can be made for saying that all apply to AI.

The frst of these criteria is pervasiveness. Although AI's perfusion of the economy and wider society has gathered pace only in recent years, the technology is already used in a variety of sectors and products. Earlier in this part of the report (2.2), we presented a range of examples illustrating how AI is being used in manufacturing, agriculture, the public sector, entertainment, fnancial services and medical practice. Given that versatility, it is already apparent that AI is well on the way to pervading society and the economy.

The second criterion is inherent potential for technical improvement, leading to lower cost and increased effciency. Again, it is evident that AI passes this test. In Chap. 1 we highlighted how Moore's Law states that computing power doubles every 2 years, opening the way for the further improvement of AI technologies. We also saw how scientifc research has fuelled the development of new and improved technologies. As a result, the application of AI has passed numerous milestones in recent years. Furthermore, as highlighted in our discussion of the future of the lab, promising new technologies are being developed, which are expected to further boost the performance and effciency of AI.

Finally, classifcation as a GPT depends on the presence of complementary innovations that lift general productivity. Numerous signs of a positive infuence on general productivity can already be discerned, but AI is simply too young for us to demonstrate conclusively the existence of complementary innovations. Nevertheless, various authoritative research bodies and consultancies, including Accenture, PwC, McKinsey and Deloitte, have forecast major productivity increases over the decade ahead. We set out the three defning characteristics of system technologies in Fig. 4.1. 18

17 In its report Health signifcance of nanotechnologies, the Health Council of the Netherlands describes nanotechnology from the perspective of an enabling technology, an approach that has interesting intersections with our discussion of GPTs.

18For a detailed analysis of AI as a GPT, see Bakker & Korsten, 2021.

# *4.1.3 AI as a System Technology*

We can conclude, then, that AI satisfes the three criteria for classifcation as a general-purpose technology. The GPT concept and the wealth of literature considering AI as such a technology provide useful starting points to understand what kind of technology we are dealing with. Nevertheless, we have chosen not to apply the term 'GPT' here. Rather, we have elected to defne AI as a 'system technology'. That choice refects signifcant focal differences between our analysis and the literature on GPTs.

Firstly, the GPT literature from the earliest Kondratiev wave sources to the recent NBER study has a strong focus on the macroeconomic effects of the technologies in question. Many researchers seek to quantify the effects of the technologies they study. That gives rise to debate as to whether and how a GPT can be shown to support a prolonged increase in economic growth. Given the huge number of variables to be accounted for, a model capable of demonstrating such an effect has to be extremely complex. By contrast, we have chosen to concentrate not on the quantitative effects of system technologies but primarily on the qualitative changes they bring about.

Secondly, the literature on GPTs pays particular attention to historical classifcations. As indicated earlier, there is considerable debate as to how many historical technologies may be considered GPTs. One researcher recognizes dozens, Perez distinguishes fve clusters, authors such as Chandler refer to three Industrial Revolutions,19 Schwab identifes four and Brynjolfsson and McAfee speak of two 'machine ages'. Furthermore, many authors make use of highly schematic timelines with precise start and end dates for individual technologies. This report differs from those approaches in that we refrain from introducing such demarcations. Because we are concerned mainly with qualitative impact rather than quantitative effects, we do not need to commit ourselves to a strict classifcation system or defnite start and end dates for technologies. What we are seeking to do is highlight general patterns. To that end we concern ourselves primarily with a small number of previous system technologies – the steam engine, electricity, the internal combustion engine and the

19Freeman & Louçã, 2001: 145.

computer – and draw pragmatically on historical sources to identify relevant parallels.

Another reason for not adopting the term 'GPT' is that it emphasizes a technology's versatility. We prefer 'system technology' because we wish to emphasize the systemic nature of certain technologies and to broaden the focus to their systemic effects on society. In the context of 'system technology', therefore, the word 'system' has two implications. First, it implies that the technology consists of a system with multiple components. Electricity, for example, works in conjunction with generators, cables, batteries and so on. Similarly, AI is part of a wider technical system of data and hardware. The second implication of 'system technology' is that the technology infuences a variety of systems and processes within society. Exercising such infuence involves a complex process of adaptation, trial and negotiation. In other words, our chosen term refects the process of societal integration and the associated qualitative effects.

# *4.1.4 Similarities and Differences Between AI and Earlier System Technologies*

AI is a system technology and therefore comparable with earlier technologies of that type such as the steam engine, electricity and the internal combustion engine (Fig. 4.2). Furthermore, we can defne AI even more precisely given that particular characteristics make it more similar to one technology than another in certain respects. For example, the internal combustion engine and the steam engine are tangible, whereas AI is like electricity in being intangible to some extent. It does not exist in isolation but only as part of a product or service. In that sense, devices such as toasters, lamps and radios that work by means of electricity are comparable with thermostats, watches and machines that work by means of AI.

**Fig. 4.2** AI as a new system technology

Another respect in which AI is more like electricity than the internal combustion engine is that it is 'technology-radical', rather than 'use-radical'. The descriptor 'technology-radical' is applied to technologies driven primarily by technical and scientifc progress; their development is propelled by the curiosity of researchers, without any clear notion of how or for what purpose the technology will ultimately be used. By contrast, 'use-radical' implies a clear understanding of the applications from the outset, with commercial factors playing a role early on. The development of use-radical technologies is goal-oriented. That was the case with the internal combustion engine. Like with electricity, researchers were working on AI long before people recognized the lucrative applications we are now aware of.

A distinction can also be drawn between system technologies in which governments play an obvious role from the start and those whose development has no such involvement. The frst group includes technologies developed specifcally for defence purposes and dependent on the defence sector for their further application and development. This differentiates them from 'civilian-frst' technologies, whose development is attributable mainly to their economic potential. Governments have more control over the development of the frst group of technologies than the second. Space technology is an example of a technology developed with direct government involvement, while biotechnology is an example of one whose development fts the second model. However, both are examples of what Jade Leung calls strategic GPTs.20

With AI, the US military institute DARPA was a key fnancier in the early stages. Nevertheless, early government-funded AI research was of a fundamental nature and military applications represent only a small portion of its full range of uses. In that respect AI is more akin to biotechnology than aerospace technology. However, it differs from biotechnology insofar as the latter's developers are largely attached to major (academic) laboratories, whereas innovation in the feld of AI is more decentralized. That has implications for researchers' ability to defne universal standards.

It is important to consider not only such technical similarities and differences between AI and other system technologies, but also how AI compares in terms of its societal and temporal context. Take the role of the government, for example. The steam engine was developed in a laissez-faire climate in the UK, with the government playing only a very limited role. On the other hand, the combustion engine and the motor car were developed in an era when government economic policy was led by Keynesian thinking. Although governments now exercise considerable infuence over the economy by means of standardization and legislation, AI emerged at a time when there was signifcant resistance to strong guidance of the economy. It is important to bear those circumstances in mind when seeking to identify historical patterns that are instructive in relation to AI.

The societal context of AI also differs from that of earlier technologies in terms of the mobilization of social actors. Increasing prosperity and the progress of

20Leung, 2019.

democratization have empowered more people to express themselves in the public arena. Whereas enterprises and governments could once shape society with relative ease, nowadays civil society, the academic community, individuals and the media have much more infuence than in the past. The mobilization of these actors therefore plays a more signifcant role in relation to AI and its integration into society than it did in relation to earlier system technologies.

This phenomenon ties in with what Trajtenberg calls the 'democratization of expectations': factory workers during the Industrial Revolution had little power because most struggled to make ends meet. We return to this point in Sect. 4.5, in relation to the Luddites. Today far more people participate in public life and workers have much better representation. Moreover, people are less inclined to bear the cost of technological change while also having greater expectations in terms of sharing in the benefts of such change.21

The world today is not only more democratized than in the past but also more globalized. Consequently, the issues associated with AI have always been more global. The extensive nature of modern markets and the consequently wide geographical impact of AI's applications are relevant in this context, as is the existence of all manner of international constraints such as trade agreements, human rights and technical standards. Interestingly enough, the rise of earlier system technologies has often been an impulse for the formation of new international organizations for standardization,22 and these are now playing a role in relation to AI. Examples include entities active in the felds of telecommunications and the internet, standardization bodies such as the ISO and international engineering associations such as the IEEE. Although the development and embedding of earlier technologies had an international dimension, the signifcance of that dimension has increased over time under the infuence of globalization.

One fnal difference between the development context of AI and that of earlier system technologies is the increased level of organization and communication amongst scientists. The scientifc community was not well integrated at the time of the steam engine's development, whereas academic organizations, codes of conduct and standards now exert signifcant infuence.

# *4.1.5 The Techno-Economic Paradigm of AI*

Finally, Carlota Perez's notion of the techno-economic paradigm warrants attention.23 She argues that major technological change leads not only to new products and services but also to new ways of thinking and working and new principles of organization. For example, the Industrial Revolution led to the rise of factories

21Trajtenberg, 2018: 178.

22Kaiser & Schot, 2014.

23Perez, 2003.

while electricity enabled 'networked' production. Similarly, the invention of the internal combustion engine gave us not only cars but also the conveyor belt. Fordism, Taylorism and just-in-time production are all derived from organizational principles. Although it is too early to characterize the techno-economic paradigm of AI in defnitive terms, we can already discern certain outlines that follow earlier forms of digitalization but also exhibit new features.

We believe that three aspects of the techno-economic paradigm of AI are already distinguishable. The frst relates to changes in the nature of objects and products. As discussed at the end of the previous chapter, in the digital domain we are dealing not so much with end products as with semi-fnished ones. A digital product is never fnished. Unlike traditional products and services, which ultimately leave the factory and are sold, digital products are constantly being revised and adapted. By means of updates, digitally-enabled objects such as computers, cars, cameras and medical devices are always changing. In the words of Kevin Kelly, everything is in a continuous 'state of becoming'.24 Or, as Luciano Floridi puts it, 'things' are being replaced by '-ings', such as interact-ing, process-ing, network-ing, do-ing and be-ing.25

Related to this is the phenomenon that physical objects that acquire a digital aspect cease to be discrete entities. In this regard Adam Greenfeld highlights porosity as a common characteristic of modern-day technologies. The boundaries between objects and between user and platform, and even the walls of our homes, have become porous due to bilateral interconnection and intermingling. Numerous actors are therefore involved with and present in all those products. The changes to the nature of physical objects raise a variety of security, privacy and responsibility questions.

A second feature of the technical paradigm of AI is, paradoxically, that while objects associated with individuals are becoming more transparent, much of the technology is becoming invisible. At a meeting in Davos in 2015, Eric Schmidt predicted that the internet would disappear. He did not mean that it would fall into decline but was referring to an idea derived from an infuential 1991 article by Mark Weiser entitled The Computer for the twenty-frst Century.26 That introduced the concept of 'ubiquitous computing', an omnipresent architecture of digital technology. According to Weiser, "The most prominent technologies are those that disappear. They become integral to the fabric of daily life, with the result that we cease to be aware of them." Hence, "computers can disappear into the background".27

Luciano Floridi has made the same point using a metaphor. He suggests that we are now living on the 'piano nobile', the central upper storey of a Renaissance home visible from the outside. However, below us are numerous servants – in our case

24Kelly, 2017: 9–27.

25Floridi, 2014: 183.

26Zuboff, 2019: 200.

27Weiser, 1991.

digital servants – at work in the service rooms.28 An interesting feature of this spatial metaphor is that it emphasizes the existence of a vertical structure. The building has multiple superimposed levels, not all of which are visible. Benjamin Bratton sees in such verticality the core of digital technology.29 He argues that we used to live in a horizontal world, with people, objects and countries adjacent to each other on the map. Digitalization has introduced a vertical structure, however, the layers of which are formed by internet addresses, cloud services and data centres running through everything largely unnoticed. In the world of technology, the 'stack' is a familiar concept: an entity made up of superimposed layers of hardware, software, network and applications. The existence of that largely invisible layering raises questions regarding power relationships and dependencies.30 Jose van Dijck uses another metaphor to describe the vertical structure of digital technology. She refers to the treelike structure of platformization, focusing attention on the power concentration associated with, for example, vertical integration.31

One fnal aspect of the technical paradigm of AI that warrants attention relates to Floridi's concept of technology. He argues that the idea of technology as an instrument is problematic because it suggests that a person uses an instrument, and by doing so exercises infuence over the outside world. That obscures the fact that much of our technology today acts not act on an external physical world but on other technologies. Our attention should therefore be directed towards that 'intertechnological' dynamism. Floridi calls technologies that act on other technologies 'second-order technologies'. One example is a brake, which acts on the wheel of a car. In that case the process is activated by a person pressing the brake pedal.

However, the world of AI is complicated by the existence of 'third-order technologies': technologies that cause other technologies to act on yet other technologies, without human intervention. In an autonomous vehicle, for example, the decision to activate the brake is taken by the vehicle's control system. Wherever an AI system can make decisions autonomously, a third-order structure may be formed. Many road-traffc penalty systems are already characterized by such structures. A vehicle is photographed infringing a traffc regulation – breaking a speed limit, for example – triggering the issue of a penalty notice, which is sent to the address of the vehicle's registered keeper. The autonomy that technology acquires with the integration of algorithms and that is ultimately integral to the defnition of AI used in this report gives rise to questions about matters such as human control, responsibility and legal liability.32

28Floridi, 2014: 37.

29Bratton, 2016.

30The AI Now Institute has also made an extensive study of the invisible layers of AI: from human algorithm trainers in other countries to the material requirements that lead to all sorts of raw material supply chains. See for example Joler & Crawford, 2018.

31Van Dijck, 2020: 1–19.

32Hage, 2017.

## **Key Points – AI as a System Technology**

# **4.2 The Societal Integration of System Technologies**

Having defned AI as a system technology, we now consider what is required for its integration into society. By analysing the history of earlier system technologies, we identify a number of characteristic patterns that can be instructive in relation to AI. In this section we consider the general lessons of the past as a precursor to examining the fve overarching tasks a society faces in relation to a new system technology.

# *4.2.1 Co-evolution of Society and Technology*

Initially, a process of societal integration or 'embedding' involves prolonged coevolution of the society and the technology concerned. Such a process requires practice, experimentation and negotiation, all of which take time. That immediately places the strongly polarized debate regarding AI in perspective. There are, for example, techno-optimists who believe that AI can fundamentally enrich society in the short term with autonomous vehicles, sophisticated medical diagnoses and automated production systems. By contrast, sceptics argue that AI is overhyped and highlight the lack of evidence regarding the technology's impact to date and the continual revision of predictions about matters such as the speed at which applications like autonomous vehicles will be realized.

Both viewpoints contain an element of truth. As the optimists suggest, AI has many potential applications. However, it would be a mistake to suppose that integrating these into society will be straightforward. Sceptics rightly draw attention to the problems that AI presents in the foreseeable future, but those problems do not justify generalized scepticism about the technology. A system technology necessitates a bilateral process of social and technological adaptation and that takes time, even in the modern era of rapid technological development and globalization. Although technologies nowadays spread around the world more quickly than in the past, embedding them, ensuring that they work and that people trust them all depend on societal processes that are not necessarily faster-moving now than they used to be. Such processes tend to proceed in fts and starts, and often span decades.

# *4.2.2 Unpredictable Development and Impact*

A related observation is that the introduction of a new system technology is to a large extent an unpredictable process. New technologies are often used for purposes other than those for which they were originally intended or initially adopted. Don Ihde accordingly refers to 'multistability' and Wiebe Bijker to 'interpretative fexibility'.33 Cars were originally used for sport and medical purposes in the belief that the 'thin air' breathed when driving at speed was good for the lungs.34 Similarly, Thomas Edison did not develop the gramophone with entertainment in mind but envisaged his 'talking machine' as a business tool akin to a dictaphone.35

Shoshana Zuboff refers to the inability to predict accurately how a technology will be used and the consequent underestimation of its effects as 'horseless carriage syndrome'.36 Major technological revolutions involve unpredictable novelties, an understanding of which is inevitably shaped by the familiar. Hence, the car was initially seen as a horseless carriage. By regarding it as a more effcient version of something familiar, people underestimated both the car's ultimate impact on society and the associated hazards. From a present-day perspective the thinking of the time may seem naïve, but we could well be making the same mistake when we speak of 'autonomous vehicles'. We may be guilty of viewing a new technology merely as an enhanced version of something we know, whose true impact we are unable to foresee.

Another misconception prevailed at the time of the car's introduction in the early twentieth century: it was widely assumed that the new vehicles would reduce urban

33Verbeek, 2014: 31.

34Verbeek, 2014: 71.

35Gordon, 2016: 186.

36Zuboff, 2019: 12. Zuboff also describes how businesses sometimes deliberately present something radically new as if it were old, in order to encourage use. One example is Google Glass surveillance technology, designed to look like ordinary spectacles (Zuboff, 2019: 156).

pollution.37 The reason being that the use of horse-drawn transport generated large amounts of animal droppings, which caused unpleasant odours and spread disease. The motor car was consequently seen as a faster and cleaner form of transport. The removal of horse dung from the urban environment did indeed make cities cleaner and more pleasant. However, people failed to realize that the car would cause its own forms of pollution and its own liveability challenges. The history of its introduction thus illustrates that new technologies often have unintended side-effects. The installation of running water and domestic sewerage was originally intended to prevent disease, but also relieved women of one of the most laborious domestic tasks: fetching and disposing of water.38 An unintended side-effect of electric lighting was a signifcant fall in deaths in domestic fres, many of which were associated with oil lamps.39

Moreover, technological changes can also lead to behavioural changes whose effect is the opposite of what was originally intended. For example, energy-saving light bulbs were developed to reduce energy consumption but in fact increased it because people started using them in places that previously had no lighting, such as gardens.40 This is what Edward Tenner calls the 'rebound effect' of technology.41 Another example is the introduction of domestic appliances, which made housework much less physically arduous but also raised expectations – regarding clean clothing, for example – and thus increased workloads in the home.42

The unpredictability of system technologies stems in part from the long-term structural changes they bring about, which are impossible to foresee. Railways had a major impact on urban planning, for example, because their arrival meant that people no longer had to live within walking distance of their work. Later the car helped to shape youth culture in the 1960s and enabled new leisure facilities such as drive-in cinemas, drive-through restaurants, motels and roadside diners.43

All these uncertainties have implications for AI's integration into society. It is therefore important to recognize that many developments cannot be predicted. We must be very cautious about framing scenarios in defnite terms or making linear extrapolations from the past because such approaches are inherently liable to disregard the unexpected effects of new technology.

37Bakker & Korsten, 2021.

38Gordon, 2016: 123.

39Gordon, 2016: 237.

40Verbeek, 2014: 22.

41Tenner, 1997.

42Gordon, 2016: 278.

43Gordon, 2016: 166.

# *4.2.3 Impact on Civic Values*

That lesson underpins our decision to consider AI's impact on civic values on the basis of structural overarching tasks. Our rationale is that a system technology's effect in this regard is impossible to predict in defnite terms and is often unclear. That is evidenced by the examples presented above: the effect of the car on urban liveability, the effect of electricity on female emancipation and the effect of railways on town planning. The general nature of a system technology makes it impossible to determine what civic values it will affect – in fact, such technologies have the potential to infuence them all. In that respect AI is like any other system technology. We can, for example, be confdent that it will infuence security and health, autonomy and freedom, civil rights and the rule of law, justice and inclusion. However, it is impossible to predict what form that infuence will take.

Nevertheless, numerous attempts are being made to identify the values, principles and rights infuenced by AI. Such initiatives are an important means of surveying topical issues as a basis for targeted intervention. But if we want to protect our social values in the long term, it is also necessary to look beyond the present and AI's current infuence. Our approach, centring on societal integration, is intended to contribute towards the current discourse by focusing attention on the long-term process whereby society and technology infuence one another, with potential implications for all civic values.

# *4.2.4 Regulation and Success Are Not Mutually Incompatible*

One fnal general observation is in order regarding the societal integration of system technologies: there is no inherent tension between civic values and their regulatory protection on the one hand and the economic success of a technology on the other. As we shall see in the next chapter, the frequently cited incompatibility of regulation and success is a myth. The history of technological revolutions shows us that the coexistence of normative parameters and innovation is entirely possible. Of course, regulations can sometimes inhibit technological development by imposing explicit prohibitions, as with the use of nuclear technology for military purposes. Often though, regulation and standardization help make a technology more reliable. That in turn increases public and corporate willingness to utilize and embrace it.