**Topical Collection "Challenges and Opportunities in Applied System Innovation"**

Editor

**Christos Douligeris**

MDPI • Basel • Beijing • Wuhan • Barcelona • Belgrade • Manchester • Tokyo • Cluj • Tianjin

*Editor* Christos Douligeris University of Piraeus Greece

*Editorial Office* MDPI St. Alban-Anlage 66 4052 Basel, Switzerland

This is a reprint of articles from the Topical Collection published online in the open access journal *Applied System Innovation* (ISSN 2571-5577) (available at: https://www.mdpi.com/journal/asi/ topical collections/Chall Oppor).

For citation purposes, cite each article independently as indicated on the article page online and as indicated below:

LastName, A.A.; LastName, B.B.; LastName, C.C. Article Title. *Journal Name* **Year**, *Volume Number*, Page Range.

**ISBN 978-3-0365-4401-4 (Hbk) ISBN 978-3-0365-4402-1 (PDF)**

© 2022 by the authors. Articles in this book are Open Access and distributed under the Creative Commons Attribution (CC BY) license, which allows users to download, copy and build upon published articles, as long as the author and publisher are properly credited, which ensures maximum dissemination and a wider impact of our publications.

The book as a whole is distributed by MDPI under the terms and conditions of the Creative Commons license CC BY-NC-ND.

## **Contents**

Reprinted from: *Appl. Syst. Innov.* **2021**, *4*, 36, doi:10.3390/asi4020036 ................ **185**

#### **Kamel Mohamed Rahla, Ricardo Mateus and Lu´ıs Bragan¸ca**

## **Preface to "Topical Collection "Challenges and Opportunities in Applied System Innovation" "**

This book contains eleven articles published in Applied System Innovation which stand out due to the way they approach the issues they address. The main idea that connects these articles is that innovation is occurring in a variety of industries and that there is a strong interdependence on the introduction of new technologies and their acceptance in various societal functions. Thus, this book introduces and provides solutions to a variety of problems faced by society, companies and individuals in a quickly changing and technology-dependent world.

The wide acceptance of artificial intelligence (AI), the upcoming fourth industrial revolution and the newly designed 6G technologies are seen as the main enablers and game changers in this environment. The book considers these issues not only from a technological viewpoint but also on how society, labor and the economy are affected, leading to a circular economy that affects the way people design, function and deploy complex systems.

The main focus of the first three articles is artificial intelligence. The first article considers how AI will influence modern societies, while the other two articles discuss specific applications in the provision of sustainable development solutions and fast, accurate and ready-to-deploy applications in problems faced in traditional bridge problems.

Even though 5G has only recently become part of our everyday lives, new 6G ideas are explored in the next two articles to further expand the Internet of Things (IoT) concept into the Internet of Everything (IoE) paradigm. The need for a theoretical framework that would allow researchers and practitioners to design effective and efficient systems in the healthcare sector is emphasized.

The advancements in Information and Communication Technologies (ICT) have enabled a variety of innovations and have revolutionized many industrial settings. Three-dimensional (3D) printing technologies and cognitive manufacturing are two ideas that are explored in the next two articles. However, this revolution has raised new questions in terms of the problems of labor productivity and employee retention. Data science solutions as well as innovative approaches based on human factor research are explored in the following articles.

All of the ideas discussed above would definitely benefit from new modeling structures that ICT technologies allow. The digital twins concept is one such structure—the article entitled "Digital Twin: Origin to Future" provides a pathway to this concept with a strong emphasis on solving crucial and important problems in the manufacturing research and development area.

Even though these developments are rapid and innovative, their environmental effect should not be ignored. An article within this book, which discusses the circular economy, can be seen as a guideline for all industries and organizations, even though it only discusses buildings.

I hope that this collection of articles will provide a background for fruitful discussions, a plethora of idea exchanges and the basis for further research and development. Innovation is a never-ending process, full of surprises and exciting new ideas.

> **Christos Douligeris** *Editor*

## *Article* **Chances and Risks of Artificial Intelligence—A Concept of Developing and Exploiting Machine Intelligence for Future Societies**

**Reinhard Hutter 1,\* and Marcus Hutter 2**

**Abstract:** Artificial Intelligence (AI): Boon or Bane for societies? AI technologies and solutions—as most revolutionary technologies have done in the past—offer negative implications on the one hand and considerable positive potential on the other. Avoiding the former and fostering the latter will require substantial investments in future societal concepts, research and development, and control of AI-based solutions in AI security while avoiding abuse. Preparation for the future role of AI in societies should strive towards the implementation of related methods and tools for risk management, models of complementary human–machine cooperation, strategies for the optimization of production and administration, and innovative concepts for the distribution of the economic value created. Two extreme possible "end states" of AI impact (if there is ever an end state) that are being discussed at present may manifest as (a) uncontrolled substitution by AI of major aspects of production, services, and administrative and decision-making processes, leading to unprecedented risks such as high unemployment, and devaluation and the underpayment of people in paid work, resulting in inequality in the distribution of wealth and employment, diminishing social peace, social cohesion, solidarity, security, etc., or, on the contrary, (b) the freeing of people from routine labor through increased automation in production, administration and services, and changing the constitution of politics and societies into constituencies with high ethical standards, personal self-determination, and the general dominance of humane principles, as opposed to pure materialism. Any mix of these two extremes could develop, and these combinations may vary among different societies and political systems.

**Keywords:** artificial and human intelligence; security; risks and risk management; quality of life; common welfare; socio-political assessment

#### **1. Introduction**

Artificial intelligence (AI) and all its theoretical, technical and operational ramifications offer a vast and rapidly developing potential to serve humankind. The authors were motivated by realizing that most discussions of AI stress either technical matters or the possible negative impacts of AI's expansion into society. The ideas presented here deviate to from today's mainstream public discussions regarding the future role of AI. Although they are a bit revolutionary, or at least unusual, the concepts presented here promise a high payoff for societies derived from AI, while, at the same time, avoiding quite a high number of risks of and controversies. Figure 1 illustrates that, sooner or later, all relevant societal spheres will be impacted by AI-based solutions, and that, in turn, the path of AI development will require guidance and control. This will require an unprecedentedly concerted effort among all societal forces with respect to the preparation, introduction, operation and maintenance of AI solutions, and with respect to the principles that should guide their development and use.

**Citation:** Hutter, R.; Hutter, M. Chances and Risks of Artificial Intelligence—A Concept of Developing and Exploiting Machine Intelligence for Future Societies. *Appl. Syst. Innov.* **2021**, *4*, 37. https:// doi.org/10.3390/asi4020037

Academic Editor: Andrzej Białas

Received: 3 April 2021 Accepted: 12 May 2021 Published: 2 June 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

**Figure 1.** Cooperation model.

Developments may imply significant improvements to the human condition, but may also constitute risks of the contrary, including abuse or bias from groups with competing interests. Furthermore, the ultimate variants in future scenarios could include the risk of full takeover of societal governance by AI and robots, relegating mankind to a remnant of the past, enslaved or tolerated at best, or, at the other extreme, the creation of a full biblical Land of Plenty, with AI services and robots providing a paradise for humans to live in.

The novelty presented in this paper is that it takes a realistic view, while trying to develop an optimistic vision: a feasible strategy that would improve security, quality of life and human cohabitation, while maintaining dignity and self-determination and diminishing the fear and uncertainty of people, societies and politics when discussing how to manage the risks that may emerge from these concepts. The accompanying benefits could and should include climate control, optimal use of natural resources and the sustainability and recreation of biological diversity, along with improved overall safety and security and adequate risk and crisis management.

In a nutshell, AI is developing abilities that will allow for progress in many directions. The path towards which mankind strives varies substantially depending on region, cultural background, political system and principles, available resources, and environmental conditions. The manner in which politics, economy, science and societies prepare themselves for, agree upon and manage the elementary prerequisites for the transformation and shaping of future "cyber societies" will be essential. The basics need to be determined sooner rather than later, and this should be started now, at both national and global scales.

Section 2 provides a brief overview of how AI has developed over the decades, and shows that it is now at a stage where collaborative (CI) models between artificial and human intelligence are key to future societal applications and their expected effects, as discussed in Section 3. Section 4 discusses the impacts of these CI solutions on the economy, on politics, and on data requirements. Sections 5 and 6 discuss promising potential future applications of CI methods, while in Sections 7 and 8, the benefits and possible risks of such solutions are evaluated. As these ideas are rather challenging, the prerequisites are discussed and their feasibility is assessed in Sections 9 and 10, respectively.

#### **2. Starting Point and Background**

Having followed and being involved in public and scientific discussions and the professional development of AI for some 25 years, we believe that its potential has reached a level that will make a big difference in future applications. We, therefore, derive some ideas and draw some conclusions that could be useful for future debate and progress in the AI domain, and that hopefully will be substantial for future human society at large.

AI started some 60 years ago with the main intention to somehow understand human rationality and cognition technically, and to build human-level AI. In the 1980s, it developed into replicating human rationality and cognitive power in rather limited application areas. More recently, AI R&D has strived towards a solution to complex problems and generalizing theories and practices beyond the limits of human capability [1]. We thus put forward the question of whether AI and its extension to Artificial General Intelligence (AGI) is the intelligence of a machine that could successfully perform any intellectual task that a human being is capable of; see, e.g., https://en.wikipedia.org/wiki/Artificial\_general\_intelligence, accessed on 20 December 2020) should be pushed more, in the sense that it will design ways to solve problems in areas where human organizations are limited in speed, capacity, and ability, as well as manage the involved risks and create optimum solutions; in other words, problem areas which, in present political systems, are constrained through societal and political restrictions by the need for compromise and by the limitations in designing and executing sustainable long-term concepts. In this sense, after phases of both enthusiasm and nightmares (such as George Orwell: "1984"), we feel it is time to think of a positive far-end of AI.

Past public and political recognition and discussions of AI have occurred in waves. In the 1980s, hope and enthusiasm was concentrated on the prospects of extending computer ability to problems that are logically structured and rather deterministic (e.g., Expert Systems). The disappointments caused by the huge cost of the required computer power and limited scientific experience with complex logical networks and the self-learning ability of systems, etc., resulted in disillusionment. Only a few, rather modest, practicable solutions evolved and survived.

The present public discussion focuses more on what AI really means for our future society. Some effort has been spent defining what AI is really meant to be. Comparing AI with, and trying to simulate, Human Intelligence (HI) alone is, to some extent, frustrating, and may lead us in the wrong direction, at least as long as there are still so many loose ends regarding how human cognition and rationality really work, let alone their dependence on basic brain structure, genetically determined or not, for the interaction with sensory input, influence of mental/psychological states and the inter-operation of all these [2]. AI research and brain research are delivering a number of insights that will help to improve AI approaches and efficiency (e.g., neural networks, interfacing machines with biological neural systems, complex pattern recognition, language interpretation and many more). Scientific progress in artificial neural networks and machine learning have also contributed. We have a deep respect for the achievements reached with systems such as Deep Blue, AlphaGo/AlphaZero or Watson, and robotics that increasingly adapt or exceed human capabilities. However, our core question has arisen as to whether these should be the priority objectives of AI R&D and if they are the right approaches and exploitations for the more general benefit of mankind. Except for the sake of scientific progress, the core driver of and the related investments into RTD on AI should not primarily deal with, e.g., fancy dancing robots or playing games, to name only two of the most spectacular examples at present, which we, a bit disrespectfully, call "toys".

We do not want to arrogantly disregard any achievements gained in AI to date. However, AI, in the future, from a political and socio-economic perspective, should focus more on solving the problems of mankind, societies, groups and individuals; in short, on improving the quality of life and common welfare. Compared to the "toys" referred to above, there are already fascinating and very helpful AI-based applications, such as pattern recognition (speech, face), language translation or improvements in self-driving cars. However, what we are advocating here is to further focus investments, public attention and success stories on solutions that more strictly follow a humankind-oriented approach. AI should not concentrate too much on things that the human brain can do already, but on things that humans cannot perform, or perform too slowly or imperfectly, such as solutions to over-population, fairness in the distribution of resources and wealth, sustainable traffic and transportation concepts or genome sequencing for healthcare purposes. This, however, is a moving target, as ultimately AI may perform better than humans at almost everything. Nevertheless, there seem to be qualities in HI, such as "Emotional Intelligence" [3] or intuition, that may never, or only in an undefined future, be achieved by machine intelligence. Of course, this is not a black-and-white issue. Recognizing faces, automating production lines, improving security through tamper-proof identification, or optimizing the search for a parking lot have personal and economic benefits. The bottom line, however, of our admittedly rather fundamental considerations is that machine intelligence should no longer focus on imitating HI, but better capitalize on tasks that cannot, or can only rather sub-optimally, be handled by humans, in particular tasks for whole societies, politics, and large constituencies, such as the global financial system, commerce, trade, and alleviating boring work, risks, duties and suffering for mankind on a larger scale. "Too many problems (in most urgent real-world challenges) suffer from painfully slow progress" [4].

The potential of big payoffs for the advocated future AI applications should not be created via blunt automation alone, but involve cooperation between human and machine intelligence. It should not be a substitution for human intelligence and labour, but complement and improve humans' mental and physical skills. In that sense, we will refer to CI. CI here stands for Complementary, Collaborative Intelligence, not Computer Intelligence as it is also sometimes used.

#### **3. CI Applications Effects**

Research and industry should concentrate CI on the exploitation of machine properties, such as basic computing power, fast networking and algorithmic concepts and solutions to complex problems. These CI application areas should be prioritized over those which obviously have not, or have very unsatisfactorily, been solved by humans to date, but for which promising solutions and payoffs are developing in AI (see also Table 1: Some Examples of Promising CI). From there, application domains should be identified where CI can produce help and solutions that will be significantly superior to human-made approaches tried in the past. The payoff will not just be high economically and commercially, but also in terms of enhanced societal qualities, including risk management.

**Table 1.** Some Examples of Promising CI (for further examples, see also [3]).

#### **Table 1.** *Cont.*

**Table 1.** *Cont.*

Of course, related concepts (and, later, the solutions) need to be assessed against a variety of impacts they will undoubtedly have, including:

The basics of such an "EELPS" assessment have been developed in several international research projects and are further discussed in [6].

The superiority of CI then would mean the superiority of solutions, possibly superiority over human abilities, but also superiority over legacy organizations and political systems. Debating threat scenarios where AI substitutes the human (labor, cognition, rationality) "enjoys" huge attention from, and has a huge attraction to, the public and media at present (see, e.g., [7], strategy # 1; 7; 10). Instead of this, a positive and constructive concept of AI and CI should aim for solutions that are developed for, and implemented and operated in, cooperative models: here called Intelligent Complementary Human-Machine Collaboration (ICHMaC or CI). For the time being, we leave open the question of whether this will materialize in direct bio–machine interaction, organization of close cooperation, or the complementary use of brain and machine intelligence. The degree of integration of HI and AI will depend on the selected problem area and the related technical solutions being developed, as well on the risks and political will.

#### **4. Validation and Consequences**

#### *4.1. Human vs. Machine Capabilities*

When following this basic approach, there would be no need for further (mostly cramped) discussion on what is human and what is artificial. In CI concepts, they simply coexist and cooperate. This will, however, require a kind of general "societal contract" or agenda on the weighting of societal values and human skills relative to AI. As AI will become increasingly smarter, this is not a one-time fixed agreement; rather, it needs to

become a continuous process, including "sandboxes" for experimenting with all related societal groups, and clear criteria for the validation of benefits and risks.

There may be necessary limitations, beyond which AI has no say. Then, the somehow still fictitious discussion of whether AI is, or will ever become, superior to HI, and even endanger human existence, should become irrelevant. Inherent to the concept should be regulations to avoid false or counterproductive competition between AI and humans. False and counterproductive here means with new risks or insufficient compensation for those people or organizations affected (see also Sections 7 and 8 on benefits and risks). In the longer term, and from a more philosophical perspective, as in our concept, issues which are not so relevant today, but may possibly in the future, should be discussed, such as whether machines may ultimately acquire properties of consciousness, free will, a "spirit", or whatever "mental" characteristics may be attributed to AI above or beside rationality. Attributing consciousness to AI systems, however, would require a clear definition and understanding of what consciousness really means, physically and logically [8]; this is not available at present, and it could take a long time to fully understand (see e.g., [9]). The consciousness of AI, then, would lead to extended elaborations on ethical impact of AI, and the evolution of superintelligence, transhumanism and digital "Singularity" [10], terms that form a different level of philosophy, which we will not further detail here.

However, these speculations will lose their menacing nature when societies realize the boons of AI and CI. "These challenges may seem visionary, but it seems predictable that we will encounter them; and they are not devoid of suggestions for present-day research directions" [11]. More on this topic is also available in [12,13].

Further practical pros and cons should be considered in socio-political discussions on future CI concepts, including:

In a cooperative AI concept, as sketched here, there will be no need to fear risks of AI (CI) becoming adverse or hostile to humanity. The precautions of creating only "benign" systems and for preventing adverse, uncontrollable or dangerous AI solutions will be, and must be, an integral part of this CI, and of any AI concept [13].

CI will leave humans to complete those tasks that he/she can do better than a machine, which require abilities and skills which are hard to implement in computers (e.g., intuition, life experience, highly sophisticated skills and knowledge such as medicine, creativity, arts, and decision-making under contradicting arguments). This will be a dynamic process, as AI/AGI skills will develop and further capturing these human properties, and AI skill profiles and human societies' trust in AI will evolve over time and integrate with human abilities to form advanced cooperative models, e.g., virtual reality arts.

#### *4.2. CI and Economy*

Economic and socio-political models of human–machine cooperation and partnerships, other than models of competition between human labour and machines, need to be created and established (sometimes called "Centaur Teams" with "Hybrid Intelligence", e.g., in [3]), controlled and maintained. Interdisciplinary collaboration of the AI community with economics, and political, social, psychological and philosophical science will become indispensable in the design, development, implementation and operation of those models. Their benefits and shortcomings continuously need to be monitored and evaluated from these very different disciplinary perspectives (see also Sections 3 and 7).

A cost–benefit assessment (short-, medium- and long-term CBA) needs to be established and regularly applied. As these CI concepts will create, or at least facilitate, rather innovative business models and political scopes of action, classical CBA will no longer be adequate. New assessment methods need to be created that have to include in their equation—besides the monetary/materialistic payoffs—the socio-economic, socio-political and even ethical implications of CI concepts in society [6].

#### *4.3. CI, Politics, Taxation*

As we have seen in the industrial revolutions of past centuries, automation is key to progress. First, industrial machines replaced raw labour, then refined machines produced goods, then general-purpose industry robots could diversify and adapt quicker to market needs, then the computer helped further automation, then the advent of the internet (not an automation tool in itself, but a productivity booster) and, finally, AI, have allowed for the creation of an increasing number of goods and services, much more cheaply than was possible with human labour.

Any country not investing in AI technology or trying to slow it down will seriously fall behind, as did any country that missed the past industrial revolution, probably even more so. In the future, computers in general, and AI in particular, will dominate the economy of every developed country.

Some believe that AI will take our jobs and cause mass unemployment and poverty. Others believe that, for every job taken over by AI, new jobs will be created. We believe that both sides of the debate are misguided. The primary goal is not to have a job and for businesses to maximize profit, but to gain a decent standard of living for as many people as possible. Un(der)employment should not be the focal problem; one may even argue that it is desirable and affordable (for more rationale see e.g., [14]). Crucial to maintaining or increasing global wealth is that the total amount of quality goods and services that are created stays the same or increases, respectively. If a robot or an AI replaces a human worker, the same amount of goods and services are produced, and likely even more, or fewer, but with higher quality. It is then "just a political matter" to redistribute these goods and services fairly, ensuring that nobody is worse off—most likely, everyone will be better off and, in the long run, much better off. The importance of immaterial benefits will increase. Few people seem to appreciate this simple economic fact today.

This redistribution can be done in a number of different ways:

Probably, a well-thought-through combination of these and other measures, suitably adjusted to the increasing automation over time, could avoid any larger outfall, as we have also largely managed technological advances in the past. Of course, the right government and governance is necessary for this transition to be possible and peaceful, but if this occurred, nearly everyone would profit from it (see more in Sections 9 and 10). Generally, policies should be set and managed in a way that the poor could become richer and others, such as investors and entrepreneurs, would profit, but not unduly, from this new societal paradigm.

The replacement or improvement of human work with IT has been ongoing for more than half a century, with benefits for all, causing temporary layoffs but generally producing new jobs and societal and economic benefits [15]. Tendencies, intentions and plans, however, to replace human jobs with AI, without adequate compensation and tradeoffs between benefits and losses, need to be avoided through political and economic precautions, and substituted with ICHMaC concepts. Supporting social programs and lifelong academic and/or vocational training will need to accompany most career profiles. The primary goal of human existence will no longer be to have a job, but to have a decent standard of living. The required human working skills will shift from routine to creativity, from standard work, replaceable by robots, to higher quality work that exceeds the abilities of AI, that will not be bound to a fixed number of working hours, but will require high flexibility from employees, innovation and the abandoning of standard procedures. CI will alleviate the compulsion for gainful occupation. Unemployment must no longer be considered a problem; it may even become desirable and affordable! Politicians, economists and entrepreneurs, however, will need to adapt to these changes in societal paradigms and agree on different taxation and social security models that will avoid "arms races" and other risks of conflict between human and complementary machine intelligence.

#### *4.4. CI and Data*

AI/CI concepts will rely on the availability of data to learn from and use in their applications, usually vast amounts of data. This requires the preparation and assessment of available or emerging databases and the creation of the necessary:

Data compression and information theory are not only important for saving storage space and channel capacity for transmissions, they also play a key role in modern approaches to artificial intelligence [16].

We will not be able to discuss AI using complex data sources within this article. Nevertheless, it is common understanding that all practical and useful AI applications begin with data. As an example of AI in manufacturing, see, e.g., https://www.forbes.com/sites/ willemsundbladeurope/2018/10/18/data-is-the-foundation-for-artificial-intelligence-andmachine-learning (accessed on 20 May 2021).

A typical classification scheme of "big data", as seen today in the era of internet with social media, with cloud services and many others, is depicted in Figure 2.

The top categories in this picture, Velocity, Volume and Variety of data, are only three of the most important technical classification criteria for mass data characterization.

Further data characteristics need to include:

**Figure 2.** Big Data Classification Parameters (adapted from: [17] where, in addition to the Figure, a full suite of big data characterstics are defined).

Among this rich spectrum are the key requirements concerning data that will be used for AI. When we discuss the applications of AI/CI in the social context, as we do in this article, we believe that, for preparation at the socio-political level, certain elements are indispensable:

These efforts require political initiatives and need to be started now and sustained and supported by impartial research capacities.

#### **5. Beyond CI**

Futuristic visions are challenging and their risks need to be perceived and managed. This paper assumes the continued existence and (hopefully) the continued exploitation and improvement of political systems such as "western" democracies. Of course, changes may occur, even major ones, of constitutions, of priorities and rankings in economy, and of society and the political system, some of which will be recommended here. More revolutionary scenarios, however, such as singularities in the sense of [10,19] machine takeover or conscious machines [12,20], and their possibilities and probabilities, lie beyond the scope of this article.

Computers have always partially outperformed human performance by factors of millions and more, e.g., in calculation and communication speed, automation of office and production labour, and nobody has cared too much or would describe this as revolutionary rather than an evolution, mainly to the benefit of society (for many examples, see, e.g., [15]). "AI Takeover" scenarios by "hyper-intelligence" and "superhumans" are likely to remain, for decades and maybe longer, in the category of Science Fiction. The basic question, at present but particularly in the future, is where to draw the line between technology's blessings and curse, how to detect or forecast them and how to reflect on them. If the line drawn too narrow, we may forfeit chances for higher living standards—not just in terms of monetary gain, but also possibly a cure for cancer, the elimination of hereditary diseases, improvements in longevity, the prevention of war, and/or saving the environment and the climate. If drawn too wide, a "singularity" or other catastrophic developments may happen, the consequences of which we, and even their proponents, cannot assess and predict with the present way of thinking about the future.

#### **6. Some Promising Examples of Future CI**

#### *6.1. Survey*

The spectrum of applying AI in the CI sense, as discussed here, appears to be almost unlimited. The few selections of promising applications given in Table 1 are rather arbitrary candidates, however likely, and not exhaustive. Columns 2 and 3 approximately indicate how work could be shared between human and AI partners in different domains. This should demonstrate that the basic principle of CI is for HI and AI to complement rather than compete with each other. The chances, challenges and benefits are left to the judgement of the readers or experts in the subject domain. Requirements, risks and benefits in the implementation of CI will need detailed tradeoffs. However, some basic guiding rules for the implementation are mentioned in the right-hand column. Solutions and their realization will require extensive research and analysis, planning and consensus-building among all involved and/or affected stakeholder groups.

#### *6.2. CI for Risk Management and Security*

The present discussion about security comprises, in principle, three different aspects:

Here, we elaborate on the latter: security as a promising application area of CI, one of the many candidates shown in Table 1.

Security is the most fundamental need of humankind, following the basic physiological needs for food and water (https://www.simplypsychology.org/maslow.html#gsc. tab=0) (accessed on 20 May 2021); this is true for societies, groups and individuals. The protection and integrity of lives and possessions are deeply anchored in the psyche of mankind, due to evolution's survival and selection principle, and are present in mankind's organizational constructions. Treats are usually classified as natural disasters, technological risks and man-made disasters. The basic prerequisites for establishing and maintaining security are subsumed in the term Risk Management. A typical organization scheme of risk management is presented in Figure 3.

Almost all tasks in this activity cycle require considerable skills and resources. Despite various approaches to security and an enormous, historical trove of experience from wars and other disasters, precautions for and measures to maintain security have always been all but optimal. This is due to the huge complexity of the various problem areas, the fuzziness of data and the short reaction times required, as well as the diverging or conflicting interests of the individuals and groups involved, a lack of incentives for long-term planning in government and industry policies, and the tendency to ignore very rare but global, truly catastrophic events that could endanger humanity in its entirety [21].

**Figure 3.** Security and Risk Management.

Even during the last 20 years, in which security increased in priority due to the 9/11 catastrophe and follow-up terrorist activities, measures taken to establish or modify organizations, in terms legislation and responsive actions, often appear ill-designed, or lacking in rationality. The authors appreciate all efforts taken and do not claim to know better, but with AI/CI, there is hope that we can do much better.

Nevertheless, the skills required in risk management, including data acquisition and analysis, organization, plans and actions, situation monitoring, and inferring and applying best practices, distinguish security as a preferred candidate for the extensive application of AI and CI. Arguments for this focus on security include:

Overall, the delineated spectrum of tasks and potential make security a prime candidate for the exploitation of convincing AI/CI solutions. They could range from intelligent monitoring and detection and the assessment of warning indicators to the use of robotics instead of humans in dangerous areas; from optimal strategies to deal with pandemics to proven good practices in allocating patients to medical resources in disaster situations; from intelligent analysis of operational situation data to optimal strategies for damage reduction and mitigation. There is an urgent need to proceed in this direction.

Elaborating on the examples of Table 1 would exceed the limits of this article and go beyond the focus of this special volume on Risk and Crisis Management. The scheme of CM discussed here, however, may serve as a role model for other subject domains listed in Table 1, many of which also address the security of societies in a wider sense.

As discussed at the very beginning of this chapter, security is a fundamental need. The other side of this coin, however, offers a huge potential for abuse: of all domains of political responsibility, and broadly independent of the political system in place, security implies the most traps and loopholes, to circumvent warp legal and societal control mechanisms. This is because the need for security often overrides the need for assessment by individuals as well as administrations, and limits rational discussions and the balancing of diverging interests and alternative options. Numerous such cases, particularly after 9/11, have occurred in security, leading to:

Societies and politics need to support the preparation abilities of AI/CI in security to prevent rather than support the phenomena which are counterproductive to a prosperous and peaceful society, which is the aim of this contribution.

#### **7. Evaluation of Benefits**

Human brains and HI are wonders of evolution. Nevertheless, they also embody huge shortcomings that manifest in conflicting interests and controversies among institutions and decision makers, emotion-based preferences, and personal struggles with mental and/or physical force and injuries, which can lead to wars and global conflicts with apocalyptic outcomes, such as weapons of mass destruction, serious environmental impacts and social disruptions. The technical deficits in HI lie in the operating speed, replication accuracy of knowledge, diverging preferences of individuals and groups, limited objectivity, limitations in solving complex problems, and finding, introducing and sustaining optimal solutions to complex societal issues. Additionally, the limited sustainability and replication accuracy of human and societal memory and of the longer-term planning, control and maintenance of decisions and their impacts can cause problems. Religions (this in no way intends to diminish the beneficial effect of religion) and un-scientific or biased mindsets, conspiracy theories, etc., also contribute to the deficits in rationality [15]. Other shortcomings produce emotional drama or allow for the creation of ideologies that may be catastrophic for mankind or for groups and individuals.

Generally, the mankind's ability to anticipate, manage and mitigate risk seems to be inadequate, or at least suboptimal, in today's sophisticated, technological and organizational, highly developing world [21].

The CI approach presented here should concentrate on compensating for, or at least ameliorating or attenuating, these kinds of deficit. It is the basic postulation of this concept that this should produce the highest benefits for the well-being, standard of living, quality of life and the further development of societies into civilizations of the highest standards (some visions of what a "highly developed society/civilization" could actually mean are given in [23] (some 400 years ago) and [24]). Monetary profits are included, but should not dominate the proposed benefits. These should also entail the adequate control of climate, environmental exploitation and biodiversity. The possible benefits are obvious; however, they need to be verified and evaluated in a continuous process. Some promising research ends are given in the Stanford Human-Centred AI Initiative, stating that "the ultimate

purpose of AI should be to enhance our humanity, not diminishing or replacing it" [25]. Other initiatives follow similar principles, e.g., [14,26].

The benefits will be a mixture of material/monetary and immaterial/qualitative gains. Advanced methodologies need to be developed to assess the benefits, particularly life-cycle models and mixed qualitative/quantitative cost—benefit tools to balance the investments and benefits. This will also create new paradigms in the definition of "cost" and "benefits". For example, if a "robot" can do what a human can for half the cost, it is "just" a political matter to ensure that everyone benefits, including those people that would otherwise lose their jobs and income.

In an earlier European Union project [27], the principles of a balanced assessment of measurable benefits, cost and societal impacts were developed. Different kinds of computer-based model focused on:

These were defined, implemented, tested and successfully demonstrated in a number of use cases. The final challenge was to integrate the results on risk assessment, cost, and societal impact evaluation into an integrated decision support tool. Although, in that project, the applications were limited to investments in security, the basic methodology and plurality of the evaluation criteria can be viewed as a baseline, which can modified to evaluate future CI concepts. Unfortunately, the willingness of public institutions to submit their decisions to such a systematic assessment was rather limited.

Nevertheless, using a comparable methodology, such wide balances between autonomy and controllability, as well as between investment, risks and benefits, need to be found for all CI solutions. This leads to the discussion of the risks of CI solutions.

#### **8. Risks of CI**

There is no technology without risk. The risks from fast-developing and internationally/globally spreading technologies include those to and from AI, as AI will increasingly penetrate all domains of society worldwide. "Global problems need global answers" [28]. Apocalyptic nightmares about the future risks of AI are frequently publicized; however, they are usually biased and based on speculation rather than sound analysis and, therefore, are rarely helpful or likely to happen. Nevertheless, precautions need to be considered and implemented to ensure AI development and employment remains "benign" and avoid hostile or dangerous applications. An example of a potential risk is that deep learning AI system algorithms may one day have the potential to go beyond human control, or at least human comprehension of what they are actually doing, with the potential to abuse economic and political power, or even sabotage and undermine democratic constituents. Even worse could occur if such systems are hacked and used to cause military controversies or abused for large-scale criminal or terrorist purposes. This means that CI regimes will require internationally agreed-on standards, validation and certification rules, and a reliable regime of governance, before they are established. Some examples of CI applications and precautions are given in Table 1. Promising developing role models for governance can be seen in the European Union's CIP, European Programme for Critical Infrastructure Protection policy (ECIP) (https: //ec.europa.eu/home-affairs/e-library/glossary/european-programme-critical\_en, accessed 20 May 2021) or in the General Data Protection Regulation (GDPR) of the European Union, which is compulsory for EU member states but appears to be becoming widely accepted as an international standard [29]. Another positive example comprises the agreements on cyber-security, which was first developed in the 1970s and has resulted in a number of internationally accepted standards in IT security (for the most prominent examples, see [30]). It should, however, be noted that the processes behind these initiatives took

decades, from the initial idea to putting the regulations into force; we may not have these timeframes with AI.

Political prerequisites, chances and obstacles regarding the further development of AI technology, and governance of CI solutions, are discussed in the next chapter.

#### **9. Prerequisites for Success of CI Concepts**

This CI concept could develop into a peaceful revolution of the human condition into the next generation of the socio-economy, related politics and modified constituents, and should be incorporated in state constitutions and national and international agendas (e.g., the UN). The basic vision of CI that should be created and communicated to all societal groups, organizations and their people is the creation of win–win situations between society and technology; between economic interests and those of the "working people"; between the maximization of consumption and the "higher values of life"; of a controllable financial sector with the aim to fairly balance the distribution of monetary resources among societies.

The classical neo-liberal concept (driven by capital, market and profit), to a large extent, needs to be replaced by, or transformed into, a concept of optimum benefit for societies (driven by social and humane values). The intent of this concept and its risks and problems is not, in any respect, to limit or hamper work on AI. Instead, it aims to employ and exploit AI to the mankind's benefit. Defining what is "best" for humanity, however, is an even more complex, manifold, ambitious endeavor. Different cultures and political systems have different values and behavioral rules and processes; the interests of states or regions vary between nations, and often within nations. The same is true for different groups, be they the financial sector, associations of interest such as unions, religions, environmentalist movements and many more.

Nevertheless, we believe that the basic statements of the Universal Declaration of Human Rights of the United Nations (1948) and the European Convention on Human Rights (1953) could set basic guidelines, to be complemented by AI-specific aspects, e.g., those based on Isaac Asimov's "Laws of Robotics" and adjacent principles, discussed by other great thinkers [31]. Without placing AI at the same level of risk as nuclear technology (civil and military), the international negotiations, treaties and organizations on nuclear control could provide useful lessons for a future international AI control regime. For a basic outline, see [21], chapters. 15 and 18. Beyond, and supporting, principles of humanity, cooperative CI models also need to define governance principles and roles for human–machine interfacing and partnerships, authority and hierarchy, management, and command and control loops.

Collaborative CI concepts will only prove successful if designed, developed and implemented in a joint effort of political, economic, commercial and societal forces, with massive support and participation from the scientific disciplines, including philosophy, sociology, psychology, economy, commerce and the Cyber and AI research community. This integrated concept needs to be established by an "elite movement", furnished with resources and authority. Basic rules need to be developed and agreed on, starting now. It is, however, hard to imagine that the adequate organizations and processes could be implemented by present-day political, administrative and economic systems, and the people in power (a typical, negative example of such organization is the final disposal of high-intensity radioactive waste, and the GE example: political discussions and temporary experiments for 70 years, a promised time horizon of site selection in the 2030s, site establishment "starting in the 2050-ies", i.e., roughly 100 years after the technology was introduced). A common understanding among all relevant groups will be necessary in all the discussed areas. Only a common agreement on the objectives of an ICHMaC strategy will facilitate solutions and better decisions than those of today. Political decision makers, legislation and responsible administrations need to accept the fact that CI can produce systems in support of many societal domains, superior to the possible solutions in present economic–political systems.

Consequently, the basic objective functions in the economy and politics need to be changed, to emphasize care for people and society, instead of saving traditional jobs, maximizing material wealth and becoming re-elected. Economic systems will need to focus on human–machine collaborative models, instead of maximizing profit at the expense of the "working", as is common today in terms of a too-high divergence in income levels, opportunities and social states. The financial system needs to be regulated and controlled.

This all needs to begin with a modified vision of quality of life that coincides with the AI vision of relieving humans from the current primary need for jobs rather than to live much self-determined lives. This is not a "neo-communist" utopia in a new tapestry. Such basic ideas on quality of life were discussed by ancient Greek philosophers and in the vision of Thomas More [23] in the early 16th century. Present-day visions are also available, e.g., in [14]. This will, however, require new ways of thinking about societal constituents, modified or new models of creating and maintaining jobs and incomes, different concepts of education and lifelong learning that keeps pace with technological and societal developments, and modified or new concepts of general social security and care.

The income profiles of governments need to be adapted. Classical income taxes on wages and other earnings, consumption, etc., may shrink, and need to be ramped up in other ways, as discussed in Section 4.3. Alternative sources may be needed for unproductive speculation, real estate, the exploitation of rare natural resources, etc. Otherwise, the diverging distribution of income between the current, so-called "elite", i.e., those who would "own" power over production, data, intelligence and capital goods, and the poorer and less privileged would accelerate, possibly to a ratio that may endanger social peace.

International organizations (such as the EU, UN, IMF, World Bank, OECD, G8 and global finance and commerce) need to support the ICHMaC concept. Political and scientific support will require increased funding of these organizations. In the same order of relevance, for the protection of the earth's environment, biosphere, resources and climate, ICHMaC needs to establish an international planning regime and binding agreements, which will be sustainable over periods far exceeding the current electoral/legislative intervals and planning horizons.

#### **10. Conclusions and Feasibility**

The basic overall requirement must be insight into, and propagation of, the idea that CI will be able to overcome the flaws in past and present socio-political systems, and the consensus and will of "all" that better and collaborative solutions can be achieved and create unprecedented win–win-models.

Future solutions for society in the cyber world will require international, improved global rules and arrangements. This is true for the cyber domain including AI, as well as for climate protection and compliance with human rights. Unfortunately, there are always reverse tendencies, with populist movements prospering and propagating absolutistic, traditionalist illusions of "the past was better" agendas. With their limited appreciation of science and their backwards orientation, they will support the CI ideas. Instead, they will obstruct them.

Concerning the diligent treatment and exploitation of AI, we have only briefly outlined an option for the future. Its realization would need to start now, with an assessment of the prerequisites discussed above. Implementation of the concept or subsets of AI should be smooth and peaceful. However, realizing the pace of development in all cyber areas, including AI, we strongly suggest that politics and the economy prepare for such solutions sooner rather than later. Regarding the coming challenges and enormous potential of AI, present "western-type" governments are more or less dormant; real strategic thinking and political concepts are missing, and transferring progressive ideas into practice takes too long and is hampered by bureaucracies. "We are not a pre-emptive democracy, we are a reactive one" should be seen as a wakeup call [32]. The concepts discussed here are rather revolutionary. Therefore, policies need to follow precautionary and pre-emptive strategies. Unfortunately, more autocratic regimes, in principle, might be better-suited to enforce,

implement and sustain them, provided their basic objective and agenda is the creation of better conditions for all. However, historically, this has rarely been the case. Less desirable "motivators" for change could be global catastrophic risk (two examples: Fukushima; Coronavirus pandemic), forcing politicians to act. In any political framework, CI will require new and more efficient ways of consultation and consensus-building between peoples, governments, industry and research.

There are two extreme options, at the opposing ends of the spectrum, and many in-between. Things driven by AI, automation and robotics could cause deep tensions and further inequality in societies, and the government and economy will face increased pressure in a "3rd economic revolution".

The first option may result in the further fragmentation of society, with people losing trust in existing political systems and technologies, feeling lost and no longer represented, needed, or taken care of. Increasing sympathies with the regional, autocratic systems and occurring at present are indications of this direction. At worst, this may lead to societal catastrophe, revolutionary mass movements, damage to democracy, and more (we have seen this in the 20th century and before).

The more optimistic second option, which is hopefully more likely and, of course, possible, would require unprecedented effort from political and economic sectors to create solutions guided by the prerequisites discussed above (and more). This would need to start now [33]. Unfortunately, there are almost no signs in politics, particularly not in education, that this will be prepared in time. As long as the high-level political discussion—as in Germany—is concentrated on internet access bandwidth, some local and rather modest investment in AI R&D, or the question of whether and how to allow or forbid smartphones in schools, we are tickling the little toe of the giant.

Speaking of giants, the largest ones at present, such as Google, Apple, Facebook, Amazon, Microsoft (GAFAM) and more, e.g., in China, are moving ahead. They have visions of future "cyber societies" and invest enormous amounts, including in AI (see e.g., [34]). Time will tell whether or not their results benefit mankind and individual societies. The truth is that their (GAFAM's) power is increasing, while that of democratic governments is shrinking, although we are all aware that IT and AI are THE drivers of the present and future economy and everyday life. In any case, the future role of AI/CI in societies cannot be left to the forces of the market.

This may sound like pessimism, with two apocalyptic options. Nevertheless, through the influential progress of scientific and technological development, political systems will either be forced to adapt to new governance models and rules, or industry power will increasingly dominate, or even dwarf, the decision space of government plans and action. Seneca had a fitting recommendation for such a phenomenon 2000 years ago: *"Fata ducunt volentem, nolentem trahunt"* (Fate leads the willing and drags the unwilling). Only strong coalitions between the IT giants, governments and international organizations will be able to create future CI-based societies, as stressed in this article. As HI and AI should be viewed as two complementary abilities, the political systems need to view societal regimes such as quality of life and the economy as integrated parts of the same medal, rather than competing ones, as typically occurs at present.

An easier transformation into CI—and transformation is what we are discussing should follow the "Think big, start small" principle: beginning an "Evolution" with rather small, non-spectacular, and politically and societal non-controversial sample projects (e.g., legalizing self-driving cars, improving healthcare systems, further automating routine bureaucracy, optimizing traffic flows and public transportation, fully automating taxation), learning from them and incrementally evolving politics and societies into CI domains of a larger size and greater impact (e.g., in law enforcement, legislation, rationalized political decisions guided by science, optimized and customized schooling, autonomous systems for safety and security, control of the finance system). In fact, cooperative IT-based models,

on a smaller scale, are already on their way, e.g., in software, for better administration, or robotics in production lines and supply chains (The Australian, 25 Jan 2019, pg. 18 on the automation of warehouse supply chains: " ... more sites are combining robots and humans."). The current and oncoming AI evolution, however, will not work at a biological pace, taking hundreds of years or more for small changes to evolve. We will have to cope with big changes within a few decades.

As realist–optimists, however, we are sure that the human race and society will survive for a long time, even without CI. Nevertheless, the question is HOW. That is what we are trying to advocate for, with some ideas for the betterment of society.

**Author Contributions:** R.H.: conceptualization of article, evaluation of mid- to long-term effects of AI on, and its potential for, future society; M.H.: AI methodologies and models, overall validation. Both authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** No institutional/ethical review required; no studies involving humans or animals.

**Informed Consent Statement:** No studies involving humans or animals.

**Data Availability Statement:** No supporting data beyond those of the references were needed.

**Conflicts of Interest:** The authors declare no conflict of interest existing.

#### **Abbreviations**

#### **References**

## *Review* **Influence of Artificial Intelligence in Civil Engineering toward Sustainable Development—A Systematic Literature Review**

**Bilal Manzoor 1, Idris Othman 1, Serdar Durdyev 2,\*, Syuhaida Ismail 3 and Mohammad Hussaini Wahab 3**

**Abstract:** The widespread use of artificial intelligence (AI) in civil engineering has provided civil engineers with various benefits and opportunities, including a rich data collection, sustainable assessment, and productivity. The trend of construction is diverted toward sustainability with the aid of digital technologies. In this regard, this paper presents a systematic literature review (SLR) in order to explore the influence of AI in civil engineering toward sustainable development. In addition, SLR was carried out by using academic publications from Scopus (i.e., 3478 publications). Furthermore, screening is carried out, and eventually, 105 research publications in the field of AI were selected. Keywords were searched through Boolean operation "Artificial Intelligence" OR "Machine intelligence" OR "Machine Learning" OR "Computational intelligence" OR "Computer vision" OR "Expert systems" OR "Neural networks" AND "Civil Engineering" OR "Construction Engineering" OR "Sustainable Development" OR "Sustainability". According to the findings, it was revealed that the trend of publications received its high intention of researchers in 2020, the most important contribution of publications on AI toward sustainability by the *Automation in Construction*, the United States has the major influence among all the other countries, the main features of civil engineering toward sustainability are interconnectivity, functionality, unpredictability, and individuality. This research adds to the body of knowledge in civil engineering by visualizing and comprehending trends and patterns, as well as defining major research goals, journals, and countries. In addition, a theoretical framework has been proposed in light of the results for prospective researchers and cholars.

**Keywords:** artificial intelligence; sustainable development; construction; civil engineering; machine learning; construction engineering

#### **1. Introduction**

Artificial intelligence (AI) is playing a critical role in civil engineering in the direction of digitalization and intelligence, allowing for substantial increases in automation, performance, and reliability, as well as establishing an active connection between physical and digital construction [1,2]. As a large economic sector, construction can influence national growth and development as well as help long-term growth [3]. According to a McKinsey Global Institute survey conducted in 2017, the global construction industry accounts for roughly 13% of global GDP, with that amount projected to rise to 15% by 2020 [4]. Furthermore, governments around the world, as well as the world's technology leaders, are putting more effort into the implementation of AI in order to gain a competitive advantage [5]. For example, the U.K. government recently signed an agreement to put the country at the forefront of the AI industry by investing an additional €1.8 billion to make the country more creative and innovative [6,7]. Similarly, France intends to spend €1.5 billion on AI science. First, such investments provide advanced economies with a

**Citation:** Manzoor, B.; Othman, I.; Durdyev, S.; Ismail, S.; Wahab, M.H. Influence of Artificial Intelligence in Civil Engineering toward Sustainable Development—A Systematic Literature Review. *Appl. Syst. Innov.* **2021**, *4*, 52. https://doi.org/10.3390/ asi4030052

Academic Editor: Teen-Hang Meen

Received: 29 May 2021 Accepted: 30 July 2021 Published: 6 August 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

competitive advantage at the national level; however, they can have a negative effect on the globalization of production and services [8,9]. In the current scenario, industrialized economies outsource resources, such as call centers and manufacturing, to developing economies due to cost advantages [10]. Companies that fully use the capabilities of AI, on the other hand, will no longer have to consider outsourcing costs or living wages [11,12]. Furthermore, rising overhead costs would limit or eliminate the outsourcing of services and output in the developing market, bringing services in-house. This change could result in a reduction in product prices and other company expenditures [13–16].

AI can be described as a distributed, statistical, and symbolic technique that focuses on simulating human functions before moving on to empathy and is making an effort to mimic acceptable communication and focuses on cognitive tasks before moving into the domain of previous approaches, such as deciding analytical ones, and devotes prior effort to modeling [17,18]. In addition, there are three levels of AI, as follows: (a) artificial narrow intelligence (ANI), in which machines only perform one mission, and thus are capable of making decisions on that one aspect of their environment, (b) artificial general intelligence (AGI), the categories of "AGI", which are also known as "powerful AI", "human-level AI", and "real synthetic intelligence" are computers that can find and comprehend the way, think abstractly, plan, figure out problems, learn quickly, and acquire experience quickly. Technologically, economically, or environmentally efficient vehicles and (c) artificial super intelligence (ASI), an intelligence that is far superior to the best human brain in all areas, including scientific ingenuity, general knowledge, and social skills [19–22].

Despite the fact that emerging innovations are opening up new avenues for community participation and governance, cities are still a long way from being genuinely smart and sustainable due to issues of disengagement and exclusivity [23,24]. Despite technological advancements (particularly in computer vision, robotics, and speech recognition), scientists, executives, and government officials have recently expressed concern that AI might endanger the livelihoods and capabilities of businesspeople, automate military operations, and potentially undermine the latter's pride in human superiority [25,26]. The world is entering a new period of sustainable growth, in which nations must work together to solve the most intractable issues such as persistent global poverty, social exclusion, economic inequality, poor governance, and environmental degradation [27,28]. AI is rapidly opening up a new frontier in the disciplines of sustainable development as a result of the employment of machines and robotics with deep learning capabilities. Deep learning skills have both disrupted and enabled governments and society [17]. They also have an impact on broader global sustainability trends. As AI affects our society, it has the potential to usher in a constructive future in which people cohabit peacefully with robots. More immediately, AI can help us make better progress toward the United Nations (UN) Sustainable Development Goals (SDGs) and push us even further toward sustainability goals [29].

A collection of ideas for understanding and caring for long-term prosperity, in addition to economic growth, with a focus on social security and environmental sustainability [30]. Much research has been conducted on the rise of AI from the beginning of the modern era to the present [31–34]. Throughout the 2000s, with their increasing contributions to AI exploration in the world of concepts, the media and academic journals added AI to a vast body of data, information, and solutions to problems in fields all over the world [35–37]. In addition, AI can use sophisticated algorithms to learn from big data and then apply what it has learned to help the construction industry [38]. Furthermore, AI offers numerous opportunities for substantial efficiency gains by analyzing large amounts of data rapidly and accurately [39]. Furthermore, AI systems and technologies can deal with complex, nonlinear functional problems and, once educated, can make predictions and generalizations at a high rate in digital construction [40,41]. AI has sparked significant interest in a wide variety of areas, including computer science, mechanical engineering, and civil engineering, attracting the attention of researchers [42,43].

Some reviews on this subject have been published as a result of the rapid growth of AI applications in civil engineering [44–47]. However, the majority of them only emphasize the importance of AI in a particular subfield, such as structural engineering [48], building information modeling (BIM) [49], automated construction manufacturing [50], computer vision [51], and others. Furthermore, Darko et al. [52] used scientometric analysis to perform a study on AI in the architecture, engineering, and construction (AEC) industry in order to increase awareness of AI in AEC. However, it does not include a thorough introduction to AI techniques and realistic AI applications in civil engineering. Yan et al. [53] conducted a literature review focusing on data mining in the construction industry. Data mining, on the other hand, is a subset of AI that is used to automatically process data and extract valuable insights. A bibliometric analysis of AI engineering applications was recently carried out [54]. However, their research is limited to only one journal's publications and provides a summary of previous work without offering directions for future work.

Therefore, the objectives of this study are:

In order to overcome this research gap, a systematic literature review (SLR) was used to seek responses to these questions for the benefit of academia and industry. It is anticipated that the answers to these questions would give rise to contributing knowledge in literature.

#### **2. Research Methodology**

A SLR was carried out through a systemic analysis to achieve the objectives suggested in the introduction of this paper. This approach has been chosen to reduce the risk of errors and make reliability possible [55]. The evaluation was conducted in five stages in accordance with the guidelines of Tranfield et al. [56] and Moher et al. [57], also known as PRISMA statement (i) selection of objectives; (ii) selection of databases; (iii) identification of the keywords; (iv) compatible papers selection; and (v) extraction of data.

Regarding the (i) objective, this study is to explore the influence of AI in civil engineering toward sustainable development as well as civil engineering awareness, civil engineering activities, features of civil engineering, and sustainable assessment. As to (ii) database selection, the authors decided to search within the "*Scopus*" database. The reason for using the Scopus database is that scientific journal papers are collected more widely than other databases. (iii) keywords were searched through Boolean operation "Artificial Intelligence" OR "Machine intelligence" OR "Machine Learning" OR "Computational intelligence" OR "Computer vision" OR "Expert systems" OR "Neural networks" AND "Civil Engineering" OR "Construction Engineering" OR "Sustainable Development" OR "Sustainability". The first search provided the result of 3478 Scopus papers. (iv) the selection process began, and documents were omitted other than engineering, for example, in connection with medicine, agriculture. Moreover, the papers related to publishing in conferences were also omitted because the conferences papers were not gone through the peer-review process. The number had now been reduced to 320 Scopus documents. Now, another screening process was conducted by evaluating the suitable title and abstract. The final outcome was 105. Afterward, (v) data extraction was subsequently performed. The detailed theme of research methodology is shown in Figure 1.

**Figure 1.** Research flowchart design.

#### **3. Results and Discussion**

The follow-up section explores in detail the annual publications trend of AI in civil engineering toward sustainable development, contributions of leading journals in terms of AI in civil engineering toward sustainable development, geospatial distribution, and comparison between developed and developing countries, civil engineering activities toward sustainability, features of civil engineering toward sustainability, AI, and sustainable assessment in order to fulfill the objectives of this study.

#### *3.1. Annual Publications Trend of AI in Civil Engineering toward Sustainable Development*

The study included 105 publications spanning the years 1995 to 2021(April). It was revealed that the trend of publications received its high intention of researchers in 2020. However, in 2021 (April), the number of publications was 22 in the domain of AI in civil engineering, and this number is expected to increase because the currently included publications were from January–April (2021). The detailed image is depicted in Figure 2.

#### *3.2. Contributions of Leading Journals in Terms of AI in Civil Engineering toward Sustainable Development*

Numerous studies emphasize and expound on the significance of conducting research in any field that involves the analysis of academic journals [58–60]. Such information can be useful to scholars in their search for knowledge tools, as well as to readers, who can then assist them in determining which journals are appropriate for publishing works related to AI in civil engineering [61,62]. It was revealed that the contribution of *Automation in Construction (61%)* was the greatest achievement, followed by *Advances in Civil Engineering (10%), Buildings (10%), Journal of Building Engineering (6%), Energy and Buildings (3%)*, and *International Journal of Civil Engineering (2%),* with the remaining journals contributing 1% equally. The detailed picture is shown in Table 1.

**Table 1.** Contribution of journals toward sustainable development.

#### *3.3. Geospatial Distribution and Comparison between Developed and Developing Countries*

The United States was the most influential in terms of publications, followed by China, the United Kingdom, Singapore, Australia, the United Arab Emirates, Iran, South Korea, Denmark, and Malaysia. However, developed countries (the United States, the United Kingdom, Singapore, Australia, the United Arab Emirates, South Korea, and Denmark) had significantly more impact on publications related to AI in civil engineering than developing countries (China and Malaysia). It is recommended to collaborate with developed countries to share ideas and works that will help to increase the shortage of publications in developing countries. Figure 3 depicts a more detailed picture.

**Figure 3.** Geospatial distribution of research publications.

#### *3.4. Civil Engineering Activities toward Sustainability*

Civil engineering activities have an effect on construction during the project's life cycle. These impacts occur during the construction process, operating process, and final demolition as a construct reaches the end of its lifespan.

#### 3.4.1. During Construction Process

To ensure a smooth construction phase, comprehensive plans of the project creation related to resources, schedule, cost, and others must be secured prior to the construction process [63,64]. There is a need to develop well-documented strategies that include cost reduction, duration, and scheduling of the realistic process. A suitable example is how a graphical representation of the building structures and systems will provide a complete overview during the construction process [65,66]. To ensure the effectiveness of these efforts, a comprehensive, flexible technique that serves as a framework for improvement activities is essential. This organized strategy was created as a consequence of the knowledge and experience gained from executing various sorts of improvement operations on numerous construction sites over the last six years. All of these measures have been designed to improve the construction industry's performance by decreasing waste and eliminating non-value-added activities from the building process [67].

#### 3.4.2. Operating Process

The project will have entered a new phase when construction is completed, known as operation, and the construction team will begin the operation process when this project is up and running [68,69]. The operating method not only helps to operate and maintain a constructed building while also maintaining the safety [70] and comfort of users, but

it also performs the expected functions during its life cycle [71]. To begin, maintaining a clear service flow of day-to-day operations helps track the facility's hazard and keep it safe while lowering the facility risk by ensuring that the systems work efficiently and that the number of failed operations is kept to a minimum [72,73].

#### 3.4.3. Demolition Process

A building may be demolished using various methods or techniques, including (a) blast, (b) wrecking balls, (c) hydraulic crushers and pulverizers, and (d) the top-down process. The method of demolition chosen is determined by the project requirements, site limitations, and equipment availability [74,75]. Demolition sites are typically located in congested areas where space for large machinery such as cranes is often restricted [76]. As a result, the first three demolition methods are typically unsuitable for the majority of demolition projects, and the top-down approach is the most widely used demolition method [77,78].

#### *3.5. Features of Civil Engineering toward Sustainability*

The key features of civil engineering consist of interconnectivity, functionality, unpredictability, and individuality.

#### 3.5.1. Interconnectivity

The interconnectivity of construction projects is most often created from the amount of work and the work combined with the degree of engagement of participation needed for the project [79,80]. Furthermore, in the construction industry, multiple workers are allocated, all of the uniform density and interconnected, which means that complex scheduling can be a problem [81,82]. The operation process should take a variety of elements into account, such as security, weather, operating hours, and time limit, and in order to deliver successful results smoothly, operations should be designed to take into consideration [83,84]. Furthermore, AI contributes to the automation of critical but time-consuming and repetitive operations, allowing humans to concentrate their time and energy on higher-value work activities. As a result, AI reveals insights that would otherwise be concealed in massive volumes of huge data sets that previously required human management and analysis, such as data generated by videos and images, as well as written reports and business papers, social media posts, and e-mail communications.

#### 3.5.2. Functionality

The initial scope and end results of projects are decided upon when they are established. If there are delays, however, the original end-goals become hazy. Changes or circumstances that occur during the lifespan of projects create a variety of needs or for a variety of reasons [85,86]. Any shifts in size, such as expansions or decreases, must be accompanied by similar changes in the schedule and budget of the project in order for all aspects of the project's dynamic reality to remain in balance [87,88]. The development of automated environmental sustainability studies of products and regions, as well as the optimization of energy use and distribution, is underway. Robotic vehicles that optimize routes and driving styles to reduce carbon emissions are also being developed. While static environmental benefits are determined by the construction, manufacture, and transportation prior to purchase, autonomous environmental objectives are provided from post-purchase autonomous interactions between an AI-enhanced product and its environment, which include knowledge and decision-making, and are determined by the design, production, and distribution of products prior to purchase. A domestic robot, for example, may clean the house and its surroundings autonomously using tools and gadgets that it purchases and collects on its own timetable [89,90].

#### 3.5.3. Unpredictability

In most cases, unknown risks must be detected before they manifest, which means they will invariably jeopardize the overall performance of the project. Notably, complex construction projects have a high degree of unpredictability, which is closely related to a number of variables [91,92]. Furthermore, forecasting and preparation must be carried out prior to construction under considerable uncertainty; specifically, determining the dates, times, and costs are often subject to change [93,94]. Some questions about the architecture design remain unanswered, such as if it can pass audits, if clients are happy with it, and so on. In addition, potential risks may be reduced if uncertainties are recognized and evaluated early on, raising the likelihood of a successful construction project [95–97].

#### 3.5.4. Individuality

Construction projects vary from one another due to variations in client specifications, project size, environments, influences, and constraints, which increases the complexity of project management [98]. When applied to a new project, repeating the schedule, design scheme, finance, and strategy is insufficient; therefore, there is no point in taking the first project's budget and plans and attempting to implement them on the new one [99]. Furthermore, designers, architects, contractors, and service providers are essential in any project since a product or service can only be designed with design work and a prototype [100,101]. It means that each project is carried out by a distinct group of people, and each group has its own set of characteristics, such as ability, knowledge, experience, and communication skills [102].

#### *3.6. AI and Sustainable Assessment*

With the current popularity of AI, the widespread availability of sensors, the advent of big data, the growth of e-commerce, the rise of the information culture, the interconnection and convergence of data, knowledge with society, physical space and cyberspace have fundamentally altered the information climate for AI development, ushering in a new evolutionary process [103–106]. New technologies have also permitted the emergence of a new form of AI. These key features include the advent of data-driven perception, Internet-mediated group intelligence, and technology-based human-machine hybrids for reinforcement learning, which both agree on the prediction of what human and media data will support and where each other converge [107–109].

People comprehend with their nervous systems and respond with their bodies. This is useful in explaining AI, which is based on cognitive computation and follows a different design paradigm than that which incorporates human cognition, emotional reasoning, and realistic judgment [110,111]. This new age of AI opens the door to a multitude of civil engineering applications in daily life, enabling the construction industry to diversify its goods and products while also using AI to enter uncharted territories for long-term growth [112]. Though impressive, these technologies are highly customized to specific tasks. Each application usually necessitates years of advanced study and thoughtful, one-of-a-kind sustainable construction [113,114].

New information and communication technology can be used to usher in new methods to promote sustainable growth, such as AI and robotics, which includes product design, production, management, testing, and integration of systems in new ways of the entire lifecycle [115–118]. In addition, sensor connectivity, understanding, thinking, perception, control, material, and environmental awareness, as well as human reasoning, must occur during the manufacturing life cycle. Human-machine communication is required, life cycle decisions must be taken, and machine and data analysis and controls must be enforced [119–121]. It is recommended that with application technology, the level and capability of infrastructure construction, single applications, synergy applications, and business growth must be evaluated. The incorporation of AI into sustainable products represents an opportunity to improve these products' environmental sustainability and, as a result, increase consumers' purchasing intentions while also appealing to new consumer segments who are not attracted to conventional environmental sustainability in the traditional sense. As a result, AI-enabled environmental sustainability can help firms build new competitive advantages and sell their products more effectively to consumers. Simultaneously, the effectiveness of a product varies based on the type of user and the product. When compared to traditional environmental sustainability, AI-based environmental sustainability has the potential to attract more consumers. This may enable firms to employ environmental sustainability to more broadly engage consumers across social boundaries, resulting in increased sales while improving the environment.

In the field of civil engineering, the value of AI is extremely great. However, the value of AI toward water resource management and sustainability is discussed here.

#### 3.6.1. Value of AI toward Water Resource Management

Water is essential for human survival. Thousands of years of human settlement and improvement decisions have been inextricably related to a reliable supply of safe and nutritious water [122,123]. One-third of the world's water is used in irrigation, with the remainder used for drinking and domestic purposes. Cities and societies all over the world work on a staggering amount of freshwater, processed water, and wastewater every day to meet the needs of human society [124,125]. Water must be treated and transported in accordance with hygiene and health standards to ensure that its material and properties meet the specifications of end-users [126,127]. Since artificial intelligence now enables managers to perform two things for water utility management: first, combine long-term growth projections with existing facilities and systems to anticipate future supply, and second, invest in new infrastructure based on results [128]. In general, the results of AI are only as useful as the data fed into it, and the performance generated by it is only as fully interpreted as the intentions of the organization that has it. As with all AI, as AI asks the questions and offers the answers, humans lose some of the information gained from solving the problems themselves [129–131]. The overall issue may be more visible, but it is not required for any particular answer to be given now. Solutions to other problems already perform several times better than human interaction, where things have a plurality of solutions, the tipping point moves from human interaction to automated responses.

#### 3.6.2. Value of AI toward Sustainability

As discussions, conversations, awareness, and campaigns have expanded over the years, and new and emerging technologies, such as AI, have entered the civil engineering market, the emphasis has shifted away from the environment and toward AI [132,133]. Though pushing the limits of human creativity, several organizations, including Microsoft, Google, and Tesla, have made significant efforts to create "earth-friendly" AI systems [134]. Other cases have shown that it is possible, such as Google's own DeepMind AI assisting the organization in reducing data center energy usage by 40% while also lowering overall (greenhouse gas emission) GHG emissions. Advancing AI architecture has added 3% to global energy demand through data centers and has also led to ensuring sustainable energy access for people living in rural and off-grid areas while also being active in micro grids and exploring ways to integrate renewable resources [135–137]. Furthermore, in cities, AI can be used to track and control the use of electrical power distribution, with artificial grids being used to generate less electricity, making them less unreliable [138]. It is the responsibility of researchers and scientists to ensure that the data collected from AI systems are completely transparent, unbiased, reliable, and credible [139]. Greater demand for automation, as well as more stringent environmental requirements and phenomena, is accompanied by rising demand for technologies with higher levels of analysis and study-driven precision, which necessitates increased investment in R&D, development, both in multinationals and in education and government, in the latter sectors.

#### **4. Summary of Findings**

A brief summary of results is discussed in the domain of civil engineering sub-fields. It can be seen that AI has had a major effect and made impressive advances in all sub-fields of civil engineering. The details are shown in Table 2.

#### **5. Theoretical Framework and Future Directions**

The overview of results serves as the foundation for the conceptual framework. It has been discovered that AI has a tremendous effect on construction activities such as computer vision, robotic construction, electronic augmentation, machine learning, digitalization, collaboration, and synchronization. While civil engineering includes several sub-fields such as structural engineering, construction engineering and management, transportation engineering, hydraulic engineering, geotechnical engineering, and environmental engineering, the role of AI in digitalization cannot be overstated. Machine learning techniques, in coordination with remote cameras and unmanned aerial vehicles (UAVs), provide promising non-contact solutions to civil infrastructure condition assessment in structural engineering.

Furthermore, in the transportation field, computer vision-based condition management techniques are used on railway networks. These approaches may be used to obtain data from computers for the purpose of tracking rail conditions [140]. The use of AI in the modeling of semi-arid area infiltration is important in hydraulic engineering. Infiltration is vital to stream flow, groundwater recharge, subsurface flow, and the quality and quantity of surface and subsurface water. In addition, landslides and slope failures are serious issues because they can result in the loss of life and property [141]. As a result, slope stability analysis is one of the most critical geotechnical engineering issues. Slope failures, like most geotechnical problems, are complex. The slope stability research had evolved in tandem with advances in computational geotechnical engineering [142,143]. Furthermore, reinforcement learning is a broad class of optimum control systems that increase performance in a highly dynamic, stochastic environment by estimating value functions obtained from experience, modeling, or search. The interactive environment promotes reinforcement learning, resulting in greater learning ability and adaptability. Because reinforcement learning does not need the use of a system dynamics model, it is a potential choice for ensuring long-term sustainability. The use of reinforcement learning in sustainability is certain to change the old way of energy use, and the system will gain intelligence.

Similarly, the rise of AI is expected to have an effect on global competitiveness, environmental assessment techniques, and long-term sustainability [144]. AI will produce data for more intelligent intervention targeting, minimize waste and losses in production and use, build new technologies that will change whole industries and careers, provide the requisite improvements in communication and cost savings, and bring the benefits of the rapid pace of technological growth to many people around the world. Furthermore, this theoretical framework laid a foundation for future researchers, policymakers, and practitioners to explore the limitations in future research. Figure 4 shows the findings and future directions in the theoretical framework. Furthermore, a recent study conducted by AlArjani et al. [145] built the framework for Saudi Arabia's sustainable development goals. It has made significant contributions to the body of knowledge by concentrating and spending greater resources on alternative energy sources such as solar, wind, biomass, and nuclear energy. In comparison to the current study, greater attention is placed on AI in civil engineering by emphasizing publishing trends, journal contributions, and aspects of civil engineering toward sustainability.

**Figure 4.** Theoretical framework.

#### **6. Conclusions and Limitations**

It was found that the pattern of publications in the domain of AI in civil engineering was received high attention in 2020. Furthermore, *Automation in construction* had played a significant role in AI-related publication works. However, it was discovered that developed countries paid more attention to AI research than developing countries. The contributions of this review are (i) to have a fundamental understanding of civil engineering and to demonstrate the possible importance of AI in supporting and enhancing construction work (ii) to discuss the influence of AI and sustainable assessment, which provides substantial evidence highlighting the benefits of AI techniques (iii) to provide a theoretical framework that will aid potential researchers in incorporating findings. It was revealed that AI had demonstrated the potential to accelerate the learning process, as well as to simplify and augment reasoning, which is useful in construction projects that differ in their basic characteristics. Technology and creativity have resulted in tremendous improvements in our living conditions. Technology development and innovation will also be at the forefront of the transition to a more sustainable future. Productivity-enhancing digital technologies such as AI can help to increase productivity, reduce production costs and emissions, reduce the intensity of production process resources, improve market correspondence, and enable the use of big data to make public services more accessible. From a theoretical approach, the recommendations presented in this research provide guidance on how to address any shortcomings in the process of defining additional research. From a practical aspect, this research can provide practitioners a modulated reference point that is easily accessible, as well as assist them in grasping the most recent techniques and methodology of artificial intelligence research aimed at sustainable development. This research has limitations, despite its contributions. The information is initially obtained from the Scopus database. In the future, additional data will be gathered by combining information from various databases for quantitative and qualitative studies (e.g., Google Scholar, Web of Science, and so on). Second, the focus of this study was limited to journal articles. As a matter of fact, the study's findings do not completely reflect the available AI literature. The limitations mentioned above offer excellent opportunities for further study, but they should be considered when assessing the research results.

**Author Contributions:** B.M.: Conceptualization, Investigation, Data creation, Writing—original draft; I.O.: Supervision, Writing—review and editing; S.D.: Writing—review and editing; S.I.: Writing—original draft; M.H.W.: Writing—review and editing. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**

## *Article* **Detection and Identification of Expansion Joint Gap of Road Bridges by Machine Learning Using Line-Scan Camera Images**

**In Bae Kim 1,2, Jun Sang Cho 2, Goang Seup Zi 1,\*, Beom Seok Cho 3, Seon Min Lee 3 and Hyoung Uk Kim 3**

**Abstract:** Recently, the lack of expansion joint gaps on highway bridges in Korea has been increasing. In particular, with the increase in the number of days during the summer heatwave, the narrowing of the expansion joint gap causes symptoms such as expansion joint damage and pavement blow-up, which threaten traffic safety and structural safety. Therefore, in this study, we developed a machine vision (M/V)-technique-based inspection system that can monitor the expansion joint gap through image analysis while driving at high speed (100 km/h), replacing the current manual method that uses an inspector to inspect the expansion joint gap. To fix the error factors of image analysis that happened during the trial application, a machine learning method was used to improve the accuracy of measuring the gap between the expansion joint device. As a result, the expansion gap identification accuracy was improved by 27.5%, from 67.5% to 95.0%, and the use of the system reduces the survey time by more than 95%, from an average of approximately 1 h/bridge (existing manual inspection method) to approximately 3 min/bridge. We assume, in the future, maintenance practitioners can contribute to preventive maintenance that prepares countermeasures before problems occur.

**Keywords:** bridge; expansion joint; joint gap; smart bridge maintenance equipment; sensor; structural health monitoring; line-scan camera; machine vision; machine learning

#### **1. Introduction**

The Korea Expressway Corporation is an institution that builds and maintains expressways in Korea, and the number of highway bridges in use reached about 9800 as of 2020. Recently, the number of narrow expansion joint gap occurrences has been increasing. It increased the most in 2018, when the number of days of summer heat waves was the greatest (see Figure 1, Table 1). If such a lack of expansion joint gap occurs, it may adversely affect the structural behavior of the bridge and can be a major threat to traffic safety [1]. Considering the characteristics of the climate of Korea, which has four distinct seasons, we can see that the importance of maintenance and the role of the bridge expansion joint responding to temperature changes between the cold and hot seasons is growing very high.

The causes of narrow expansion joint gap occurrences vary. Representatively, there are construction errors, inappropriate pre-setting, deformation of backfill, and alkaline silica reaction; recently, abnormal high temperatures have also emerged as one of the causes. The year 2018 had the highest number of heatwave days since 2013, and accordingly, the number of cases of lacking bridge expansion joint gaps and pavement blow-up damage rapidly increased. The Korea Expressway Corporation conducted a complete survey of bridges under its management and found that narrow expansion joint gap occurred in 276 bridges (2.96% of the total of 9334 bridges). Table 2 presents the main causes of the occurrence of narrow expansion joint gaps as analyzed through onsite investigations [2].

**Citation:** Kim, I.B.; Cho, J.S.; Zi, G.S.; Cho, B.S.; Lee, S.M.; Kim, H.U. Detection and Identification of Expansion Joint Gap of Road Bridges by Machine Learning Using Line-Scan Camera Images. *Appl. Syst. Innov.* **2021**, *4*, 94. https://doi.org/ 10.3390/asi4040094

Academic Editor: Luís Bragança

Received: 18 October 2021 Accepted: 16 November 2021 Published: 18 November 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

**Figure 1.** Examples of major damage at road bridge expansion joint: (**a**) blow-up in road pavement, (**b**) lack of joint gap in expansion joint, (**c**) damage to wing wall of bridge abutment, and (**d**) damage by excessive bridge support movement.

**Table 1.** Average annual temperature, average annual maximum temperature, and number of heatwave days (national).

**Table 2.** Analysis of causes of insufficient expansion joint gap.

Most importantly, after bridge expansion joint damages occur, a huge budget is required for maintenance, and restoration is virtually impossible unless the bridge is remodeled. Experience shows that the life-cycle costs of a bridge expansion joint, over the life of the bridge, are many times greater than the initial costs of supply and installation, especially when consequential impacts such as traffic disruption during replacement works are considered [3].

Therefore, to solve these problems, the overall design/materials/construction/maintenance should be re-examined to supplement the relevant standards, but this has the problem of consuming a large amount of budget and time. Gaining a full understanding of the demands on the bridge's expansion joints, and how well they are performing, can enable adjustments and maintenance measures to be tailored to maximize the length of the service life with a minimum of maintenance effort [4–7].

In this study, to detect problems and minimize the damage that may occur in the bridge expansion joint at an early stage, an optimal inspection method was developed by converting a vast number of bridge expansion joint gap maintenance inspection methods from existing manual inspections to automated inspections. To this end, it was decided to incorporate an M/V technique to switch from the existing manual inspection method of the bridge expansion joint device to a precise high-speed inspection method.

M/V refers to the technology and methods used to provide imaging-based automatic inspection and analysis for such applications as automatic inspection, process control, and robot guidance, usually in industry. It refers to many technologies, software and hardware products, integrated systems, actions, methods, and expertise.

Definitions of the term M/V vary, but all include the technology and methods used to extract information from an image on an automated basis, as opposed to image processing, where the output is another image. The information can be used for such applications as automatic inspection, robot and process guidance in industry, security monitoring, and vehicle guidance [8–10]. This field encompasses a large number of technologies, software and hardware products, integrated systems, actions, methods, and expertise [10,11]. M/V is practically the only term used for these functions in industrial automation applications. It attempts to integrate existing technologies in new ways and apply them to solve realworld problems in a way that meets the requirements of industrial automation and similar application areas [10,12].

The primary uses for M/V are imaging-based automatic inspection and sorting and robot guidance [13,14]. The imaging device (e.g., camera) can either be separate from the main image processing unit or combined with it, in which case the combination is generally called a smart camera or smart sensor [15,16]. While conventional (2D visible light) imaging is most commonly used, alternatives include multispectral imaging, hyperspectral imaging, imaging various infrared bands, line scan imaging, 3D imaging of surfaces, and X-ray imaging [13]. We used the method of line scan imaging, which we think is most effective in technology that can automatically acquire images while driving at high speeds. [17–20].

One of the purposes of the research is to use a line-scan camera in M/V to record road surfaces as images with a resolution of 1.0 mm per pixel in a high-speed (100 km/h) environment. Existing line-scanning cameras are used for the purpose of identifying counterfeit bills and inspecting semiconductor wafers' surfaces on belt conveyors, so they can be said to be the best technology for introducing to fast-moving vehicles on roads [21,22]. This is suitable for this study because it is free from the problem of lens distortion that may occur in an area-scan camera [23]. According to this study, we developed a survey system equipped with line-scan cameras, Global Positioning System (GPS), and a distance measurement instrument sensor (DMI) in the vehicle, and as a result, we could obtain a safe and accurate image without blocking the road [24–28].

Recently, deep-learning-based approaches [29] have been applied to many problems in various industrial and academic fields. Visual recognition tasks, which extract information of interest from images, such as image classification [30], object detection [31], and semantic segmentation [32], have been actively studied. In particular, CNN (convolutional neural networks) have shown successful results in many visual recognition applications. In this study, models based on CNN and some deep learning techniques are applied to image analysis problems to determine the distance between expansion joints. Two CNN models are used to analyze the distance step by step. First, image classification categorizes images semantically into sub-groups. An image-classification CNN extracts important image features like texture cues and shape cues from each image, and a logistic regression [31] distinguishes them between categories based on the extracted features. Successful studies on image classification CNNs have been conducted focusing on the ImageNet benchmark dataset [33], and some design patterns have been proposed to achieve technical goals such as learning more complex patterns (ResNet) [34], light model weights (MobileNet) [35], and efficient scalability (EfficientNet) [36]. These design patterns, called CNN architectures, are applied to general image classification to reduce the need to design new models every time. Second, semantic segmentation is the process of classifying each pixel in an image belonging to a particular class. The recent success of CNN has also driven outstanding progress in semantic segmentation [37–46]. Based on CNN architecture, the semantic segmentation model extracts local contexts (a small area centered in a pixel to classify) and global contexts (overall semantics of input image) from an image and reconstructs contexts to create a class heatmap of the same size as the original image, which shows the probability of which class each pixel is. Therefore, it can be thought of as a classification problem for each pixel, considering local and global contexts.

Another purpose of the study is to develop software that measures gap width with more than 90% accuracy in the road image of the survey so that many bridges can be monitored quickly. Therefore, an algorithm was developed to measure the gap width after first finding the expansion point in the road image using machine learning. As a result, the identification accuracy was more than 95%, and the investigation time was reduced by more than 95%, from an average of about 1 h/bridge (the inspector's existing manual inspection method) to about 3 min/bridge.

The main contribution of this study is to demonstrate the possibility of developing smart maintenance techniques for road structures, in other words, the successful development of a smart maintenance system that combines machine vision and machine learning technology, for the following purposes: first, to ensure the traffic safety of vehicles on roads that are already in use; and second, to obtain the condition of road structures at the level the investigator wants with figures and accurate images.

The article is organized as follows:

#### **2. Development of Monitoring Technology for Bridge Expansion Joint Using Line-Scan Cameras**

Recently, the Korean government has been actively encouraging the introduction of the latest structural safety inspection technologies through the revision of various laws and regulations. For example, Annex 10 of the Enforcement Decree of the Special Act on the Safety and Maintenance of Facilities was amended in March 2020 to newly establish a provision for "appearance investigation and image analysis using new technology or inspection robot, etc.". In other words, new technology can be applied for structure inspection. Further, the revision of Article 167 of the Occupational Safety and Health Act emphasized the safety management of internal employees by strengthening the punishment standards for employers (a maximum fine of \$100,000 USD) in case of workplace accidents caused by negligence in worker safety management [1,2].

As a result, the structure management conditions of the Korea Expressway Corporation are becoming increasingly strict. The establishment of facility and performance evaluation following the revision of the Special Act on Safety and Maintenance of Facilities newly necessitated the performance evaluation of 2611 Type 3 facilities (bridges) and 4626 structures. As a result, structural inspectors performed 25,219 regular safety inspections, 2205 precise safety inspections, and 258 precise safety diagnoses in 2020, including in-house and external services. A shortage of management personnel is arising owing to this excessive workload (see Table 3).

**Table 3.** Number of structural inspections (2020).

1 Regular safety inspection: an average of approximately 450 EA/yr/branches performed by the structural staff. 2 Precision safety inspection (2020): self-manpower performance (684EA), performing external services (1521EA).

Given that the stricter structure management conditions are imposing greater demands on manpower, it is crucial to promote inspection methods and smart technologies that can replace manpower.

The Korea Highway Corporation Road Traffic Research Institute analyzed past safety inspection and technical advice data and found that 78.3% of expansion joint devices that had to be replaced lacked a gap. Since 2016, we have been researching monitoring methods to preemptively maintain a suitable gap between expansion joints [2].

For this purpose, by integrating high-speed line-scan cameras, which are widely used in various fields such as semiconductor inspection and road pavement investigation, image-processing technology to determine the length of expansion and contraction through AI-based (i.e., machine learning) image sensing in the analysis process, and automatic control technology, we developed the **N**onstop bridge **EX**pansion joint gap measuring **U**tility **S**ystem (NEXUS) to measure the distance between expansion joints with 1.0 mm resolution while driving at a high speed of 100 km/h [1] (see Figure 2).

**Figure 2.** Development system (NEXUS) and main function configuration.

Existing methods manually measure the gap in highway bridge expansion joints through traffic control and partial blockage. By contrast, the proposed method uses a highspeed line-scan camera mounted on a vehicle driving at 100 km/h to acquire an image across a 40 cm-wide car lane. This system uses the geographical information system data of the bridge (above, longitude coordinate system) to perform automatic measurements in conjunction with the mounted GPS while driving at high speed and thereby creates a database of the expansion gaps based on accurate survey images without affecting the traffic flow. In this study, a test survey was conducted on approximately 5000 bridges along the highway, and the analysis results were used for big-data-based machine learning for developing algorithms to accurately determine the length of the expansion joint gap depending on its type and site conditions (see Figure 3) [45–62]. The NEXUS system and the on-site test survey introduction is available on our YouTube channel [63].

**Figure 3.** UI configuration of the analysis program: measurement point image; survey trajectory (link to GPS); front-view status; value of the joint gap.

The use of the NEXUS for performing automatic surveys of the bridge expansion joint device gap while driving at high speed reduces the survey time by more than 95%, from an average of approximately 1 h/bridge (existing manual inspection method) to approximately 3 min/bridge. In addition, if the accumulated gap is monitored and preemptive maintenance is performed before gap narrowing occurs, it can eliminate the risk factors in the future temperature expansion behavior of the bridge and contribute to traffic safety and cost reductions.

#### **3. Initial Gap Measurement and Evaluation of New Bridge Expansion Joint Device**

Previously, it was difficult to use data when necessary, as they were managed in the form of reports and fields. By contrast, by using the NEXUS device, initial data can be accumulated, and the adequacy of highway bridge expansion joints can be checked and evaluated online at any time. To confirm this, we tested the initial data for the adequacy of the pre-setting after 300 days, when the initial drying shrinkage and creep deformation roughly converge; in this regard, it can be beneficial to use the initial gap database (D/B) for the three routes completed at the end of 2016. As the pre-setting of the expansion joint device is not consistent with the daily average temperature during winter and summer at the time of construction, it is essential to adjust the gap in advance such that it has an intermediate value at the reference temperature (average daily temperature of 15 ◦C).

The route to be investigated has 302 expansion joint devices of three types, namely, rail type, steel finger type, and mono cell type (see Table 4). We converted the expansion gap and average temperature (obtained from the Korea Meteorological Administration) at 15 ◦C during the investigation and evaluated the adequacy of the pre-setting. After analyzing the initial values of the new bridges, we investigated some bridges that were not properly preset, potentially due to lacking gaps during summer (35–40 ◦C) or excessive gaps during winter (−15 ◦C) (see Figures 4 and 5).

For example, the design temperature range ΔT is from −5 ◦C to 35 ◦C in the prestressed concrete girder type expansion bridge design movement calculation standard. Therefore, the reference temperature for pre-setting can be seen as 15◦C, which is the median temperature. In fact, the temperature at the time of construction of the new joint is under various temperature environments, not 15 ◦C, so pre-setting is performed at the time of installation to increase or decrease the spacing at the time of installation. The y-axis "Percentage of the capacity of the new joint" is to ensure that pre-setting was performed properly at the time of construction:

This value means the following:

As such, if it is possible to investigate and analyze the expansion joint device gap while driving, without blocking a separate route, from the beginning, and to convert it into D/B in the bridge management system, by monitoring the changing trend, it is possible to take preemptive measures in case of emergency (such as when the gap narrows in a short time).

**Table 4.** Bridge expansion joint devices installation status.

**Figure 4.** Pre-setting suitability evaluation of new bridge expansion joint devices.

 **Figure 5.** False-detection case of expansion joint gap.

#### **4. Advanced Identification of Expansion Gap Using Machine Learning**

Expansion joints device has different details for each development and sales company, and typical expansion joint devices constructed on road bridges in South Korea include mono cell type, finger type, and rail type. Table 5 lists the types of expansion joint devices installed on highway bridges in South Korea as of 2014.

**Table 5.** Expansion joint device installation status (2014).

The analysis of the expansion joint gaps of 4821 bridges and 12,825 devices on the highway mainline from 2017 to 2019 using NEXUS equipment revealed various types of expansion joint devices and gap identification errors depending on the condition of the expansion joint device body installed in the field. False-detection errors include the case of erroneously identifying another gap in the expansion joint body as a gap and the case of detecting a specific part of the road surface (portholes, repair marks, cracks, etc.). On average, the gap identification accuracy was 67.5%; that for the finger type with a complex shape was the lowest at 51% (see Table 6, Figure 5). False detections were attributed to limitations of the traditionally coded image analysis algorithm. Therefore, the image identification method must be improved using machine learning, as it is superior to traditional coding.

**Table 6.** Gap identification accuracy by type of expansion joint (2017–2019).

We aimed to overcome the limitations of the traditional gap identification algorithm. Machine learning can be used to learn previously investigated images as big data, and to thereby identify the expansion joint device from an input image and more accurately determine the gap value. The input image obtained using the NEXUS equipment was recorded as multiple 10,000 × 1024 pixels images, depending on the irradiation distance. The image analysis system consisted of cascaded AI vision modules, in which the work process analyzed high-resolution input images for realizing effective calculations. The AI process involves the following steps: original image input > expansion joint area extraction AI network (classification) > expansion joint area image area (cropping) input > gap extraction AI (segmentation) > gap measurement (algorithm) (see Figure 6).

**Figure 6.** Expansion joint device analysis process for AI method (i.e., machine learning).

To measure the gap value, the position of the expansion joint device must be specified in the image. Further, for achieving high measurement accuracy, the expansion joint device needs to be recognized at the pixel level. This can be done using image segmentation methods used in computer vision. Further, a deep learning model with extensive data learning is used for problems with large changes in illuminance, angle, image shape, and noise generation, such as road environments.

The AI machine learning model for expansion joint device discovery was implemented to solve the classification problem using CNN as a feature extractor. The original image with a size of 10,000 × 1024 pixels was cut into an image patch with a size of 1000 × 1024 pixels to be inputted to the AI model with minimal distortion. Then, 19 image patches were generated for each line-scan image, and classification was performed to find the image patches in which the expansion joint device exists (see Figure 7).

**Figure 7.** Image segmentation and classification to search for expansion joint device.

To find the point with the minimum gap distance, it is necessary to precisely segment the gap region from the image patch where the expansion joint device appears. The AI machine learning model for gap region extraction used U-Net, a representative segmentation model. The first AI model generated and learned training data by labeling pixels in the corresponding area to extract metal parts with characteristic textures from the line-scan images of the expansion joint device. The second AI model was trained to find the gap region between the extracted joint devices (see Figure 8).

**Figure 8.** Segmentation process for extracting gap area.

After obtaining a binarized image with the gap region and the background, we found the point with the minimum gap distance. The minimum gap distance in pixels was converted to the final gap value in millimeters by multiplying the information in millimeters per pixel.

#### *4.1. AI-Based Image Analysis*

Line-scan images obtained using NEXUS were converted into high-resolution digital images with a length and width of 10,000 and 1024 pixels, respectively. The unit length (mm) of the pixels can be calculated by considering the vehicle speed at the time of the shooting. If the expansion joint device can be measured in pixels, the actual value of the gap distance can be inferred within the error range. Therefore, recognizing the position of the expansion joint device in the line-scan image using a program and accurately measuring the size of the gap in pixels determine the measurement accuracy.

To perform highly accurate analyses, a large number of line-scan images were trained using deep learning. To efficiently analyze high-resolution input images, cascade-type analysis procedures were configured. The analysis involved the following steps: (1) ultrafast line-scan image input, (2) extraction of small image patches in a square matrix, (3) recognition of expansion joint device among the image patches, (4) expansion gap division, and (5) gap distance analysis and actual value calculation (see Figure 9).

**Figure 9.** Spacing measurement process of expansion joint device.

#### *4.2. Expansion Joint Device Recognition*

Line-scan images have a high resolution of more than 10,000,000 pixels and require considerable memory and computational resources when being analyzed using a deep neural network. Therefore, it is necessary to divide the image into small image patches for realizing effective computations. Through the window sliding method, images were extracted every 1000 pixels in the longitudinal direction to generate a total of 19 image patches with a size of 1000 × 1024 (see Figure 10). Each image patch is again converted to a size of 224 × 224 pixels and subjected to binary classification.

**Figure 10.** Line-scan image segmentation to search for expansion joint devices.

Each image patch is again converted to a size of 224 × 224 pixels for binary classification. By using a CNN, an effective neural network for image analysis, we labelled patches with the expansion joint device as y = 1 and patches without the expansion joint device as y = 0 and classified them by logistic regression. The logistic regression followed by CNN infers the probability P that there is an expansion joint in the image patch. The error of the model prediction for each image patch is calculated with binary cross entropy [64–69], which compares each of the predictions to the actual class, which can be either 0 or 1. The errors of each set of training data are summed up. In stochastic gradient methods [70–77], the cost and sum of errors is used to update current model parameters to reduce the distance from the optimal point in the parameter space. The equation of binary cross entropy is shown as follows:

$$L = -y \log P - (1 - y) \log(1 - P) \tag{1}$$

Because class imbalance exists in a ratio of 1:9, a weight is assigned to each class.

$$L\_{\text{weight}} = -\varepsilon \times y \log P - \mu \times (1 - y) \log(1 - P) \tag{2}$$

where *-*= 9.0 , *μ* = 1.0.

The cost function *J* is calculated by averaging the errors for *N* training data and adding the L2 regularization to reduce overfitting [78–85] of the model.

$$J = \frac{1}{N} \sum\_{j=1}^{N} L\_{\text{weighted}}^{j} + \lambda |w|^2 \tag{3}$$

where *λ* = 10−4.

The gradient descent method updates the model parameters in the direction of reducing the cost function *J* as follows:

$$w \Leftarrow w - \ n \cdot \mathfrak{g} \tag{4}$$

where *α* = 10−4, *g* = *∂J ∂w* .

We tested ResNet, MobileNet, and EfficientNet, all of which are well-known CNN architectures [22–24]. Gradient vanishing is more likely to occur as the layers of the deep learning model deepen, and ResNet solved this problem by performing residual learning using skip connection. The structure obtained high accuracy compared to the scale of the model. MobileNet proposed a depth-wise separable convolution that reconstructed the existing convolution method to reduce the computational amount of the model. Compared to the popular models at the time of proposal, the amount of computation was significantly reduced, and the same accuracy was maintained. EfficientNet, which empirically reports a methodology to increase model complexity to improve performance, updated the state of the art for benchmark datasets.

We trained the three models to a binary classification of expansion joints and nonexpansion joints (see Figure 11). Through the test, the EfficientNet model showed the highest performance, with recognition accuracy of 97.57% for the expansion joint device. Further, it was better to start learning from randomized initial parameters than through transfer learning for ImageNet. This is because the analysis image has different characteristics from the general characteristics of ImageNet. We employed EfficientNet for expansion joint detection, as it has the highest accuracy.

**Figure 11.** Recognition accuracy test results for each CNN structure.

#### *4.3. Expansion Joint Gap Segmentation*

Even if the image patch of the expansion joint is extracted from the line-scan image, image segmentation is required within the detected image to accurately measure the expansion gap. Image segmentation is a pixel-level classification that deduces the class each pixel belongs to (i.e., expansion joint device or background). A masking image

representing the pixel corresponding to the expansion joint device can be generated by the image segmentation algorithm.

Figure 12 shows the image patch (left) and correct mask (right) of the expansion joint device. The neural network structure receives image patches as an input and performs binary classification of the individual pixels. The deep learning model receives image patches as the input and learns to predict a masking that is close to the correct answer. In this case, the class of a pixel should be determined by considering the global and local characteristics of the image rather than individual pixel values.

**Figure 12.** Image of flexible expansion joint and mask of correct answer: (**a**) original image, (**b**) masked image.

This study uses a U-Net model, which was previously developed for sophisticated analyses of organ lesions in the field of biomedical science [36]. U-Net is suitable for the problem of precisely detecting the shape of the analysis target by simultaneously learning the global and local information of the image. The structure of U-Net was modified to develop EfficientNet, a neural network that extracts image information.

U-Net infers the probability that an arbitrary pixel is an expansion joint device and learns the correct answer for each pixel from the correct answer masking. Therefore, the prediction error for all pixels *M* of the image patch expressed as BCE is as follows:

$$L\_{\text{path}} = \sum L\_{\text{pixel}} = \sum\_{i=1}^{M} (-y\_i \log P\_i - (1 - y\_i) \log(1 - P\_i)), \text{ } i = 1, 2, 3, \dots, N \tag{5}$$

The final cost function *J* is expressed by summing the mean error for *N* image patches and the L2 regularization term:

$$J = \frac{1}{N} \sum\_{j=1}^{N} \left( L\_{patch}^{j} + \lambda |w|^2 \right) \tag{6}$$

where *λ* = 10−4.

The gradient descent method updates model parameters in the direction of reducing the cost function *J* as follows:

$$w \Leftarrow w - \ n \cdot \mathfrak{g} \tag{7}$$

where *α* = 10−4, *g* = *∂J ∂w* .

We compared the performance between U-Net's masking image and the correct masking image in the test dataset. Table 7 shows the pixel-level classification performance for the test set; the pixel precision of the expansion joint device was 96.61%, recall rate was 94.38%, and f1-score was 95.49%. The f1-score of the expansion joint device pixel detection was within 5%. In the post-processing process, by correcting the error of the

predicted masking image of U-Net, the minimum gap point was detected, and the distance was measured.

**Table 7.** Gap identification accuracy by type of expansion joint (2017–2019).

#### *4.4. Gap Distance Analysis Algorithm*