**Funding:** Research of L. Lhotska and O. Stepankova was supported by institutional resources of Czech Technical University in Prague.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**

## *Article* **Intersectional Study of the Gender Gap in STEM through the Identification of Missing Datasets about Women: A Multisided Problem**

**Genoveva Vargas-Solar**

CNRS, University Lyon, INSA Lyon, UCBL, Ecole Centrale Lyon, University Lyon 2, LIRIS, UMR5205, 69622 Villeurbanne, France<EMAIL_ADDRESS>
**Abstract:** This paper discusses the problem of missing datasets for analysing and exhibiting the role of women in STEM with a particular focus on computer science (CS), artificial intelligence (AI) and data science (DS). It discusses the problem in a concrete case of a global south country (i.e., Mexico). Our study aims to point out missing datasets to identify invisible information regarding women and the implications when studying the gender gap in different STEM disciplines. Missing datasets about women in STEM show that the first step to understanding gender imbalance in STEM is building women's history by "completing" existing datasets.

**Keywords:** data; missing datasets; gender gap; data-driven studies; women in artificial intelligence; women in data science; women in STEM

#### **1. Introduction**

Many science and engineering occupations are predicted to grow faster than the average rate for all professions. Workforce projections in 2018 by the U.S. Department of Labour showed that nine of the ten fastest-growing occupations that require at least a bachelor's degree would need significant scientific or mathematical training [1]. This prediction has been confirmed in recent years, and computer science fields with great potential like artificial intelligence (AI) and data science (DS) can be certainly included. More recent sources anticipate that 75% of future jobs will be related to these fields (https://lac.unwomen.org/en/dig iteca/publicaciones/2020/09/mujeres-en-ciencia-tecnologia-ingenieria-y-matematicas-enamerica-latina-y-el-caribe, accessed on 1 June 2022), given that 7.1 million jobs were expected to be displaced by 2020, and half of existing jobs will disappear by 2050. Some of the most significant increases will be in engineering (and computer-related) fields in which women currently hold one-quarter or fewer positions [2,3]. Indeed, only 22% of all professionals working in AI around the world are women (World Economic Forum. Global Gender Gap Report 2018, https://www.weforum.org/reports/the-global-gender-gap-repo rt-2018/, accessed on 1 June 2022). The ubiquitous male dominance in countries in different regions results in a feedback loop shaping gender bias in AI and machine learning systems used in DS experiments [4–6]. These fields are particularly fast-moving [7] both in industry and academia, so it is essential to map how gender gaps (Davenport, T.H. and Patil, D.J. (2012) Data Scientist: The Sexiest Job of the 21st Century. Harvard Business Review. Retrieved from: https://hbr.org/2012/10/data-scientist-the-sexiest-job-of-the-21st-century, accessed on 1 June 2022) [8] are manifest in data-driven solutions comprehensively. If not addressed soon, the gender gap in STEM will widen during the Fourth Industrial Revolution. Science and engineering address key challenges of our time:

• Finding cures for diseases like cancer and malaria, tackling global warming, providing people with clean drinking water, developing renewable energy sources, and understanding the origins of the universe;

**Citation:** Vargas-Solar, G. Intersectional Study of the Gender Gap in STEM through the Identification of Missing Datasets about Women: A Multisided Problem. *Appl. Sci.* **2022**, *12*, 5813. https://doi.org/10.3390/

Academic Editors: Aida Valls and Karina Gibert

Received: 1 April 2022 Accepted: 2 June 2022 Published: 8 June 2022

app12125813

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

• Engineers design many of the things we use daily, including buildings, bridges, computers, cars, wheelchairs, and X-ray machines.

The presence of women in teams seem to increase collective intelligence, and if those women have decision making positions, success probabilities increase in startups. When women are not involved in designing products and addressing social and political problems, then needs and desires unique to women may be overlooked. For example, a predominantly male group of engineers tailored the first generation of automotive airbags to adult male bodies, resulting in avoidable deaths for women and children [9]. With a more diverse workforce, scientific and technological products, services, and solutions are likely to be better designed and more likely to represent all users.

The consolidation of AI and DS (World Economic Forum. Global Gender Gap Report 2018. Retrieved from: https://www.weforum.org/reports/the-global-gender-gap-report -2018, accessed on 1 June 2022, World Economic Forum. Global Gender Gap Report 2020. Retrieved from: https://www.weforum.org/reports/gender-gap-2020-report-100-years-pay -equality, accessed on 1 June 2022, World Economic Forum. The Future of Gender Parity: A Labour Market Shift. Retrieved from: https://reports.weforum.org/global-gender-gap-re port-2020/the-future-of-gender-parity/?doing\_wp\_cron=1612906910.182411909103393554 6875, accessed on 1 June 2022, as backbone tools to promote data-driven solutions opens the risk of overlooking women in the loop. Women are underrepresented in observation and data collection tasks. Their workforce in AI and DS is small, and they are underrepresented in algorithm design and testing teams. Thus, their characteristics are not represented because of how data are engineered. Their needs are not visible for designing units, even if more than the 50% of the planet's population are women (The National Centre for Women & Information Technology (NCWIT https://ncwit.org, accessed on 1 June 2022.) argues about the importance and interest for organizations to search for gender and race equality. Besides the economic welfare, teams respecting gender equality seem to be more creative and effective in performing tasks and sharing knowledge).

When science is combined with economy, politics and social impact, it is possible to classify disciplines and reason about the capital they produce and who produces that capital. To activate the economy, every asset that is potentially productive must participate in the active STEM market. Women are a massive part of the assets that remain excluded from the equation, weakening the economy. Countries with dynamic economies like Germany lack human force in these disciplines and import brains to fulfil the requirements of their economy. According to the economist François Lenglet in his TV show "La guerre des âges", two strategies are possible for these economies: either hiring an immigrant brain force or integrating women into the production market. Surprisingly, Germany is developing a very aggressive strategy for attracting immigrant experts (!) (Is the old German saying that associates women to the three K's (i.e., Kirche, Küche, Kinder), still valid? The German gender issues are out of the scope of this paper. Thus, we leave this topic open and we invite the interested reader willing to seek for more information to visit [10,11]). Why are women excluded from the equation? Are they recognised as a potential labour force, or are they hidden Cinderellas invisible to statistics? Are gender bias and discrimination playing a role in women exclusion in STEM?

We believe that it is critical to study the phenomenon of data and women's absence in STEM from many different perspectives. Since women come in many different sizes, colours, and formats, the study must consider several communities from the global north and south instead of general studies.

This paper studies missing data under a qualitative approach [12] that combines grounded theory and use case strategies for answering the following research questions:

RQ3 Do missing intersectional perspectives prevent performing representative quantitative analysis about women's activity and contribution in STEM?

Our work adopts a qualitative methodology that does not rely upon mathematical, data mining or artificial intelligence models. Still, this "alternative" analytics choice can be seen as a preparation protocol for data science experiments that use AI models because of the critical role of data in ensuring "representative" results. These models are adapted in cases with datasets that are representative observation samples of a phenomenon, even if raw. Yet, the hypothesis that drives this study is that missing datasets giving insight into female contribution are missing. We question the possibility of producing statistical and induced or deduced knowledge from incomplete observations.

#### *1.1. Context and Methodology*

For performing a study on missing data and female absence as two perspectives of a mirror, it is essential to consider the conditions in which data were absent before and after the pandemic years. In the global north, there are programs like UN Women (https://lac.unwomen.org/en, accessed on 1 June 2022), She Figures (https://ec.europ a.eu/assets/rtd/shefigures2021/index.html, accessed on 1 June 2022), and Davos reports (https://www.eversheds-sutherland.com/global/en/what/articles/index.page?Ar ticleID=en/Data-Protection/World\_Economic\_Forum\_Report\_2022, accessed on 1 June 2022) that have collected data about women in the economy, particularly in STEM. They perform serious data collection, and they often rely on governmental data. Other works like [13] perform analytics processes to measure the gender gap index in schools (before university and possible activities that can generate scientific contribution) by combining qualitative and quantitative methods. However, data are not intersectional (Intersectionality is an analytical framework for understanding how aspects of a person's social and political identities converge to create different modes of discrimination and privilege, https://en.wikipedia.org/wiki/Intersectionality, accessed on 1 June 2022); companies and institutions do not share them, so they are not included in studies. In consequence, despite these efforts, data about women are still missing. Thus, it is capital to understand the forces that play in the persistence of this discriminatory situation and the distribution of opportunities [14]. We need datasets as complete as possible, and as fine-grained and representative as possible, to perform clear data-driven studies about gender balance. Yet, these studies' fuel (i.e., datasets) is partial, incomplete or absent. It is not a matter of the analytics processes (data preparation, cleaning, engineering) nor the analytics models applied to extract knowledge. The point is that without intersectional data, which is often missing in available datasets, the gender issue cannot be modelled and understood in its complexity. These studies can lead to conclusions but do not wholly understand the gender gap problems. Yet, to efficiently address gender gap problems, we believe that we need to identify which datasets are missing. Our work focuses on tracking data that provide evidence about women's participation in STEM, particularly in CS, AI and DS.

Therefore, our work focusses on missing datasets instead of analysing the gender gap with available ones. These studies have been already done at the national level in different countries (e.g., in the UK [15], or specific datasets [16]) by international bodies like UN Women, the European Commission with the "She Figures" report, and by editorial bodies. These studies remain partial because they concentrate on quantitative approaches with datasets of one type (i.e., publications, student numbers); they lack intersectional approaches for collecting data [17,18].

Our first strategy is to identify women's influential role and contribution to the global north (The concept of Global North and Global South is used to describe a grouping of countries along socio-economic and political characteristics. The Global South identifies the regions of Latin America, Asia, Africa, and Oceania. It denotes regions outside Europe and North America, mostly (though not all) low-income and often politically or culturally marginalised countries on one side of the so-called divide, the other side being the countries of the Global North (often equated with developed countries). The term does not inherently refer to a geographical south) in CS represented in the annals of relevant awards in the field. We have performed an exploratory study of the sources where data lies that is explicit or hidden about women in STEM.

#### *1.2. Contributions*

As a result of the grounded theory and use case based approach:

The choice of Mexico as a case study is because our vision is to perform regional studies about women, mainly located in the global south.

Accordingly, the remainder of the paper is organised as follows. Section 2 introduces existing projects and initiatives devoted to building the history of women in CS with a particular interest in works addressing their role in AI and DS. It gives a timeline of women that have contributed to pin milestones in the history of CS. Section 3 describes a use case about Mexico highlighting possible sources where data refer to female scientific and technological contributions. Section 4 concludes the paper and discusses open issues and future work.

#### **2. Making Women's Contribution in Computer Science Visible**

Studying and understanding the gender gap in STEM is a complex problem because each discipline and scientific community has different characteristics and histories. To analyse missing datasets about the gender gap in STEM and identify the type of datasets missing, we decided to focus on CS and two related sub-disciplines, AI and DS. However, at some point, we refer to other disciplines, which shows to which extent data are missing that we need to look at the datasets that we have at hand and derive guesses about related fields.

Figure 1 shows the workflow of our qualitative approach and how it is related to the steps of a data science workflow. Our approach opens the data collection box and exhibits three main phases, namely, (1) selecting and choosing data providers and (2) building datasets both systematically interacting with a (3) qualitative assessment phase. In our approach, the first phase applies the grounded theory methodology to identify providers and the type and characteristics of provided data. It drives conclusions about them, launching the qualitative assessment phase. Once data providers have been selected, datasets are built by applying fusion, integration and correlation to the data stemming from different providers. The objective of this phase is to "complete" data representative for answering a research question that drives a data science experiment. Recall that data science uses quantitative methodology implemented by applying data mining, machine learning and other artificial intelligence techniques.

Qualitative phases of data science and data-driven studies are mainly done manually or with an active presence of a human in the loop. This characteristic explains why this study does not report the application of (semi)-automatic techniques for performing analytics. Still, we argue that qualitative methodologies preliminary to the automatic phases of a data science workflow are critical because they show the interaction between DM, ML and AI and human tasks.

**Figure 1.** Qualitative methodology for identifying missing datasets.

We instantiated this approach in our study as follows. For selecting and choosing data providers (phase 1), we adopted the hypothesis that female contribution (from the global north and south) in STEM should be visible through the data representing their work and contribution. Data should refer to actions intended to create women collectivities in the area, the female workforce in promising topics like AI and DS, and prestigious awards. Research questions (RQ1, RQ2, RQ3) guide our grounded theory strategy, and it leads to conclusions about the qualitative characteristics of available datasets presented in the following sections. Similarly, the building datasets phase is driven by a use case strategy. Therefore, we report on observations and datasets available in Mexico that can compose a representative dataset about female contribution in STEM, particularly in computer science (see Section 3).

#### *2.1. Women in STEM Studies and Inclusion Initiatives*

The exclusion of the contribution of women scientists in STEM history is a concern in specific academic and industrial sectors around the world. Some studies suspect that the lack of critical mass of visible female scientists and professionals is one of the reasons for the increasing desertification of women in the field over the last three to four decades.

Different academic organisations in the global north like IEEE (women in engineering), ACM (women in ACM) in the US, the European Institute of Gender Equality in Europe, the CNRS through the "Mission pour la place des Femmes" and the "Comité paritéégalité", the Institute of Gender in France, and major technology companies (e.g., Microsoft, Google, Facebook) have recognised the importance of understanding and organising actions that can promote a more gender-balanced, diverse and inclusive STEM (science, technology, engineering and mathematics) community. The opportunities and economic share, and return of investment in different disciplines in STEM are not homogeneous.

Some international actions in industry and academia have been organised to make women's contributions to computing visible. One of the most established actions is the Anita Borg Institute, founded by computer science PhDs Anita Borg and Telle Whitney to recruit, retain and advance women in technology. Other international organisations are working to promote the visibility of women in computing:

• Similar organisations in Africa, the Middle East, Asia-Pacific and Oceania have emerged. This list is not representative of the diversity of groups and forums working for the inclusion and visibility of the contribution of women in computing. There are too many, and there is no integrated map enumerating them. This diversity demonstrates both the interest in the issue, the lack of coordinated actions, and the lack of scientific gender studies applied to women in computing and women in STEM considering fine-grained analysis, for example, by field or geographic region. Some governments and universities have created gender studies institutes with dedicated chairs for science, technology, engineering and mathematics. Yet, despite exceptions, the topic remains considered of second class and not an absolute priority.

**Missing datasets 1: Who is studying and making women's work and contribution in STEM visible?** In the datification era, the gender imbalance issue, women's contribution in raw datasets and representative samples for statistics are missing. For example, who is measuring the role of women in professional social networks, in DS forums like Kaggle, in technical discussions on Stack Overflow? The "absence" of (integrated) datasets and sources about women in STEM studies is a big issue. Without intersectional datasets, it is difficult to respond to the initial question: Where are women in STEM? What are the topics and problems they are addressing? How are they contributing to the economy of STEM? Who is collecting evidence and promoting studies to answer these questions?

#### *2.2. Women in Artificial Intelligence and Data Science*

The persistent absence of women employed in the AI and DS fields is troubling. According to the report of the World Economic Forum in 2018 (https://www.weforum.org/re ports/the-global-gender-gap-report-2018, accessed on 1 June 2022), over three-quarters of professionals in these fields globally are male (78%); less than a quarter are women (22%). What about other underrepresented communities [19]? How are they represented in the DS and AI workforces, and which are the part of opportunities offered by these promising areas taken by these communities? Of course, to acquire a complete understanding of this phenomenon, it is necessary to treat the female community [1] and other underrepresented communities [19,20] as multifaceted and heterogeneous groups, with a plurality of experiences, and where gender intersects with multiple aspects of difference and disadvantage [21]. Discrimination at work must indeed be studied with an intersectional approach to acquire better and fine-grained understanding of the problem [22].

In the last decade, the role of data and scientific and technical skills used to exploit it have created promising career and economic spaces. AI and DS have emerged as promising areas for developing careers with critical financial benefit perspectives. Nevertheless, professional career perspectives in these disciplines are not equal depending on gender [23,24] and other criteria [25], including race/ethnicity, socio-economic level, the institution's reputation where people did their studies, etc. For example, in the United Kingdom, the House of Lords Select Committee on Artificial Intelligence in 2018 advocated increasing gender and ethnic diversity amongst AI developers. In France, several companies like Renault and Engie, through the "Laboratoire de l'Egalité", signed a call for widespread awareness of the discriminatory effects of AI and a commitment by its supporters to correct them. It is addressed to leaders in the public and private sectors, research and training organisations, companies that produce digital technology, companies that use digital technology and AI consultants. In 2020, the European Commission (European Commission (2019). 'Women in Digital Scoreboard'. Retrieved from: https://digital-strategy.ec.europa.eu/en/library/women-digital-scoreboard-2020, accessed on 1 June 2022. European Commission (2020a). Opinion on Artificial Intelligence opportunities and challenges for gender equality. Advisory Committee on Equal Opportunities for Women and Men. (18 March). European Commission (2020b). Gendered Innovations 2: How Inclusive Analysis) noted that it is time to reflect on the interplay between AI and gender equality. French, European, and international organisations and agencies [8,26,27] perform studies for observing workforce shares in industry and sometimes in academia from a global perspective. Few fine-grained studies have studied underrepresented communities' workforce evolution and gaps from the gender perspective in disciplines such as AI and DS [6,28].

A thorough understanding of the way the workforce accesses the AI and DS opportunities in industry [29] and academia [30,31] is essential for building fair and inclusive societies [32]. This understanding can also be crucial for ensuring that countries obtain the full benefits of developing these areas to achieve better economic and social conditions and leading positions in the international arena through technology self-sufficiency.

Despite the economic and symbolic capital investment seeking a fair distribution of AI and DS opportunities for women and underrepresented communities, the crystal ceiling must still be broken [33]. Part of the explanation resides in data (!) [34]. Indeed, many studies agree to consider that quality, disaggregated, intersectional data are still missing. These data are essential to interrogate and tackle inequities in the AI and data science labour force [6]. As stated in the Alan Turing Institute study, "Where are women?" [6], the Royal Society has noted that a significant barrier to diversity is the lack of access to data on diversity statistics. The AI Roadmap recognises diversity and inclusion as a priority to make data-driven decisions to determine where to invest and ensure that underrepresented groups are given equal opportunity.

**Missing datasets 2: Intersectional data about AI and DS female labour force.** The existing evidence base about gender diversity in the AI and DS workforce is minimal. The available data is fragmented, incomplete and inadequate for investigating the career trajectories of women and men in the fields. Public data sets often rely upon data produced through proprietary analyses and methodologies. Governmental statistics lack detailed information about job titles and pay levels within ICT, computing, and technology. This partial vision of the workforce status is a significant barrier to examining the emerging hierarchy between data science, AI, and other subdomains. Furthermore, available data about the global AI & DS workforce is often aggregated and rarely broken down by age, race, geography, (dis)ability, sexual orientation, socioeconomic status, and gender. As stated by [6,35,36], "this is particularly concerning since it is those at the intersections of multiple marginalised groups who are at the greatest risk of being discriminated against at work and by resulting AI bias".

Integrating intersectional datasets allows understanding discrimination from different perspectives and mainly exhibits the various aspects that contribute to such discrimination. The absence of datasets about the female labour force in AI and DS is a form of discrimination. We believe it is necessary to promote actions applying mathematical and machine learning techniques to integrate complete and high quality, privacy-preserving data collections. Studies can use these data collections to drive conclusions about the gender gap in these disciplines.

#### *2.3. Awarded Women in Computer Science*

The history of computing seems to have an equivalent gender balance issue [9], as it has acknowledged with difficulty and marginally the contributions of women or at least their participation in the advances of this young science. Few documents outline the history of computer science, including women, and documents systematically include men: Alan Turing, Charles Babbage, Herman Hollerith, etc. The documents that list female computer scientists include Ada Lovelace, Hedy Lamarr, the ENIAC programmers (although their names and faces are unknown), Grace Hooper, Mary Allen Wikes, Lois Haibt or Radia Perlman. However, even members of the computing community are probably unaware of who these women scientists were and what their contributions were besides Ada Lovelace and Grace Hooper (the Appendix A lists a non-exhaustive but a more extended set of contributions authored by female computer scientists).

The history of women's contribution to computing is spread across blogs, websites and news articles. Some films acknowledge women's role as "calculators" [9,37–39], and female labour is mentioned as a curiosity in the history of science, insisting on "pencil-dragging" tasks rather than on how they developed as programmers and became part of the core of digital computing advances [40]. The objective in referring to the term "calculator" is intended to show that contribution of women has been considered paper dragging even if they played a relevant role in the projects they participated in. Despite the importance of their contribution, they were not regarded as leaders of projects, and they remained invisible for a long time for history. The films have contributed to give visibility to these women. Still their stories and their contributions must be studied with methodological approaches and then included in books and in study programs, etc.

Histories of computing are not abundant, but they do exist; for example [41,42]. Reconstructing history is a complicated undertaking, to the extent that there are special series on the subject in well-known publishers such as Springer and IEEE. In Springer, the series is entitled "History of Computing" and in IEEE, it is entitled Annals of the History of Computing. These papers name the works of the illustrious scientists who have become the pillars on which computing is founded. Not surprisingly, the most prestigious prize in the field, the Turing Award, has only been awarded to three female scientists since it was first awarded in 1966. This event first occurred in 2006 (40 years later!). Can you name three or five names of people who have received the award? Are there any women's names on your list? The names of the three female Turing Award laureates are:

The cases of other awards are similar. For example, the ACM SIGMOD Contributions Award in the area of databases, initiated in 1992, has recognised the work of Maria Zmankova (1992), Laura Haas (2000 with Michael Carey), Marianne Wenslett (2012) and Meral Özsoyo ˘glu (2018), Juliana Freire, Ioana Manolescu with four other male colleagues in (2020). The ACM Edgar F. Codd Innovations Award, also in databases initiated in 1992, has recognised Patricia Selinger (2002), Jennifer Widom (2007), Laura Haas (2015), and Anastasia Ailamaki (2019)—four in thirty years by 2022. The Internet recognises approximately thirty-six women who contributed to advances in computing; Wikipedia lists about sixty-four women's contributions between 1842 and 2022 (in 180 years). Since the beginning of this science, outstanding contributions have been made by women in programming languages and programming (Fortran, Smalltalk C, C# Ruby). Many famous women programmers are clustered in video games, operating systems, software engineering and software evaluation. There is also an ambition to disseminate knowledge and active participation in computer education. These names do not include, for example, the recipients of the ACM SIGMOD Award Contributions or the ACM Edgar F. Codd Innovations Award. This situation demonstrates the dispersion when it comes to reconstructing history and accessing the memory of forgotten science when it comes to remembering the names of women scientists [43].

**Missing datasets 3: Female contributions in CS organised by sub-discipline and geographic region.** The discipline has different areas, including AI, DS, and much more, yet there is no database collecting significant contributions per discipline. Datasets are missing about the test of time and best paper awards concerning papers that have been highly cited during a ten-year interval. Indeed, major database conferences VLDB, SIGMOD/PODS, EDBT/ICDT, ICDE, and major data mining conferences like KDD have adopted this practice. Few databases collect information about papers with female partial or complete authorship, including, for example, the exhibition Women in Computing of the Science Museum in the UK (https://www.sciencemuseum.org.uk/objects-and-stories/women-computing, accessed on 1 June 2022). What happens at national conferences? What is their role in the local generation of significant knowledge? What is the part of female contribution to this knowledge production?

**Missing datasets 4: Contributions in CS of female scientists of the global south.** We remark that these lists do not include women scientists working in the global south (i.e., Latin America and the Caribbean, Asia Pacific, Africa, and the Middle East). Who are the Latin American, Caribean, Asian, and African women who have contributed to advances in computing science in different fields? What have been their contributions? How have their contributions improved their regions' development and knowledge? The first action by the United Nations recognised female scientists in Latin America and the Caribbean today. There is considerable work to collect data about female CS and STEM contributions.

#### *2.4. Discussion*

As a result of the identified datasets and observations done, we can derive answers to our research questions:

#### *[RQ1] Do missing data concerning women's STEM research participation prevent a comprehensive view of their contribution across history?*

Through the conclusions discussed in the three sources that we have analysed, we observed that women's history in STEM is partial and mainly located in the global north. We also observed that data are not organised according to disciplines and subdisciplines. This situation is particularly true in "young" sciences such as computer science and its subdisciplines. Regarding data science, if contributions to statistics and numerical methods should also be considered, it is true that very few and sparse data concern women, while women have been there contributing to mathematics and other sciences for ages. Data is sparse in that there are few or almost no datasets devoted to the topic, collected and built according to scientific methodologies. Any theory about women's contribution and absence in science history is anecdotal with these missing datasets.

#### *[RQ2] Is it possible to design viable data science experiments that can estimate the female workforce in STEM and answer the question, "where are women in STEM"?*

Data about the female workforce in STEM is provided in big grain, and the presence of women in different disciplines indeed changes a lot. The study presented in [44] shows, for instance, that the female authors in different computer science disciplines are not evenly distributed; areas like software engineering report more female publications coauthorships than in human performance architectures, for instance. With current datasets, even combining different datasets, such as awards and census of women scientists in universities, it is impossible to provide realistic cartography of women in the various STEM disciplines and subdisciplines. Besides, the location aspect remains since fewer datasets are available regarding the female workforce in the global south, even if they have widely contributed to STEM evolution. It is not realistic to design data science workflows for modelling women in STEM. Using, for example, clustering methods nor correlating variables that determine the choice of women choosing STEM careers and the evolution of their careers can find behavioural patterns that can explain their presence and absence in STEM. We should devote efforts to building datasets about women in STEM that are (i) intersectional, (ii) geographically representative, (iii) organised by discipline and subdiscipline, and (iv) show their experience in academia and industry.

#### **3. The Mexican Case**

Our vision is to address the study of missing datasets for studying the gender gap in STEM focuses on specific countries, considering that the context is essential to understanding the issue. Thus, we chose the Mexican case as a case study. The methodology adopted was to look for data about visible scientists in CS. The case study includes scientists recognised by the National System of Researchers (SNI), the Mexican Academy of Sciences, female scientists awarded by the National Award of Sciences, the history of Mexican women in STEM, and female professional situation and perspectives in STEM. We use these references because they provide an integrated national view of the Mexican

scientific community. A complete list of the acronyms used in this section is given in Appendix B.

#### *3.1. Mexican National System of Researchers*

According to their scientific production, the SNI classifies Mexican scientists with PhDs into five levels (candidate being the lowest one, and emeritus being the highest one). The evaluation focuses mainly on publications appearing in the ISI Thompson List and the impact factor of these publications. For the candidate level, applicants (less than 40 years old) must have published three journal papers recognized in the JCR or the list of journals of the CONACyT. To apply to level I, the scientist must have published 5 JCR papers, three of them in the last three years before the application. For level II, scientists must have published 15 JCR papers, 5 or 6 in the previous 5 or 6 years before the application. The scientist must also have advised graduate students. Finally, to apply to level 3, scientists must have published 15 to 30 JCR journal papers, and 8–9 papers must have been published during the last three years before the application. The awards can be renewed, and for SNI 3, emeritus scientists can have lifetime fellowship. Awarded scientists at SNI are granted fellowships that range between 375 USD– and 1750 USD.

Beyond the research are domains that cover a large spectrum of disciplines, organised in seven areas: I. mathematics, physics and earth sciences; II. biology and chemistry; III. medicine and health sciences; IV. humanities and behavioural sciences; V. social sciences; VI. biotechnology and agricultural sciences; and VII. engineering. What attracts attention is the distribution of the population across the SNI classification levels between men and women. The numbers show that 34% of Mexican scientists are women (!). What happened to all those women doing graduate studies, more significant in number than men? From this 34%, the majority of the female scientists are classified as grade I. This grade corresponds to people with at least 3 or 4 years of experience. Next, 15% of females are classified as grade II and 5% as grade III. These statistics show that it seems complicated for women to consolidate their careers and access recognised senior positions independently of the discipline. To gender inequality, we could add the age dimension, as Paloma Alcalá stated (Paloma Alcalá, seminario "El sexo de la Ciencia" held in the Faculty of Philosophy and Education Sciences of the Universidad del País Vasco, San Sebastián, 1 y 2 March 2000) that when women achieve the highest positions in the organisation, they have invested 16 or 20 years more than men. STEM, as other profession possibilities for women, are fields where women make less money and advance through the ranks more slowly [45]. Women's absence at the top provides arguments to understand women's underrepresentation in STEM.

During the new presidential six-year term (2018–2024), the SNI has made an effort in including women in evaluation committees. Recent raw data has been exported recently on the official governmental site; our study can be eventually completed with this new release. The gender gap is still wide, particularly in the highest grades of the SNI.

In 2014, the Mexican Council of Science and Technology (CONACyT) published the distribution of scientists doing research in the Mexican system recognised by the SNI. Only 15% of researchers in Mexico were women, with the criterion of having obtained the distinction of emeritus granted by the SNI. Later, according to official report of the CONACyT in 2020, there were 33,165 researchers recognised by the SNI, of which 8727 were candidates (26.31%), 17,091 level I (51.53%), 4793 level II (14.36%) and 2584 level III and emeritus (7.79%). In 2021, the percentage of women in the SNI achieved 38.2%. In 2022, the SNI granted a group of senior scientists with the emeritous grade: 183 scientists in all disciplines; of this number, 38 correspond to female researchers, the highest number of women who have been awarded this distinction. In 2022, there are 102 female researchers with emeritus status in the SNI out of a total of 462.

Let us analyse people doing research in computer science recognised in the SNI. We find eighty-two women researchers identified out of four hundred and thirty-seven recognised between candidate and three consecutive grades. There is only one female

scientist in grade three, Elba Patricia Melin Olmeda of the Instituto Tecnológico de Tijuana, a specialist in artificial intelligence, compared to thirteen male researchers. There are two female researchers in grade II, against thirty-eight male researchers with the same grade. The rest are distributed among forty female researchers at grade one against two hundred and seventeen male researchers. There are thirty-eight female researchers with grade "candidate" against ninety-seven male researchers. Regardless of observing an evident and pronounced gender disparity, sociological and economic studies are still needed to explain this phenomenon in the Mexican context.

The CONACyT and the Mexican government implemented inclusion policies seeking gender balance, promoting it in evaluation committees with the controversial idea that with more women in committees, they would encourage gender balance. In 2016, the SNI decision-making commissions, composed of 14 members each, were published on the CONACyT Web Site (http://conacyt.gob.mx/index.php/convocatorias-conacyt/informaci on-importante-sni/miembros-de-comisiones-dictaminadoras/12183-integrantes-de-las-c omisiones-dictaminadoras-2016/file (accessed on 20 August 2016)). The gender proportion was as follows:

In 2021, the distributions were as follows. Note that the classification of disciplines evolved, more women have the presidency of the commission, and their participation is higher in areas where they are underrepresented:

Figure 2 shows a comparison between the number of female and male members of the evaluation commissions. The difference is still significant even if it lowered between 2016 and 2022. Note that the CONACyT merged engineering and technology and created a multidisciplinary area. This reorganisation of disciplines might be why the number of women in engineering and technology, for example, increased.

**Missing datasets about the female scientific force in STEM at SNI:** The Mexican government has adopted an open data initiative that includes the SNI. This initiative gives access to the SNI community, and it is possible to perform simple statistics about women in different disciplines. However, no contextual data can make it possible to perform more profound studies about the professional career vs. the evolution of SNI awarded grades. Are women working in universities located in urban areas? Are women working in institutions located close to the capital more likely to be awarded by the SNI? What kind of contact and working network do SNI-awarded women have? Does this network play a role in their evolution in the SNI grades? There are also missing datasets about the situations in which scientists, particularly women, lose the SNI grant for some time and then are granted again when their scientific production takes off. Additionally, datasets about the metrics used for evaluation are missing. They could be helpful to observe how their values evolve according to gender and SNI level of scientists. This data could provide a more representative understanding of the "productivity" of scientists and its evolution correlated with the development of their career (e.g., administrative positions, classification in their institutions, projects coordination, student advising).

**Figure 2.** Comparison between the composition of commission members by gender and discipline between 2016 and 2020. Commissions with a \* have female presidents. The complete names of the items with ". . . " are given in the text.

#### *3.2. Mexican Academy of Sciences*

Another reference showing the unbalanced number of men and women in STEM is the current membership numbers in the Mexican Academy of Sciences (AMC). The AMC is a non-profit organization comprised of distinguished Mexican scientists attached to various institutions in the country and several eminent foreign colleagues, including different Nobel Prize winners. According to the Web site of the AMC, by March 2022, there were 1376 members in "Exact Sciences" (astronomy, physics, engineering, chemistry and geosciences)—1158 men and 218 women. The discipline with fewer women was engineering, with 43 women and 318 men (see Figure 3).

**Figure 3.** Members of the AMC by gender and discipline.

These numbers show that the AMC is even progressive considering that women's access to university was granted recently. For example, in Europe, between 1860–1900, Cambridge accepted women without restrictions until 1947, and the National Academies of Sciences opened spaces for women later. Marjory Stephenson and Kathleen Londsdale were the first scientists to join the Royal Society in 1945, an institution with 300 years of tradition. Yvonne Choquet-Bruhat was the first scientist accepted in the Academie de Sciences in France in 1979. This institution was founded in 1666. The first Spanish women received in the Royal Academy of Pharmacy and the Royal Academy of Sciences, Physics, and Natural Sciences were María Cascales (1987) and Margarita Salas (1988).

The AMC has already had its first female president, Rosaura Ruíz, who actively promoted the Science L'Oréal-UNESCO-AMC Award. She also encouraged the creation of fellowship programs for women studying Humanities and Social Sciences. She was awarded the National Award of Sciences and Arts in Technology and Design in 2009. In 2007, Esther Orozco was elected director of the Institute of Science and Technology of Mexico City, and she was awarded the L'Oréal UNESCO award in 2006. In 2009 Yoloxóchitl Bustamante was named the first female president of the Mexican Institute of Technology (Instituto Politécnico Nacional), 73 years after being founded. Some Mexican female scientists are influential and have occupied important positions in national and international institutions, including, for example, Ana María Cetto in the International Agency of Atomic Energy. She is a senior scientist at the Institute of Physics of the National Autonomous University of Mexico (UNAM). She has been head of the Faculty of Sciences and president of the Pugwash Conferences, a committee member that received the Nobel Peace Prize in 1995. In 1991, Ana María Cetto was president of the Organization for Women in Science for the Developing World (OWSD) for the Caribbean and Latin American Region. Mayra de la Torre was elected vice-president of the same institution. She collaborates in Science and Technology and the renewal of the program for Biotechnology/Biosecurity for the Americas. She has been awarded the National Award for Sciences and Arts, the Award of Sciences in the Third World, the Manuel Noriega Morales Award of the OEA and the CIBA GEIGY award in Innovation, Technology and Ecology.

**Missing datasets about females in the AMC.** Detailed datasets are missing regarding the contributions of women in the AMC. The open question is similar to data about women at the SNI: their connection networks, possible mentors, and mentoring roles. Which Mexican female scientists did they support to be accepted as members of the AMC?

#### *3.3. Mexican National Award of Sciences*

Let us take the Mexican National Award of Sciences as another possible data provider that shares little data about awardees. This award started being granted in 1945, and it recognises individuals or groups working in the following areas:

Female scientists started to be awarded in 1979 with Guillermina Bravo, the first woman awarded for her work in Beaux-Arts. In 1996, María Luisa Ortega Delgado was awarded the National Award for Technology and Design with Adolfo Guzmán Arenas. In 2008, María de los Ángeles Valdés Ramírez was awarded (biology), and in 2009, Blanca Elena Jiménez Cisneros was awarded (environmental engineering) with José Luis Leyva Montiel. Thus, only one woman in CS has been awarded in the whole history of the awards program. Gender inequality is evident, and importance is given to gender issues in the country, considering that women acquired the right to vote in 1945 and that sexual liberation happened in the 60s.

The situation is even more precarious when making decisions and defining plans, policies and programs in committees. Indeed, few women participated in the structure of the Scientific and Technological Consultative Forum that coordinated the development of the National Science and Technology 2006–2012 (María Valdés, La mujer mexicana en la ciencia, https://www.cronica.com.mx/notas-la\_mujer\_mexicana\_en\_la\_ciencia-946726- 2016.html, accessed on 1 June 2022).

**Missing datasets about female contributions in STEM and its implications.** We have highlighted the lack of data about female contributions in STEM. The lack of data leads to the underrepresentation of women in visible databases that can help choose them to be granted awards. Their role is not highlighted in scientific teams and labs, project coordination, keynote speeches, or innovation authorships. Therefore, they disappear from the scientific committees' memory, and thus, they are rarely granted prestigious awards.

#### *3.4. History of Mexican Women in STEM*

The history of Mexican women in science is still to be written and disseminated as part of the education of young Mexicans. Women have contributed to scientific production since ancient times. An example of one of the first Mexican women to attend University and obtain a diploma is Matilde Montoya (1859–1939). President Porfirio Díaz granted her permission to perform her professional examination, becoming the first Mexican Surgeon. Later, Helia Bravo (1901–2001) was the first Biologist in Mexico to develop scientific work on cactus. She published more than 160 papers and 3 books about these plants and the arid Mexican ecosystems. She described 57 species, 2 genders and 8 species that bear her name. Paris Pishmish (1911–1999) was an astronomer. She proposed a theory for explaining the origin and the spiral structure of galaxies. She discovered three open stellar cumuli, now named by her name. She also found that stellar galactic associations move away from the galaxy's centre. Then, Luz María del Castillo Fregoso, chemist and pioneer in biotechnology and zymology in Mexico, was internationally recognized for her work on physical chemistry, particularly on enzymatic reactions. She was the first woman to obtain the Sciences Award of the AMC, and she was a member of the SNI grade III from her first application.

**Missing datasets about Mexican women in Science.** The work of reconstructing the history of computing in Mexico from a gender perspective remains to be done. Naming all-female forgotten science contributors to STEM is important because this can let emerge forgotten activities and contributions that would be erased from the collective memory. It can make it possible to understand the conditions in which STEM fields produce results and capital, the real opportunities for professional careers, what people (men and women) should expect and how they can become agents of change.

#### *3.5. Female Professional Situation and Perspectives in STEM*

Women's representation varies by discipline and professorship status in the academic labour force. The majority of full-time female faculty in STEM disciplines are significantly low in computer and information sciences, maths, physical sciences, and engineering. In the life sciences, an area in which many people assume that women have achieved parity, women made up only one-third of faculties. The professional expectations do not seem to be very promising for women if we analyse the numbers reflecting female professionals accessing decision-making positions, including, for example, the number of female deans in engineering schools in universities and the number of university presidents in technology institutes globally. Of course, these numbers change among countries depending on culture, race, political trend and prominence of the institutions.

Studies show how women leaders in the US have a more significant impact on a company's bottom line (https://www.fastcompany.com/90733328/the-secret-to-women s-leadership-that-can-drive-such-a-positive-impact, accessed on 1 June 2022). However, only an estimated 15% of C-suite executives and 51% of managers are women. Additionally, women make up only 30% of full professors and 26% of college deans in US universities (https://www.catalyst.org/research/women-in-academia/, accessed on 1 June 2022). However, in STEM, gender inequality is about leadership and inequality in terms of critical mass. It is impossible to choose female leaders where few women have the professional and/or scientific consolidation to access such positions. Referring to our use case, in Mexico, according to the Asociación Nacional de Universidades e Instituciones de Educación Superior (ANUIES (http://www.anuies.mx), accessed on 1 June 2022) between 2014–2015, 1 million 842 thousand 978 women and 1 million 876 thousand 17 men did studies in STEM fields. Thus, more or less, the numbers are balanced. Moreover, more women (167,967 women) than men do graduate studies in STEM (146,030 men). Therefore, what happens in professional life?

According to "Attitudes and experiences of engineering alumni" by [46–48] workplace environment, bias, and family responsibilities all play a role. Many women appear to encounter a series of challenges at mid-career that contribute to their leaving careers in STEM industries. Women cited feelings in studies of isolation, an unsupportive work environment, extreme work schedules, and unclear rules about advancement and success as significant factors in their decision to leave. Departmental culture includes the expectations, assumptions, and values that guide the actions of professors, staff, and students. Individuals may or may not be aware of the influence of departmental culture as they design and teach classes, advise students, organise activities, and take classes. For example, people tend to view women in "masculine" fields, such as most STEM fields, as either competent or likeable but not both, according to Madeline Heilman, an organizational psychologist at New York University [49,50]. Although being both competent and well-liked are essential for advancement in the workplace, this balance may be more difficult for women than men to achieve in science and engineering fields. STEM fields are perceived as male, including fields like chemistry and maths, where almost one-half of degrees awarded now go to women. Heilman's research shows how, in the absence of clear performance information, individuals view women in male-type occupations as less competent than men [49,50]. There is a need to understand how people are evaluated in professional settings and understand which features in organisations give men more success opportunities. The first false assumption is that superior intelligence is required to address STEM fields. More than intelligence that can be numerically measured, the prejudice is that STEM calls for talent and that there is only one single talent that ensures success in such fields. The discrimination further argues about the possibility of numerically measuring such aptitude. The worst is the idea that such talent is a fixed gift, that talent is not malleable. Only those people with the highest standardised test scores and the most confidence will step into STEM fields and develop a career. Many believe that all these prejudices reduce the number of people deciding to make a career in STEM fields and even break the glass ceiling. The good news is that this talent is not a unitary thing. It is multidimensional and difficult to quantify and measure; many different skills are critical to step into STEM fields; talent can be developed and enhanced by education, encouragement, self-confidence, and hard work [45].

Few institutions promote reflection about policies for addressing gender issues in their organizations. In Mexico, there are three representative centres: the Centro de Investigaciones y Estudios de Género (CIEG) and the *Unidad Politécnica de Gestión con perspectiva de Género* (Polytechnic Unit for Management with a Gender Perspective) (http: //www.genero.ipn.mx/, accessed on 1 June 2022) at the Universidad Autónoma de México and the Instituto Politécnico Nacional; and the Project on Gender Issues of the Humanities and Social Sciences School of the Monterrey Institute of Technology (Tecnológico de Monterrey). The CIEG has a unique chair on Gender in Science, Technology and Innovation, in addition to the Mexican Network of Science, Technology and Gender.

**Missing datasets about working conditions of women in STEM.** Women's labour conditions when they develop a professional career in STEM are taboo both in academia and industry. Few or no data are collected by institutions, industry and independent bodies about the labour conditions: working hours, positions, evolution, environment, stress, entrepreneurship, glass ceiling, etc. How can the gender gap be understood and reduced with little input to drive studies and design policies? Data about these issues are disseminated in online forums (e.g., Quora, Medium, Glassdoor), blogs and social networks. They must be collected and integrated into datasets that can drive seminal studies.

#### *3.6. Discussion*

After applying a grounded study strategy for approaching datasets about women in STEM in general and then focusing on computer science, we observed that datasets are missing. The history of female contribution in STEM is often silent about women. We concluded that currently available data make it questionable to apply DM, ML and AI methods for picturing women's history in STEM and discovering patterns to understand their apparent absence. With these observations, we approximated conjectural answers for research questions RQ1 and RQ2. This first study could not provide elements to answer RQ3 because the studied datasets do not provide variables or perspectives about the conditions in which reported contributions were produced, only scientists' year, institution, discipline, and administrative nationality (nationality of birth and age). RQ3 questions about intersectionality, and therefore we chose a concrete case, that of Mexico, for which we observed the contribution statistics that show representative scientific production and excellence in the country. We enumerated the criteria used to evaluate scientific contribution in the National System of Researchers (SNI). The question was whether these criteria could provide elements to build an intersectional perspective and a quantitative study to approach an intersectional gender gap index.

*[RQ3] Do missing intersectional perspectives prevent performing representative quantitative analysis about women's activity and contribution in STEM?*

Existing datasets in Mexico providing data about women in STEM are sparse, partial and reduced to a tiny group of scientists (both female and male). For example, scientists working in private universities were excluded from the possibility of obtaining a fellowship related to SNI recognition. This exclusion hides an essential group of scientists from statistics. Universities, polytechnic centres, and other institutions that develop science provide different facilities to scientists to do their research. This difference depends on their location, type, and the disciplines they support. Female scientists and their conditions almost disappear in this complex context. Building intersectional datasets combining all these elements is a project to run. In a country such as Mexico, these socio-economic and political factors produce clusters of scientific production with different characteristics and conditions. Being a female scientist at the CINVESTAV (the most important research centre in Mexico) in Mexico City is entirely different than being a scientist in Huajuapan de Leon in Oaxaca (the poorest state in Mexico). The studies about women must address these aspects that call for thorough data collection strategies (by region and by the institution) and then running analytics at a small scale before scaling to the whole country. To answer RQ3, it is true that without intersectional datasets built with methodological data collection strategies, women will continue to be hidden and difficult to locate in the scientific cartography.

#### **4. Conclusions**

Gender gap issues in STEM are a matter of money and power and, more precisely, about capital in the sense of the theory of the French sociologist, Pierre Bourdieu (who defines three types of capital: education, material and cultural. Pierre Bourdieu proposes a complex machinery of interrelationships of these three types of capital that impose power and influence on each other and organise activities and social markets). Indeed, gender gap issues are all about an unbalanced problem among the different capitals activating STEM, particularly in promising disciplines like the AI and DS markets. These issues have been animated by bias associated with the roles and capacities related to men and women. Society believes and promotes that men are more apt to address STEM disciplines than women. However, the gender divide in STEM is more complex because of sociocultural and racial factors. The official social order has viciously controlled access to specific knowledge disciplines.

The absence of women in STEM denotes the possibility of significant losses of brains with potential. If the gender perspective is not relevant or a priority from a human point of view, perhaps it can acquire value when it translates into millions of dollars lost by not valuing and encouraging the scientific contribution of a part of its population. In any case, this lack of balance in the STEM professional market attracts the attention of different actors willing to activate the global economy. For example, AI and DS are widely regarded as critical to the national economies in many "developed" countries. Concern about America's ability to be competitive in the global economy has led to several calls to action to strengthen the pipeline into these fields [51–53]. Thus, this unfair situation in countries of the global south like Mexico creates an economic loss. Countries where inequality is more critical compete with less capital and thus with fewer possibilities to obtain benefits.

There is still a lot to be done, mainly because the capital conditions in which women as minorities enter the STEM professional and scientific market are very different from those in which men do. This issue is also related to education in STEM, the way young students perceive it and how they decide to step into STEM studies and careers depending on whether they are men or women. Many departments and universities worldwide develop studies about the aspects that move female students to inscribe in STEM majors and the conditions of their permanence from bachelor to graduate studies eventually. This collected data must be integrated and shared, but it has the merit of existing. Postcolonialism and decolonialism have already explained that minorities are never the same and cannot be studied and analysed as a whole. Racial aspects must be considered as well. In Mexico, indigenous women living in small villages cannot aspire to the same opportunities as women with more European racial heritage living in a big city. Nevertheless, the participation of women in STEM is still considered a special event that should have special treatment.

Data-driven studies are crucial to understanding the gender gap from an intersectional point of view, considering the particular conditions of the global north and south countries. This paper has shown that data are the fuel that can provide perspectives about the conditions in which women participate in the STEM market. The paper has also demonstrated that datasets are cruelly missing and that the challenge of collecting representative data is significant. Data-driven studies also need algorithms that can exploit them. These algorithms must be designed with care so that they do not bias observations and conclusions about the gender gap and do not further marginalise and discriminate against women in STEM. We need to open the door and allow and encourage women to contribute to the development of STEM, and this perspective depends on data and fair algorithms. Here is the view that should drive gender studies. Our current work addressed these issues in the context of the JOWDISAI and SINFONIA projects that are willing to create intersectional data collections about the women labour force in AI and DS in academia and industry in France. Other initiatives are seeking to reason about datasets, their content and collection conditions from decolonisation and feminist perspective, for example, the movement Tierra Común (https://www.tierracomun.net, accessed on 1 June 2022), and the A+ Alliance (https://feministai.pubpub.org, accessed on 1 June 2022). Our future work will include data collection strategies and algorithms to propose an intersectional analysis of the gender gap in STEM.

**Funding:** This research was partially funded by the projects JOWDISAI program AAP of the CNRS, France and SINFONIA of the interlaboratory program of the Institute of Gender, France.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Not applicable.

**Conflicts of Interest:** The author declares no conflict of interest.

#### **Appendix A**

The following list shows contributions in CS authored by leading women. This list is complementary to the list of women granted with prestigious awards. The list shows the controversy about the relation between awards and contributions to knowledge with a gender perspective.

#### **Appendix B**

This section provides a list of acronyms with their definition.

#### **References**

## *Article* **Ethical Issues in AI-Enabled Disease Surveillance: Perspectives from Global Health**

**Ann Borda 1,2,\*, Andreea Molnar 3, Cristina Neesham <sup>4</sup> and Patty Kostkova <sup>5</sup>**

**Abstract:** Infectious diseases, as COVID-19 is proving, pose a global health threat in an interconnected world. In the last 20 years, resistant infectious diseases such as severe acute respiratory syndrome (SARS), Middle East respiratory syndrome (MERS), H1N1 influenza (swine flu), Ebola virus, Zika virus, and now COVID-19 have been impacting global health defences, and aggressively flourishing with the rise of global travel, urbanization, climate change, and ecological degradation. In parallel, this extraordinary episode in global human health highlights the potential for artificial intelligence (AI)-enabled disease surveillance to collect and analyse vast amounts of unstructured and real-time data to inform epidemiological and public health emergency responses. The uses of AI in these dynamic environments are increasingly complex, challenging the potential for human autonomous decisions. In this context, our study of qualitative perspectives will consider a responsible AI framework to explore its potential application to disease surveillance in a global health context. Thus far, there is a gap in the literature in considering these multiple and interconnected levels of disease surveillance and emergency health management through the lens of a responsible AI framework.

**Keywords:** AI; disease surveillance; pandemics; global public health; ethics

#### **1. Introduction**

Infectious diseases such as COVID-19 pose a global health threat in an interconnected world. In the last 20 years, resistant infectious diseases such as severe acute respiratory syndrome (SARS), Middle East respiratory syndrome (MERS), H1N1 influenza (swine flu), Ebola virus, Zika virus, and now COVID-19 are impacting global health defences, and aggressively flourishing due to the rise of global travel, urbanization, climate change, and ecological degradation.

In parallel, this episode in global human health highlights the potential for artificial intelligence (AI)-enabled disease surveillance to collect and analyse vast amounts of unstructured and real-time data to inform epidemiological responses. The exploration of the requirements of a global data-driven and epidemic intelligence surveillance system finds precedence in the literature [1], but the implementation has still a long way to go. A 2020 Lancet commentary provides insights into what might be possible in a collective approach to address the unpredictable and evolving COVID-19 transmission rates using AI [2]. However, this promise is also hampered by the challenges of AI, including the opaqueness of the outputs derived from informal sources, and the commensurate need for transparency at multi-stakeholder levels [3,4].

AI as a component of global health surveillance systems is relatively new and largely bounded by both specific applications and geographies [5,6]. Global health itself is built

**Citation:** Borda, A.; Molnar, A.; Neesham, C.; Kostkova, P. Ethical Issues in AI-Enabled Disease Surveillance: Perspectives from Global Health. *Appl. Sci.* **2022**, *12*, 3890. https://doi.org/10.3390/ app12083890

Academic Editors: Karina Gibert and Aida Valls

Received: 15 February 2022 Accepted: 8 April 2022 Published: 12 April 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

on national public health actions and institutions, primarily concerning population-wide interventions. However, global health is ideally concerned with all strategies for health improvement, whether population-wide or individual-based health care actions. In other words, it goes across geographical administrative boundaries. There is no formalized definition for *global health*, but Koplan et al. [7] is widely cited for defining it as: "an area for study, research, and practice that places a priority on improving health and achieving health equity for all people worldwide". Taking on board the parameters of this approach, we are faced with both opportunities and ethical stumbling blocks of an AIenabled disease surveillance for global health. For example, how might we take into account the interconnected levels of individual and population disease contexts and stakeholders, multiple data sources, and geographical boundaries, and the implementation of equitably applied interventions?

#### **2. Disease Surveillance**

Surveillance is already a core function of both public health and global health practice. It is defined by the World Health Organization (WHO) as the "continuing scrutiny of all aspects of the occurrence and spread of disease that are pertinent to effective control". It is characterised by "methods distinguished by their practicability, uniformity, and frequently by their rapidity, rather than complete accuracy" [8].

The enabling components of surveillance systems may include laboratory diagnostics to detect or confirm health conditions; information technologies to support the surveillance processes of data collection, analysis, and dissemination; clinician consultation and reporting; public health education and training; and legislation, regulations, and policies that support the conduct of surveillance and response. Communicable disease surveillance can operate at international, national, state, and local levels. The primary responsibility for public health action often lies with state and national health departments or public health agencies (e.g., Public Health England in the UK). For the purposes of this paper, WHO [9] has defined a disease epidemic as "the occurrence of cases of disease in excess of what would normally be expected in a defined community, geographical area, or season". In practice, infectious disease data collection and analysis is complicated and encompasses a multi-stage process with several stakeholders across various organizational boundaries [10]. The main objective of infectious disease surveillance is to identify changes in incidence, either in the form of an acute outbreak (e.g., COVID-19) or a change in long-term trends [11].

Disease surveillance intelligence includes activities related to the prompt identification of potential health hazards and their verification, assessment, and investigation to enable public health control recommendations [12]. The 2005 International Health Regulations (IHR) are designed to ensure the timely recognition of outbreaks of infectious disease with the potential to spread widely [13]. The IHR 2005 also serves as a foundation for the Global Health Security (GHS) agenda [14]. The GHS agenda is "an effort by nations, international organizations, and civil society to accelerate progress toward a world safe and secure from infectious disease threats; to promote global health security as an international priority; and to spur progress toward full implementation of the IHR". They require WHO member nations to report outbreaks of international concern to the WHO within 24 h of discovery [15].

For case-based surveillance, member states can report using the WHO outbreak toolkit [16] to develop an initial case report. For example, consistent with the IHR, during the initial months of the H1N1 pandemic in 2009, the WHO requested that countries report the initial cases and, thereafter, the number of confirmed cases and deaths throughout the H1N1 pandemic. The resulting database represented one of the most comprehensive and timely outbreak reporting databases available to the public on the Internet at the time [13]. By contrast, the overwhelming human resource demands to effectively undertake COVID-19 data collection and analysis over the initial months of the pandemic swiftly required a shift from case-based reporting to aggregate and accelerated forms of reporting [17].

A critical component of disease surveillance which is growing in importance due to the availability of big data is 'epidemic intelligence' [18,19]. Epidemic intelligence incorporates two components: an indicator-based component and an event-based component [9,12,20,21]. The goal of indicator-based surveillance is to find increased numbers or clusters at a specific time, period, and/or geolocation that may indicate a threat [22]. The indicator-based component refers to structured or formal data collected through routine surveillance systems, such as the number or rates of cases based on standard case definitions, and the computation of indicators upon which abnormal disease patterns to investigate are detected [12]. Traditional indicator-based surveillance systems are based on the obligatory reporting of certain diagnosed diseases to a central health agency.

Event-based disease surveillance systems use information on events impacting human health from Internet-based sources, including news aggregators and social media channels [21–23]. Event-based surveillance complements traditional approaches to public health surveillance and can provide early warning of emerging events, whereas there may be a lag in other forms of data aggregation due to delays in sample collection, laboratory confirmation, and country reporting, for example [24,25]. The Medi + Board is an example of a dashboard framework which integrates public health surveillance data streams with zoonotic surveillance data [26], illustrating multiple geo-referenced and time series data.

There are several established and active disease surveillance systems utilized to monitor disease trends, mainly using online news sources, across the globe. Among these are: the Global Public Health Intelligence Network (GPHIN), HealthMap, and ProMED [20,21,25,27,28]. A short description of these systems is outlined in Table 1.

**Table 1.** Disease Surveillance Systems.

#### **Global Public Health Intelligence Network (GPHIN)**

#### **HealthMap**

#### **Program for Monitoring Emerging Diseases (ProMED)**

The IHR has emphasized the importance of both indicator-based and event-based components of epidemic intelligence for the early detection of events [30]. Informal information sources are important in this context, with the WHO reporting that more than 60% of initial disease epidemic reports come from unofficial sources and this percentage continues to increase with the availability of internet-based and social media sources [25]. During the COVID-19 crisis, the frequently changing government platforms used to publicly disseminate data included Facebook and Twitter, as well as government-authorized websites [31].

Common informal sources found on the internet include search queries, news feeds (e.g., Google and Baidu news), blogs, and social media channels [25,32]. Social media platforms, such as Twitter, are perceived as an increasingly useful form of real-time data with a geolocation resolution that can be systematically mined, aggregated, and analysed by algorithmic applications to inform public health agencies [30,33,34]. The increasing number of smartphones worldwide and lower policy barriers in many lower-middle income countries (LMICs) also provide an opportunity for the usage of AI-enabled mobile applications in event-based surveillance [18,35,36].

There are also emergent forms of epidemic intelligence [12] which are arising from newly identified microbes through advances in DNA sequencing technology. This primarily relates to genomic epidemiology in which lab-based molecular diagnostics and genotyping methods are being enhanced or replaced by rapid genomics- and AI-based methods to aid epidemiologic investigations of communicable and outbreak diseases [37]. From DNA to environmental exposures, climate change is similarly modifying the public and global health environment such that disease spread can take on new patterns, and subsequently new types of surveillance data are required, e.g., meteorological data [38].

#### **3. AI and Public Health Disease Surveillance**

Rapid epidemic detection and real-time monitoring are critical objectives of eventbased surveillance to minimize the morbidity and mortality caused by infectious diseases and to enable the preparation of rapid response public health services and personnel. Automated event-based surveillance systems have the potential capacity to aggregate and sort vast amounts of heterogeneous data for patterns and abnormalities using forms of AI [39].

AI machine learning classifiers play an important role in real-time analysis and trend predictions [40]. AI applications used with social media sources can be effective in providing trends for climatic and socio-economic contexts [38,41]. Social networks can further provide an efficient method for risk communication to prevent disease outbreaks and serve as real-time dissemination channels for declaration of pandemic events [42].

Machine learning models analysing internet and open health data sources have precedence across a number of disease surveillance projects to improve infectious disease surveillance and prediction, e.g., malaria [43], dengue fever [44], and cholera [45]. In the example of influenza, algorithmic and machine learning models have been applied to influenza tracking for nearly two decades. Bernardo et al. [32] indicate that the use of internet-based sources for disease surveillance is traceable to 2006, with early work focusing on influenza. Influenza can be highly contagious and easily spreads as people move about and travel, for instance, making tracking and forecasting flu activity a challenge. To address these challenges, researchers have proposed methods utilizing internet-based sources, mobile phones, and other event-based surveillance methods. Research in the use of internet data has demonstrated a positive predictive value of the incidence of infectious diseases, e.g., seasonal influenza [46], but it has shown in some cases to have a low predictive value [47].

To monitor the volume of influenza cases, there is an increasing use of "flu tracker" apps, whereby flu symptoms are reported using crowdsourced platforms to improve the global surveillance of influenza [35,48]. The U.S. Centres for Disease Control and Prevention (CDC) developed the FluView app, which draws on the U.S. influenza-like illness surveillance network (ILINet) that records the percentage of patients reporting to outpatient

clinics with symptoms of influenza-like illness (ILI), such as fever, a sore throat, or cough, over the total number of patient visits, for example. The data are integrated in the weekly joint WHO and European Centre for Disease Prevention and Control (ECDC) COVID-19 and influenza (www.flunewseurope.org (accessed on 15 January 2022)) bulletins [49].

These symptom measurements are an established indicator of historical ILI activity, but an important drawback is that they require several days to collect from individual healthcare providers, causing a delay and reducing the opportunity for real-time situational awareness and data analysis. In an evaluation of surveillance systems, it was found that the systems issued alerts for human cases between 1.9 and 6.1 days, on average, before WHO reports [30].

As monitoring is improved over time, the heuristics improve as their outputs are confirmed against a set threshold for the epidemiological attributes of extracted health events [22]. Prior to returning the extracted information, the surveillance system aggregates the extracted events into outbreaks, across several documents and sources based on this 'learning'.

The Computational Health Informatics Program (CHIP) at Boston Children's Hospital is among numerous research groups developing forecasting techniques combined with machine learning to provide more predictive indicators and estimates of local flu activity. An application called ARGONet (autoregression with general online information) leverages information from electronic health records, flu-related Google searches, and historical flu activity in a given location. In a research study, ARGO was shown to outperform Google Flu Trends (GFT) [50], a flu forecasting system developed by Google that operated from 2008 to 2015, but notably failed to accurately predict peaks in the flu season [25,48], as described by Daughton et al. 2020. To improve its accuracy, ARGONet also draws on the spatial-temporal patterns of flu spread in neighbouring areas. "It exploits the fact that the presence of flu in nearby locations may increase the risk of experiencing a disease outbreak at a given location" [50].

In disease outbreaks before COVID-19, the application of AI was limited because of a shortage of the data needed to provide rapid updates. The millions of feeds about coronavirus on social media and news sites are allowing algorithms to generate near-real-time information for the public health services tracking its spread. For instance, the disease surveillance system HealthMap based at Boston Children's Hospital uses natural language processing and machine leaning to analyse data from government reports, social media, news sites in 15 languages, and other informal sources to constantly track the spread of outbreaks [51]. The future developments at the time of this study include building a symptom-checker to assess the symptoms of coronavirus that distinguish it from seasonal flu.

In the emergence of COVID-19, risk analysis companies using AI-enabled tools were able to provide early detection of the coronavirus, before the WHO was informed. An AI epidemiologist in the Canadian-based company BlueDot (https://bluedot.global/ (accessed on 1 November 2021)) is widely reported as the first source to reveal news of the COVID-19 outbreak in late December 2019 [2]. BlueDot analysed data from news reports, airline ticketing, and animal disease outbreaks to predict areas that would be prone to the outbreak, expanding from regions in China [51]. Data sources and AI played an important role in various countries' responses to the pandemic [18].

Another company using AI capabilities is Metabiota (https://www.metabiota.com/ (accessed on 1 November 2021)) based in San Francisco, which offers an epidemic tracker and a near-term forecasting model of disease spread. Metabiota, which also used analytics to track flight data accurately, anticipated that countries such as Japan, Thailand, Taiwan, and South Korea would be at risk of a coronavirus outbreak days before any case was reported in any of those countries [51,52].

Interest in syndromic surveillance based on social media data has greatly increased in recent years, leading to what might be termed "nowcasting" [50,53,54], that is providing predictions of disease levels which incorporate data from current social media activity but can include rolling data sources such as ever-hospitalized cases and persons tested [53]. The early detection and analysis of epidemics based on mining Twitter messages is a common approach in nowcasting [53,54]. The VAC Medi + Board dashboard provides visualizations of vaccination threads extracted from Twitter, for example, illustrating the diffusion of information, the most influential users, and impact groups [55,56].

Over the course of the swine flu (H1N1) pandemic in 2009, novel big data streams from social media channels, in correlation with traditional surveillance reporting, demonstrated a potential for early-warning systems for infection disease outbreaks detecting a peak in an outbreak of swine flu up to two weeks before the official public health authorities [57]. Broniatowski et al. [34] reports that including Twitter data on influenza or influenza symptoms can improve nowcasting performance to a greater degree than was possible with search surveillance using Google Trends. During the 2012–2013 influenza season, a Twitter-based AI surveillance application also consistently predicted and mirrored CDC data with 85% accuracy [34].

COVID-19 big data applications are drawing on increasing aggregations of both formal and informal sources, including Twitter and major published open data platforms, e.g., Johns Hopkins University's COVID-19 global map dashboard. Pilot initiatives for COVID-19 forecasting, modelling, and machine learning rose exponentially in 2020, demonstrating the capability of AI for disease surveillance applications [58]. The Global Partnership on Artificial Intelligence (GPAI) identified 84 initiatives supporting some form of AI tool, application, and/or platform, from across the global north and south, related to the pandemic [59].

#### **4. Ethical Considerations**

There is a noted asymmetry in the literature on the ethics of AI in healthcare, with less attention granted to global health, particularly disease surveillance [4,19,60,61]. Much of the AI-driven intervention research in global health has limited descriptions of the ethical, regulatory, or practical considerations specifically required for cases of widespread use or deployment at scale [59,62]. Not least, there is the interconnectedness of individual and population levels of disease surveillance, diverse data types, and sources which are required for reporting and decision-support [17–19].

Making AI systems transparent, fair, secure, and inclusive is essential in this context, but how these systems are interpreted and operationalized can vary [6]. The OECD has recognized that AI systems deployed in specific contexts require different approaches to policy-making and governance, and have produced an evaluation toolkit to address variations using scenario-based examples [63]. The OECD toolkit shares principles with increasingly pervasive responsible AI frameworks [59,61,64,65], although this still remains a relatively unexplored area when applied to disease surveillance systems.

The recent literature focusing on the COVID pandemic has only just touched on the potentially important application of responsible AI [59,64,65]. The Future Society in collaboration with the GPAI launched a responsible AI assessment framework aimed at applying a number of normative criteria to AI-related initiatives arising in the COVID-19 pandemic. This assessment is based on human rights, inclusion, diversity, innovation, and economic growth [59].

In an earlier and wider-ranging study, a meta-analysis of ethical AI frameworks by Floridi et al. [66] resulted in the development of the AI4People framework for a "good AI society". The five principles underlying the AI4People framework are [66]:

Of particular relevance to this study is the grounding of the AI4People framework in bioethics principles, which more closely aligns it with an ecosystem of agents, patients, systems, and environments [60–62,66]. In applying the AI4People principles in the context of global public health disease surveillance, we reflect on how such principles could inform

both ethical and practical considerations for future disease surveillance system practices and applications.

#### *4.1. Beneficence: Promoting Well-Being, Preserving Dignity, and Sustaining the Planet*

This principle establishes the requirements for doing good, in the sense of pursuing the good pro-actively, as a constructive outcome, adding value to existing benefits. In this context, the role of a public health disease surveillance system in itself depends on a wider global health network, through which we can promote collaborative actions for the protection of people and the environment [38,63]. Those systems of IHR member states already have some form of embedded transparency in protecting and promoting the value of human dignity in the time of pandemics and in the process of disease surveillance. However, the implementation of public health responses, particularly as post-surveillance events, as well as whole-of government approaches to public health disease outbreaks, need to also support and ensure a continuity of beneficence [4,61,63,66].

#### *4.2. Non-Maleficence: Privacy, Security and "Capability Caution"*

This principle promotes the idea of avoiding harm, in particular to people, human society, and the natural environment. In the context of this paper, key aspects of this "do no harm" norm are data sharing, data tracking, data quality, and algorithm benchmarking.

#### 4.2.1. Data Sharing

The quality and completeness of the data on which the WHO relies depends on the capacity of its member states for surveillance and their willingness to share surveillance data. As the COVID-19 pandemic progressed, member states were asked by the WHO to submit detailed case data via line lists on more than 80 variables, the vast majority of which were incompletely reported [17]. This recent pandemic, in particular, has highlighted significant challenges in sharing, storing, and linking data from public, private, and quasipublic sectors, as well as between, across, and outside jurisdictions. Depending on the nature of the data, there may also be restrictions on linking or sharing outside of the geographical source of that data [19]. In the context of new technology and innovation, it is becoming increasingly important for people to share their data, so that disease prevention advances can be more impactful. For example, the sharing of SARS-CoV-2 genetic sequences has allowed scientists to track bio evolution and variants [37,67].

#### 4.2.2. Data Tracking

Here, there are concerns related to the privacy of the people whose data is being used, especially when the data is linked from different sources such as self-reporting mobile apps or trackers, or CCTV video, making it easier to identify the people involved. Many countries relied on an extrapolation of infection-control and public-health measures to contain the COVID-19 pandemic. These have ranged from extreme quarantine measures in China to detailed contact tracing in South Korea to enforced stay-at-home policies and travel restrictions [51,68]. Such measures have been variously implemented as COVID-19 contact tracing has revealed ethical trade-offs between public health and privacy. For instance, in South Korea, there were human rights concerns that the excessive disclosure of private information could cause people with symptoms to avoid testing [68].

Bluetooth privacy-presenting, anonymous, voluntary opt-in apps have been championed by human rights and data protection groups and supported by the WHO [69]. The decentralized privacy-preserving proximity tracing (DP-3T) protocol developed by several European academic institutions was among those developed in response to the COVID-19 pandemic to safely facilitate digital contact tracing of infected individuals at the population level [70]. These were intended to offer a comprehensive set of technical and legal safeguards against the potential misuse of personal data. However, a majority of individuals have not downloaded national digital contact tracing apps [70]. According to the Future Society report [59], public hesitancy querying the purpose of COVID-19 contact

tracing apps has also emerged, caused by mistrust in public authorities and by fears around the establishment of continuing forms of government surveillance. Additional means of tracking and screening such as thermal image scanning are among the potential privacy breaches and creep of mass surveillance [71].

What should the ethically acceptable trade-offs be between stopping the spread of disease (a public good) and protecting a person's privacy from potential breaches (a private good)? This is a sensitive issue.

#### 4.2.3. Data Quality

Real-world disease data tends to be noisy and incomplete. Although reporting of most notifiable diseases through various public health agencies is required by law, for the most part hospitals, laboratories, and clinicians participate voluntarily. This reinforces the need for careful evaluation of data sources and collection procedures. Moreover, event-based surveillance systems can generate a sizable amount of information on any given outbreak topic, sometimes overwhelming the systems themselves [21]. For example, the Argus system generated approximately 22,000 reports on the pandemic (H1N1) 2009 from April 2009 to March 2010 [20]. In stark contrast, the COVD-19 case reports captured early in the pandemic tallied 49,659 daily cases and 2739 deaths for 27 March 2020 only [17]. Given the serious consequences of using poor quality data to manage global health related decisions, improving data quality becomes a crucial ethical concern [63].

#### 4.2.4. Algorithm Benchmarking

Each syndromic surveillance system also implements a unique set of outbreak detection algorithms. This requires a better understanding of the strengths and limitations of various detection techniques and their applicability, which may require capability caution in the process [71]. For instance, algorithms can perform differently in the field than in the lab, and this may create a major challenge in ensuring high performance, accurate benchmarking against any current standard care, and continuously assessing performance after deployment [63,72]. Given the wide and powerful impacts of the decisions informed by such benchmarking, again, considerations of technical accuracy become ethical issues.

#### *4.3. Autonomy: The Power to Decide (Whether to Decide)*

The use of AI-based technology for global health surveillance also raises ethical questions around who should have the power to decide on these matters, and who should decide whether (and when) to decide at all. Concerns have been expressed about significant shifts of the locus of control on such decisions from humans to machines. In the context of global health surveillance, we note specificity, misinformation, consent, and data governance as the main aspects that would require further ethical analysis (and, possibly, regulation).

#### 4.3.1. Specificity

In event-based surveillance, reports may be generated by automated machine-based processes, e.g., HealthMap, or by human analysts or subject matter experts, e.g., GPHIN, ProMED [25,27,28]. For the automated systems, manual report examination for relevancy typically occurs post-dissemination. ProMed utilizes local observers on the ground for some of its outbreak reporting; otherwise event-based biosurveillance systems often disseminate reports that are not observer or laboratory verified [27,37]. Thus, although the reports provide near real-time alerts to users, the data they provide may not be specific enough. One consequence of this on human autonomy may be that individual human agents end up being constrained by limited one-size-fits-all (general) data and may not be able to see where (and how) they can act in different circumstances [63].

#### 4.3.2. Misinformation

Syndromic surveillance systems often generate false detections because it is sometimes difficult to distinguish natural data variations resulting from outlier and dynamic system changes, e.g., with time or space variables, from real outbreaks [73]. Human reviews and follow-up investigations are necessary for verification, which can be costly in time and labour. The prompt identification of novel microbes is often not matched with equally prompt epidemiological assessments [37]. The scarcity of such data, and the lack of transparency in algorithms could compound challenges of misinformation or false alarms that could potentially occur with AI-enabled platforms, especially if they rely on social media or non-authoritative data sources, for example. False detections can erode public trust in these surveillance systems, and such misinformation could weaken an agency's or individual's power to effectively make decisions [74–76]. The role of social media in trending public attitudes can also be put to effective use to disseminate risk communication in real time [26,42], but could be a double-edged sword increasingly prone to misinformation and the spread of fake news [74–76].

#### 4.3.3. Consent

In an article by Blasimme and Vayena [60], the authors raised ethical questions around consent when employing AI-driven social media analysis for digital epidemiology. For example, ubiquitous surveillance for use by AI systems through personal devices, such as mobile phones, introduces the concern that granular data can be re-identified, and personal health information can be hacked [35,71]. The main concern here is that this personalized data can be used by third parties without the subject's knowledge or approval. In the case of contact tracing apps, for example, there are ethical considerations to gather and act on information without consent [69]. Such digital surveillance on individual-level data can lead to unique complications [74], and relates to the trade-offs between the public good and individual rights [75].

#### 4.3.4. Data Governance

The COVID–19 pandemic has exposed long-standing data governance issues such as intellectual property rights, data sharing, reuse, and ownership [59,63,76,77]. Globally, persistent data gaps and fragmented approaches to governing disease surveillance data need to be addressed through global cooperation and clear, unified direction. The Global Pandemic Data Alliance (https://gpdahub.org/, (accessed on 1 November 2021)) is a collaboration of data-focused organizations across the G7 countries meeting the challenge of establishing health data as a global public good. This need for globally integrated data governance for the public good is highlighted in the Future Society report [59], the OECD Framework [63], and in the WHO Health Data Governance Summit [78].

#### *4.4. Justice: Promoting Prosperity and Preserving Solidarity*

It is increasingly accepted that principles of justice should apply when AI is used to support event-based surveillance efforts in global health policy. Justice is not just a lofty ethical ideal, and an end in itself. It is also a means to distribute prosperity widely within society and to foster social cohesion and solidarity across different social groups. Geographic scope, human rights, and equity in predictive decisions are the three main aspects of ethical concern to be discussed here.

#### 4.4.1. Geographic Scope

The geographic scope of an event-based disease surveillance system may vary [21,27]. It could cover a region, a country, a continent, or the entire globe. The following jurisdictions have established, hosted, and maintained such systems: United States, European Union, Canada, Japan, Australia, Brazil, Singapore, Mexico, and Thailand [11,15]. While there is immense potential opportunity for AI to support event-based surveillance of aggressive disease outbreaks such as COVID-19, the issue of missing data is especially present in lowand middle- income countries (LIC and LMIC), which may also lack the infrastructure and human capital required to maintain these systems [17,36,74,75,79]. Geographic inequalities in event-based system maintenance may constitute significant barriers to achieving justice

in global health outcomes, hence improvements in geographic coverage should become a priority.

#### 4.4.2. Human Rights

The protection of human rights is paramount and, as stated in the UN Declaration of 1948, should be universally guaranteed. However, individuals from vulnerable groups and stateless communities may discover that current disease surveillance practices are still a long way from safeguarding them from discrimination and oppression [79]. The IHR [15] has introduced important safeguards to protect the rights of travellers and other persons in relation to the treatment of personal data, informed consent, and non-discrimination in the application of health measures under the regulations. The effectiveness of these measures on the ground remains to be determined.

#### 4.4.3. Predictive Decisions

With respect to ensuring equity in predictive decisions, there is some debate as to whether responsible AI frameworks can address the explicit and implicit biases embedded within these systems [62]. To verify that an AI system is not using data in ways that result in bias or discriminatory outcomes, some level of transparency is necessary to explicitly addresses diversity and inclusion [59,60].

#### *4.5. Explicability: Enabling the Other Principles through Intelligibility and Accountability*

It is now recognized that the 'black-box' nature of first-generation AI systems can be a source of injustice, dominance, and harm [4,72]. The need for explainability (or explicability), in the sense of having algorithmic processes and outcomes explained in clear terms to the human intellect, has become evident. None of these processes and outcomes can be submitted to human judgment and scrutiny without certain explainability requirements being met [72]. The main aspects we discuss here are: data noise; meeting regulatory standards or policy requirements; assessing risks, robustness, and vulnerability; and understanding and verifying the outputs from a system.

#### 4.5.1. Data Noise

Noisy data are a result of data corruption which can carry a large amount of additional or meaningless information. High noise data are a weakness of real-time data streams analysis which can introduce small errors that in turn can have an outsized effect on largescale predictive models [80]. Some systems, such as HealthMap, relieve noise by integrating data from an assortment of online sources that have been moderated already [73].

#### 4.5.2. Meeting Regulatory Standards or Policy Requirements

Transparency is essential to legal compliance. Without it, enforcing individuals' legal rights in relation to the uses and applications of a technological system, establishing that a service meets regulatory standards, and determining liability may prove impossible [41,63,75]. While most nations have in place the procedures needed to foster compliance with existing data protection and privacy regulations, many such procedures are time consuming and create significant barriers to explicability [17,41,74].

#### 4.5.3. Assessing Risk, Robustness and Vulnerability

Understanding how a system works can be important in assessing risk [41]. This can be particularly important if a system is deployed in a new environment, where the user cannot be sure of its effectiveness [63,80]. Interpretability can also help developers understand how a system might be vulnerable to so-called adversarial attacks, in which actors seeking to disrupt a system identify a small number of carefully chosen data points that they could alter in order to prompt an inaccurate output from the system [72,81,82].

#### 4.5.4. Understanding and Verifying the Outputs from a System

Interpretability can be useful in verifying the outputs from a system, by tracing how modelling choices, combined with the data used, affect the results. In some applications, this can help developers understand cause-and-effect relationships in their analysis. Furthermore, people are more likely to trust an algorithm when they can easily understand its implications and can modify it [72,81,82].

#### **5. Key Issues of Normative Ethics**

We preface this discussion by noting that our conceptual approach inevitably presents limitations in the application of responsible AI principles in general. Challenges that may not be covered by such principles, for example, point to finer grained considerations relating to the complexity of disease surveillance systems, from a system alert to reporting to different levels of health authority with varying governance capabilities and a reliance on automated data generation and analysis in a global health emergency scenario.

What is summarily highlighted in the process is that AI is both a technical problem and a human problem. AI models are increasingly used in decision-making contexts in global health disease surveillance. At the same time, they "are more complex and less interpretable than ever" [83]. Scientific and practical work on the application of AI in supporting disease surveillance must consider that it is ultimately humans who need to understand the technological parameters. From a normative-ethical perspective, we see two issues bearing on AI-based decision-making in disease surveillance systems: (1) it is 'ethically blind' unless curated for value alignment (i.e., values shared by the human community are embedded in the algorithm); and (2) it complicates the issue of who is responsible for the consequences/outcomes of this decision-making process and their impact on various stakeholders.

The application of responsible AI has considerable potential for supporting transparency which, in itself, promotes respect for the human dignity of stakeholders in the decision-making process. It also makes value alignment and judicious responsibility attribution possible in the first place. Without responsible AI principles, meeting the two conditions systematically would be impossible.

#### **6. Conclusions**

Introducing *responsible AI* applications could support a more ethical and equitable approach [66] to combatting pandemics and maintaining global public health [59]. However, the layered complexity and scale of global disease surveillance systems raise challenges for the integration of *responsible AI* at different levels. These go beyond data, algorithm, prediction, and privacy concerns. The long-term goal relies on responsible health actors limiting the power of private corporate interests, ensuring transparency and protection of individual rights, and not least fostering trusted networks of partnerships and shared commitment through responsible global governance [63,66,84]. Further research should be engaged in developing effective approaches for value alignment in AI-assisted global health surveillance systems, as well as in normative-ethical analyses of responsibility attributions in the context of global health decision-making—the latter particularly in reference to the management of volatile pandemics, such as COVID-19.

**Author Contributions:** Conceptualization, A.B., A.M., C.N. and P.K.; methodology, A.B., A.M., C.N. and P.K.; investigation, A.B., A.M., C.N. and P.K.; writing—original draft preparation, A.B., A.M., C.N. and P.K.; writing—review and editing, A.B., A.M., C.N. and P.K. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Not applicable.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**

## *Article* **The Digital Revolution in the Urban Water Cycle and Its Ethical–Political Implications: A Critical Perspective**

**Lucia Alexandra Popartan 1,\*, Àtia Cortés 2, Manel Garrido-Baserba 3, Marta Verdaguer 1, Manel Poch <sup>1</sup> and Karina Gibert <sup>4</sup>**

**Abstract:** The development and application of new forms of automation and monitoring, data mining, and the use of AI data sources and knowledge management tools in the water sector has been compared to a 'digital revolution'. The state-of-the-art literature has analysed this transformation from predominantly technical and positive perspectives, emphasising the benefits of digitalisation in the water sector. Meanwhile, there is a conspicuous lack of critical literature on this topic. To bridge this gap, the paper advances a critical overview of the state-of-the art scholarship on water digitalisation, looking at the sociopolitical and ethical concerns these technologies generate. We did this by analysing relevant AI applications at each of the three levels of the UWC: technical, operational, and sociopolitical. By drawing on the precepts of urban political ecology, we propose a hydrosocial approach to the so-called 'digital water ', which aims to overcome the one-sidedness of the technocratic and/or positive approaches to this issue. Thus, the contribution of this article is a new theoretical framework which can be operationalised in order to analyse the ethical–political implications of the deployment of AI in urban water management. From the overview of opportunities and concerns presented in this paper, it emerges that a hydrosocial approach to digital water management is timely and necessary. The proposed framework envisions AI as a force in the service of the human right to water, the implementation of which needs to be (1) critical, in that it takes into consideration gender, race, class, and other sources of discrimination and orients algorithms according to key principles and values; (2) democratic and participatory, i.e., it combines a concern for efficiency with sensitivity to issues of fairness or justice; and (3) interdisciplinary, meaning that it integrates social sciences and natural sciences from the outset in all applications.

**Keywords:** Artificial Intelligence; urban water cycle; hydrosocial urban cycle; urban political ecology

#### **1. Introduction**

Digital technologies, particularly those related to Artificial Intelligence, are dramatically transforming our lives and the environment. Urban Water Cycle (UWC) management is a key part of this transformation. The UWC covers all the services related to the extraction, supply and sanitation of waters in a city (see Figure 1). The management of the UWC includes all the water that is present in urban environments: natural surface water, groundwater, drinking water, sewage, stormwater, flood overflow water, and recycled water (stormwater harvesting, managed aquifer recharge, etc.) [1]. In recent years, UWC management has been dramatically transformed by the development and application of

**Citation:** Popartan, L.A.; Cortés, À.; Garrido-Baserba, M.; Verdaguer, M.; Poch, M.; Gibert, K. The Digital Revolution in the Urban Water Cycle and Its Ethical–Political Implications: A Critical Perspective. *Appl. Sci.* **2022**, *12*, 2511. https://doi.org/10.3390/ app12052511

Academic Editor: Jega Veeriah Jegatheesan

Received: 31 January 2022 Accepted: 23 February 2022 Published: 28 February 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

new forms of automation and monitoring, as well as the use of AI data sources and knowledge management tools. This so-called "digital revolution" [2,3] has received considerable attention in both research and practitioner circles. The studies focused on this transformation can be grouped into three main categories: first, the tech companies developing digital applications in the water sector present an overwhelmingly positive angle, emphasising the benefits of digitalisation (for an overview, see [3]). Second, scholars coming from water, environmental engineering, and data science fields, who mainly focus on developing and refining existing tools [4–6], also frame digitalisation as a fundamentally positive trend. While some legal and ethical issues are marginally mentioned, this type of analysis presents the intertwining of AI with Big Data science and water management as a 'revolution' in the sector, which opens new ways to analyse, organise, and extract information from large volumes of varying types of data [2,7]. Finally, an emerging literature unpacks the more problematic aspects of water digitalisation [8], but the discussion of the broader sociopolitical implications of this process remains overly sketchy.

The lack of critical work on this topic is surprising, given the interest in digitalisation and the proliferation of scientific works devoted to concerns related to AI in general. Scholars of ethics, politics, and technology have unveiled the outright dangers of the unchecked deployment of AI applications, with notorious examples of thinkers such as Shoshana Zuboff [9,10], Noah Yuval Harari [11], and Byung-Chul Han [12] devoting entire volumes to—often dystopian—AI futures. The increase in the references focused on ethics in AI is another indicator; in this sense, see [13–17]. The lack of specific critical attention to the applications of AI in water management is even more puzzling if we consider the importance and ubiquity of water not only as a source of life, but also as a carrier of data and, as an increasingly scarce resource, an object of political conflict. In the context of the overwhelmingly positive accounts of AI application in water management, this paper aims to offer a more nuanced view of the impact of digitalisation in the sector and to look at 'both sides of the coin', which any transformation of this magnitude necessarily possesses. Drawing on the precepts of urban political ecology (Section 2), we propose a hydrosocial approach to digital water management which aims to integrate and overcome the one-sidedness of the state-of-the art approaches to this issue. Thus, the contribution of the article is a new theoretical framework which can be operationalised to analyse the ethical–political implications of the deployment of AI in urban water management.

The paper performs a critical overview of the state-of-the art scholarship on AI applications in water management, looking at both the opportunities and the concerns they

generate. However, selecting relevant examples can be problematic, given that the influence of AI and related technologies in the water sector is extremely wide: data science, augmented intelligence techniques, and automation have enabled innumerable applications, from virtual representation of the water system and near-real time flow and quality monitoring to asset management. Therefore, following our methodology (Section 2), we decided to focus on examples that had more potential to result in ethical or political problems. Moreover, we covered each of the three levels of complexity vs. impact in the water sector proposed by Poch et al. [18] and inspired by Funtowicz and Ravetz [19]: the technical, the operational, and the sociopolitical (Figure 1).

The structure of the paper is organised according to these three levels. Thus, in Section 3, we take two examples of technological developments enabled by digitalisation: smart metering and sewer epidemiology. The former refers to the instant measurement of households' water consumption. The latter is the extraction of biological data from wastewater to identify geographical patterns of disease spread, substance consumption, etc., a technique that has become especially popular during the COVID-19 pandemic. The second level corresponds to the design and operation of urban water cycle infrastructures. In Section 4, we refer to Decision Support Systems (DSS), a technology widely used to help in the choice of treatment plants to be included in the urban water cycle. We also look at how AI enables increased optimisation in the water cycle and operation of the infrastructures, but also transforms the workforce in the water sector. The third level is related to wider sociopolitical aspects of the urban water cycle. At this level, we consider two examples (Section 5): the gender issue and water governance more generally. In the discussion section, we explore pathways for the responsible use of AI and other digital technologies in the UWC, making the case for a 'hydrosocial' approach to A. We argue that this approach overcomes the artificial separations between nature, society, and technology, while integrating ethical and sociopolitical considerations from the outset.

#### **2. Theoretical and Methodological Underpinnings**

This investigation relied on a three-step method which involved the following: (1) desk research to review the theoretical literature (political ecology and hydrosocial science); (2) the development of criteria for the initial selection of applications and areas of impact; (3) interdisciplinary expert consultations—apart from the authors, consultations included renowned scholars from political science, environmental justice, philosophy, AI ethics, and environmental engineering—to validate the list and discuss the possible implications of digital water management.

The approach proposed in this paper takes its cue from urban political ecology (UPE) [20–23], a theoretical perspective that can be used to help unpack some of the main assumptions of the 'positive' or 'technocratic' approach to water digitalisation. UPE has traditionally explored the entanglements of power with the infrastructural provision in cities: instead of looking at infrastructure as a mere technological and apparently unproblematic issue, UPE scholars explore the political and conflictive dimension of technological and infrastructural projects [24]. Thus, the urban environment can be conceptualised as a socionatural hybrid. It embodies the relations, technological artefacts, networks, and flows that make urban life possible [25]. Building on the same metaphor of the hybrid, the concept of 'hydrosocial' aims to transcend nature–society binaries and envisions the circulation of water as a combined physical and social process [20,23]. Thus, water infrastructures and technologies of any kind—i.e., including the digital—need to be understood as an inseparable intertwining of cultural, political, and economic relations [25]. It follows that an evaluation of the impact of AI on the water sector needs to consider this hybrid nature and, importantly, place these applications in a wider sociopolitical context.

In this sense, urban political ecologists have begun to study the relationships between the deployment of water infrastructures, technologies, and governance within the shifting forms of capitalism and neoliberalism [21,26]. The growing importance of digital technologies in our society has brought about new forms of capital accumulation. Under so-called

"cognitive capitalism", the industrial mode of production becomes obsolete and, at the same time, "the object of accumulation consists mainly of knowledge", which is now the primary source of value [27] (p. 57). A dysfunctional form of cognitive capitalism could be considered "surveillance capitalism", a term coined by Shoshana Zuboff, who refers to it as a "fully institutionalised new logic of accumulation" based on the prediction and modification of human behaviour [9]. A fundamental shift in this new regime is that day-to-day practices—therefore water-related ones are included—become a primary target of commercialisation strategies [10]. Importantly, and given the rapid pace at which this trend is advancing, our society is yet to find a way to govern Big Data: how will it be used, to what purpose, who decides, and who decides who decides are fundamental political questions that remain largely unanswered [9].

These questions are also profoundly relevant for how the 'digital revolution' in water will be understood and managed. Digital technologies such as AI, data-mining, and advanced monitoring have the potential to fundamentally transform the way we understand and collect data from water, as the sector becomes riddled with new ethical and political concerns. Bringing together UPE and hydrosocial approaches, the paper focuses on the choreographies of power, nature, and capitalism encapsulated in 'digital water' applications. The article asks: notwithstanding the benefits of these applications, to what extent are they obscuring instances of power and discrimination and what can be done to overcome these concerns?

#### **3. The Impact of AI on UWC: The Technical Level**

This section focuses on the technical level of 'digital water' issues and the associated concerns, as exemplified in two applications: sewer epidemiology and smart metering.

#### *3.1. Sewer Epidemiology*

Water has the potential to become one of society's key sources of information; according to Garrido-Baserba et al. [2], cities will soon be able to 'mine' their sewer systems for information in order to understand lifestyle habits and the overall health status of the population by measuring human biomarkers. Importantly, by collecting the data available in wastewater, public health decision making will be improved through rapid access to potential sources of epidemiological threat. According to 'optimist' accounts, in exchange for this information, users will be "continuously informed and guided about health (e.g., disease predisposition, recovering follow-ups, tailored health programs, etc.) and lifestyle (e.g., dietary recommendations, personal-suggested activities, sport-recommendation with its corresponding associated products, etc." [2].

However, if we look at this technology and its potentiality from a UPE perspective, several problems arise. As our current experiences with Big Data show us, the relationship between users and the firms which process and commercialise data is more like an extractive process than a fair exchange [9]. It is precisely this one-way process that makes Big Data possible, and it typically occurs in the absence of dialogue or consent. Similarly, in the case of data obtained via sewer mining, optimistic accounts of the benefits of advanced capacity for early detection of diseases do not consider the potential risks of this commodification of data.

Moreover, as the management of COVID-19 has shown, access to intimate data can be used as a tool of advanced surveillance. For instance, the ability to profile different neighbourhoods according to their lifestyle choices raises important threats to privacy, but furthermore, the access to this data can legitimate, even without the threat of health hazards, policies which reproduce inequalities and discrimination against the more vulnerable. For instance, it is known already that the pandemic disproportionately affected poor, black communities, not because of their 'irresponsible' habits, but because of their structural life conditions: dwelling in crowded houses increased the incidence of contagion. The selective lockdown of poor neighbourhoods is just an example of how biopolitics can be spun against these communities [28].

Allowing access to personal lifestyle data can have significant impacts on the lives of the citizens, since in some health systems—notoriously in the USA—lifestyle habits determine the degree of access to health services. A smoker pays up to 50% more for health insurance, depending on the state [29]. Still, so far, the system relies on the 'honour' of the respondents and does not actually perform any checking; in other words, people can hide their habits, even if lying about them constitutes light fraud. A 'smart' system such as sewer epidemiology, which is able to detect eating habits, substance ingest, or other conducts which pose a threat to our health, could not only 'recommend' changes in diets, but also stigmatise and hinder medical autonomy.

Another important concern raised by sewer epidemiology is the potential publication of the databases and the eventual uses of this information. Even if the individual data are not made available—and therefore the European regulations on privacy are safeguarded the publication of data even at a large geographical aggregation level can have an impact on certain communities. Wastewater can provide information about the level of intake of certain medical drugs in certain areas, which is in fact a valuable commercial information and can help target marketing campaigns for certain 'health' products. These instances require additional cautions that might be beyond the General Data Protection Regulation (GDPR) regulations, since they may be contrary to the principles declared in the recent AI Act of the European Commission from April 2021 (Article 5 of Title II) about the prohibition of any practice oriented to manipulate human behaviour [30].

#### *3.2. Smart Metering*

Smart meters installed in every household allow for continuous measuring of water consumption. Through this technique, more accessible data are available to both consumers and companies, and this can impact the way people relate to the service while opening countless possibilities for influencing consumption behaviour. From the utility point of view, it can allow for reductions in peak demands, improved demand forecasting, promotion of efficient appliances, and performance indicators. On the consumer side, smart meters are considered useful as they provide information on how and where water is used to allow the reduction of consumption or quick leak detection.