The structural characteristics of an elementary act of practice can be recognized if the labor process is taken as a pattern. Marx in his work "The Poverty of Philosophy" [14], discussing the cost of labor, reveals its following elements: man with his aims, knowledge, and skills; operations of his rational activity; and objects included in definite interactions in the course of these operations. Paying special attention to the labor object, he considers, that "*labour is qualitatively defined by object and object in its turn is defined by specific labour qualities*" [14, p. 32].

This pattern can be applied to the structure of practical activity and introduced as the unity of subjective (man with his abilities, aims, and rational actions) and objective (production means, initial materials, products of labor from initial materials) aspects of the cognition process. There are various forms of human practical activity: according to Marx, the initial form of practice that lies in the foundation of other kinds and forms of human life activity, in general, is material production as a way of getting material welfare, which, in its turn, becomes the main driving force of social development.

#### **4.4 Technical Activity as a Special Form of Practice**

Forming and developing public relationships is also a necessary form of practical transformative activity: it does not influence the environment surrounding a human, but it involves people themselves and their relationships. However, this form of practice is initially connected with the material production practice. According to Marx, "the way of producing material life defines its social, political and spiritual processes in general" [15, p. 7]. Hence, this view proposes that there is a unified practical activity that includes two aspects: man's attitude to nature and his attitude to himself. It is worth noticing that during public interactions these aspects differentiate further and further since, due to the universal character of public activity, the subject can be both pieces of nature undergoing transformation and people whose "properties" change because of their inclusion in various social subsystems interacting in society as an integrated organism. Studying the first possibility, we can conclude that humankind deals with the human influence on nature, but the second postulated subject concerns the "subjective" character of practice as aimed at the transformation of social objects. From this point of view, humans can be considered as both the subject of practical activity and as its object.

The early stages of social development witnessed the unification of subjective and objective aspects of practical activity. As the labor process became more complex, the operations that had been performed by man started to be materialized. They were carried out as a sequence of influences of one tool on another and after that on the object being transformed. Science as a special form of practice, having more narrow social significance than production or social instances, gained an increasing influence in society. It then turned into a direct production power and a means for control of social processes.

Scientific cognition by itself possesses such qualities as a reflection of existing objects and these can be received or reproduced by different forms of practical activity. It also has projective and constructive functions, i.e., it provides knowledge about the objects being mastered during production and during social activity in the future. To check the validity of the knowledge requires a scientific experiment as a special form of practice: as a rule, this is typically based on production opportunities and human social experience within the current stage of development of society, but it overcomes the contemporary level of entities and foresees future technologies and ways of control of social life activity.

#### **4.5 Social and Cultural Perspectives of Communication Technologies**

In view of the accelerating pace of technological advancement, it is appropriate to consider ideas about the likely evolutionary paths of technologies that strongly influence society (especially communications technologies) before returning to consideration of the correlations with Marx's concepts.

At first, it is necessary to understand what the term "trans-science" means. It is often used by participants of international philosophical forums and conferences. According to Weinberg, the trans-science period started with the first A-bomb tests in the middle of the twentieth century. It stipulated the shift from researching nuclear power by some scientists to team projects:

Many of the issues which arise in the course of the interaction between science or technology and society, for example, the deleterious side effects of technology, or the attempts to deal with social problems through the procedures of science – hang on the answers to questions which can be asked of science and yet which cannot be answered by science. I propose the term trans-scientific for these questions… [16, p. 209].

Biological effects of weak radioactive influence on the environment, consequences of applying nanotechnologies to prolong the life expectancy, emergence of virtual reality, creation of artificial intelligence, changes of human ethics under new technological impacts are among such questions.

The main trans-scientific peculiarity is to unite problems from various fields of human knowledge, i.e., it is based on a trans-discipline approach. An outstanding current example is emerging and spreading of the Internet technologies. In comparison with rather rational technological methods, used in the past, modern technologies are able to provide for numerous negative effects due to the availability of mechanisms possessing an opportunity to manipulate the human mentality. Moreover, there has been no comprehensive reflection of their properties so far. So, they might be accepted as the starting point of philosophical deliberations due to the fact that the traditional approach to understanding the technology cannot reflect all the range of arising challenges. As a result, the main question is how all the spectrum of technologies and the social life are changed with penetration of the Internet technologies into absolutely all aspects of human existence.

Scientific knowledge, as viewed from the position of trans-science, is understood as suggesting probable hypotheses, which pass through bifurcation points where the choice of trajectory for further development occurs. This subject of research particularly addresses complex, dynamic systems that include technical, managerial, social, and other levels. Any cognition process turns into a social act, since communication among representatives of different knowledge fields stimulates the emergence of special norms and standards that are not connected with a specific author, but acknowledged as valid by all the scientific community involved in the process. As a result, they become a characteristic of a specific public style of thinking, with a consequential transformation of the communication language, which then acquires a universal character and influences linguistics and linguistic methods. These norms and standards permit cognizance of the complete range of science functions:

However, these models have a number of drawbacks. Their acceleration, the extension of their mobility and flexibility and their reduction of local attachment stimulate the arrival of location-independent micro-societies and new social institutions of global communications that can exchange information globally, e.g., in the form of text messages or, more powerfully, as social media. According to Nasarchuk, "*man becomes a message generator*" [18, p. 69], hence, a message performs a role as the fundamental item of the society, not a human. The whole set of messages, depending on their intention and content, form the lifestyle of a person, a micro-society or a social institution. It does not matter who is a message carrier, because it is the content that gains a dominant significance. This means that its content and its author's (supposed) competence are more important than its sender. Yet, a message and its perception are not always correlated with each other. The example of Internet technologies indicates a conclusion that a sender often knows nothing about his/her message recipient. In turn, the recipient is often not sure that the message has been sent to him/her personally. The recipient thus frequently cannot deduce what should be perceived as the message information as contextualized by some intention, i.e., the objectification of the discussion acquires some fuzziness. Antonovsky and Emelin suggest that "*communication on the society level is still possible, but can't form self*-*adjusting stable successions of messages*" [19, p. 106], since the knowledge presented loses its discrete boundaries and metamorphoses regularly.

According to the above conceptions, it is possible to highlight the following social and cultural perspectives of trans-humanism predictions under the influence of communication technologies:

Let us consider the final thesis in detail. An active role of cognition is the most important aspect of the application of these concepts to trans-humanism methodology [20]. It supposes the dominance of activity of the mind in all levels of perception, while nonstructured and nonclassified sensor data are absent.

According to the ideas of trans-science [21], modern technologies will provide new opportunities to create artificial intelligence and, as a result, to realize new social forms and psychosocial processes. The website of the Russian Trans-Humanism Society [22] cites some of its "radical possibilities" that could follow from the abovementioned trans-science issues and ways of their realization, such as the following:

Trans-humanism manifests itself in three aspects: practical activity; technological achievements and social transformations. Trans-humanism can be considered as a synthesis of discrete doctrines being applied in technological practice, especially in the field of communication technologies. In this conception, a human being is substituted by a virtual existence, i.e., there is the potential elimination of anthropocentrism principles, which were laid in the foundations of traditional humanism and traditional science.

Thus, the exploration of trans-humanism issues needs to be coordinated with the "stable development" conception [23], which can be understood as the necessity of the immediate protection of the environment, on the assumption of further convergence of natural and humanitarian sciences aimed at getting a more comprehensive understanding of a viable forward path. This approach raises a question concerning the boundaries of man's constructive activity, its involvement, and correspondence to the real world. A human being is becoming more and more "technological" with the evolution of communication technologies; however, s/he doesn't stop being sentient: the human, its body and consciousness turn into an integral part of complex eco-, sociocultural, and socio-technical systems [24].

In the context of the developing ecological and humanitarian crisis, the problem of the human future can be considered as a task of preventing the degradation of human and natural systems, achieving co-evolution of science and society, and forming a civilization based on stable evolution by means of searching out sensible answers on topical matters: this correlates with a comprehensive worldview outwith trans-humanism. Communication technologies have a great potential to dominate these issues and define future perspectives of not only science, but the whole of the development of civilization. They can be both production tools and the final product of human fabrication activity. That is why they are becoming more and more important components of the production process, defining its character and content (e.g., "Industry 4.0" [25]). Communication technologies require a range of specific skills from their participants and as a result, imply the dialectic unity of material artifacts on the one hand and human skills and experience on the other hand.

#### **4.6 Extrapolation into the Future**

The development and introduction of new technologies have led to the emergence of a new sociocultural reality that raises new ethical issues. They are closely connected with the realization of various projects, e.g., complete description of thinking processes and perception of reality by the human brain; the slowing down of aging processes; the opportunity of human organism rejuvenation; the development of brain/brain or brain/computer interfaces; the creation of robots and other devices possessing at least partial individuality; and similar speculations. Ethical problems arise from the realization of the above projects and the ethical principles that are conventionally followed nowadays will be transformed. Development and penetration of these new technologies will provoke a cultural effect that is likely to see the intensification of some ethical values and the devaluation of others.

For example, the future achievement of neurointerface technology would lead to the unification of human and machine on a qualitatively new level. It could change the level of virtualization of the human mind and social relations. Penetration of virtual technologies into human sensibility will create the situation of "hybrid reality", which could obliterate distinctions between man's virtual personality and his physical localization in a body. However, the current virtual world of social networks, and especially social media, leads to egocentrism and the human's preoccupation with itself and its thoughts: the result of this can be the loss of relationships between the human and reality. For this reason, discussion of change of the spatial conception concerning the physical boundaries of interpersonal communication and identification is very desirable. This change will involve a reconsideration of the human presence in the communication environment: whether it should be treated in both real and virtual forms simultaneously. Such an approach indicates a potential trend toward a completely new phenomenon of human existence: the boundary between real and virtual exists rather clearly nowadays.

Thus, sociocultural perspectives of the current development of communication technologies include the following:

Marx died in 1883. The telephone was invented in 1876, although the practical deployment of the telegraph dates from 1838. Thus Marx was alive at the beginning of near-instantaneous electrical communications, although the means of propagation of ideas most familiar to him would have been printed works: books, newspapers, and broadsheets. Extrapolation of his ideas to the circumstances of the present day must, therefore, be seen in this light. For him, the industry would have meant the making of hardware products and the concept of the "knowledge economy" would have been difficult to embrace: it is significantly different from material production as a way of getting material welfare (see Sect. 4.3, above). On the other hand, Marx's assertion "the way of producing material life defines its social, political and spiritual processes in general" (Sect. 4.4, above) represents a more information-oriented view and his view that the realization of any forms of practical and transformative activity occurs only under conditions of social cooperation (Sect. 4.1, above) correlates closely with modern concepts of social media and co-production of media artifacts.

#### **4.7 Conclusions**

The philosophical exploration of social and cultural results of current technological development is becoming more and more topical. Today there is a real necessity to bring out the distinctive features of these technologies and to analyze their impact on social reality. It is also very urgent to consider a new approach to humanism, as still understood traditionally, to clarify transformations of social values and the meaning of human life in the perspective of these developments, to study new cultural stereotypes emerging nowadays under the influence of communication technologies and emerging so-called "virtual reality", social media, and artificial intelligence.

Latour demonstrated the relation between technology and society [26] and the theme of democratization of technology has recently begun to receive the attention it deserves. As Latour has argued, the exclusion of technology from the social scientific concept of society is untenable. But once technology enters the picture, the issue of rationality appears in a new guise. It was Weber who introduced the concept of rationalization to explain many of the processes Marx had earlier identified as central to modernity [27]. Whereas in Marx's works such processes were examined as potentially opposing (capitalist or socialist),Weber argued that they were the same for all modern societies. When there is a strong focus on technologies as there is today, a basis for questioning Weber's simplification emerges again. It is clear that the modern world does not provide a unified view on the question of whether mankind is able to get a single homogeneous outcome to both technological and social development. The traditional practice activities as highlighted by Marx are blurring more and more as there is no sharp division between objective reality and the subjects of practical activity. The scheme "object– objective –practical activity–subject" as the foundation of scientific cognition and the criterion for the correspondence between knowledge and reality introduced by Marx has undergone a number of transformations in modern epistemology. Communication technologies with their new sociocultural reality erase differences between material and nonmaterial, nature and culture, theoretical and practical, etc. [26]. We do not need them anymore as the transition to "hybrid reality" and trans-humanism due to the equal interaction of living objects, social structures, theories or scientific-research programs, etc., generates a network consisting of a number of active elements. Its main characteristic is their collective actions. The coordination of their functioning in the network occurs due to the connections among elements, which makes all components interact.

It is possible to conclude that there is a special meaning in these circumstances for such a form of practice as technical activity in the field of communication, and the analysis of Marx continues to give useful insights. This can be directed to the creation of not only material values and artificial environments, but also thinking, cultural, and cognitive processes.

#### **References**

**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Chapter 5 Use of Digital Holography to Re-Encode and Image Chinese Movable Type Printing**

**Shuo Wang, Ardeshir Osanlou, and Peter Excell**

**Abstract** In the long history of the development of civilization, Chinese movable type printing has played a pivotal role for disseminating information and it can be seen as the ancestor of the modern information age. From traditional engraving to type, from clay to lead type, from laser printing to 3D holographic printing, the techniques of encoding have been gradually developed and recorded, and these printed texts convey the progress of mankind. A brief introduction to encoding in the context of Chinese cultural communication is presented and, specifically, the role of encoding will be explored. In addition, digital holographic printing is introduced as a new approach and new medium to explore the enlightenment conferred by Chinese movable type printing on today's society and especially the information revolution. Further, the future of encoding through the advanced technology of digital holographic printing is discussed. Two holographic artworks are presented to illustrate the principles expounded and the response to their display in national museums discussed.

**Keywords** Chinese movable type · Digital holography · Holographic printer · Printing technologies · Textual encoding

#### **5.1 Chinese Movable Type Printing**

#### *5.1.1 Introduction to Chinese Movable Type Printing*

The invention of Chinese movable type is a remarkable milestone in the history of printing, as it ushered in a new printing era, well before the well-known developments in Europe. Chinese movable type printing has been used for nearly one thousand years and greatly promoted the development and exchange of world culture; in addition,

A. Osanlou · P. Excell (B)

S. Wang

Beijing Institute of Graphic Communication, Beijing 102600, China

Centre for Ultra-Realistic Imaging, Wrexham Glynd ˆwr University, Wrexham, UK e-mail<EMAIL_ADDRESS>
it contributed to the historical progression of world civilization [1]. Movable type involves making individual movable characters in advance, which can then be chosen according to the manuscript, arranged in line on plates and finally printed on paper. All of the movable characters may be used repeatedly, as each of them can be separated and rearranged.

According to historical records from the ancient book MengXiBiTan [1] (see Appendix), Chinese movable type originated in the Qingli period (1041–1048 CE), during the Song Dynasty, with clay movable type characters invented by Bi Sheng (Fig. 5.1) [2]. It should be noted that this was around 400 years before the introduction of movable type in Europe by Gutenberg [3]. Later, with the progress of technology, movable type characters made of wood, tin, lead, copper, and further derivatives appeared [4]. Figure 5.2 shows the Uighur wood movable type examples which were found in cave sediments of the northern Dunhuang area of China in 1908: these were the earliest surviving examples of wood movable type in the world [5].

The invention of Chinese movable type printing enabled the simplification of the process of printing plate creation, reducing labor requirements, and improving production efficiency. However, Chinese movable type printing did not immediately replace engraving-type printing until the Qing Dynasty, when the government began to promote text-based cultural transmission. After this, the number of movable type books greatly increased. Printing techniques also varied, as the Qing government directly organized human labor and material resources to support the development.

**Fig. 5.1** Replica of clay movable type characters based on Beisheng's technology, in the Chinese Printing Museum

**Fig. 5.2** Uighur wood movable type examples, as found in 1908

For instance, during the Yongzheng period in the Qing Dynasty, copper movable type was used to complete as many as 160 million words on the integration of ancient and modern books [6] (Fig. 5.3). Further, in the Qianlong period, wooden movable type was used to print several thousand volumes of the Hall of Martial Valor book [6] (Fig. 5.4). Government participation was central in promoting the application of movable type and cultural communication with other countries.

Today, movable type, embracing many of the ideas of the ancients, still has great value. Its most valuable point is its encapsulation of creative ideas, that is, the transfer of whole-plate engraving printing into individual units, whether phrases or characters. In today's language, this may be called a code, symbol, or material carrier. This idea, in computer typesetting, is widely accepted as a general principle, which drives forward modern printing techniques [5]. Moreover, movable type printing is still popular among Chinese people.

**Fig. 5.3** The reference book—Qingding Ancient and Modern Integration

**Fig. 5.4** The book—Hall of Martial Valor (Wuyingdian), one of the largest quantity of books produced by wooden movable type printing in China

#### *5.1.2 The Influence of Chinese Movable Type Technology in the Wider World*

Movable type was introduced into Korea not long after Bi Sheng invented it during the Song Dynasty. The original reason for using such an innovative printing technique in Korea was that Buddhist scriptures and Confucian classics imported from China were in short supply. In the beginning, Korea used ceramic movable type characters, but they were made in different sizes and this led to unsatisfactory printing quality until the Goryeo Dynasty, when copper movable type was gradually developed, which not only enhanced cultural identity between the two countries, but also promoted a new development of Asian culture [7].

After the invention of Chinese movable type printing, Japan adopted the technique, mainly using wooden movable type printing characters to produce Buddhist scriptures for use among monks. A large number of Japanese books were printed, including the Japanese scripts modeled on Chinese cursive script [8]. Movable type printing was a major turning point during the process of social development as it improved the efficiency of mass book production and greatly promoted the dissemination of scientific, technical, and cultural knowledge. The spread of knowledge expanded from the aristocratic classes to the general population [8].

Vietnam commenced the use of movable type around 1841–1847 when the Vietnamese Dynasty bought a pair of Chinese wooden movable type sets and printed a large number of national legal books with it. This can be seen as the direct manifestation of Chinese type printing in Vietnam [7].

Besides Korea, Japan, and Vietnam, Chinese movable type printing also had a great influence on nations in other countries. In particular, Christian books in the Philippines were originally made by the ethnic Chinese people in local printing presses; however, after 1608, the Filipinos set up their own printing houses. In addition, Iran was a transit station on trade routes, facilitating the introduction of China's printing technology into Europe and hence making great contributions to the westward spread of the technology [6].

Europeans' first contact with Chinese printing began in the Yuan Dynasty (1271– 1368 CE), with the paper money brought by European businessmen and travelers returning from China [7]. Around 1450, A German inventor, Johannes Gutenberg, derived inspiration for a form of movable type printing from the technology of wine pressing machines [3]. Although this was more than four hundred years after Bi Sheng's invention, the European movable type printing techniques introduced significant progress in the use of fatty ink, movable type made of lead, etc. Its mechanical printing technology laid the foundation for modern printing. These machines greatly increased the speed of production and promoted the formation and dissemination of new ideas: this became the driving force of modern civilization, and advanced the modernization of European society.

#### **5.2 Encoding in the Context of Chinese Cultural Communication**

#### *5.2.1 Re-Definition of Encoding Concepts*

Encoding in semiotics can be interpreted as the process of creating a message transmission from an addresser to an addressee [9]; in psychology, encoding can be interpreted as the process of entering the memory system, storing, and subsequently retrieving [10]; in computers, encoding can be interpreted as the process of efficiently transferring or storing a sequence of characters (letters, numbers, punctuation, and symbols) into a professional format. In the book "Encoding and Decoding in the Context of Television Discourse," by contemporary cultural researcher Stuart Hall, encoding is interpreted as meaning production of a media message and this plays an important part in cultural communication, and similar processes [11]. Based on these concepts, encoding is a specific process and has different definitions in different discipline areas.

The present work attempted to identify the encoding of movable type printing in the context of Chinese cultural communication, hence encoding is defined for the present purpose as a transferring process from one role to another during the historical development of China's movable type printing. In a physical view, movable type printing breaks the earlier whole-plate engraving printing into individual units and this may happen during the processes of deconstructing phrases or words. In the psychological view, movable type printing evolves from transmitter to protector of information (in cultural context), and from protector to inheritor.

#### *5.2.2 The Role of Enlightenment*

The appearance of words can be regarded as the gestation phase of encoding. The invention of words was a great advancement for human civilization. In primitive ages in China, humans tied a number of different knots or different shapes of knots on ropes to express specific meanings: the so-called knot note (Fig. 5.5). Afterward, ancient humans portrayed certain things on stones and stone walls to convey certain meanings. This kind of pictogram directly gave birth to the origin of human writing: the archeological discoveries of ancient Egyptian hieroglyphic characters, Sumerian cuneiform characters, Chinese characters, and similar are all such.

Chinese characters have a long history, but the exact time of their emergence is still a historical mystery [5]. Judging from the archeological findings, the Oracle script system was fully developed in the Shang Dynasty, around 3500 years ago. Chinese characters are ideograms whose form and meaning are closely connected. Early characters were of pictographic form: with such a glyph, people could easily know what it stood for.

The usage of Chinese characters is closely linked with their transmission medium, but there were some limitations for expedient information transmission [12], for instance, scattered oracle bones were composed of animal bones with engraved symbols (Fig. 5.6); bronze inscriptions were heavy and difficult to move (Fig. 5.7); the words on bamboo slips were hard to modify (Fig. 5.8); silk as a writing substrate was expensive and susceptible to be eaten by insects (Fig. 5.9). The invention of paper was convenient for text recording, but it was not able to change the transmission mode of manual copying and thus still constrained the efficiency of transmission. Until the invention of whole-plate engraving printing, information could not be spread widely. Compared with manual copying, engraving printing had great advantages in saving time and improving efficiency, but it also had shortcomings: it was very costly as each version of the text required creation of a new set of plates. However, with the invention of movable type printing, whole-plate engraving printing was divided

**Fig. 5.5** An example display of a knot note

**Fig. 5.6** Oracle bone with inscription in the China Printing Museum (late 2nd millennium BCE)

**Fig. 5.7** Bronze tripod bowl cast for Duke Mao during the King Xuan period of the western Zhou Dynasty (828 BC–782 BCE): the vessel has 497 characters in 32 lines. (China Printing Museum display)

**Fig. 5.8** Words on bamboo slips in China Printing Museum (140–110 BCE)

**Fig. 5.9** Lao Zi silk manuscripts in the China Printing Museum, dated around 2000 years ago

into separate re-usable units, overcoming many of the above problems and starting widespread information transmission.

#### *5.2.3 The Role of Encoding*

With the development of information dissemination, the role of encoding evolved from transmitter to protector then to inheritor, or in a global view it can be considered as a navigator, directed toward civilization.

Whether as whole-plate engraving or movable type printing, the most obvious role at the beginning is as the information transmitter. In this process, information is transferred from one form or format to another. It is also a medium for a communicator to translate its message or meaning into languages, images, symbols, and so on. In view of relationship between language and literacy, printing provides a platform for converting spoken language into written language, which promotes the development of language, especially the development of written language in the Chinese literary and grammar frameworks. Considering the relationship between written language and communication efficiency, movable type printing provides an acceleration effect, and a new language experience can be built on its platform. This process of evolution with experience is also a role of the transmitter. In other words, the transmitter influences the efficiency improvement between language and literacy.

If the transmitter only transmits information and lacks cultural connotation, it cannot assist in advancing human civilization. Therefore, during the encoding process, the dominant role is held by the transmitter while the recessive role is that of the culture protector. In ancient times, the advancement derived from printing could be identified with history, religion, politics, military matters, and many other issues where printing had important historical and cultural values. The role of protector is crucial in the transfer process, for instance there are still preserved the Vajra cchedik¯a Prajñ¯a P¯aramit¯a (in English: The Diamond Sutra) (Fig. 5.10) based on engraving printing. This was made in the Tang Dynasty (868 CE), while the encyclopedia MengXiBiTan [1] (see Appendix), based on movable type printing, was written in the Northern Song Dynasty (written c. 1086–1093 CE) [6]. There can be little doubt that the ancient imperial system also contributed in establishing special rules and orders for specific cultural preservation.

Protection aims at re-dissemination and then assured inheritance. As the foundation of modern printing, movable type printing is the basis of pre-digital era printing technology (digital printing can be argued to be closely connected—see Sect. 3.1).

**Fig. 5.10** Vajra Cchedik¯a Prajñ¯a P¯aramit¯a (The Diamond Sutra), printed in the Tang dynasty: this is a well-preserved piece with the name of the carving master and the year of carving (868 CE), it was discovered in the cave of Dunhuang in 1900. The picture on display is of a replica

**Fig. 5.11** Brush pressing by a local craftsman onto wooden movable printing type in the Rui'an area of Zhejiang Province

Here, another role of encoding is linked to deep ethnic cultural conventions. For instance, wood type printing was derived from the custom of renewing genealogy records, especially in the southern part of Zhejiang Province, China: the clan concept of root-seeking remains quite strong there. Every clan's family tree is renewed every 12 years or so. Figure 5.11 shows the use of wooden movable type printing in the Rui'an area of Zhejiang Province. In the wider view of communication theory, the expansion of social and cultural communication has promoted the popularization of education and the exchange of knowledge. Additionally, mass production of printed books increases the chances of book retention, reduces the possibility of handwritten copies becoming extinct due to limited collections, and has a huge positive influence on cultural heritage.

Looking at the role of movable type printing in the history of civilization, it has always guided humans toward civilized societies. As a driver of social progress, movable type printing has effects on almost every aspect of modern civilization. Due to its roles as transmitter, protector, and inheritor, new ideas are quickly disseminated and popularized, and speculative thoughts are promoted.

#### **5.3 Digital Holographic Printing**

#### *5.3.1 Modern Digital Printing*

The rise of computer-mediated printing has caused a radical change in printing technologies, although this did not happen immediately. For many years the standard computer printers were the line printer and the electric typewriter, both of which used metallic characters of a similar form to movable type, although not assembled in the same way as in a printing press: the daisy wheel printer can be said to be similar. Toward the end of the twentieth century, however, raster-based printers (e.g., laser and inkjet) became common and now totally dominate the market. These do not use hardware type characters in the same way as the earlier technologies, but the philosophical difference is not so great, since they store a vast range of characters in software, made available to the user as "fonts", exactly mimicking the jargon of the traditional movable type printing technician. The fundamental point here is that human readers have become completely attuned to the use of discrete characters, as established by movable type, and the raster-based systems have to display text in the same way for it to be meaningful.

A more radical difference with this technology is the ease of inclusion of illustrations, as these can be printed with exactly the same raster technology.

#### *5.3.2 Digital Holographic Printing*

Holograms can represent objects and scenes in three dimensions without the need for special viewing glasses. Computer-generated or computer-mediated holograms are intrinsically digital and hence also have a raster structure, but in three dimensions, hence the two-dimensional raster pixels are replaced by three-dimensional voxels in the software. However, the hologram is a two-dimensional object which contains the three-dimensional information. This means that a different implementation of raster pixels is required. Thus digital holographic printing is composed of a matrix of holographic pixels, also termed holopixels or hogels. Holopixels are created by the interference of three-color lasers in a holographic medium, such as a photopolymer or silver halide film.

A digital printed hologram can visually represent an object in a realistic way and it also can be a creative medium to re-interpret the object in a narrative way. In addition, digital holograms can be duplicated repeatedly through a digital holographic printer.

A digital holographic printer includes several components, such as the data source, tri-color lasers, a spatial light modulator (SLM), beam-steering optics, and a computer to show the images on the SLM and to control the whole system. The data sources should include a selection of resources, such as general artwork and illustrations, photographs taken by people, or digital imagery produced by software such as Maya, Sketch-up, 3D Studio Max, point clouds, and laser scans. The laser-writing scheme utilizes an object and reference beam pair. The object beam is modulated through the SLM using two-dimensional (2D) information synthesized from a scene.

Digital holographic printing as an artistic medium has been practically developed and applied in cultural presentations. For example, Fig. 5.12 shows a holographic artwork created by author Shuo Wang: in this artwork, the digital printed hologram, as an innovative medium, not only enables the viewer to recover an archeological excavation space as it appeared in 1980, and to have a realistic spatial visualization of it, but also the losses of archeological detail owing to the difficulties of conditions

**Fig. 5.12** Digital holographic artwork representing a reconstruction of an archeological scene, by Shuo Wang

at that time could be bridged, bringing a profound influence upon archeological, historical, geological, and artistic representation of the priceless cultural relics that were recovered [12].

#### *5.3.3 The Evolution of Chinese Movable Type Printing to Digital Holographic Printing*

Understanding the past is an essential component for projecting developments in the future. Chinese movable type printing has developed in the course of its historical changes. The process has been complex and diverse with both positive and negative aspects, which are closely related to the social system, philosophies, aesthetics, and humanity of Chinese society in the context of historical development at the time [13]. With the initiation of the modern information age, the rise of digital holographic printing also needs to be explored in the context of today. Since the development of digital holographic printing and movable type printing occurred in very different historical periods, it is hard to compare them. However, if they are regarded as transmission media and linked in their aspect as information carriers with their respective medium characteristics, then they appear to have some similarities in particular periods in their history. Thus, some comparisons and suggestions of similarities may be attempted:

On the positive side:

printing, digital holography carries more information or data and has the potential for the inclusion of highly realistic graphics.

3. Both movable type printing and digital holographic printing have opened up a new mode of information transmission in their respective times. With the development of movable type printing, efficient replication using various materials has been created; digital holographic printing will also replicate with more materials in the future, not be limited to glass and photopolymer, but the digital generation of characters gives infinite scope to replicate not only the entire Chinese alphabet, but characters from other languages, mathematical symbols and new, fictitious, creations.

On the negative side, neither of them has had a great speed of popularization. Chinese movable type printing was invented in the Song Dynasty but it was not widely used to replace engraving printing, which required intensive labor [4]. Currently, digital holographic printing is not widely used either. The main reasons are as follows:

#### **5.4 The Exploration of Encoding in the Future**

The encoding of digital holographic printing can be seen as paralleled by movable type printing in some cases. In view of the medium features and advanced applications, digital holographic printing will also experience the roles of transmitter, protector, and inheritor. However, due to the rapid growth of information and the diversification of information media, digital holographic encoding ideally needs to be combined with artificial intelligence, so as to optimize it. For example, throughout the whole process of digital holographic modeling, this process can be performed with a large number of databases based on people's creative ideas, functioning as a replacement of manual modeling, improving the models, material, lighting, rendering and other steps, and designing the frame narrative through smart thinking. During the printing process, smart devices can hasten transmission speed, shorten printing time, and achieve efficient replication.

#### **5.5 Artistic Works**

#### *5.5.1 Three Manifestations of a Chinese Movable Type Text*

Based on the research on encoding of movable type printing, as well as the relationship between movable type printing and holographic printing from the perspective of media analysis, an artwork entitled "CUBES" was created to further explore the different roles in the future. The name "CUBES" involves two levels of significance: the straightforward meaning is descriptive of the artwork represented by different pieces in the form of small cubes; secondly, CUBES is an acronym, meaning "CUlture Broadly Encoding Spreading". The work was reproduced in the form of collocated wooden movable type, 3D printed movable type and holographic movable type, and the record of movable type printing in MengXiBiTan [1] was taken as the content (see Appendix). This follows the philosophy of the four "RE" creation methods (Re-definition, Re-creation, Re-construction, and Re-presentation) devized by author Shuo Wang [12], as well as holographic montage ideas. Each individual text component has been created in the 3D software: initially, the length, width, and height of the components have been calculated based on the wooden texts; furthermore, the polygonal modeling method follows the style of Chinese wooden engraving and the corresponding writing style, Fig. 5.13 shows holographic modeling of the content of MengXiBiTan and Fig. 5.14 shows the holographic rendering result, which simulates the effect of Chinese wooden engraving.

5 Use of Digital Holography to Re-Encode and Image Chinese … 75

**Fig. 5.13** The "CUBES" artwork: holographic modeling in Maya

**Fig. 5.14** The "CUBES" artwork: rendering effect in Maya

#### *5.5.2 Four Manifestations of Chinese Movable Type in a Hybrid Timeline*

This artwork (Figs. 5.15 and 5.16) explores the different roles in the context of encoding. In addition, digital holographic printing was used as a new approach and

**Fig. 5.15** Artwork: Hybrid Plate as a timeline

**Fig. 5.16** Interpretation of this artwork

**Fig. 5.17** The psychological and philosophical view of encoding

new medium to explore the enlightenment conferred by Chinese movable type printing on today's information revolution. This artwork consisted of a wooden plate and 266 individual blocks, including wooden movable type blocks, analogue holographic blocks, digital holographic blocks, and black blocks, these four parts (Fig. 5.2) briefly present a timeline of the information dissemination from ancient times to the future.

In creating this work, the author Wang assumed two roles/identities in the creation process: information receiver and information presenter. Furthermore, he attempted to explore the relationships among transmitter, receiver, presenter, protector, and inheritor to address the psychological and philosophical views of encoding (Fig. 5.17).

Considered as a receiver, the information received was drawn from the content of the "Ancient Chinese Wikipedia"—MengXiBiTan [1], In the process of receiving the information, the whole content has been summarized into 70 characters, and the these characters have been engraved into wooden blocks, based on the traditional method of Chinese wooden engraving: this aims to present the ancient method of information transmission for audiences.

Considered as a presenter, two approaches have been explored, based on the medium characteristics of analogue holography and digital printing holography. Initially, the 70 characters were produced using a Holo-Camera (Fig. 5.18): these holographic blocks can be seen as one of the modes of protection for wooden blocks (via substitution, since the 70 wooden blocks are considered as heritage objects); additionally, they can encapsulate the moment of creation of the wooden blocks, which provides a spatial view for audiences.

As a further insight, in the process of the digital holographic modeling, other spatial aspects of the characters can be discerned, as well as the opportunity to reinterpret the character in a narrative way. In this artwork, when each character was

**Fig. 5.18** Using a Holo-Camera to produce the analogue hologram

in the processing of being created, the structure revealed a layer of new meaning and spirit.

In such an era, facilitating the production of the hybrid printing plates, each individual block carries different dimensions of time and space, from the ancient wooden type block to the latest digital holographic blocks. Thus technology, medium, and culture have been developing together: they act the different roles in the process of information dissemination. Hence the development of printing technology and culture has always been developing in a synergistic way.

#### **5.6 Conclusions**

The historical development of Chinese movable type printing has been reviewed and its influence on the World's culture emphasized. The encoding of movable type printing has been explored and this expanded to include the evolution of Chinese movable type printing to digital holographic printing, including reinterpretations of the concept of encoding. Art works have been presented that demonstrate the coexistence of ancient movable type printing and modern digital holographic printing and the insights that can be drawn from this. The power of the holographic image has been demonstrated since, when the artwork was illuminated by the appropriate light source, virtual (holographic) character cubes, and real character cubes were combined together, thus presenting a new form of encoded space within the culture of human communication. In addition, the artwork reflects the passage of thousands of years of historical space and time, linked through the continuity of Chinese characters. The re-encoding of Chinese movable type printing reveals that this printing technology continues to develop from the past to the future—in digital holographic forms—with the potential to inform new interpretations of human communication.

#### **Appendix: Records of Movable Type Printing in MengXiBiTan**

MengXiBiTan is a book written by Shen Kuo, a scientist in the Northern Song Dynasty. It was written around 1086–1093 (CE) and documented Shen Kuo's views and opinions in his life. It is regarded as an Encyclopedia of ancient China by western scholars, with many foreign language versions published around the world.

In the Tang Dynasty, people had not yet adopted engraving printing to print books on a large scale.  After that, various classics and books were printed. During the Qingli period, a civilian named Bi Sheng created movable type printing. His method was to use clay to engrave characters and the thickness of each word was like the edge of copper coins. Each character was a matrix, hardened by firing. Firstly, he set up a piece of iron plate, covering it with turpentine and wax mixed with burnt paper ash. When printing, he put an iron frame on the iron plate, then arranged the word (i.e., character) matrix appropriately and closely. He filled the iron frame as one printing plate, then held it near the fire to bake; when the rosin and other bonding materials began to melt, he pressed the surface with a flat plate. In this way, the pattern on the plate was flat and smooth. If people wanted to print only two or three books, this method would not be convenient; but this method would be particularly efficient for tens or even hundreds of books. Usually two iron plates were produced while printing. One plate was used for printing and the other one for arranging the matrix. As soon as one plate finished printing, the other one would be ready to use. The process was quite fast with two plates. Each word had several matrix elements. For example, the Chinese words " 之" and "也" usually had more than twenty matrix elements in case of repetitive characters in one plate. When the words were not used, they would be marked by pronunciations with paper strips. Each rhyme part was marked with a label, then stored in a wooden lattice. If an unfamiliar character was needed but unprepared, people engraved it and then grilled it with a grass fire, hence it could be made very quickly. The reason why people did not use wood to make movable type matrices was that the texture of the wood varied between sparse and dense. Once stained with water, it would expand, and easily got stuck to the frame, making it hard to remove. Thus, clay was a better choice. After the matrix was used, the coating material could be melted by fire. Wiped by hand, the matrix would fall off and not be contaminated by coating material at all. Bi Sheng's matrices were understandably still treasured by his cousins and nephews after his death.

#### **References**

**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Part III Interactions Between the Arts and Data**

# **Chapter 6 The FOREVER-DO** *Game*: **A Big Data Fishing Expedition**

**Jill Townsley and Carlo Ferigato**

**Abstract** The FOREVER-DO *Game* is a participatory visual artwork, that takes the form of a relational game and installation. It is one outcome from an art science collaboration supported by the European Commission's Joint Research Centre in Italy. The artwork was developed under the theme of 'Big Data and Identity' it considers the causal chain that our individual and social actions may have as they flow across the interface of the human and the digital network. The work focuses on data flow that has some correspondence with coordination and communication, as identified by Petri Nets theory. Carl Adam Petri introduced the Nets in the 1970s, they are one of the techniques used today for the analysis of Process Data. When data is ordered in time, the causal chains of data are collected in 'processes'. One simple representation of a process net is the 'bucket chain' used to represent two fundamental aspects of data flow: selection and transfer. By using the bucket chain as a platform this project is focussed on a discipline of data where some 'causal flow', possibly represented by nets, has a role in the data analysis game. The resulting artwork encourages the viewer/participant to flow through an installation following a set of rules on a box, their actions coordinate according to the incidence of chance, selection, transfer and the similarity or difference of everyday objects.

**Keywords** Petri Nets · Coordinated behaviours · Data · Flow · Communication discipline · System · Time · Space · Repetition · Difference · Unfolding · Folding · Selection and transfer · Participation

J. Townsley (B)

University of Huddersfield, Huddersfield, UK e-mail<EMAIL_ADDRESS>
C. Ferigato European Commission, Joint Research Centre (JRC), Ispra, Italy

e-mail<EMAIL_ADDRESS>
#### **6.1 Introduction**

Artistic activity is a game, whose forms, patterns and functions develop and evolve according to periods and social contexts; it is not an immutable essence [1].

The FOREVER-DO *Game* is a socially relational public artwork or happening, in which people flow around a space, moving between a network of coloured doormats, according to a set of instructions printed on a box. This seemingly absurd game activates interactions between players, who meet randomly in space and time. The coordinated nature of the rules-based system dictates the path that the players travel, it is in itself a socially related construct. A construct that rationalises data flow, making visible the hidden nature of all data flowing within our big data systems. The work presents us with a symbiotic relationship between scientific theory and art practice, while also raising questions about socially mediated, rules-based interactions, especially those that cross the physical and digital interface.

The game is positioned within today's digitally networked age, it mirrors the theoretical construct of a 'Bucket Chain' network, a simple form of Petri Net system, proposed by the computer scientist Carl Adam Petri in the 1970s. In theoretical Computer Science, nets are instruments used for the analysis and design of systems, distributed in time and space. The strength of these nets is their explicit representation of fundamental situations of coordination and concurrency among system agents. Agents can be anything: objects, computers and/or human beings.

The first manifestation of this work was played/exhibited during the Milan Digital Week in March 2019, in the *Palazzo dei Giureconsulti*, near the Duomo, in collaboration with the MC3 research group on concurrent systems at the University of Milan Bicocca. The game was repeated in October 2019 for the Resonance III festival, at the European Commission's Joint Research Centre (JRC) in Italy, this time with additional digital visualisations developed through the application of Radio Frequency Identification tracking (RFID). Trackers were placed inside each of the 2,000 boxes used to play the game, the feedback from the trackers was detected by antennas with the data then displayed digitally alongside the installation (Fig. 6.1). It is the first presentation of the FOREVER-DO *Game* in Milan that we discuss here.

The cross-disciplinary collaboration between artist Jill Townsley and computer scientist Carlo Ferigato was important to the way that the artwork has developed. The work was commissioned by SciArt, an art science organisation embedded within the JRC. The JRC is the European Commission's science and knowledge service which employs scientists to carry out research in order to provide independent scientific advice and support to EU policy. The SciArt mission:

brings together scientists, artists and policy makers to discuss matters of concern from various points of view, not only to the JRC and the European Commission but also more widely to society [2].

Additionally, SciArt set the agenda of 'big data and identity' as the theme for the collaboration, stating:

**Fig. 6.1** The FOREVER-DO *Game* (2019 October JRC exhibition) Copyright © J. Townsley 2019

Big Data have erupted in our daily lives and we want to join the discussion on how to turn Big Data into culture, not leaving it only in the hands of technicians, trolls and corporations. We want to become hackers of our own future. We think it is time we make them ours, these Big Data, this digital transformation, in a vast collective effort open to all [2].

To do this SciArt have proposed a new concept 'DATAMI':

We see the datami as a virtual tatami [a Japanese mat], made of the data that we cherish, our data and those of our families and friends, our discoveries and curiosities, our roaming and conversations, all of these making up our new identities [3].

This is the theme for the Resonances III Festival and Exhibition held at the JRC in October 2019, and it is within this overarching context that the FOREVER-DO project was formed (Fig. 6.2).

In order to help convey the significance of the FOREVER-DO *Game* we look to the work of art critic and curator Nicolas Bourriaud and his concept of relational aesthetics. This supports the identification of some formal qualities in the artwork especially the interpersonal cooperation that is present within the FOREVER-DO *Game* and the significance of this model of interaction as a creative tool, to critically reflect on

**Fig. 6.2** The FOREVER-DO *Game* (2019 October JRC exhibition) Copyright © J. Townsley 2019

society. We also look to the media theorist, critic and philosopher Boris Groys and his writing on 'flow'. The flow of people and objects within the game is important and Groys helps identify how flow may define moments of temporary happening. Happenings that can have some governance over social interaction, presenting as 'an event not as a thing' [4]. Groys's theory helps us draw some conclusions about the way that the game mediates the viewer/participant in order to highlight the interface of the digital and physical realm, especially in relation to big data operations and to net theory.

Finally, to help us consider how the artworks analogue process interacts with the human digital interface we will be looking not only to Petri Nets theory, (as the inspiration for the way that the game has been developed), but also reading Carl Adam Petri's writing as a critical tool. Looking specifically at his *Communication Disciplines* delivered as a set of twelve conceptual tools. He proposed them in a lecture in 1976 to support reasoning in communication systems design, and suggested that they could help build the ideal computer system [5]. A system that could enhance human cooperation and overcome some of the many problems he foresaw for a computerised world going forward. Petri's principal aim was to help realise a computer system whose applications could support humanity to develop its best possible future. It is this call from the 1970s that dovetails so significantly with the concept of DATAMI proposed by the SciArt team and motivates the creation of the FOREVER-DO project.

#### **6.2 A Fishing Expedition—the Method**

Meeting within the context of the JRC, and in response to the SciArt theme of Big Data and identity, we (artist Jill Townsley and computer scientist/researcher Carlo Ferigato) realised that 'process' held an important position within both of our working practices. Ferigato's research around the process design of communication systems, specifically systems identified by Petri Nets theory, and Townsley's artwork emphasising the process of the art object, definitive beyond subject and object. The process presented us with common ground to articulate our practices in language that cut across our respective fields, producing a rich dialogue and exchange of creative thinking. It identified important key words beyond the process itself, such as *system, time, space, flow, coordinated behaviour, repetition, difference, unfolding, folding, selection and transfer.*

Through analysing our processes, nets became a key link between the art and science used in our collaboration. Put poetically, we became fishermen casting our nets, to fish across disciplines into the sea of big data sets in order to extract coherent patterns and procedures. We found patterns pertinent to Petri Nets theory, that could be visualised as art. From this, we have developed two symbiotic artworks the FOREVER-DO *Game* and the FOREVER-DO *Infestation.*

While we consider only the FOREVER-DO *Game* here, it is important to know that the placement of boxes within the game became data that has driven another artwork, the FOREVER-DO *Infestation*. The data, gathered from one piece of work flows on to inform the realisation of another. In this way, authorship is shared with every person who has played the game by placing a box on a pile.

While the FOREVER-DO *Project* as a whole (the *Game* and *Infestation*) explores the idea of 'fishing' into data sets, the nets we use are multifariously allegorical. Beyond being a fisherman's tool, the word 'net' has reference to our digital networks, the Internet. Nets are also the title given to the formal computer science tools 'Petri Nets' (as outlined in the introduction). One of Petri's examples, the 'Bucket Chain' is the main source of inspiration for the FOREVER-DO *Project*. It is outlined in some detail in the theoretical context below but is an explanation of the mechanics of the FOREVER-DO *Game*.

#### **6.3 The FOREVER-DO Game—the First 'Catch'**

This artwork propels its human participants on a physical journey that mirrors a 'causal flow' of data (Fig. 6.3). The basic idea of causal flow is to understand how factors influence one another. The game seeks to envision the invisible influence between human action and data. As the game is played, a sculptural installation consisting of towers of boxes develops in the gallery space. These box-towers simulate a data pile, presenting a physical object that records the coordinated meetings of each individual who (following a set of instructions on the box) encounters another

**Fig. 6.3** The FOREVER-DO Game (2019 October JRC exhibition) Copyright © J. Townsley 2019, included by permission

player at a coloured mat and finds (through random chance) that their boxes contain identical objects.

The mechanics of the game are as follows. In a large room there are placed a set of objects:

	- 4 Yellow,
	- 4 Red,
	- 4 Blue and
	- 4 Black doormats.
	- 1,000 brown boxes—randomly dispersed through the installation,
	- 1,000 white boxes—randomly dispersed through the installation.
	- There are 6 sets of rules offering slight variations.
	- The 6 sets of rules are randomly distributed (shuffled) throughout the 2000 boxes.
	- a knife,
	- fork or,
	- spoon.

With this absurd combination of objects, the game is ready to be played.

Visitors to the exhibition are invited to pick up a box and follow the instructions printed on the top (Fig. 6.4). The instructions send people on a journey travelling

**Fig. 6.4** Boxes and Instructions Copyright © J. Townsley 2019

through the space from coloured mat to coloured mat. There are 6 sets of instructions, all presenting a different order of coloured mats to visit:

Players physically 'flow' through the space of the gallery, moving according to the rules on their box. The brown or white boxes contain an everyday object: a knife, a fork or a spoon. When individuals meet at a mat, they open the box to compare the contents. Depending on the 'local' circumstance around the similarity or difference of those contents, players encounter one of two consequences:

	- this happens when their compared objects are different (non-repeated)—then the objects are exchanged and the flow continues according to the rules.
	- if the objects are the same (repeated) then players are asked to leave their box and object behind, placed on a tower at the point of coordination.

**Fig. 6.5** Players of the game Copyright © J. Townsley 2019, photograph Pierre-Stuart Rostain, included by permission

In this way a 'causal flow' happens, when non-repeated coordination occurs, objects are exchanged and continue their journey, independent of player or box flowing on, to continue the FOREVER-DO. Alternatively, when repeated coordination occurs (people meeting with the same objects), the flow is halted and boxes build up around the mats. Over time the number of boxes in the towers gather randomly around the mats, infesting the space and forming a sculptural installation (Fig. 6.5).

The installation makes visible a poetic data trail of coordinated human interactions, recording the incidence of two people meeting at a mat, to find that through random chance, their boxes contain identical objects.  Receiving the flow of information as people and boxes move from mat to mat, enabling coordination and a place for human interaction (Fig. 6.6).

As the box-towers grow, data may be extracted from the game in many different ways. The piles of white and brown boxes could be interpreted as binary code. The number of boxes clustered around a coloured doormat also give that mat an integer value. From this data, new digital, conceptual and physical constructs may be developed. Reorganised data is used to form a new sculpture: the FOREVER-DO *Infestation*. Alternatively, a new event or happening could be derived as presented in the FOREVER-DO *Game* 2. In other words, a new fishing expedition can begin (Figs. 6.7 and 6.8).

More generally, the gathering, reorganisation and extraction of information, as presented in the FOREVER-DO *Game*, is indicative of the way data flows today. Data (collected in a myriad of ways) flows forever through our digital systems. Systems and data unimaginably vast in form, yet invisible in time and space. This data can be organised and extracted in whatever way society chooses: it can be reorganised and

**Fig. 6.6** Players of the game and the box-towers Copyright © J. Townsley 2019 included by permission

extracted to help us do great things for humanity, such as understand diseases; or it can be used less responsibly, to smartly target us to purchase more mass-produced goods.

#### **6.4 Petri Nets, Communication and the Bucket Chain**

The structure of the FOREVER-DO *Game* takes its principles from Petri Nets theory, in experiential and analogue terms it makes visible a form of communication that is happening across the interface of our physical and digital worlds. The installation presents a data trail made from the flow of people around a room, flowing until random instances of coordination happen between players. This has resonance with Petri Nets theory, in its desire to rationalise information flow as a resource in communication.

The history of Petri Nets can be traced back to the Ph.D. thesis of Carl Adam Petri in 1962 [6]. His aim was to establish a new theory of communication-based on two assumptions:

These assumptions can be considered as natural ones today, but they were not so natural in the 1960s, when the theory of abstract computing machines (automata) was

**Fig. 6.7** Image of towers around a mat Copyright © J. Townsley 2019 included by permission

developing rapidly and, sometimes without regard for its potential limits in actual hardware realisations.

As a consequence of these natural assumptions, Petri analysed the subject of 'information flow' in a new way, based on the relations between the structure and behaviour of automata and on the role of information as a resource in communication. In his view, communication becomes an *organised activity* involving automata and people. *Communication with automata*, the title of the thesis, refers precisely to this organised activity where 'with' has the double meaning of 'between automata and people' and 'using automata as communication medium'. Considering automata as *organised communication mediums*, as we do today through the Internet for example, Petri opened the way to new applications of the discipline 'pragmatics', as a branch of linguistics, in view of a modern theory of communication systems [7].

Petri's bucket chain is a simple example of net theory and one way of rationalising data flow. It presents a scenario of coordinated behaviour between firemen extinguishing a fire. Figure 6.9 shows figures coordinating a task to carry water from a tank to a fire, using a chain of buckets.

The sequence in Fig. 6.10 explains how the coordination of behaviour (the figures) and the flow of data (the water from the tank to fire) operate. The theoretical net

**Fig. 6.8** Image of towers around a mat Copyright © J. Townsley 2019 included by permission

**Fig. 6.9** The Bucket Chain (drawing from Ferigato's blackboard, 2019) Copyright © J. Townsley 2019

**Fig. 6.10** The Bucket Chain and its unfolding in space and time (Ferigato 2018) Copyright © J. Townsley 2019

'unfolds' through time and space to illustrate how the coordination of the figures putting out the fire operates. This is Petri's clever graphical way of explaining simple net theory through the bucket chain scenario.

As the information, in this case, the water in bucket B, flows smoothly from the water tank to fire, it is also possible to see how each fireman moves left and right until they meet to exchange the buckets (Fig. 6.11).

The proposition: '*A walks in the direction right*' can be represented, following the formalism of Petri's Occurrence Nets, by a circle, representing in an abstract space the persistence of the proposition.

Importantly, the truth (or value) of the proposition '*A walks in the direction right*' is changed by the occurrence of events. For example, as in Fig. 6.10, two men walking in opposite directions along the same line will eventually meet at points in space represented as squares in Fig. 6.11. This 'meeting event' changes the respective directions of the two men, as if they were bouncing balls. So, we can represent a chain of firemen extinguishing a fire by carrying back and forth buckets of water in an abstract way, as in Fig. 6.11.

The net presents *events,* the meetings between two firemen, or, at the edge between the first fireman on the left and the reservoir of water and the last fireman on the right and the fire. Note that meetings between firemen do not happen in exactly the same place, nor are they regulated by any definite intervals of time.

This is the principle illustrated in the game, people flowing from coloured mat to coloured mat according to a set of rules on a box. They are moving through the space as if in a deep fog, meeting other players outside of a determinate system of time and space. The rules written on the box propel the player according to a networked system, but when another player is encountered, the meeting event happens outside of the rule of a clock or defined by a specific determination of place. Also, the proposition for flow is changed with each encounter as objects are exchanged. In our digital world information also flows forever, it can be extracted at any time or place that interaction happens. This model also presents the basic principle of artificial intelligence, where the information has the ability to change according to an occurrence or meeting event.

#### **6.5 Theoretical Context: Design and Analysis of (Coordinated) Communication Systems**

computer technology supplies us not [just] with a medium for artificial intelligence nor with a machine which may be used solely for computation, but with a medium for communication and for strictly organised information flow, a medium which may induce major changes in the modalities of co-operation between human beings [5].

The contemporary designer of communication systems is forced, in his profession, to consider relations between computers and society that are not yet consolidated in a unique practice or competence. Along the history of design of communication systems, plain consideration of measurable components like memory, throughput and availability, has shown its weakness in respect to less measurable subjects like delegation of tasks to computers, keeping control on copies of data and identify data as belonging to a given category.

Petri proposed the first set of twelve *communication disciplines* that are today still a valid starting point for designing and analysing communication systems. He addressed the design of communication systems from a wide perspective, as coordinated activities involving humans and computers.

The 'Communication Disciplines' were presented in September 1976 during a lecture given at Newcastle University and later published in 'Computer Systems Design: Proceedings of the Joint IBM-University of Newcastle upon Tyne Seminar (B. Shaw editor). He stated:

I tried to classify practical problems from my long list, according to my stated viewpoints of the role the computer [5].

His categories were:

His principal aim was to look into the future of computer systems and their applications, in order to support humanity to develop towards a brighter future. His conclusion was that computer system design should ideally be developed to 'organise information flow' in order to support 'cooperation between human beings'. As the true maverick he was, Petri proposed a list that he hoped computer systems developers could potentially use to 'concentrate our attention on' building a computer system that would enhance human cooperation and solve some of the problems he foresaw for a computerised world in the future.

#### **6.6 Fishing the Net of the Communication Disciplines**

To identify some of the processes and functions operating within the FOREVER-DO *Game*, we will use a selection of the 12 Communication Disciplines (highlighted in bold) as critical tools. In doing so we are also contextualising the game within a broader critical framework.

Petri states that the discipline of **identification** '*covers the question of pattern recognition'* [5]. The installation is a result of pattern recognition, specifically, a record of all the players whose **synchronised** meetings resulted in the **identification** of **copied** objects. Identifying sets of objects also indicates a destination at a mat. This is most obvious as the **composition** of stacked boxes grows around the mats in the room. Petri proposes that **composition** '*is concerned with determining the structure of documents relative to a material or conceptual carrier*' [5]. In traditional art terms, there is a strong correlation here between 'function and form' in material composition. The towers are subject to the rules of the game, the rules act as a 'conceptual carrier' determining the number of boxes in a pile. This takes the form of an ever-emerging flow of data, offered through time, as boxes move around the installation.

Contemporary art escapes the present not by resisting the flow of time but by collaborating with it. It produces artistic events, performances, temporary exhibitions that demonstrate the transitory character of the present order of things and the rules that govern contemporary social behaviour. Imitation of the anticipated future, may manifest itself only as an event not as a thing [4].

Groys's observation is a good way of thinking about the game, especially in the many ways flow can be identified. Flow happens in the **reorganisation** that ensures the slow evolution of the installation. Petri talks about reorganisation in relation to mechanised tasks within a nuclear power plant, he highlights the potentially 'disastrous' consequences if the flow of tasks and information are not adequately reorganised to meet changing situations. In the game, reorganisation is the process of becoming, an ever-changing contingent object. Reorganisation is embedded within the flow of the game and also enforced by external contingencies.

The idea of artwork adapting to contingent conditions is not new, artwork can exist for just a moment within a definitive contingent space, temporary artwork such as: Allan Kaprow's happenings and environments, or the installations of Doris Salcedo, Olafur Eliasson and teamLab et al. Martha Buskirk considers 'The Contingent Object of Contemporary Art' in her book by the same name [8]. She refers to the artwork *Gnaw* (1992) by Janine Antoni where the artist chews, over many days, two 600 pound cube's, one made of chocolate, the other of lard. The action of gnawing results in a constant reorganisation. Buskirk says:

Thus the experience of the work, including the relationship of its components to one another, changes depending on when and in what part of the cycle one sees it, and also diverges from photographic records of its appearance [8].

This is pertinent to the FOREVER-DO, not only from the position of a passive viewer, but also for the player who is not only witnessing the reorganisation of boxes in a room but accepting active agency within that reorganisation. This implicates the player as an author within the ongoing realisation of the game. Petri says of **authorization**:

this discipline is concerned not only assigning and schematically representing access rights but also with scheduling obligations [5].

By thinking about authorization in this way, a parallel can be drawn between authorization within a system, such as Petri indicates, and authorship commonly placed within art production, the 'authorial' process of an artwork. If, within the FOREVER-DO *Game*, authorship is shared across participants, objects, systems rules and networks, it cannot be solely located. Rather, obligations are shared, there is no single owner or receiver. Petri died in 2010 yet his inspiration is still implicit in the game, an authorial obligation through time. Authorial agency is delegated across all the agents of the game, flowing across the human, object and conceptual realm.

In this way **Delegation** is forming an ever-changing collaboration between the mechanics of the game and the society of people who participate in it, Petri identifies in his networks a '*delegation of tasks from one agency to another*' [5].

This shared social context is fluid, outside of a singular vision. There is, however, a fixed position in the **naming** of the groups of objects: a knife; a fork; a spoon. '*Naming is understood as denoting objects structures*' [5, p. 177]. Naming—becomes the subject of coordination, the similarity or difference, triggering the selection or transfer and the **cancellation** of the player in the game. As players track objects in the game they also have an experience of a route travelled and can anticipate a future route to travel. The pattern of flow through the network is in this way an experiential one, as infinite possibilities for coordinated behaviour are generated through random instances.

The **synchronisation** is 'partially' controlled by the system mechanism, which embraces chance, ensuring that synchronicity happens outside of a deterministic time or place. In this way, the game is mirroring Petri's premise that synchronisation is based on the '*partial ordering in terms of causality as opposed to an ordering to time*' [5]. This mirrors Groys's observation on flow being collaborative of time rather than fixing it [4].

Value, such as it is in FOREVER-DO, resides in the moment of participation or observation, not in a fixed object. Petri's consideration of **valuation** states: '*the instant the information is registered by the observer it already loses its value because it may not be presented anew.*' [5]. With this definition, valuation offers a very tricky problem to the traditions of an art market. Traditionally value in art has been placed on the object but, in the game, creative value is in the performative relational aesthetic of the work; it is essentially objectless, a process rather than an object. In this way, value is transient, unable to be 'presented anew' a data value, that can only be experienced within the moment.

In an attempt to contextualise art made in today's global context, the French curator Nicholas Bourriaud coined an art movement, the Altermodern. He defined it as a moment when it is possible to produce something:

from a vision of human history as constituted of multiple temporalities … a positive vision of chaos and complexity [1].

In the Tate triennial exhibition by the same name, 'Altermodern' (2009). The triennial presented artworks from artists such as Marcus Coats, Mathew Darbysher and Franz Ackerman whose artworks offered a document, narrative or engagement with change. Earlier in his book 'Relational Aesthetics' (1998), he offers examples of art practice as a game:

Artistic activity is a game, whose forms, patterns and functions develop and evolve according to periods and social contexts; it is not an immutable essence [1].

Art practice, based on the relational form, is presented in FOREVER-DO as a meeting point of exchange, a physical human interaction, within a networked system. In our digital age, these exchanges are often an opportunity for data mining. Points, relational in time and space, where data is gathered, producing much of our big data; data that is held outside of our physical realm.

The role of artwork is no longer to form imaginary and utopian realities, but to actually be ways of living and models of action within the existing real, whatever the scale is chosen by the artist [1].

The FOREVER-DO *Game* also represents a 'model of action within the existing real'. Big data, whose functions control us beyond our everyday perception, is here and is real. And yet, the otherness of the nature of big data, its invisible procedures, make it hard for us to perceive our own individual relationship with it—our essential role within the phenomena of big data systems. Without us, our relational interactions, big data would not exist while at the same time human life is becoming more dependent on the information big data feeds back to us. In reality, perhaps, we are already cyborgs in a close and symbiotic relationship with big data itself.

The art historian Edward A. Shanken, in his essay *Art in the Information Age* (2001), considers work by artists such as Hans Haacke and Joseph Kosuth, he says:

'meaning and value are not embedded in objects, institutions, or individuals so much as they are abstracted in the production, manipulation and distribution of signs and information' [9].

The process of the FOREVER-DO offers a distribution of signs and information, through the search for repetition. Repetition here is the key to unlocking the FOREVER-DO *Game*. It is the **copying** of objects (two knives, forks or spoons) that signifies the end of the player's participation, while the process of the game continues. Copying for Petri is a '*message*—*occurrence in a definite pragmatic context*' [5]. In the game, this is a process held within an unfolding never-ending chain of events, Nietzsche like in its eternal return. In '*Thus Spake Zarathustra*' *(1917)* Nietzsche describes two eternal paths '*This long lane backwards: it continueth for eternity. And that long lane forward*—*that is another eternity*' [10]. Most importantly the paths come together at a gateway: '*The name of the gateway is inscribed above: This Moment*' [10].

In the game, the most significant 'moments' are the coordinated meetings accumulating to model the towers of boxes. Petri utilises the discipline of **modelling** to guard against rigidity of implementation of established systems across different fields. He states:

our 'models of thinking', tend to gain some illegitimate independence once they have proved successful on a particular field, and are then – per analogy but without care – transferred to other fields [5].

Perhaps this is a warning against the transformative analysis between Petri's disciplines (as pertinent to computer systems) and the FOERVER-DO *Game*. However, the very temporality of the FOREVER-DO *Game* can perhaps embody modelling, where:

we are able to deal with mathematical models in which notions of a temporal ordering is replaced by that of an ordering in terms of causality [5].

Whether you are an active participant in the game or a viewer on the periphery of the game, you are in communication with the model of the game and its objects. This is not a fixed exchange but a communication flow (a causality), with all agents causally influencing one another in a relational sense.

Nicolas Bourriard draws on such symbiotic forms when he proposes that through the act of 'inclusion'—the art object is causally significant to the participation of the viewer. Indicating that there is an energy exchange between the art object and the participant.

 Petri offers a fluidity of destination with his discipline of **addressing**: '*by this we mean the description of routes or systems of paths through the net of channels and agencies*' [5]. This flow of address as a destination, is an agency of the ever becoming. Addressing within the flow of the game drives it on to new beginnings, this chimes with Groys when he says:

Being immersed in the flow of things, one cannot return to previous moments in time or experience the events of the past [4].

**Addressing** also happens in a subtler way too, as social address between players in the game. The form of address has a systemic origin beyond the game, residing in the social etiquette of human behaviour. This presents us with a wider network of behaviours and systems. Petri has proposed that his net systems could have a role in conflict resolution. He suggests, nets could help reconcile complex human problems, emotionally driven problems, beyond the mechanical, systematic or digital, by offering clear alternative channels of communication.

#### **6.7 Conclusion**

Art does not predict the future, but rather demonstrate the transitory character of the present – and thus open the way for the new [4].

By thinking about art as a process or event, as relational, societal and social, a new place is offered for authorship and realisation. This is not only the post-modern reform of authorship, deferred or shared, but authorship within the moment of a contingent ever flowing process—the FOREVER-DO.

This conceptual space made real through event also requires a temporal place. For the FOREVER-DO *Game* the place of the Palazzo Giureconsulti at Duomo in central Milan, and temporarily occupied by Milan digital week, is important. Milan Digital Week is itself a social construct, a temporal event into which interested parties' (visitors and participants) occupy space temporarily. Visitors are in this way already complicit within the flow of events.

The historically embedded definitions of art and art practice, particularly the traditional consideration of value in relation to object, are being challenged through utilising digital systems as integral tools. By visualising, in analogue terms, this relationship between the participant (the player of the game) and the system (the rules and network of the game), the FOREVER-DO highlights our individual complicity within the greater whole. Places of meeting within the game indicated by simple coloured doormats are also nodes within a networked system. When individuals meet through a flow of random incidents, objects are either exchanged or offloaded onto the data pile. The piles of boxes record the incidents of two humans meeting and randomly possessing a repeated object. This process presents a digital and analogue description of events through time and offers a visualisation of a similar scientific modelling that the Petri Nets identify.

Petri's Communication Disciplines, written in the 1970s still 'concentrate our attention on' building a computer system to enhance human cooperation and overcome some of the many problems brought about by our human digital interface. Problems for culture arising from irresponsible systems, technicians, trolls and corporations. We are all complicit in society's broader data communications. As Petri outlined, we must all consider the relations between computers and society that are not yet consolidated in a unique practice or competence. The FOREVER-DO opens a discussion about the interactive trail we leave behind and the systems acting upon us. By reclaiming our digital identity, being aware of the symbiotic relation of self to our larger networks, maybe we can hack our future in a positive way, for the benefit of all.

**Acknowledgements** The FOREVER-DO project and its resulting artwork (including the FOREVER-DO *Game*) is a collaboration between Jill Townsley (University of Huddersfield) and Carlo Ferigato (Joint Research Centre of the European Commission—JRC-Ispra, Italy). The project is supported by advice from the MC3 research group at the Milano-Bicocca University and Otolab, Milano. The project is funded by the EU Commission through its SciArt initiative.

#### **References**

**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Chapter 7 Searching for New Aesthetics: Unfolding the Artistic Potential of Images Made by the Scanning Electron Microscopy**

#### **Anastasia Tyurina**

**Abstract** The visual arts have become a powerful tool for alternative approaches to scientific outputs, but it is crucial that both science and art cultures are aware of their interdisciplinary capabilities and limitations. It is necessary to differentiate between images captured by devices designed as resources for scientific investigation and images that exploit the 'visual elements' of scientific images. A great example of a device designed for scientific exploration is the Scanning Electron Microscope (SEM), which was introduced to scientific research in the mid-1960s. This chapter outlines the procedures, processes, and methodologies used in developing a body of studio work that investigates the artistic potential of scientific images made by the Scanning Electron Microscope (SEM) with a view to its possible social and cultural impact of this practice. It also outlines the developed theoretical findings and proposes that certain images made by the SEM can have esthetic value apart from that of scientific documentation. The use of artistic manipulations in experiments with the SEM fuses science and technology with art, and the SEM-based images that result provide a new meaning for scientific photomicrographs.

**Keywords** Science-art · Visual arts · Photomicrography · Interdisciplinary · Microworld · Interactivity · Aesthetics · Self-organizing processes · Coffee ring effect · Alternate visualizations · Visual coalescing · Creative coding · New media · Scanning electron microscope · Water

#### **7.1 Artist Statement**

My art practice involves interplay between photography and scientific imaging, photomicrography in particular. Originally being a technical discovery, photography has been widely used in almost all fields of human activity, acting as a research tool and as an independent artistic direction. Modern times offer "modern" interpretations of

A. Tyurina (B)

Queensland University of Technology (Brisbane), Brisbane, QLD, Australia e-mail<EMAIL_ADDRESS>
National Research University MIET (Moscow), Moscow, Russia

scientific photographs and attitudes toward them. Scientific photomicrography is a powerful tool for receiving and storing information and for providing solutions to a variety of tasks in many areas of science and technology. At the same time, the artistic application of photomicrography is capable of revealing a set of complex and interrelated principles that underlay the materiality of the human environment. Photomicrographs also expand human visual vocabulary, revealing principles of beauty which are difficult to access with the naked eye.

Water is the main subject of my research project. I believe that an interdisciplinary approach is the most appropriate one for deepening knowledge about unique properties of water and building a sustainable practice of water management through enhanced visualization of water contamination. My focus is on interconnected artistic thinking and by developing alternate forms of visualization, I aim to transcend disciplines and contribute to the new ways of seeing water.

My visual artworks consist of series of images made by the SEM depicting the nature of water (water chemistry), which is invisible to the naked eye, through using the SEM utilizing the phenomenon of drop evaporation. However, it was not the purpose of this research to claim that the created images of water after evaporation are scientifically valid forms of documentation; rather, this research takes an esthetic approach to scientific photomicrography.

#### **7.2 Introduction**

This research was undertaken during my Ph.D. study at Queensland College of Art, Griffith University and involved esthetic approaches to scientific photomicrography. Scientific photography is commonly perceived as a way of recording scientific data through techniques such as photomicrography, high-speed photography, time-lapse photography, X-ray photography, and aerial photography, among others.

Images made by the SEM are not photographs in the traditional sense; they go beyond what can be captured with light because the process of producing a picture is camera-less. Imagery produced by the SEM can confuse the viewer because the microscopic sample seems as if it is observed in the eye aperture when illuminated, and light seems to come from a particular illuminant. SEM photomicrographs are constructed out of pixels synchronized with a distribution map of the intensity of the signal being emitted from the scanned area of the specimen [1].

Interestingly, the last decades' use of SEMs in creating scientific images formed a new, well-established visual culture within a variety of scientific disciplines. As Klaus Hentschel explains, it became "*an image centered science in the sense of being even totally dependent on photographic images as basis of all further processes of inference*" [2, p. 315].

By exploring the interplay between the indexical and iconic modalities in scientific photomicrographs, I try to imbue them with new meanings. Both art and science are experimental in nature. There are different ways of artistically representing and perceiving an object, some of which may be valuable for science. My practice aims to draw attention to the qualities of water through enhanced visual details that aid in the interpretation of and differentiation between water samples.

This project investigates how to reinterpret photomicrographic images made by the SEM of micro-scale drops of water after evaporation and thus turns scientific photography into an art form.

In recent decades, there have been increasing concerns over the ecological management of water. Waterway pollution is recognized as placing urban ecosystems around the world at risk. Rainfall that washes oils, metals, and nutrients directly from streets into rivers and seas is hard to treat [3]. This is a challenging problem for science and technology. The health and well-being of present and future generations are dependent on how quickly and well it is managed.

Photomicrography has a particular potential to respond to this issue from both a scientific and a cultural perspective. SEM-made photographs are capable of visually representing features related to water composition and, in some cases, the contamination of water. At the same time, they can transmigrate from science into the domain of art and draw attention to water issues.

#### **7.3 Water Represented by the Scanning Electron Microscope**

The variety and frequency of the unusual properties of water are determined by the physical nature of its atoms and their association in the molecule and the groupformed molecules [4, 5]. The composition of water, even that which is entirely free from mineral and organic impurities, is complex and diverse because water is constantly in contact with many substances.

During my experimentation with the water droplets collected from different aquifers, I aligned the scientific method of revealing water composition (the so-called 'Coffee Drop Effect') with my artistic practice. The drop evaporation phenomenon, the so-called 'Coffee Ring Effect', has been the subject of studies in the last few decades. It was first mathematically described by Robert D. Deegan in 1997 as a natural model for studying the dynamics of self-organizing processes and is actively used in physical experiments [6]. When the liquid is 'pinned' to its contact surface, the liquid evaporating at the exterior edge is replenished by the constant flow of liquid from the interior to the periphery. This flow carries nearly all dissolved solids toward the edge [7].

During experiments for my project, the structure of the water impurities visually transformed, leading to a unique connection between the processes of evaporation and solidification. These two processes are shown in Fig. 7.1. This natural process of drying reveals the unique informative capacity of droplets through the shapes, patterns, details, and characteristics resulting from each water sample.

At the University of Chicago Materials Research Center, scientists are exploring the driving mechanism responsible for this phenomenon that is found in many

**Fig. 7.1** Anastasia Tyurina, *Drop Evaporation and Solidification Scheme*, 2015. Image copyright © A. Tyurina 2019

varieties of liquids. This research validated my process of exploring variations in water composition for my project. On its webpage, the Center states: "*We have determined that ring formation is a ubiquitous and robust phenomenon. It doesn't depend on the solute, the solvent, or the substrate so long as the solvent is partially wetting and volatile, and the contact line is pinned*" [8].

An interesting observation made by the researchers is that "*By controlling the speed and spatial variation of the evaporation, this model predicts that we can control the shape and thickness of the deposit*" [7]. This was particularly important for my practice because it meant that experimenting with various modes of evaporation could lead to different visual formations of the drops after they dry.

Specifically, for this research 'science' can be described as a process through which new knowledge about the world is built. The scientific process relies on the testing of ideas through experimentation. Scientific photography, as a tool, aims to support the scientific process by capturing data about the world for examination.

Reflecting on the scientific findings, the primary purpose of my visual art project was to depict the inherent features of water that are invisible to the eye through using the SEM. To do so, I used the process of evaporation as an alternative and unusual artistic method of visually presenting the composition of water. My approach is unique in the specific way in which I use water to create images using the SEM. During experiments for my project, it became evident that the structure of water impurities is visually transformed after evaporation and reveals a unique connection between evaporation and solidification. This process of revealing the nature of water (water chemistry) allowed me to play with the process like an artist [9].

One of the technical requirements when taking a microscopic image with the SEM is that any object placed in the chamber of the SEM must be completely dry, because the SEM operates in a vacuum. As explained in Under the Microscope: A Hidden World Revealed (1987), "*Turning to biological specimens for the SEM, it used to be necessary first to fix and dehydrate them, and then dry them either on air or by the use of liquid carbon dioxide ('critical point drying')*" [10, p. 199]. After the evaporation process, water is no longer a liquid; dry solids or other substances become watermarks that represent its previous composition.

The main difference between 'Artistic' and 'Scientific' use of the SEM in this project is that the artistic approach has a focus on making aesthetic images. Within this is the substance of the scientific idea about showing water state and how it is changing. In my work the results of imagery processes become art pieces with both artistic and scientific applications (Fig. 7.2).

My artistic intervention into a scientific process through experimenting with the SEM was a way to find out what different things my images could reveal about water to a viewer. There was an opportunity to explore some concerns related to the environmental impact of water pollution in an artistic context and to communicate the significance of water.

By exploring the integration of art and science in this way, my project focuses on the use of scientific tools to create science-based art. However, it must be recognized that my approach was primarily one of artistic research and experimentation: a search for new aesthetics, exploring spatial and temporal dimensions, engaging with materiality, and involvement of modern technologies in the formation of esthetic experiences for the viewer. In my project, images are highly variable in what they show and how they show it because of changes in water content.

#### **7.4 Photomicrography as an Art Form**

Contemporary practices in photomicrography are making significant contributions to the dialog around aesthetics and artistic components in science. As far back as 1963, in his article "The Esthetic and Pictorial Applications of Photomicrography" for The Photographic Journal, Douglas Lawson wrote:

When mentioned within the hearing of some pictorialists they immediately think it is only for the scientifically minded, or the record worker. Nevertheless, the so-called record photograph can be made to look quite attractive. Some of you may remember Mr. H. A. Murch, one of our great pictorial photographers, once saying, 'I do understand the desire to apply pictorial ideas in record work, which is a very different objective, and we ought to welcome such an application when it can be done without losing anything of the essential factual value of the record' Photomicrography offers expression not in what the painter has already done but in what the painter cannot do. Because this application of photography is highly scientific there is perhaps a tendency to think of it as being without scope for the artistic application. May I suggest that a work of art in any medium can be the deliberate creation of unity, and nature through the microscope is one medium provides us with plenty of scope for such unity [11].

To effectively engage with photomicrography as a social phenomenon, it is crucial for an artist to demonstrate an understanding of its 'scientific' protocols of representing. But will the viewer be familiar with what he or she sees in scientific images? If

**Fig. 7.2** Anastasia Tyurina, *Watermarks Series*, 2015 (From left to right: *Brown Lake, Brisbane River, Raby Bay, Rainwater, Mount Gravatt Pond, Mimosa Creek, South Bank Pond, Cylinder Beach*). Image copyright © A. Tyurina 2019

not, how will they perceive such images? Do the images need to be explained or are they capable of conveying messages in a different context?

Artistic photomicrographs of water after evaporation call upon the viewer's creative ability to perceive previously unseen water composition after evaporation as well as to observe natural phenomena over and beyond the directly visible.

My images transform the microworld to a macro-level and evoke an interest in water chemistry that is shown as being beautiful. This causes a dilemma for viewers, particularly because the gallery space is different from a laboratory. The captions for my photographs refer to the sites where the water samples were collected. It is intriguing that they can resemble aerial photographs of topographical features of particular water reservoirs (Figs. 7.3 and 7.4).

Today, the practice of visually pleasing photomicrography is more mainstream. The most popular international photomicrography competition is Nikon's Small

**Fig. 7.3** Anastasia Tyurina, *Photo documentation of the artistic installation for H2O* + *exhibition, PoP Gallery, Queensland College of Art, Griffith University, 30 March*–*16 April* 2017. Image copyright © A. Tyurina 2019

**Fig. 7.4** Anastasia Tyurina, *Photo documentation of the Shifting the Posts exhibition, Webb Gallery, Queensland College of Art, Griffith University, 27 November*–*7 December December* 2018. Image copyright © A. Tyurina 2019

World Photomicrography Competition, which dates back to 1975. Participating images are judged for "*their scientific and artistic merits*" [12] and represent a broad range of visual studies of the microworld. Some of these photomicrographs have allegorical titles, a fact that highlights the relation of artistic intentionality in producing images and the scientific objectivity underlying the concept of the competition.

Participation in the competition is restricted to images made by optical microscopes of any kind, which are capable of revealing natural colors of objects. As mentioned above, the SEM apparatus is camera-less; there is no light involved in the process; it is not an optical instrument. Thus, colors cannot be reproduced. Rather, artificial colors can only be added with the help of graphic software such as Adobe Photoshop. While there is a whole range of possible manipulations available in Photoshop, it is worth mentioning the coloring tools. Through different algorithms, it is possible to apply different colors to different details of the image. There are many sources both in the technical literature and on the Internet providing scientists and artist with various tutorials on image coloring. But the author of the image makes the final decision of which color to apply to a particular point of the presented object and thus color choice is very individual. Experimenting with such coloring, I have tested two options: the coloring algorithm available in Photoshop and the software offered by Recolored, which was designed for the purpose of recoloring black-and-white photographs (Fig. 7.5).

**Fig. 7.5** Anastasia Tyurina, *Colour 32*, 2017. Image copyright © A. Tyurina 2019

#### **7.5 Photographs or not Photographs?**

It is important to examine the process of image formation in relation to the imagery obtained by the SEM in modern microscopy. It is also crucial to distinguish image generation by SEM considering that the process is not one that is based in code. The image generation process can be seen as a work of high-energy electrons that create a range of different signals when hit the specimen. Converted into pixels signals result in image formation appeared on the screen.

Technically, the SEM produces electron micrographs because the image is developed by either secondary electrons (electrons ejected from the material by the incoming electron beam) or backscattered electrons (electrons from the electron beam, which have 'bounced' off the material into the detector). Dee Breger explains:

Scanning electron microscopes don't merely use electron beams to illuminate objects so small they can't be seen by light. Since they are electronic devices, SEMs can manipulate isolated groups of electrons from the sample-beam interaction to create separate pictures (such as secondary and backscatter images) that contain different kinds of information about a single object. This variety can lead to a 'compound reality' or, since no version tells the whole story, a kind of 'ironic nonreality' [13, p. 11].

Images produced by the SEM are beyond light; captured by a focused beam of electrons, they are not photographs. The apparatus tries to recreate a reality that is not a visual phenomenon, which scientists then try to analyze through its visual representation: the photomicrograph.

However, the most commonly used SEM electron detector, named Everhart-Thornley (E-T),1 typically uses a material that produces light when an electron collides with it—a scintillator.2

Therefore, the SEM does use photons, but they are converted back into electrons, which are accelerated onto the electrodes of the photomultiplier, producing an increasing stream of electrons until the final collector is reached [14]. The image formation in scanning electron microscopy consists of the scanning system, the signal detectors, the amplifiers, and the display [1]. A physical, material connection of the SEM apparatus with the studied object is gained by the interaction of these four elements (Fig. 7.6).

<sup>1&</sup>quot;The E-T detector operates in the following manner: when an energetic electron (≈10 keV energy) strikes the scintillator material (S), light is emitted. Light is conducted by total internal reflection in a light guide (LG) (a solid plastic plastic or glass rod) to the photocatode of a photomultiplier (PM). At the photocatode, the photons are converted back into electrons, which are accelerated onto the successive electrodes of the photomultiplier, producing an ever-increasing cascade of electrons until the final collector is reached" [14].

<sup>2&</sup>quot;A scintillator is a material that accepts incident high-energy electromagnetic or charged particle radiation and in turn uses that energy to fluoresce photons whose peak emission wavelength is longer than the wavelength of the incoming radiation. In the case of a SEM, the scintillator disc collects the secondary electrons that are produced as the electron beam scans the surface of the sample. These electrons are converted into photons which travel through the light pipe to the photomulitiplier tube (PMT) so that the signal may be amplified to the level required for viewing" [15].

**Fig. 7.6** Anastasia Tyurina, *Scanning Electron Microscope Imaging Facility*, 2016. Image copyright © A. Tyurina 2019

During the imaging process, the energy exchange between the electron beam and the sample results in the reflection of high-energy electrons, the emission of secondary electrons and the emission of electromagnetic radiation, each of which can be measured by specialized detectors. The detectors interpret these signals by the algorithm and the resultant image appears constructed out of pixels. This algorithm is defined by each quantum of signal information but it is also implemented in the physical world in apparatus, computer, and screen [1]. In some sense, the SEM apparatus interprets (transforms) signals and visualizes signals.

#### **7.6 Scientific Images and New Media**

The artworks employ two kinds of medium, comprising two different applications of SEM photomicrography: still photomicrographs and interactive digital installations. They can be perceived differently in terms of the esthetic response they generate in the viewer, but both draw attention to the qualities of water through enhanced visual details that aid in the interpretation of water samples. I use static SEM-generated photomicrographs of water, then apply a digital code which allows the image to be altered by the audience's interaction with its touch-sensitive interface. Thus, anyone who touches the screen can create, display, and experience a ripple effect, which is very similar to the effect we can see and observe when we interact with water surface by touching or disturbing it. Interacting with the scientific photomicrographs in this way offers a layered meaning and can enhance the audience's perception of scientific data, scientific photography, and water.

Exploring the idea of 'interactivity' for my project, I looked for a method that allows viewers to interact with photographs by altering the work's visual content. Utilizing the programming language Processing, I developed an algorithm which allows viewers to physically interact with my photomicrographs so that they become direct objects of manipulation. Using the random human touch of some areas of the developed algorithm alters the static image, which in some senses visually represents non-visual elements: scientific data.

Through my research, I have explored different digital tools and realized that creative coding suits my goals because this type of computer programming is designed to create something expressive instead of something functional. Using creative coding tool such as Processing, it is possible to create art installations, projections, sound art installations, and much more within the context of the visual arts.

The interplay between virtual ripple effects and the resemblance of physical interaction with water can be seen as embodying the "artist–display–user" paradigm seen with regard to computer art [16, p. 10]. In their article "The Post-Display Condition of Contemporary Computer Art," Toby Juliff and Travis Cox attempt to re-conceptualize the relationship between artistic intentionality, coding, interface, and user input. They suggest that the relationship between the artist and computer code should be reconsidered so that the paradigm becomes "*artist*–*code*–*display*–*user.*" They state that "*the code takes input from the user and subjects it to internal semiosis between distinct elements of the code, before being output to the display. This code, having been written by the artist, or an agent of the artist, contains within its structure an inbuilt intentionality, a way of approaching input that is integral to conveying the intended meaning*" [16, p. 10]. In line with artists using code to activate new experiences and with Juliff and Cox, who argue that the code is not only a product of intention and meaning but also a producer of them, I seek in my practice to explore the role of code in the production of meaning, in particular for the scientific photomicrograph of natural phenomena.

In the series of live images for my project, the photomicrographs displayed on the screen seem to be still when viewed from a distance; yet, as the audience comes closer and start touching the screen, the ripples appear.

The photomicrograph transforms into a live picture, producing varying forms of ripple that seem both ordered and random at the same time. Such transformations occur continuously until the user-viewer stops touching the screen (Figs. 7.7 and 7.8).

**Fig. 7.7** Anastasia Tyurina, *Rainwater* (detail) 2016, Digital mixed media. Image copyright © A. Tyurina 2019

**Fig. 7.8** Anastasia Tyurina, *Rainwater* (detail) 2016, Digital mixed media. Image copyright © A. Tyurina 2019

Interacting with the image in this way transforms the work into something that transcends disciplines. It offers a layered meaning, providing audiences with the opportunity of experiencing the fluid and animated qualities of the effects that connects with these qualities in the subject matter: water. The addition of the interactivity and animation introduced by the ripple effect is the expanded mode of 'reading' or appreciation. As well as offering a visual engagement, the work offers an embodied

**Fig. 7.9** Anastasia Tyurina, *Colour 32* (detail) 2017, Digital mixed media. Image copyright © A. Tyurina 2019

engagement: an important connection to the material significance of water in our lives.

Colour 32, one of the artworks, portrays a drop of Brisbane River water. This artwork was exhibited at various venues displayed as both interactive work and video at screens of different scale (Figs. 7.9, 7.10 and 7.11).

Such digital installations allow the transformative power of water to be explored; they offer new perceptual experiences and, by the artist–code–display–audience interaction, are capable of providing a new meaning for scientific images through their ever-changing visualization.

#### **7.7 Conclusions**

Not every image is art, and art is far more than just an image. Originally a technical discovery, photography has been widely used in almost all fields of human activities, acting as a research tool and as a form of artistic practice. Scientific tools have brought new ways of seeing the world. Although usually reserved for scientific use, such technology is now being used in the creation of art.

The primary purpose of this research was to show that the artistic use of SEMmade photomicrography can shift the visual outcomes of scientific photomicrography

**Fig. 7.10** Anastasia Tyurina, *Colour 32*, North Spine Plaza, Media Art Nexus, Nanyang Technological University (NTU), Singapore, 2017. Image copyright © A. Tyurina 2019

**Fig. 7.11** Anastasia Tyurina, *Colour 32*, public screening, Adelaide Festival Centre, 2018. Image copyright © A. Tyurina 2019

to function within the context of art. Drawing on literature, visual analysis, and theoretical findings, it was possible to develop my artistic practice using the SEM to visualize the microworld of water. As Breger states, the microworld exists beyond our cognition; its beauty is hidden but it can be explored using appropriate tools, such as the SEM [13].

Not many people have access to complex research technologies that can widen their daily experience such as the SEM. Mostly, people passively receive scientists' dry results and the interpretations they are exposed to. My work in the field of photomicrography aims to expand human visual vocabulary, revealing principles of beauty which are typically difficult to otherwise access. Thus, my artistic intention is to expand the visual representation of the composition of water, its chemical features, and their patterns after evaporation. My aims are to also communicate issues related to the ecological management of water and to raise and deepen responsible attitudes toward consuming and managing water resources.

During this project, I have placed (both virtually and physically) scientific images into a 'hostile' artistic environment. In the gallery space, the significantly greater scale of photomicrographs, their impeccable details, the artistic materials used, and the digital, still and interactive modes of presentation uncover the novel artistic potential of the scientific image. This leads to the image being disconnected from pure data and objectivity and allowing the viewer to perceive it as art, helping them to interpret photomicrographs of water in different ways. Most importantly, through these images, I hope to uncover the horizon of meanings previously unseen.

Today, many institutions and organizations host scientific photography and visualization competitions (and associated exhibitions), which encourage scientists from different disciplines to contribute. The images are not only judged by professionals (scientists and photographers) but often the audience is also invited to evaluate the competing images. By doing so, all three groups apply their own esthetic judgment criteria. This illustrates how scientific images can be evaluated from different contexts. Examples of such events include the annual the Visualization Challenge, National Science Foundation; the Wellcome Image Awards, Wellcome Trust; Nikon Small World Photo Competition; The Royal Photographic Society International Images for Science, just to name a few.

This popular trend indicates the growing interest in the visual coalescing of science and art through photography, particularly through photomicrography. Nevertheless, the idea that the photographic scientific image can be artistic and that the arts can serve science and vice versa still generates skepticism, and the debate will surely continue. Regardless, it is obvious from the discussion that the arts have the potential to reveal important aspects of the complex world in their own way, which is complementary to what scientists do and may be useful for both disciplines. Scientific photography aims to record and illustrate data and experiments that differ according to specific disciplines. Although the main purpose of scientific photography is to convey accurate information rather than beauty, its ability to record material in addition to that which is merely informative allows it also to serve expressive, subjective, and esthetic purposes.

The new definition other than photographs (or photomicrographs) for SEM images should be sought considering the nature of the process of image generation by the SEM. This is particularly interesting due to the complex nature of the apparatus. Different ways in which images made by SEM can be interpreted create layered meaning in them especially if they are given in the artistic context.

#### **References**

**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Chapter 8 Interspecific Interactions: Interaction Modes Between Sound and Movement in Collaborative Performance**

#### **Manoli Moriaty**

**Abstract** Collaboration between composers and choreographers is an approach possessing a long history and expansive list of creative outcomes. In recent years, such collaborative endeavours have further engaged with knowledge and practices from scientific disciplines, consequently giving rise to an emergent field of research focusing on novel modes of interaction between the expressive media of sound and movement as facilitated by new technologies and methodologies; from analogue proximity devices permitting dancers rudimentary control over recordings in the 1965 *Variations V*, to Marco Donnarumma utilizing biophysical sensors and machine learning algorithms as means of maintaining detailed command of digital signal processing (DSP) through specific body postures in the 2016 *Corpus Nil*. Whilst the contributions by this relatively new transdisciplinary field have produced practice and research outcomes of unquestionable value, its main focus concentrates on the expression of sound through movement, with lesser emphasis placed on perspectives of practitioners utilizing interactive sound as means of informing the arrangement of movement. This chapter aims to reflect those perspectives through the research conducted during my collaborations with dance practitioners, where we have together examined areas such as each performer's role while operating interactive systems, employing different modes of system operation according to the desired determinacy of resulting material, and simplification of discipline-specific language when engaged in polydisciplinary collaboration. The contribution of this research concerns a novel taxonomy of interaction modes informed by the biological phenomenon of symbiosis. Defined as close and persistent relationships between organisms of different species, symbiosis manifests in three core types—mutualism, commensalism, and parasitism—with each type determined according to the change in fitness outcome experienced by each of the engaged organisms. In the context of my collaborative practice, the biological notion of fitness outcome is interpreted as the expressive range allocated to each performer and their respective media, while simultaneously relating to the level of determinacy from the score and choreography. The symbiotic modes of interaction are firstly described through examining three contemporary precedents, each showcasing distinct approaches in

M. Moriaty (B)

University of Salford, Salford, UK e-mail<EMAIL_ADDRESS>
<sup>©</sup> The Author(s) 2020

R. Earnshaw et al. (eds.), *Technology, Design and the Arts—Opportunities and Challenges*, Springer Series on Cultural Computing, https://doi.org/10.1007/978-3-030-42097-0\_8

collaboration and use of gesture recognition technologies (GRT), followed by a practical demonstration of each mode by its activation during a practice outcome resulting from my collaborations with two dancers, including a detailed analysis of the developed interactive system and its principles of its operation.

**Keywords** Polydisciplinary · Collaboration · GRT · Interaction · User interface · Symbiosis

#### **8.1 Introduction**