Therefore, the time window that represents the class 1 (laser) for obtaining the model will be chosen starting in the time when the maximum is detected and will have a length of 800 ms. In contrast, the time window of class 0 (non-laser) will be chosen equally for all subjects lasting also 800 ms just before the laser appearance. This way, the area of greatest difference between class 1 (laser) and class 0 (non-laser) can be captured.

### 2.4.4. Feature Extraction

After the selection of the subset of electrodes and the 800 ms time windows for class 0 and 1, feature extraction is performed (Figure 2, left). In all cases, the features have been obtained from the average of the signal of the electrodes chosen for each subject, which is called *f* 1.

From this signal *f* 1, a vector with 5 features has been extracted:

1. Area included under the curve of the absolute value of *f* 1 derivative:

$$fent(1) = \int abs(f1')\tag{3}$$

2. Variance of *f* 1 derivative:

$$fent(2) = var(f1')\tag{4}$$

3. Amplitude of *f* 1 derivative:

$$feat(\mathfrak{z}) = \max(f1') - \min(f1')\tag{5}$$

4. Amplitude of the accumulative sum of the absolute value of *f* 1 along a window:

$$feat(4) = \max(sum(f1)) - \min(sum(f1))\tag{6}$$

5. Maximum cross correlation between *g*1 and *f* 1 in each case (*g*1 is the mean of all the *f* 1 obtained in each particular laser appearance):

$$f\_i \\ f\_i (\mathfrak{F}) = \max(f1 \star g1)\_i \tag{7}$$

The features vector is obtained for all window classes 0 and 1, and is properly labelled.

### 2.4.5. Classifier

For the distinction between the 2 classes, a linear discriminant analysis (LDA) has been chosen as the classifier, which transforms a set of α-dimensional data into a δ-dimensional subspace (with δ < α), maximizing the distance between the means of each class and, simultaneously, minimizing their dispersions [24,25] (Figure 2e).

The discrimination function is as follows:

$$h\_{\beta}(\mathbf{x}) = \begin{cases} \begin{pmatrix} \beta^T \cdot \mathbf{x} + \beta\_o \end{pmatrix} \ge 0 \to 1\\ \begin{pmatrix} \beta^T \cdot \mathbf{x} + \beta\_o \end{pmatrix} < 0 \to 2 \end{cases} \tag{8}$$

$$\beta = \sum^{-1} (\mu\_1 - \mu\_2) \tag{9}$$

$$\beta\_o = -\beta^T \cdot \left(\frac{\mu\_1 + \mu\_2}{2}\right) + \ln(\frac{\pi\_1}{\pi\_2}) \tag{10}$$

where β is the vector of classification parameters, β*o* is the bias term,  is the clustered covariance matrix, μ1 and μ2 are the average vectors of class 1 and class 2, and π1 and π2 are their prior probabilities [26]. As these probabilities are not known a priori, they are also analyzed in the results section for a prior probability (π) assigned to class 0 of π = 1:4 versus π = 1 to class 1.

### *2.5. Pseudo-Online Analysis*

In order to achieve a more realistic evaluation of the BMI, a pseudo-online classification is performed using the last trial of each session.

For this purpose, a sliding window from EEG signals of 0.8 s has been analyzed, shifting each one by 0.1 s. Each window was analyzed following the procedure in Figure 2 (right) and is classified as to whether it belongs to an event of walking (class 1) or response to the obstacle (class 0).

The process is applied only to the electrodes selected during the model creation. Since the band-pass filter can distort the sides of short time windows, the length of the window is extended to the previous seconds of each moving window. Then, after applying the band-pass filter, only the last 0.8 s are kept for the next steps. Then, the data of the selected electrodes are averaged and their features are extracted. Finally, the LDA model is applied to obtain the classification of each moving window as class 0 or 1.

Due to the overlapping of windows (0.7 s), there is redundant information that can cause the detection of several successive windows as class 1, when for instance only one has been produced [27]. Therefore, a K value that represents the number of positive detections (class 1), must be taken into account to assure a correct one. This parameter has been tested for values of K from 1 to 5. For example, if an isolated detection is performed with K = 2, it will not be considered, as it would require at least 2 consecutive detections to its computation. A correct selection of K helps to reduce the number of false positives.

In addition, the real-time algorithm for detecting the stop through the Tech MCS equipment described in Section 2.3 has been included in the pseudo-online test. Due to the structure of the test, when the person restarts the gait after a stop, a noise is generated on the EEG signal. This distortion affects the BMI system, as it could be detected as a false positive. However, as the obstacle detection must be only considered when the subject is walking, the predicted class is automatically set to 0 during the windows associated with the 1.5 s after the IMU's detection of a stop. This period of time is not considered for the total count of seconds, in order to not affect the FP/min rate.

### *2.6. Subjects*

Nine healthy subjects, with ages between 21 and 54 years, without any neurological disorders and with normal vision have participated in the experiments. The experiments were approved by the Ethics Committee of the Miguel Hernández University of Elche (Spain). All subjects were informed and signed informed consent according to the Helsinki declaration.

### **3. Results**

This section starts describing the results of the stop detection through the IMUs. This is a critical result, as the feedback of the IMUs provides the tool used for the definition of a real physical stop. Section 3.2 shows the subset of electrodes obtained by manual selection as explained in Section 2.4.2. From Section 3.3 onwards, the results cover the different analysis considered for the detection of the stop intention through EEG. Results are obtained from the pseudo-online analysis of the last trial of each session. The parameters computed include the true positive rate (TP) and the false positive rate (FP/min). The different subsections correspond to some of the alternative paths shown in Figure 2. The first default analysis (Section 3.2) uses a manual selection of electrodes, the average of trials for the window selection of class 1, a prior probability of 4 and K values from 1 to 5. From those default results, successive analyses study alternative paths such as changing the prior probability of the classifier, selecting the class 1 windows by event or using an automatic selection of the electrodes. Finally, the time of detection by the algorithm is compared to the time when the stop is computed by the IMUs, in order to assure that the detection is assessed before the real stop happens.

### *3.1. Results of the Stop Detection through the IMUs*

Table 1 shows, for each subject, the percentage of stops correctly detected through IMUs (TP: true positive) and the percentage of stops not detected (FN: false negative). Both values have been calculated on average for all the sessions of each subject. The number of false positives is also included, considering FP as the cases where a stop is detected but there is not a real one. Furthermore, the average time for each subject to detect the stop from the laser activation is shown.

**Table 1.** Results of stop detection through IMUs.

From the results, it can be asserted that the algorithm properly detects the stops, with a TP percentage of 92.5% ± 7.0% and only 0.1 ± 0.2 FP. There are only false detections for 2 of the subjects (S5 and S6) while the rest of them have 0 false detections.

Regarding the reaction time for each subject, the values are in the range of 1.6 to 2.1, which could be considered a high variability range. In this sense it must be considered that all the subjects do not stop in the same way. Some of them stop their gait immediately after seeing the laser, while others go one step further before stopping. This could explain this variability, in addition to the different reaction times of each person.

It is important to remark that the objective of the research is to detect the intention of the subject to stop when an obstacle appears. Therefore, the time showed in Table 1 must be lower than the needed time by the developed algorithm to detect a stop intention.

### *3.2. Manual Electrode Selection*

Table 2 shows the list of electrodes selected for each subject with the manual method. Furthermore, the maximum of the average signal is included in the third column, in milliseconds, determined by the addition of the selected electrodes in all laser appearances.

**Table 2.** Manual electrode selection and maximum location for each subject. Electrodes coincidental with automatic electrode selection in bold text.

From the results some conclusions can be obtained. First, in 8 out of 9 subjects there is an activation of the electrode FCz, which reflects that an error-related potential is being generated when the obstacle appears [28]. Also, electrodes Cz and Fz, located in the same area (fronto-central area) appear in most of the subjects. Additionally, there is an activation of the motor area, with the inclusion of electrodes such as FCz, FC2, C1, C2, C3 and C4 due to the intention of a stop. Finally, the occipital electrodes such as P1, P2, P3, P4, POz and Pz are also activated due to the visualized obstacle. Both behaviors could be expected since we have a visual stimulus (laser), to which the subject reacts, sending an order to stop.

### *3.3. Results of the Pseudo-Online Analysis with Initial Configuration*

Table 3 shows the results obtained in the pseudo-online analysis of the tenth trial. The default configuration considered is based on the manual electrode selection and the time windows selection for class 1 after the maximum time showed in Table 2. The prior probability assigned to class 0 (non-laser) is 4 versus 1 for class 1 (laser).

**Table 3.** Results for the pseudo-online analysis with different values of K. Best values according the criteria (bold).

A detection is considered successful when it happens in the interval between the appearance of the obstacle and the average reaction of the subject. The pseudo-online analysis is calculated for K values between 1 and 5. Results for K = 1 are not detailed in the tables since the FP/min are too excessive to be considered as acceptable.

The criteria which has been followed for considering the best result is to keep a FP/min rate lower than 4 with the highest TP value. Following this rule, Table 3 shows in bold text the best result for each case and subject. Results show a mean TP value of 59.5% ± 25.9% and an average value FP/min rate of 2.7 ± 1.3 The K values depend on the subjects with values from 2 to 4.

### *3.4. Results Varying the Prior Probability of the Classifier*

Table 4 shows the comparison between the TP and FP/min rates for different values of prior probability π = 2:4 assigned to class 0 (non-laser) versus π = 1 to class 1. To condense the information, only the results of the optimum K value are shown for each subject.

**Table 4.** Results obtained in the pseudo-online analysis for different values of prior probability for class 1 (laser).

Results for equal prior probabilities (1 vs. 1) for both classes have not been included because of the high FP/min obtained by all the subjects.

According to these results, prior probability of 3 for the class 0 offers the best relation between TP and FP/min rate, so the next analysis will use this probability to calculate the results.

### *3.5. Results when Modifications are Made to the Best Case*

The results in this subsection take into account two alternatives regarding time window selection and automatic electrode selection.

### 3.5.1. Results for Time Windows of Class 1 Obtained from a Particularized Maximum by Event

Table 5 shows the results obtained for capturing the time windows for class 1 locating the maximum of the signal for each time window, according to the explanation given in Section 2.4.3. Results are generally worse with this alternative, except for the subject S1, who obtains a significant improvement. For the rest of the subjects TP has a value of 46.6% ± 27.8% and FP/min of 2.7 ± 0.8. This reflects a decrease of nearly a 15% in the obstacle detection, while the FP/min rate only decreases by 0.1. Therefore, it is not worthwhile to particularize the class 1 window by event.

**Table 5.** Results for capturing the time windows with the particularized maximum.

3.5.2. Results Obtained with the Automatic Selection Electrodes

Table 6 shows the list of the electrodes obtained with the automatic selection as it was explained in Section 2.4.2, as well as the time instance from which the time window for class 1 is captured for creating the model, according to the mean obtained from all the laser appearances.

**Table 6.** List of electrodes obtained with the automatic method for each subject. Electrodes coincidental with manual electrode selection in bold text.

If this selection is compared with the one obtained manually, it can be seen that with the automatic method, in 8 out of 9 cases the selection includes a lower or equal number of electrodes. Additionally, as it happens with the manual selection, the electrodes that appear more frequently are again the ones

related to the error-related potential: FCz, Fz (which appear in all the subjects), Cz, FC1 and FC2. The electrodes located in the occipital zone (P1, P2, P3, P4, POz and Pz), where visual information is generated, also seem to contribute to the generation of a distinctive potential in most of the cases. Electrodes located in the motor area (C1, C2, C3 and C4), appear only in subject S7, so the potential generated for the stop intention is weaker, contributing less to the detection of the potential in the automatic selection method.

Table 7 shows the results obtained with the pseudo-online analysis. Only the results for the best value of K are shown. The right side of the table shows the best results obtained with the manual selection method for comparison.

**S7** - - - 5 15.4 3.0 **S8** 4 100.0 3.6 4 92.9 3.6 **S9** 3 69.2 3.8 3 91.7 2.3

**Table 7.** Comparison between the results obtained with the automatic electrode selection and with the manual selection.

It is difficult to consider which electrode selection performs better. Two out of the ten subjects, S1 and S8, improve with the automatic selection. However, in the case of S4 and S9, the results get considerably worse. Subject S3 obtains exactly the same rates, while in the rest of the cases it is not clear as to whether the results improve or worsen, because a TP rate increase is correlated to a higher FP/min and vice versa.

As a criterion for its application in real time, the electrode selection could be first done using the automatic selection, as it is an unsupervised, faster method. After the model and selection is created, a new trial could be done to see if the TP is higher than 60% and the FP/min is lower than 3.0. If they are, the automatic selection is used, and if not a supervised manual selection is done and tested in pseudo-online analysis to see if the results could be improved.

### *3.6. Analysis of Instant Time Detection Versus Obstacle Appearance and Physical Stop Detection*

In this subsection the instance after the laser appearance, when the BMI detects the intention to make a stop through EEG signals, is analyzed. This instant is compared with the instant when the IMUs detect the stop. With the difference between these two moments it can be stated whether the stop intention is detected before the real stop of the person. For instance, this could be helpful in the command of an exoskeleton, allowing the subject to stop it faster than the reaction time.

Table 8 shows those results taking into account the optimal value of K obtained for each subject with a prior probability π = 3 for the class 0 and with the manual selection of electrodes. The time interval between the detection of a stop through the IMUs and the classification of class 1 through the BMI (2nd column) are shown. Table 8 also shows the interval between the obstacle appearance and the obstacle detection through the BMI (3rd column). Both intervals are calculated averaging all the time differences obtained with the different laser appearances. In all the cases, the detection of the potential has been produced before the real detection by the person. This means that the BMI has been able to predict the detection of the subject.

**Table 8.** Analysis of the instant time detection (in seconds): IMUs vs. Brain-Machine Interface (BMI), which indicates prediction time before physical stop of the user; and BMI vs. laser appearance, which indicates how much time the BMI needs in order to detect the stop after the obstacle appears suddenly through the laser.

The average anticipation is 0.9 ± 0.2 s, which could be considered enough anticipation to send a command to an exoskeleton and stop it in the case of an obstacle appearance.

Regarding the time interval between the laser appearance and the detection through the EEG signals, an averaged value of 1.2 ± 0.2 s is obtained. In this case, however, a great variability between subjects is shown. In fact, it must be appreciated that there is a relation between the instant when the maximum of the EEG signal was observed in each subject and the mean value when the intention to stop is being detected through the pseudo-online analysis. For instance, in S4 the peak of the signal appears 720 ms after the laser appearance (the highest value) and has the highest time interval (1.5 s).

### **4. Conclusions**

This paper has evaluated new methodologies for detecting the appearance of unexpected obstacles while walking using EEG signals. Since the final goal is to propose a method that can be used in real-time tests, the chosen alternative cannot involve excessive time for training the BMI system and obtaining the model. That is why it has been decided to personalize some of the algorithm decisions, due to the improvement of the results, and generalize others to reduce the time needed to create the model. The results have been tested in a pseudo-online scenario which is a more realistic analysis and is suitable for its future application in combination with an exoskeleton in real time.

Regarding the selection of the electrodes it is not possible to conclude which method, manual or automatic, achieves better results. Therefore, the criterion consists of applying the automatic faster selection in an initial stage, and depending on the results obtained, applying a second manual selection for trying to improve the results. The selection of the class 1 data for the model creation performs better when it is based on the maximum of the average signal of all events per subject instead of personalizing it by event. The results of applying different prior probabilities for the LDA classifier indicates that π = 3 is the most beneficial one in general terms for the class 0. Finally, the K value for consecutive detections should be personalized during the model creation between values of 2 and 5 for each subject, since the results vary widely depending on the subject.

Following all these considerations, the best results obtained for each subject according to all the tested variations are shown in right part of Table 9. The second column (M/A) indicates if a manual or automatic solution for electrode selection has been followed. This allows us to compare the results with previous work [17], showed in the left part of Table 9, where a polynomial feature extraction was performed. A significant improvement in the percentage of true positives can be observed, from 30.0% to 63.9%, and also a clear reduction in the FP/min (from 6.7 to 2.6). In addition, it is possible to affirm that the BMI system is capable of detecting the intention to stop of the person in a time prior to the actual stop, sufficient enough to be able to send an order to the exoskeleton in a reasonable time.

**Table 9.** Comparison between the bests results obtained with previous research.

Future works will focus on performing real-time experiments by applying the procedure in order to move to the final stage by testing this architecture with an exoskeleton. It will also be analyzed if a lower number of trials to create the model allows us to obtain similar or better results in order to decrease the needed training time. However, there is still work to do in order to increase success percentage and to reduce as much as possible the FP/min. This is the most critical parameter for its implementation with an exoskeleton.

**Author Contributions:** Conceptualization, M.E., E.I., M.O. and J.M.A.; methodology, M.E., V.Q. and E.I.; software, M.E., V.Q. and E.I.; validation, M.E., V.Q. and M.O.; formal analysis, M.E.; investigation, M.E. and V.Q.; resources, M.E., J.M.A.; Data curation, M.E., E.I. and M.O.; writing–original draft preparation, M.E. and E.I.; writing–review and editing, E.I., M.O. and J.M.A.; visualization, M.E.; supervision, E.I. and J.M.A.; project administration, J.M.A.; funding acquisition, J.M.A.

**Funding:** This research has been carried out in the framework of the project Walk—Controlling lower-limb exoskeletons by means of brain-machine interfaces to assist people with walking disabilities (RTI2018-096677-B-I00), funded by the Spanish Ministry of Science, Innovation and Universities, the Spanish State Agency of Research, and the European Union through the European Regional Development Fund.

**Acknowledgments:** We want to thank all the subjects that took part in the experiments.

**Conflicts of Interest:** The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

### **References**

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

### *Article* **AMiCUS—A Head Motion-Based Interface for Control of an Assistive Robot**

### **Nina Rudigkeit \*,† and Marion Gebhard**

Group of Sensors and Actuators, Department of Electrical Engineering and Applied Physics,

Westphalian University of Applied Sciences, 45877 Gelsenkirchen, Germany<EMAIL_ADDRESS>
**\*** Correspondence<EMAIL_ADDRESS>
† Current address: Xsens Technologies B.V., P.O. Box 559, 7500 AN Enschede, The Netherlands.

Received: 23 May 2019; Accepted: 18 June 2019; Published: 25 June 2019

**Abstract:** Within this work we present AMiCUS, a Human-Robot Interface that enables tetraplegics to control a multi-degree of freedom robot arm in real-time using solely head motion, empowering them to perform simple manipulation tasks independently. The article describes the hardware, software and signal processing of AMiCUS and presents the results of a volunteer study with 13 able-bodied subjects and 6 tetraplegics with severe head motion limitations. As part of the study, the subjects performed two different pick-and-place tasks. The usability was assessed with a questionnaire. The overall performance and the main control elements were evaluated with objective measures such as completion rate and interaction time. The results show that the mapping of head motion onto robot motion is intuitive and the given feedback is useful, enabling smooth, precise and efficient robot control and resulting in high user-acceptance. Furthermore, it could be demonstrated that the robot did not move unintendedly, giving a positive prognosis for safety requirements in the framework of a certification of a product prototype. On top of that, AMiCUS enabled every subject to control the robot arm, independent of prior experience and degree of head motion limitation, making the system available for a wide range of motion impaired users.

**Keywords:** assistive technology; human-machine interaction; motion sensors; robot control; tetraplegia; IMU; AHRS; head control; gesture recognition; real-time control

### **1. Introduction**

Tetraplegia is defined as the partial or total loss of motor and/or sensory function of the arms, legs, trunk and pelvic organs due to damage of the cervical segments of the spinal cord [1]. Besides traumatic injuries also disorders like cerebral palsy, amyotrophic lateral sclerosis or multiple sclerosis can lead to this severe disability [2]. The worldwide incidence of tetraplegia is estimated to lie between 3.5 and 27.7 per million inhabitants per year. The percentage of tetraplegics of all cases of spinal cord injuries increased within the last decades [3].

Tetraplegic patients require extensive home care services and often retire from working life because they no longer meet the physical conditions. The development of assistive robots for the people concerned is of major importance to at least partly restore their autonomy, substantially improving their quality of life.

Different uni- or multimodal Human–Machine Interface (HMI) concepts for tetraplegics have already been tested. These interfaces consider the user's remaining capabilities to voluntarily produce input signals, such as movement of eyes [4], head [5] or tongue [6], speech [7], breath [8], brain activity [9] or voluntary muscle contraction of neck [10], facial [11] or ear muscles [12]. The choice of the most suitable input modality depends on the preferences and physical abilities of each user as well as on the underlying control scheme. However, the goal of the research presented here was developing a solely head motion-based interface for real-time control of a robotic arm.

Common sensing modalities for head motion are ultrasound modules [13], ordinary cameras [14], infrared cameras, chin joysticks [15] and motion sensors [16]. In recent years, state-of-the-art motion sensors, such as Attitude Heading Reference Systems (AHRS) based on Micro Electro-Mechanical Systems (MEMS) technology, have gained increasing interest because they enable accurate motion measurement while being small, low-cost, lightweight, energy-efficient and self-contained, making them ideal for use in HMIs. For this reason, a MEMS AHRS has been considered the preferred choice to measure head motion within this work.

The majority of existing motion sensor-based head-controlled interfaces are limited to 2D-applications, such as control of mouse cursors [16–20], wheelchairs [17,21] or other vehicles [5]. Few attention has been paid to more complex applications, such as robot arm control. However, Williams et al. [22] use head orientation to control the Tool Center Point (TCP) of a robot arm. Furthermore, Fall et al. [23] use motion of the shoulders and neck to control the commercially available robot arm JACO [24]. The user can choose between different control modes of JACO to perform 3D-translations, arm and wrist rotations or control the fingers' positions, respectively. The Human–Robot Interface (HRI) presented by Fall et al. requires additional switches to switch between these modes, though.

In our research group, we have developed the AMiCUS system, which is the first interface that uses only head motion to produce all the necessary signals and commands for real-time control of an application with more than three Degrees of Freedom (DOF), such as a robotic arm. Some of the major criteria for the development of such an HRI are the following:

AMiCUS has been designed with special attention to these requirements. A tetraplegic with severe head motion limitation has been involved in the whole development cycle to ensure the system's relevance for the target group.

In the next section, the hardware, software and signal processing of the resulting system are described. Afterwards, the experimental setup of a user study with 13 able-bodied and 6 tetraplegic subjects to validate the system is presented. Subsequently, the results are presented and discussed. In the last section, these results are compared against aforementioned criteria for a head motion-based HRI.

### **2. AMiCUS**

Within this section we introduce AMiCUS. AMiCUS stands for **A**daptive Head **M**ot**i**on **C**ontrol for **U**ser-friendly **S**upport. The demonstrator AMiCUS is the result of several years of research within our research group. First, we will give an overview of our research activities. Then, we describe the sensor placement. This is followed by a description of the used hardware. Next, we present how robot groups have been built in order to control all DOFs of the robot with head motion. Afterwards, the two modes of AMiCUS, namely Robot Control Mode and Cursor Control Mode, are described in detail. The section closes with the presentation of the necessary calibration routines.

### *2.1. Relation to Previously Published Work*

In Reference [25] suitable control modes for real-time control using head motion have been analyzed. Based on the results, a first iteration of the general control structure has been evaluated in References [26,27]. An algorithm to detect head gestures, which can be used as switching commands as part of the chosen control structure has been presented in Reference [28]. Within this work, the whole system that resulted from all previous research is presented and validated in a user study. A usability study of an alternative control structure and GUI for the AMiCUS system has been presented by Jackowski et al. [29]. The system in this publication uses all four head gestures defined in Reference [28] to switch between robot groups, whereas the interface described here uses a control structure based on the work published in References [26,27]. The main advantage of the version presented by Jackowski et al. is that switching between groups is faster, while the main advantage of the version presented here is that it can be used by a wider range of users as explained in detail later in this work.

### *2.2. Sensor Placement*

A MEMS AHRS was chosen to measure the user's head motion. A typical AHRS outputs raw sensor data from a 3D accelerometer, a 3D gyroscope and a 3D magnetometer, as well as sensor orientation obtained from such raw data.

In Reference [30] we showed that a rigid body placed onto a human head is moved on a spherical surface during head motion (Figure 1). For the sake of simplicity, the sensor placement was chosen in a way that the sensor yaw axis and the approximated yaw axis of the user's cervical spine coincided. Given this sensor placement, changes in head and sensor orientation are identical and a transformation of sensor orientation to head orientation is not needed. Nonetheless, an offset calibration remains necessary to define the zero position of the user's head. Every head motion apart from rotation around the yaw-axis results in additional linear sensor movement as the sensor is not rotated around its own center.

**Figure 1.** Kinematics of the cervical spine. From the kinematic point of view, the human cervical spine can be approximated by a ball joint. That means, every motion can be divided into single rotations around three orthogonal axes that intersect in one point. This point, that is, the center of rotation, roughly coincides with the location of the thyroid gland.  Hardware*

AMiCUS detects head motion using a Hillcrest FSM-9 motion sensor system [31], which is placed as described in the previous section. Output data of the sensor system under static conditions and during head motion are provided in the Supplementary Materials. The acquired sensor data is processed on a desktop computer with the AMiCUS software, which is written in C++ using the Qt framework [32]. The resulting control signals are transmitted to a Universal Robots UR5 robot arm [33] which is equipped with a Robotiq 2-Finger parallel gripper [34]. By default, the position of the gripper can be controlled in world coordinates **p***w* = (*x*, *y*, *z*) *w T* as well as gripper coordinates **p***g* = (*x*, *y*, *z*) *g T* using inverse kinematics. These coordinate systems are shown in Figure 2. Rotations of the gripper are performed in gripper coordinates *α g* = (*ϕ*, *ϑ*, *ψ*) *g T*. The gripper can be opened and closed in order to interact with objects. The position of the gripper's fingers is denoted as *β*. All these DOFs, **s** *r* = **p** *w*/*g* , *α g* , *β T* , can be controlled proportionally. That means that the signal amplitude can be varied continuously. A Logitech C930e webcam [35] is mounted on the gripper to provide feedback during positioning and gripping. The camera image is part of the Graphical User Interface (GUI) of AMiCUS. The GUI is displayed on an ordinary computer screen in front of the user.

**Figure 2.** Coordinate systems of the AMiCUS system. Degrees of Freedom (DOFs) of the same color are controlled by the same head DOF. The zero orientation of the head coordinate system depends on whether the cursor or the robot is controlled. During robot control, the head coordinate system is denoted by *αhr* = (*ϕ*, *ϑ*, *ψ*) *hr T* and *α hc* = (*ϕ*, *ϑ*, *ψ*) *hc T* during cursor control.

### *2.4. Robot Groups*

Three proportional control signals are provided through head motion, that is, *α h* = (*ϕ*, *ϑ*, *ψ*) *h T*. With these signals, seven DOFs of the robot, **s** *r* , have to be controlled. That means, direct robot control in terms of a 1:1-mapping is not feasible. For this reason, groups containing maximum three DOFs of the robot have been defined, namely Gripper, Vertical Plane, Horizontal Plane and Orientation. The DOFs of the head have been mapped onto the DOFs of the robot as follows (Figure 3):

**Figure 3.** Mapping of head DOFs onto robot DOFs. Four different groups, that is, Gripper, Orientation, Vertical Plane and Horizontal Plane, are depicted. The user is able to switch between groups in order to control all DOFs of the robot.

In the Gripper group, the gripper can be opened and closed using the pitch motion of the user's head ( *ϑh* → *β*). The group Vertical Plane enables the user to move the gripper within a plane that is perpendicular to the user's line of sight. The user can move the gripper to the left or right by turning the head to the left or right ( *ψh* → *xw* ). In order to move the gripper up or down, one has to tilt the head up or down ( *ϑh* → *yw* ). The gripper can be moved back or forth in the Horizontal Plane group by tilting the head down or up ( *ϑh* → *z w* ). Additionally, the user can move the robot left and right the same way as in the Vertical Plane group.

The orientation of the gripper is controlled in the Orientation group. Each rotary movement of the head is mapped onto the corresponding rotation of the gripper. That means, the roll rotation of the gripper is controlled by the roll rotation of the head ( *ϕh* → *ϕg* ); the control of the pitch rotation of the gripper is performed using the pitch rotation of the head ( *ϑh* → *ϑg* ); and the yaw rotation of the gripper is controlled by the yaw rotation of the head ( *ψh* → *ψg* ).

### *2.5. Robot Control Mode*

During direct robot control within one of the robot groups, the position along a head DOF is projected onto the velocity along a robot DOF (*γ* → *g*˙), with *γ* ∈ *α h* and *g* ∈ **s** *r* [25].

A GOMPERTZ-function is used as the transfer function (Figure 4). Using this function, head motion close to zero position, which is likely to be unintended, is not translated into robot motion (Deadzone). If the user keeps increasing head deflection, the robot slowly starts to move into the desired direction. Small deflection angles of the head enable slow, precise robot control. An increase of head deflection leads to an exceedingly smooth transition to fast, imprecise robot control. Mathmatically, the implemented transfer function can be formulated as:

$$\begin{array}{rcl} \dot{\jmath} &=& \begin{cases} A\_{\text{max}} \cdot e^{\delta \cdot \varepsilon^{r \cdot \gamma}} & \text{if } \gamma > 0 \\ -A\_{\text{max}} \cdot e^{\delta \cdot \varepsilon^{-r \cdot \gamma\_{\text{n}}}} & \text{else} \end{cases} \end{array} \tag{1a}$$

$$
\gamma\_n = \frac{\gamma}{\gamma\_{\text{th}}} \tag{1b}
$$

**Figure 4.** Robot control transfer function. A GOMPERTZ-function is used as transfer function. The used parameters are *A*max = 1, *δ* = −30 and *r* = −10. The space between the dashed lines indicates the deadzone.

The parameter *A*max ∈ R+ indicates the upper asymptote. This corresponds to maximum velocity in *g*-direction. The coefficient *δ* ∈ R− sets the displacements along the *γ*-axis and *r* ∈ R− indicates the growth rate. The parameters *δ* and *r* are identical for all the robot DOFs. The amplitude *A*max is adjusted separately for the linear DOFs **p***w* , the rotational DOFs *α g* and the gripper *β*. The parameter *γ*th defines the range of motion along the *γ*-axis, which shall be used for control. This range as well as the zero position are obtained during a calibration routine as described later.

For every time step of length Δ*t*, the new robot state **s** *r* new is computed using the following relationship:

$$\mathbf{^r\mathbf{s}}\_{\text{new}} = \mathbf{^r\mathbf{s}}\_{\text{old}} + \mathbf{^r\dot{\mathbf{s}}} \cdot \Delta t \tag{2}$$

If the linear acceleration of the sensor during control exceeds a certain threshold, robot control is deactivated. This is supposed to guarantee operational safety as quick movements are likely to not be intended for robot control. After deactiviation, the user has to turn his head to the zero position in order to continue control. This event is accompanied by visual and acoustic feedback.

The normalized head angles *γn* are displayed in real-time on the GUI during Robot Control Mode. Additionally, a picture of the current robot group and the image of the gripper camera are shown (Figure 5).

During Robot Control Mode, all DOFs of the chosen group can be controlled simultaneously. However, switching is necessary to control the DOFs of another group. The user can leave the current group by performing a Head Gesture, which is described in the following.

**Figure 5.** Graphical User Interface during Robot Control Mode. The GUI displays an icon of the current robot group (top right), the image of the gripper camera (bottom right), an information line (bottom) and feedback about the current head angle given by the arrow (left). The square represents the deadzone in which the robot arm cannot be moved.

### Head Gesture

The Head Gesture denotes a nodding-like movement (Figure 6). More precisely, the user has to quickly move the head down starting from zero position and back. Mathematically, the shape of the gesture can be described by a Gaussian function:

**Figure 6.** Head Gesture. The gesture is displayed with its *ϑ hr* -angles over time *t*. Parameters: *d*max = amplitude, *w* = peak width, *tc* = location of the peak center.

$$d = d\_{\max} \cdot e^{-\left(\frac{t - t\_c}{w}\right)^2} \tag{3}$$

Maximum head displacement is expressed by the amplitude *dmax*, *tc* is the centroid and *w* is related to the peak width. The implemented algorithm for robust Head Gesture recognition in a real-time data stream has been presented in Reference [28]. Briefly summarized, the algorithm assumes that activity is present when the magnitude of linear acceleration of the sensor exceeds a certain threshold. If activity is present, both head angles *αhr* and sensor data are recorded. Recording stops when activity is no longer present. If the length of the recorded data lies between 0.25 s and 2 s and the dominating initial linear acceleration has been in positive *xh* -direction (Figure 2), the recorded data might contain a Head Gesture. For validation, a Gaussian function is fitted to the recorded *ϑ hr* -angles. The Head Gesture is classified if the following conditions are fulfilled:

In case the Head Gesture has not been performed correctly, visual feedback is shown on the GUI that informs the user how to adjust a movement for successful Head Gesture detection. In case of correct gesture execution, the system switches from Robot Control Mode to Cursor Control Mode.

### *2.6. Cursor Control Mode*

During Cursor Control Mode, the user controls a mouse cursor on the GUI using head motion to select a different robot group or to perform any other action, such as starting a calibration routine or pausing control.

To control the mouse cursor, the user's head orientation is directly mapped onto the position of the cursor [25]. That means, the pitch-DOF is mapped linearly onto the *y c* -axis of the screen ( *ϑh* → *y c* ), while the yaw-DOF is mapped onto the *x c* -axis of the screen ( *ψh* → *x c* ). This relationship is described by:

$$\mathbf{s}^c \mathbf{s} = \begin{pmatrix} \mathbf{c}\_x \\ \mathbf{c}\_y \end{pmatrix} = \text{diag}\left( {}^h\_{\mathbf{c}} \mathbf{m} \right) \cdot \begin{pmatrix} {}^h\_{\theta} \Psi \\ {}^h\_{\theta} \theta \end{pmatrix} + {}^c \mathbf{b} \tag{4}$$

The parameters **m***hc c* = *mx*, *my hc T c* reflect the sensitivity. The parameters **b** *c* = *bx*, *by c T* indicate the cursor coordinates when the head is in its zero position, (*ϑ*, *ψ*) *hc T* 0 . The corresponding values are obtained during a later described calibration routine.

By moving the mouse cursor, the user interacts with the GUI that is shown during Cursor Control Mode (Cursor GUI, Figure 7). This GUI contains dwell buttons, for example, to start the calibration routines, pause control, switch between coordinate systems or close the program. A dwell button is activated by dwelling on it for 2 s. In addition to the dwell buttons, the GUI contains a Slide Button for each robot group. A robot group is selected by activating the corresponding Slide Button.

**Figure 7.** Graphical User Interface during Cursor Control Mode. The GUI contains one Slide Button for each robot group. The dwell buttons in the top toolbar allow the user to perform several actions, such as pausing control, starting calibration routines or exiting the program.

### Slide Button

For successful activation of the Slide Button, the following steps are necessary (Figure 8): First, the user has to hover the Slide Button with the mouse cursor and dwell there for a certain time (State 1). Then, a rail rolls out in order to inform the user that the button can be slid to the right as indicated by the rail (State 2). If the cursor is moved to the right in a straight line, the Slide Button moves with it. If the user leaves the Slide Button area by moving the cursor up, down or left, the action is immediately aborted. After sliding the button to the right end of the rail (State 3), the user has to move the Slide

Button back to the left. The action is aborted when leaving the Slide Button area to the top, bottom or right. If the Slide Button is moved to the left along the rail correctly, the rail rolls up and the robot group associated with the Slide Button is entered. Each state of the Slide Button is accompanied by visual and acoustic feedback.

**Figure 8.** Slide Button. The following steps are necessary for successful activation: When the button is in its neutral state (*S*0) the mouse cursor has to dwell in the button (*S*1) until visual and acoustic feedback occurs. Then, the button has to be moved to the right along the rail (*S*2). At the end of the rail visual and acoustic feedback is given. Next, the button has to be moved to the left along the rail (*S*3). When the button reaches the initial position (*S*4), it is activated and the assigned action is performed.

### *2.7. Calibration Routines*

### 2.7.1. Robot Calibration

The calibration routine, which is needed to control the robot arm during Robot Control Mode, is called Robot Calibration. At the beginning of the Robot Calibration, the user has to define the zero position of the head coordinate system, *αhr* 0. The zero position should be chosen in a way that the user faces the robot arm. This position has to be held for 2 s within a tolerance of 2◦. Afterwards, the corresponding Euler angles of the sensor are saved as offset *α s* 0,*r* = (*ϕ*, *ϑ*, *ψ*) *s T* 0,*r*. The offset is subtracted from the sensor angles *α s* in order to obtain head angles *αhr* .

After offset definition, the user has to perform each one repetition of neck flexion and extension, neck lateral bending to the left and to the right and neck rotation to the left and to the right. Pictures of these positions are displayed on the GUI. The users are instructed to move their heads as far as they can while still being able to see the robot. After holding a certain position, the corresponding head angle of the relevant DOF is saved and the next position is displayed. In this way two values are obtained for each DOF: One in positive direction (*γ*+) and one in negative one (*γ*−). To guarantee a symmetrical mapping, only the smaller value is used to define the range of motion *γ*th.

Whenever a calibration point is saved, acoustic feedback is given to the user. Furthermore, a status bar indicates how long the user has to hold the head stable until a calibration point is saved.

### 2.7.2. Cursor Calibration

The calibration that is necessary to control the mouse cursor during Cursor Control Mode is denoted as Cursor Calibration. During Cursor Calibration, the user has to turn the head towards five targets, which are shown on the screen one after the other. A calibration point is saved when the user holds the head stable for 2 s within a tolerance of 2◦. The first target is shown in the center of the screen. The user's head orientation when facing this point is defined as the zero position of the head during cursor control, (*ϑ*, *ψ*) *hc T* 0 . The corresponding sensor angles (*ϑ*, *ψ*) *s T*0,*c* are defined as the offset between the head and the sensor coordinate system. For all samples, the offset is subtracted from the sensor angles (*ϑ*, *ψ*) *s T* to obtain the head coordinates (*ϑ*, *ψ*) *hc T* explicitly. This step ensures that Gimbal Lock does not occur. For every target, the head angles (*ϑ*, *ψ*) *hc T* and the position of the target on the screen

**s** *c* are saved. The remaining targets are displayed in the upper left corner, center, lower right corner and in the center again. Whenever a calibration point is saved, acoustic feedback is given. At the end of the calibration procedure, the parameters **m***hc c* and **b** *c* are computed from the five calibration points using a minimum least squares fit. Head angles corresponding to the center of the screen are acquired three times to obtain more reliable data for this point. This is important, because the correct calibration of the center point has a major impact on the subjective calibration success perceived by the user.

### **3. Materials and Methods**

The aim of the study was providing a proof-of-concept of the AMiCUS system. That means, showing that AMiCUS enabled people to perform simple manipulation tasks with the robot arm efficiently. Furthermore, we wanted to investigate if all aforementioned criteria for a head motion-based HRI were met and at which points AMiCUS could be developed further.

### *3.1. Subjects*

Thirteen able-bodied subjects and six teraplegics took part in the experiments. The able-bodied subjects were recruited via announcements on the university website and in the local newspapers. Six of them were male, seven female. Their mean age was 37.0 ± 15.0 years. The able-bodied subjects had no known neck motion limitations and carried out the experiments at the Westphalian University of Applied Sciences in Gelsenkirchen. The able-bodied subjects represented users with full Range of Motion (full ROM).

The tetraplegics were recruited via the BG-Hospital Hamburg where they also carried out the experiments. Five of them were male, one female. Their mean age was 35.7 ± 15.2 years. The levels of injury ranged from C0 to C4. Subjects with both complete and incomplete injuries were included. The tetraplegics were chosen to have severe neck motion limitations in order to represent users with restricted Range of Motion (restricted ROM). It is worth noting that all of these tetraplegics were unable to operate the system presented in Reference [29] due to their neck motion limitations.

None of the subjects had prior experience with AMiCUS. The study was approved by the ethics committee of the University of Bremen and the subjects gave their informed consent prior to participation.

### *3.2. Experimental Setup*

The subjects were seated in front of a table with an arrangement of platforms and softcubes according to the particular task to be performed. The robot arm was positioned on the opposite side of the table, facing the current subject.

The GUI was displayed on a 27 " screen, which was mounted behind the robot arm. The horizontal distance of the screen from the subject was approximately 2.4 m. The screen was positioned in a way that it was not occluded by the robot. Small deviations between the experimental setups for the able-bodied and tetraplegic subjects could not be avoided due to different spatial conditions.

The used sensor settings are shown in Table 1. The sampling rate for the raw and fused sensor data was set to 125 Hz. This is the recommended minimum sample rate for the sensor fusion algorithm to work properly. The accuracy of the orientation output was specified as 1.5◦ by the manufacturer. The sensor data was downsampled to 60 Hz for further processing to save computing power while still displaying head orientation smoothly on the screen. The calculation of the joint angles using inverse kinematics and the physical robot movement needed almost 0.04 s. For this reason, the 60 Hz control signals to update the robot's joint angles were further downsampled to 25 Hz.

Both the linear acceleration and angular position output were used for the gesture recognition. The data was processed as described in Reference [28]. The control signals for the cursor and robot arm were generated based on the angular position output as described in Section 2.

**Table 1.** Settings of the Hillcrest FSM-9.

### *3.3. Procedure*

A trial session, a predefined task and a complex task were part of the experimental study. The procedure was identical to the usability study performed with the alternative AMiCUS version as described in Reference [29]. This was done to allow for a better comparison of both versions. However, such a comparison is beyond the scope of the work presented here.

### 3.3.1. Trial Session

Prior to the trial session, the subjects were shown an introduction video that demonstrated the basic working principle and modes of operation of AMiCUS. Video instructions were chosen to make sure every subject received the same information. After the video, the subjects were free to try the system for 10 min. They were encouraged to enter each robot group at least once.

### 3.3.2. Predefined Task

After completing the trial session, a video of the predefined task was shown. For the tasks, five square-shaped platforms with 9.5 cm edge length and three softcubes with 6.5 cm edge length were arranged as shown in Figure 9.

**Figure 9.** Experimental setup of the predefined task. The subjects were clearly instructed how to move the robot. Movements 1–3 had to be performed in the Vertical Plane group, movements 4–6 in the Horizontal Plane group. After movement 6, the subjects had to perform one 90◦-rotation around each rotation axis.

First, cube 1 had to be moved from the blue to the green platform (1). After placing the cube on the green platform, the gripper had to be moved to the top of the tower ( 2) to grip cube 2 and move it to the blue platform (3). Afterwards, the gripper had to be moved to the red platform (4) and cube 3 had to be transported to the yellow platform (5). Finally, the gripper had to be moved to the blue platform (6). After arriving at the blue platform, cube 1 had to be gripped, lifted a little bit and rotated around 90◦ in positive *ϑg* -direction, then around 90◦ in negative *ψg* -direction and finally around 90◦ in positive *ϕg* -direction.

In order to solve this task, all subjects received detailed instructions about the desired control steps to be performed. Movements 1–3 had to be performed in the Vertical Plane group, movements 4–6 in the Horizontal Plane group and the rotations in the Orientation group.

### 3.3.3. Complex Task

During the complex task, 3 softcubes were placed on a table as shown in Figure 10a. The users were told to stack the cubes on top of each other in a way that softcube 1 was on the formerly empty platform, softcube 2 in the middle and softcube 3 on top of the stack. In addition, a picture of the solution has been shown to the subjects prior to the task (Figure 10b). The subjects had to find their own control strategy to solve the task.

(**a**) Initial setup (**b**) Target setup

**Figure 10.** Experimental setup of the complex task. The users had to find their own control strategy to solve the task.

### *3.4. Evaluation Criteria*

### 3.4.1. Objective Evaluation

The completion rates were obtained for both the predefined and complex task. Subjects had only one trial to complete the tasks. A task was counted as failed when manual assistance from the experimenters was needed to proceed, when the subjects used their hands or when the task was executed incorrectly. Completion time was considered unsuitable as an evaluation criterion because of too many disturbing influences, for example, due to talking.

For both the Slide Button and the Head Gesture success rate and activation time were chosen as evaluation criteria. Success rate was evaluated using data of both the predefined and the complex task, whereas activation time contained only data from the predefined task. Activation time was defined as the time from the spoken command of the experimenter until the change of mode was visible on the GUI. That means that failed attempts were also included in the activation time. As a result, activation time and success rate are correlated. The medians of the activation times and success rates were first

computed subject-wise. Based on these data, medians and quartiles were obtained for all subjects together. Differences between Slide Button and Head Gesture were evaluated utilizing paired-sample t-tests [36]. An overview of statistical tests can be found in Reference [37].

### 3.4.2. Subjective Evaluation

For the subjective evaluation of the system, all subjects completed an evaluation sheet. This sheet contained 30 statements, regarding calibration, GUI, switching, mapping, transfer function and general aspects, which could be rated between 1 (*"I strongly disagree"*) and 5 (*"I strongly agree"*) (Table 2). Three additional statements regarding the speed of operation could be rated between 1 (*"Far too slow"*) and 5 (*"Far too fast"*). The answers were compared with each other using Friedman's tests [38]. The significance level for the tests was 5 %.

### **4. Results and Discussion**

### *4.1. Objective Evaluation*

### 4.1.1. Completion Rates

All subjects, independent of available range of motion, were able to get a general understanding of the control structure and could move the robot arm in a controlled manner, as demonstrated in the trial session. It is worth mentioning that all subjects were first-time users of AMiCUS and did not have prior experience with any other head-controlled system.

The Fisher's exact test [39] indicated that there was no significant difference between the users with full and restricted ROM regarding task completion rates. Therefore, the completion rates are presented for all subjects together. All subjects were able to complete the predefined task, resulting in a completion rate of 100 %. The overall completion rate of the complex task was 72.2 % (Figure 11). In 16.7 % of the cases, manual assistance was required to solve the task at all or within a reasonable time period. In each 5.6 % of the cases, hand-use or stack-collapse led to task failure. It is notable, that 60 % of the non-completions resulted from inaccurate gripper positioning in *z w* -direction.

**Figure 11.** Completion rate of the complex task. There was no statistical difference between the users with full and restricted ROM. The overall completion rate of the complex task was 72.2 % (n = 18).

### 4.1.2. Success Rates and Activation Times

The success rate of the Slide Button was 85.80 % for the subjects with full ROM (Figure 12a). For the subjects with restricted ROM it was 82.52 %. When the first attempt failed, the subjects with full ROM activated the Slide Button successfully after the second attempt in 12.60 % of the cases. The subjects with restricted ROM succeeded after the second attempt in 14.23 %. In 1.32 %, the subjects with full ROM needed three attempts to successfully activate the Slide Button. The subjects with

restricted ROM needed three attempts in 3.25 % of the cases. In the remaining 0.28 %, more than three attempts were needed by the subjects with full ROM. The subjects with restricted ROM did not need more than three attempts.

**Figure 12.** Comparison between Slide Button and Head Gesture. The performance of the Slide Button and the Head Gesture was evaluated in terms of success rate and activation time.

In Figure 12b the activation times for the Slide Button and for the Head Gesture are shown based on the median activation times for each subject. For the Slide Button the median activation time was 5.6 s for the subjects with full ROM and 5.0 s for the subjects with restricted ROM. The two-sample t-test [36] indicated no significant difference between both groups. However, for the subjects with full ROM the median activation time of the Head Gesture was 2.0 s. This is almost three times faster than the Slide Button activation, even though all failed activation attempts were included in the activation time. This difference is significant. For the subjects with restricted ROM the median activation time of the Head Gesture was 4.6 s. That means, the subjects with restricted ROM needed significantly more time to perform the Head Gesture correctly than the subjects with full ROM (two-sample t-test). Futhermore, according to a paired-sample t-test, the activation times of the Head Gesture and the Slide Button did not differ significantly for the subjects with restricted ROM.

It is important to note that the execution time for a single Head Gesture had to lie between 0.25 s and 2 s. Movements beyond these bounds were not taken into account by the classification algorithm. That means that a certain execution speed was enforced for the Head Gesture while the Slide Button could be moved as slowly as desired. The fact that the Head Gesture had to be performed both quickly and accurately led to a significant lower success rate of the Head Gesture compared to the Slide Button for both groups (paired-sample t-tests): The success rate of the Head Gesture for the subjects with full ROM was 58.45 % and 41.45 % for the subjects with restricted ROM. In 23.68 % of all cases, the second attempt was successful for the subjects with full ROM. The subjects with restricted ROM succeeded after the second attempt in 26.27 % of all cases. Three attempts were needed by the subjects with full ROM in 9.51 % of all cases. The subjects with restricted ROM performed the Head Gesture correctly after the third attempt in 11.69 % of the cases. In 8.36 % of the cases more than three attempts were needed by the subjects with full ROM, while the subjects with restricted ROM needed more than three attempts in 20.59 % of the cases. It is notable that the maximum number of observed failed attempts was 16 for the Head Gesture while it was 4 for the Slide Button. Though the non-detection rate of the gesture was relatively high, the robot did not move unintendedly, which is very important from a safety point of view.

In conclusion, for both groups the success rate of the Head Gesture was significantly lower than the one of the Slide Button. On the other hand, the activation time of the Head Gesture was significantly lower for the subjects with full ROM. However, this difference was not present for the subjects with restricted ROM: The results indicate that the correct execution of the Head Gesture becomes more difficult when the ROM is restricted, while based on our results, the level of difficulty of the Slide Button is not affected significantly by the available ROM of the user.

### *4.2. Subjective Evaluation*

The Mann-Whitney U-test [40] indicated no significant differences between the subjects with full and restricted ROM. Hence, the ratings from both groups have been assumed to originate from the same distribution and are therefore presented together.

**Table 2.** Evaluation sheet for AMiCUS including results.

Rating: 1 = "I strongly disagree", 2 = "I disagree", 3 = "I partly agree", 4 = "I agree", 5 = "I strongly agree".

### 4.2.1. Calibration

As Table 2 shows, the majority of subjects found both the Cursor as well as the Robot Calibration highly intuitive. Furthermore, the acoustic feedback to indicate that a calibration point has been saved was perceived as very useful.

### 4.2.2. GUI

The subjects considered both the Cursor GUI and the Robot GUI visually appealing and clearly structured. They agreed that all important functions were easily accessible during cursor control. Moreover, the majority of subjects had the opinion that all important information was displayed on the GUI during robot control. A few subjects remarked that more information could be helpful. The feedback showing the current head orientation was experienced as very useful. The camera image was mainly considered useful, even though a few subjects remarked that they had not paid much attention to the camera image during control. This is likely, because it was not mandatory for the given tasks. However, there are scenarios in which the gripper camera is necessary because direct view is obstructed.

### 4.2.3. Switching

The Slide Button activation was rated very easy. This is also in line with the high success rate of the Slide Button. The vast majority of the subjects perceived the acoustic and visual feedback of the Slide Button as useful or very useful but some subjects found the acoustic feedback disturbing after a while. The Head Gesture has been found to be significantly more difficult than the Slide Button. This is also represented by the lower success rate of the Head Gesture. Some subjects remarked that they perceived the Head Gesture as a stress-inducing factor. Observations during the experiments indicate that the gesture execution was demanding for some subjects with restricted ROM. The entire switching procedure was mainly rated as easy. Most subjects rated switching between groups as too slow. The common opinion was that the Head Gesture and also the Slide Button alone were quick enough but the combination was not. All in all, the Head Gesture was not considered easy to execute and the associated feedback was not considered helpful.

### 4.2.4. Mapping

In order to evaluate the intuitiveness of the mapping, the subjects were asked how well they could imagine how the robot would respond to their head motion. The results indicate that the subjects found it highly intuitive to open or close the gripper using pitch motion. Furthermore, they strongly agreed that they could well imagine that the robot moved up or down in the Vertical Plane group when they tilted their heads up or down. The majority of the subjects could also imagine well how the robot would move if they rotated their heads left or right. Most subjects agreed that they could imagine well how the robot would move in the Horizontal Plane group when they tilted their heads up or down. However, the vast majority remarked that they would have found it more intuitive if the direction of the robot motion had been switched, that is, that the robot moved closer to the subjects when they tilted their heads down and vice versa. This is also in line with the observation that many subjects initially moved the robot in the wrong direction during movements 4 and 6 of the predefined task. The intuitiveness of the mapping for the DOFs of the Orientation group was significantly lower than for the DOFs of the Vertical Plane group or the Gripper. The subjects criticized that they found it hard to imagine rotations in local coordinates of the gripper. But they agreed that it became a lot easier after they had been instructed to imagine the gripper to have a face with the camera as the eyes and the gripper as the mouth. A possible explanation is that the gripper movements may then be processed in different parts of the brain, such as the Fusiform Face Area. Once the gripper is interpreted as a face, mirror neurons may map its movements onto the subjects' own head movements.

### 4.2.5. Transfer Function

The majority of subjects strongly agreed that it was very easy to move the mouse cursor precisely under the testing conditions. This is in line with the high success rate of the Slide Button. Furthermore, the subjects found precise positioning of the mouse cursor significantly easier than gripping or moving the robot arm precisely. A few subjects annotated that they found the mouse cursor too fast. This problem can be solved by decreasing the distance between subject and screen or by choosing a larger screen. Gripping accurately was mainly considered as easy. Some subjects criticized that the gripper could not be opened and closed smoothly. This problem was caused by the gripper and can only be solved by changing the gripper hardware. On average, subjects agreed that accurate positioning of the robot arm was easy. The subjects who partly agreed to that statement mainly argued that they sometimes lacked feedback for accurate positioning. Nonetheless, all the subjects were able to complete the tasks without colliding. The velocities for opening/closing the gripper, linear motion and rotational motion have been assessed suitable by almost all subjects (Figure 13). More experienced users might find robot control too slow. However, the AMiCUS system allows the adjustment of maximum robot velocity.

**Figure 13.** Control speed evaluation. Mean and standard deviation of subjective velocities during gripping, linear motion and rotations (n = 18).

### 4.2.6. General

The majority of subjects found it easy or very easy to assess robot position correctly in both the Vertical Plane and Horizontal Plane group. However, some subjects asked for additional feedback for accurate positioning. Assessing robot orientiation correctly was experienced as significantly more difficult than assessing robot position. Most subjects had no problems with attention loss when looking back and forth between screen, robot and test course. In general, the system has been greatly accepted by the subjects: All of them found that robot control with AMiCUS was fun.

### **5. Conclusions**

Within this work, we presented the AMiCUS system and evaluated it in a user study with 13 able-bodied and 6 tetraplegic subjects. In this section, we validate that the requirements are met and we compare the system presented here with previously published work.

### *5.1. Compliance with Requirements for a Head Motion-Based HRI*

In the following, we step through the list of criteria for a solely head motion-based HRI in order to validate that all requirements are met:

1. The HRI should be adaptive, always using the full available neck range of motion of the user.

During the calibration routine, AMiCUS is adapted to the available neck range of motion of the user. Possible constraints are only imposed by required head gestures. For our experiments, 6 tetraplegics have been recruited that were unable to operate the AMiCUS version in Reference [29] due to severe head motion limitations that restricted their ability to perform head gestures. All of these subjects were able to successfully operate the robot arm with the version presented here. This makes our version available for a wider range of users. Furthermore, the control structure here can easily be adapted to users that can only use one or two head DOFs by introducing more robot groups with fewer DOFs in the mapping.

2. The relationship between performed head motion and resulting robot motion has to be intuitive.

In general, the relationship between performed head motion and resulting robot motion is intuitive, as all first-time users were able to operate the robot arm. However, further improvements are still possible: The direction of the mapping onto the *z w* -axis should be inverted in the future. That means, the robot should move closer to the user when turning the head down and further away when turning the head up. Moreover, performing rotations was generally considered as challenging. This resulted mainly from the fact that subjects found it difficult to imagine rotations in gripper coordinates. However, after they were instructed to imagine the gripper as a face, they perceived rotations as easier. In order to facilitate the interpretation of the gripper as a head, the design of the gripper including the camera may be adapted in a way that the gripper resembles a face more. Further attempts to increase intuitiveness may also focus on performing rotations in world coordinates instead of gripper coordinates. Additionally, future versions of AMiCUS may include an option to disable simultaneous rotations because the vast majority of subjects was confused by coupled rotations. However, observations in our research group strongly indicate that rotations in local coordinates as well as simultaneous rotations are not an issue for more experienced users.

3. The HRI must reliably distinguish between unintended head motion, head motion intended for direct control and head motion to generate switching commands.

It could be demonstrated that the robot did not move unintendedly. That means, AMiCUS could reliably distinguish between head motion intended for robot control and head gestures or unintended head motion. Furthermore, unintended head motion has never been classified as a switching command.  However, in some cases head motion intended to perform a Head Gesture or Slide Button activation has not been detected because the head movements have not been performed as defined. Given these inevitable imperfections of the user to perform head movements in an accurate and repeatable manner, the control elements and switching process have been analyzed to identify possibilities for improvement:

The Slide Button turned out to be a good control element with both high success and satisfaction rate. In contrast, the success rate of the Head Gesture was significantly lower, resulting in lower user acceptance. Furthermore, motion limitations negatively affect a user's ability to perform the Head Gesture correctly. The entire switching process was perceived as easy but slow, leaving room for improvement. As the Head Gesture has been perceived as difficult and has been in particular challenging for user's with restricted ROM, it should be discarded or improved in the future.

4. The HRI has to give sufficient and useful feedback to the user to allow safe and efficient operation.

The results of the study indicate that the calibration procedures are easy to understand and that sufficient and useful feedback is provided. The GUI was generally experienced as visually appealing and clearly structured. Moreover, the subjects agreed that all important functions could be accessed easily and all important information was displayed. A possibility to turn the sound off would be appreciated by most long-term users, though. In some cases, assessing the gripper position along the

*z w* -axis was perceived as difficult. However, our experience indicates that users will learn to interpret the gripper camera image better after working with AMiCUS for a longer period of time. In conlusion, AMiCUS gives overall sufficient and useful feedback to the user to allow safe and efficient operation.

5. The HRI must enable the user to perform smooth, precise and efficient robot movements in Cartesian space.

Overall, the transfer function was well-adapted to most users during linear motion, rotation and gripping as it enabled accurate positioning while providing a suitable speed of operation. Therefore, AMiCUS enables the user to perform smooth, precise and efficient robot movements. In case the default settings are not suitable, the parameters of the transfer function can be adapted easily to meet the preferences of the user, in particular regarding operation speed.

6. The user should enjoy using the HRI.

User acceptance is often a critical point for assistive devices, jeopardizing their success as a product. AMiCUS, in contrast, received high user acceptance since the subjects of this study considered robot control with AMiCUS as fun.

In summary, AMiCUS fulfills all the requirements of a head-controlled HRI for robot arm control and has been greatly accepted by the subjects.

### *5.2. Comparison with Previously Published Work*

When using the interface presented here, the user has to perform a Head Gesture and then activate a Slide Button in order to switch between robot groups. Even though the switching process can still be improved, switching will inherently take longer than for the interface presented by Jackowski et al. [29], which uses each one out of four different gestures to switch between robot groups. This direct group access makes switching very quick. However, the correspondence between gestures and robot groups depends on the current robot group, which is a general source of confusion and sometimes makes users select the wrong group. Furthermore, performing head gestures is complex and involves many muscles, some of which might not be available for tetraplegics with severe head motion limitations. As a matter of fact, out of 13 randomly picked tetraplegics, only 6 were capable to use the gesture-based interface presented by Jackowski et al. This strongly limits the applicability of this interface for individuals with head motion restrictions. Even more, if the restrictions are such that one or two head DOFs cannot be used for control: The interface presented within this work offers the possibility to change the mapping as described previously. This is hardly possible for the interface presented in Reference [29] because each new group resulting from a lost head DOF requires a new head gesture, whereas the ability to produce head gestures decreases with increasing head motion restrictions. For these reasons, we conclude that the interface presented by Jackowski is promising as a hands-free interface for people without head motion limitations, whereas the interface presented here is more suitable for individuals with head motion restrictions because it can be adapted to their special needs.

**Supplementary Materials:** The following are available online at http://www.mdpi.com/1424-8220/19/12/2836/ s1, Static Data S1: 5 min of static data recorded with Hillcrest FSM-9, Dynamic Data S1: Head motion data recorded with Hillcrest FSM-9.

**Author Contributions:** Conceptualization, N.R. and M.G.; Data curation, N.R.; Formal analysis, N.R.; Funding acquisition, N.R. and M.G.; Investigation, N.R. and M.G.; Methodology, N.R.; Project administration, N.R. and M.G.; Resources, M.G.; Software, N.R.; Supervision, M.G.; Validation, M.G.; Visualization, N.R.; Writing—original draft, N.R.; Writing—review & editing, M.G.

**Funding:** This research was funded by the Federal Ministry of Education and Research of Germany grant number 01158324.

**Acknowledgments:** The authors would like to thank all the people involved, in particular Roland Thietje and his team from the BG-Hospital Hamburg, Axel Gräser, Icons8 and all the subjects who took part in the experiments. **Conflicts of Interest:** The authors declare no conflict of interest.

### **Abbreviations**

The following abbreviations are used in this manuscript:

### **References**

c 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

MDPI St. Alban-Anlage 66 4052 Basel Switzerland Tel. +41 61 683 77 34 Fax +41 61 302 89 18 www.mdpi.com

*Sensors* Editorial Office E-mail<EMAIL_ADDRESS>www.mdpi.com/journal/sensors

MDPI St. Alban-Anlage 66 4052 Basel Switzerland

Tel: +41 61 683 77 34 Fax: +41 61 302 89 18


**Łukasz D ˛ebowski**

Institute of Computer Science, Polish Academy of Sciences, ul. Jana Kazimierza 5, 01-248 Warszawa, Poland<EMAIL_ADDRESS>Tel.: +48-22-3800-553

Received: 20 November 2019; Accepted: 8 January 2020; Published: 9 January 2020

**Abstract:** We supply corrected proofs of the invariance of completion and the chain rule for the Shannon information measures of arbitrary fields, as stated by D ˛ebowski in 2009. Our corrected proofs rest on a number of auxiliary approximation results for Shannon information measures, which may be of an independent interest. As also discussed briefly in this article, the generalized calculus of Shannon information measures for fields, including the invariance of completion and the chain rule, is useful in particular for studying the ergodic decomposition of stationary processes and its links with statistical modeling of natural language.

**Keywords:** Shannon information measures; fields; invariance of completion; chain rule

**MSC:** 94A17


*Article* **Mining Indonesian Microbial Biodiversity for Novel Natural Compounds by a Combined Genome Mining and Molecular Networking Approach**

**Ira Handayani 1,2,†, Hamada Saad 3,4,†, Shanti Ratnakomala 5, Puspita Lisdiyanti 2, Wien Kusharyoto 2, Janina Krause 1, Andreas Kulik 1, Wolfgang Wohlleben 1, Saefuddin Aziz 3, Harald Gross 3, Athina Gavriilidou 6, Nadine Ziemert 6,7 and Yvonne Mast 1,7,8,9,\***

**Abstract:** Indonesia is one of the most biodiverse countries in the world and a promising resource for novel natural compound producers. Actinomycetes produce about two thirds of all clinically used antibiotics. Thus, exploiting Indonesia's microbial diversity for actinomycetes may lead to the discovery of novel antibiotics. A total of 422 actinomycete strains were isolated from three different unique areas in Indonesia and tested for their antimicrobial activity. Nine potent bioactive strains were prioritized for further drug screening approaches. The nine strains were cultivated in different solid and liquid media, and a combination of genome mining analysis and mass spectrometry (MS)-based molecular networking was employed to identify potential novel compounds. By correlating secondary metabolite gene cluster data with MS-based molecular networking results, we identified several gene cluster-encoded biosynthetic products from the nine strains, including naphthyridinomycin, amicetin, echinomycin, tirandamycin, antimycin, and desferrioxamine B. Moreover, 16 putative ion clusters and numerous gene clusters were detected that could not be associated with any known compound, indicating that the strains can produce novel secondary metabolites. Our results demonstrate that sampling of actinomycetes from unique and biodiversity-rich habitats, such as Indonesia, along with a combination of gene cluster networking and molecular networking approaches, accelerates natural product identification.

**Citation:** Handayani, I.; Saad, H.; Ratnakomala, S.; Lisdiyanti, P.; Kusharyoto, W.; Krause, J.; Kulik, A.; Wohlleben, W.; Aziz, S.; Gross, H.; et al. Mining Indonesian Microbial Biodiversity for Novel Natural Compounds by a Combined Genome Mining and Molecular Networking Approach. *Mar. Drugs* **2021**, *19*, 316. https://doi.org/10.3390/md19060316

Academic Editors: Ipek Kurtboke, Orazio Taglialatela-Scafati and Max Crüsemann

Received: 29 April 2021 Accepted: 25 May 2021 Published: 28 May 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

**Keywords:** Indonesia; biodiversity; novel antibiotics; drug screening; bioactivity; gene cluster networking; GNPS

### **1. Introduction**

It is now 80 years ago that Selman Waksman and Boyd Woodruff discovered actinomycin from *Actinomyces (Streptomyces) antibioticus*, which was the first antibiotic that was isolated from an actinomycete [1]. Since then, actinomycetes have been widely used as sources for drug discovery and development [2]. Most antibiotics and other useful natural products applied in human medicine, veterinary, and agriculture are derived from these filamentous bacteria [3,4]. Within the family of Actinomycetales, *Streptomyces* is the most prominent genus in respect to the production of bioactive secondary metabolites since it is the origin of more than 50% of all clinically useful antibiotics [5]. Successfully, the intensive screening campaigns of soil-derived streptomycetes yielded many currently recognized drugs, such as the antibacterial substance streptomycin, the antifungal metabolite nystatin, and the anticancer compound doxorubicin during the golden era of antibiotics [6,7]. However, in the last few decades, discovering and developing new drugs from these soil microorganisms has declined immensely, while the need for new drugs to overcome multidrug resistance has become greater than ever [8]. Nowadays, one of the major problems in antibiotic screening programs, in particular with streptomycetes, is the high rediscovery rate of already-known antibacterial compounds through the classical bioactivity-guided paradigms [3].

Sampling actinomycetes from conventional environments such as soils often leads to the rediscovery of known species producing already-known antibiotics [9]. Thus, gaining access to unusual unique habitats with the pursuit to isolate new strains as sources of novel bioactive compounds represents a current barrier in drug discovery research [9]. In recent years, the bioprospection of underexplored niches such as extreme or marine environments has become an efficient approach to find novel *Streptomyces* species that might produce novel compounds [10,11]. *S. asenjonii* strain KNN 42.f, isolated from a desert soil sample, is one example of a novel *Streptomyces* species from an extreme habitat, which produces the three new bioactive compounds asenjonamides A–C [12]. Another example displays the marine *S. zhaozhouensis* CA-185989 that produces three new bioactive polycyclic tetramic acid macrolactams [13]. *Micromonospora* sp. as turbinimicin producer represents a further example of prolific marine bacteria that can deliver new antifungal compounds [14]. These are only a few examples demonstrating that unusual or aquatic territories can be promising avenues as new natural products reservoirs.

Indonesia is the world's largest archipelagic country, spanning into three time zones, covering more than 17,000 islands, with 88,495,000 hectares of tropical forest, 86,700 square kilometers of coral reefs, and 24,300 square kilometers of mangrove areas [15,16]. It has the second-highest level of terrestrial biodiversity globally after Brazil [17], while being ranked as first if marine diversity is taken into account [16,17]. With the given species-rich flora and fauna besides endemic and ecologically adapted species, mega biodiversity of microbial species is gratifyingly represented across various unique habitats [18–20], such as acidic hot springs [21], peatland forests [22], the Thousand Islands reef complex [23], Enggano Island [24], fish species [25], and leaves of traditional medicinal plants [26]. Thus, since unique Indonesian niches are expected to deliver untapped potential actinomycetal strains that may produce novel bioactive secondary metabolites, different locations were targeted for the sampling of actinomycetes in this study.

The latest analyses of genome sequence data from actinomycetes revealed a remarkable discrepancy between the genetic potential of the secondary metabolism, known to be encoded by biosynthetic gene clusters (BGCs), and the actual natural compound production capacity of such isolates, upon their growth under standard laboratory conditions. This is attributed to the fact that numerous BGCs are not expressed under conventional lab

parameters and occur as so-called "silent" or "cryptic" BGCs [27]. The activation of these silent clusters allows one to unlock the chemical diversity of the tested organisms and enables the discovery of new molecules for medical and biotechnological purposes [28]. Thus, several efforts, e.g., involving genetic and cultivation methods, are employed to activate the expression of silent gene clusters [29]. One cultivation-based approach to exploit the metabolic capacity of the natural compound producers is the "one strain many compounds (OSMAC)" strategy [3,28,30]. Such a strategy simply relies on the variation of media compositions as a basis to test for different natural compound production profiles since global changes in the specialized metabolic pathways can occur under variable cultivation conditions [31]. The OSMAC concept represents a well-established model that was suggested nearly two decades ago; however, it still leads to the discovery of new chemotypes, such as the novel aromatic polyketide lugdunomycin from *Streptomyces* sp. QL37 [32] or an eudesmane sesquiterpenoid and a new homolog of the Virginiae Butanolides (VB-E) from from *Lentzea violacea* strain AS 08 [33]. Along the lines of the OSMAC concept, an elicitor screening approach has recently been suggested, which intends to mimic natural trigger molecules that can induce the biosynthesis of formerly unknown metabolites. This format has been conducted in a high-throughput approach and was coupled with MALDI-MS analysis. In the case of *S. ghanaensis*, this strategy led to the discovery of the antibiotically active depsipeptide cinnapetide [34].

Besides the variable trials to elicit the BGCs via pleiotropic approaches, a mass spectrometry dereplication step is frequently included in the current screening programs to address the formerly stated challenge of the high rediscovery rate prior to the tedious screening, isolation, and purification processes [35–37]. The utility of such a platform is to pinpoint known compounds in the initial phase of the discovery pipeline and leverage the process of finding new drugs. Integrated genomic and metabolomic mining methods have proven as an efficient dereplication strategy for compound identification in recent years [38–41]. While genome mining involves the identification of putative BGCs based on the genome sequences of the natural compound producers [42,43] using in silico bioinformatics tools such as antiSMASH [44], metabolome mining encompasses sorting out the chemical compounds in extracts of natural compound producers via their mass fragmentation patterns. Counting on the fact that metabolites with a similar chemical architecture tend to generate similar mass fragmentation patterns in mass spectrometry (MS) analysis, the implementation of the computational platform Global Natural Product Social (GNPS) to group the structurally related entities, often derive from a common biosynthetic origin, as a connected set of a molecular family cluster is an overgrowing necessity [45]. Such a platform iteratively proves its effectiveness to arrange seamlessly large numbers of samples enabling dereplication and tentative structural identification and/or classification [46]. The combinatorial employment of both computational tools side by side empowers the rapid identification of new substances, which can be highlighted by discovering the antibacterial substance thiomarinol from *Pseudoalteromonas luteoviolacea* [38] and microviridin 1777, a chymotrypsin inhibitor from *M. aeruginosa* EAWAG 127a [47].

Taken all together with the promises that highly biodiverse habitats can offer in synergy with an effective and practical mining technique, this study aimed to characterize the secondary metabolomes of selected actinomycetes isolated from three different locations within Indonesia. A collection of 422 actinomycetes from Lombok, Bali, and Enggano Islands were sampled and preliminary filtered with different bioactivity tests, where nine actinomycetes with the most bioactive potential were nominated for a hybrid genome mining and molecular networking approach in order to assess their biosynthetic capacity for the production of novel natural compounds.

#### **2. Results and Discussion**

#### *2.1. Isolation and Characterization of Indonesian Actinomycetes*

To isolate actinomycetes, soil samples were collected from two specific habitats (terrestrial and marine) in three different geographic areas of Indonesia using standard isolation

protocols [48–52]. Enggano Island was chosen as a sampling location for terrestrial habitats since it is a pristine island with many endemic species and high biodiversity [53,54], whereas Bali and Lombok Island were selected as sampling sites for marine habitats resulting in 422 strains in total (Table 1). Among all sampling locations, the Enggano Island soil samples contributed to the highest number of actinomycetes isolates (56.2%), followed by sediment samples from Lombok (37.2%) and Bali island (6.6%).

**Table 1.** Indonesian strains, isolation method, source of isolation (compare Figure 1), and most closely related species (%) based on 16S rRNA gene sequence phylogenetic analysis with EzTaxon.

Within the frame of a preliminary bioactivity screening, all 422 isolates were evaluated for their antimicrobial activities in agar plug diffusion bioassays against selected Grampositive (*Bacillus subtilis, Micrococcus luteus*, and *Staphylococcus carnosus*) and Gram-negative bacteria (*Escherichia coli* and *Pseudomonas fluorescens*). The 16 most potent isolates were selected based on their antimicrobial activity against the tested organisms, indicated by the largest inhibition zones around the agar plug. All 16 isolates showed bioactivity against the Gram-positive test organism *B. subtilis* (Figure 2A), and nine exerted further activity against Gram-negative test strains (Figure 2B), while only four strains (BSE 7–9, BSE 7F, I3, and I6) displayed potency against both (Figure 2).

**Figure 1.** Map of Indonesia showing three geographical regions (**A**). Sampling site location in Enggano Island (**B**), Bali Island (**C**), and Lombok Island (**D**). Red dot shows the sampling locations at Enggano Island, B1: Desa Meok; B2: Desa Banjar Sari; B3: Bak Blau Waterspring, Desa Meok; B4: Desa Boboyo; B5: Desa Malakoni; at Bali Island C1 for Kuta; and Lombok Island D1: Pantai Cemara, Lembar; D2: Pantai Tanjung Kelor, Sekotong; D3: Pantai Koeta.

**Figure 2.** Antimicrobial bioassays with 16 Indonesian actinomycetes strain samples against Grampositive (**A**) and Gram-negative test strains (**B**). Inhibition zone diameters of agar plug test assays are given in mm. Agar plugs were used after ten days of growth of the respective actinomycetes strains. Data shown are as the result of three independent biological replicates.

To investigate the phylogenetic relationship of the 16 bioactive actinomycetal isolates, 16S rRNA gene sequence analyses were performed. For this purpose, the genomic DNA was isolated from each and was used as a template in a PCR approach with 16S rRNA genespecific primers. The resulting 16S rRNA gene amplifications were sequenced, and the 16S rRNA gene sequences were compared using the EzTaxon database (www.ezbiocloud.net/, accessed on 28 May 2018) to determine the phylotype of the strains [55]. EzTaxon analysis revealed that all isolates belong to the genus *Streptomyces* with similarity values amongst the various predicted related species ranging from 98.44–99.89% (Table 1).

Subsequently, nine strains were prioritized based on their bioactivity profile and taxonomic position. Strains SHP 22-7, BSE 7-9, BSE 7F, I3, I4, I5, and I6 were selected since they showed antibacterial activity against Gram-positive and Gram-negative bacteria (Figure 2A,B). DHE 17-7 and DHE 7-1 were selected as they exerted bioactivity against at least two different Gram-positive test strains. DHE 6-7 and DHE 5-1, which showed bioactivity against all Gram-positive test strains, were not chosen for further analysis because both strains showed a close phylogenetic relationship to *Streptomyces parvulus* (Table 1), which is a known producer of the polypeptide antibiotic actinomycin D [56]. In an initial attempt with HPLC-MS analysis of the methanolic extracts of culture samples from DHE 6-7 and DHE 5-1, actinomycin D was detected as a product (Figure S1), ruling out both strains from further investigations.


**2. Theoretical deliberations of the economic game**

This section provides a theoretical background that motivates the economic game. First, based on the theoretical predictions of the game-theoretic model of Morris and Shin [1], we develop the framework that will be used in the experimental design. Then, within this framework, we derive the main results.

#### **2.1 Speculative attack as a coordination game under heterogeneous information**

Speculative attacks can be modeled as a coordination game with strategic complementarities. In the experiment, we employ a simple beauty contest model of Morris and Shin [1] with a finite number of economic agents N who decide simultaneously whether to attack or not.2 Let us Y denote the fundamental state of the economy. The higher Y is, the better the state of the economy.

The gains of agents are given as follows:

	- Y, if the proportion of agents who attach exceeds a(Y).
	- 0, if the attack fails

Agents decide to attack or not based on two types of information (signals) given the state of the economy Y. Hence, the central bank disseminates posterior private and public information, where the public signal *Z=Y* ! *N* 0, *σ*2 *η* . Individually, they

2 It is a concept developed by Keynes [18] to explain the fluctuations in equity markets. It is the view that much of investment is driven by the expectations about what other investors think, rather than expectations about the fundamental profitability.

receive private signals *xi* ! *N* 0, *σ*2 *ε* . Morris and Shin [19] and Hellwig [20] show that there is a unique equilibrium with a critical state below which currency crisis occurs with certainty, that provided private information is sufficiently precise relative to the public information. Metz [21] showed that the signals' interaction may have adverse effects. Her theoretical predictions can be summarized as follows: