User study discussion
pronation, muscle fatigue, supination
REGRESSION, MINOR
Overall, it could be shown in the user studies that both the standard kNN scheme as well as the DSM-reduced technique yielded significantly higher success rates than RR-RFF and RR in most of the scenarios. The behaviour that kNN-based methods performed significantly better at higher exertion levels in the basic user study could be due to the fact that gestures of low intensity are more often subject to misclassification. This might result from a too high rest magnitude threshold which causes movements with low signal amplitudes being classified as rest state. In the extended user study, there was no significance for this difference. However, the effect could probably be curtailed in general by a learning process where the subjects would get used to the specific behaviour of the algorithm and adapt to it. Furthermore, the limb position effect might influence the results with respect to the average rest signal magnitude, although the subjects sat in a standardized pose.Extended user study (7 classes, 4 subjects), online classification time performance per subject, averaged for all prediction samplesSince the principle idea behind the use of random Fourier features is to fit cosine functions in the regression space, this might lead to unwanted behaviour at intermediate levels while showing better performance for especially the full gesture exertion (and slightly also for low levels). For RR, a similar principle holds, with the exception of using linear instead of cosine functions. Probably, the assumption of linear dependency is valid for small intensities, hence showing better success rates for For the kNN-based methods, the highest success rates were achieved for wrist extension (and in the second user study also for wrist flexion). The reason for this could be that power grasp and pointing gesture are most probably mainly exerted by the same group of muscles, but the wrist gestures are not—thus leading to better separability of those classes. In the basic user study, precisely, the success rate difference between pointing gesture and wrist extension was significant; for DSM-kNN, power grasp also differed significantly from wrist extension. This might point towards the described explanation. In the extended user study, there was no significance for that.Since the muscle groups activated in pointing gesture and power grasp are spatially close to each other from a biomechanical point of view, it could be explained that these two yielded the lowest success rates in the first user study, probably due to misclassifications between the two classes (they differed not significantly). The gestures are only distinguished in one degree of freedom (index finger), while the other degrees of freedom are the same. In the extended user study, pronation and supination also performed at the same lower level; however, there was no significance provable. Standard RR showed the same behaviour in terms of individual gesture performances as the kNN-based methods (at a generally reduced success rate baseline), with the exception of wrist extension performing worse than flexion, although this difference was not significant in any case. Extension and flexion address the same degree of freedom which might therefore cause smaller deviations. With that, the differences between the wrist movements in RR-RFF could also be explained. One reason for pointing yielding the most successes in RR-RFF (although not significantly) could be that it was exerted by the subjects in a different manner than when the other methods were tested. Basically, the exertion can take place by also using the muscle group used for extension (stressing the index finger movement), instead of the muscle group used for flexion and power (where the activity patterns of the flexed fingers are stressed). If this was the case, for RR-RFF, this might also explain the reduced performance of wrist extension. Nevertheless, the question would be why this would have been only the case for RR-RFF. A reason therefore might be traced back to the specific properties of random Fourier features when subjects try to reduce overshooting or similar. However, for the success rate differences between individual gestures in RR and RR-RFF, no significance could be substantiated.In the extended study, pointing and pronation performed with the lowest success rates for the kNN-based methods, probably due to potentially addressing the same groups of muscles by these gestures, although there was no significance. The observation that RR-RFF performed worst in the extended user study (with the main exception of pronation where there was no significance between RR-RFF and the other schemes) could potentially be related to its capability of predicting multiple degrees of freedom in parallel. This might be leading to unstable predictions when it comes to predicting only a single degree of freedom. In order to realize a higher extent of comparability, only single degrees of freedom might be checked in the target achievement tests instead of all of them in future experiments. All in all, the extended user study could confirm the general observations made in the study’s previous main part so that further in-depth experiments including more than the originally tested four gestures are recommended. Except for RR-RFF, there was no drop of success rate in comparison to the basic user study.The fact that the DSM-reduced kNN did not perform worse (and in some cases even better) than the non-reduced kNN could be attributed to a possibly better subject’s adaptability to the algorithm since there are less samples available whose decision borders are more clearly defined than when making use of the whole non-reduced dataset with possibly more abrupt changes in the decision borders the user can hardly learn. Another reason might be that noisy instances are discarded in the reduction process so that samples leading to misclassifications and worsening the performance are not considered anymore. Nevertheless, the slightly better performance of DSM-kNN might also be just resulting from stochastic factors. There is no statistical significance for that observation.The generally better performance of the chosen kNN-based techniques can be also addressed to the fact that they are classification-based methods—extended by proportionality schemes. This is why in the presented manner, they are not suited for simultaneous control, i.e. predicting mixed states of different gestures. In contrast, the RR-based methods also consider these states as an inherent property of regression. This means, as soon as multiple degrees of freedom are trained, RR and RR-RFF can get influenced by multiple degrees of freedom in parallel, although in the prediction tasks, only a certain degree is tested at once. Therefore, the advantage of simultaneous control is at the expense of stability and robustness and vice versa for kNN.In order to find a good measure of comparability with regard to the development of the success rate over time for the individual methods, the available time slots during one experiment for a subject have been split into eight time subgroups. The differences over time are rather minor. The main perceivable difference originated from the choice of method. However, some interpretation of minor tendencies is given in the following.It could be observed for the kNN-based methods that the performance in the first time slot was below the ones in the later timeslots. This could potentially be explained by a learning effect, i.e. the subject adapting to the specific behaviour of the algorithm. The same effect could explain that in DSM-kNN, the highest success rates were achieved towards the end of the experiment. At the very end, the performance decreased again, potentially due to muscle fatigue setting in. It could be seen that RR-RFF and RR exposed the same monotonicity when it comes to the mean performance over time, namely a sequence of possible learning effect, muscle fatigue and learning effect again. The learning effect might have set in again after each break. Apart from this, the RR-based methods showed a constant median of 33% success rate over time (with the exception of the first time slot in RR). However, to gain representative insights, it might be useful to look into the time performance of individual gestures.