Vision and Fusion Laboratory Institute for Anthropomatics Karlsruhe Institute of Technology (KIT), Germany<EMAIL_ADDRESS>
### **Abstract**

Fine-grained vehicle classification is an important task particularly for security applications like searching for cars of suspects who abuse stolen license plates. However, data privacy and the large number of existing car models render it highly difficult to create a large up-to-date dataset for fine-grained vehicle classification with surveillance images. While a large number of images of vehicles are available in the web due to car selling sites, they have a perspective which is vastly different to surveillance images. Domain adaptation is the field of research that uses domain-wise inappropriate images for training of classification models with the target of running accurate inference on images of a different domain. Since the widely considered unsupervised and semisupervised domain adaptation settings are unrealistic for fine-grained vehicle classification, we establish a baseline for cross-domain fine-grained vehicle classification in a supervised partially zero-shot setting. Our results indicate that existing domain adaptation methods like domain adversarial training and triplet loss are still advantageous for this setting and we show the benefit of distance-based classification for this task.

## **1 Introduction**

Fine-grained classification tasks like vehicle make and model recognition are relying on large datasets for training. These are needed since the small inter-class variance compared to the large intra-class variance are required to be properly approximated by the learned model. While in the web, a large amount of images for different cars are provided by e.g. car selling sites, fine-grained classification is often applied in different domains. For example, vehicle make and model recognition is useful for security applications like manhunt when applied to cameras on highways which provide a surveillance perspective. However, for these perspectives, the availability of data is scarce. The situation is worsened by the high rate of car manufacturers proposing new vehicle models .

To approach the lack of data, domain adaptation methods can enable the use of the large-scale availability of data of different domains like web-nature images to perform tasks like classification in domains which have a limited availability of data like surveillance. While domain adaptation has been widely approached [33, 45, 48, 22, 4, 25, 11] and also specifically for fine-grained classification [10, 31, 34, 35, 44] applications, an unsupervised or semi-supervised domain adaptation setting is commonly assumed. In these settings, a large number of images is present in the target domain for all classes but the labels aren't present for any or only a part of the images. However, for real-world use-cases, the assumption to have data for all classes is hard to fulfill since it can only be assured if labels would be present. Thus, we focus on a different domain adaptation setting: a supervised partially zero-shot setting [38]. This setting assumes that for a large number of base classes, images and labels are available for both domains while for a small number of novel classes, images and labels are only available for the source domain. For these novel classes, no images are available at all for the target domain during training. However, the evaluation on these novel classes with images from the target domain is the main focus of the setting.

Since the research for such a setting is rather small [32], we provide an extensive evaluation of existing domain adaptation methods to find a good baseline for further research. Besides the widely applied domain adversarial learning [8], we explore the use of metric learning with a triplet loss which also has shown advantages for classification across domains [20, 32].

Based on these experiments, we found that a typical softmax classifier only achieves a low classification accuracy for the novel classes. However, a domain adversarial loss heavily increases the accuracy. A distance-based classifier with a combination of a cross entropy loss and a triplet loss showed promising results which can further be improved by the use of a domain adversarial loss resulting in the overall best model.

In Section 2, existing works in the fields of fine-grained classification, crossdomain classification and cross-domain fine-grained classification are introduced. In Section 3.1, the evaluated methods are described and the evaluation results are shown in Section 4. A conclusion of this work is given in Section 4.

# **2 Related work**

In this chapter, an overview of the literature in the fields of fine-grained classification and cross-domain classification as well as works which employ cross-domain classification for fine-grained classification tasks is given.

### **2.1 Fine-grained classification**

Various approaches have been used to improve the accuracy for fine-grained classification. While all recent approaches share their basis of deep neural networks, there are several different extensions and they can be structured into the following categories. **Part-based models** first detect relevant regions like specific parts of a vehicle before the crops of these parts are fed into a convolutional neural network (CNN) [7, 15, 28, 41]. This reduces the feature space to significant parts and thus, reduces the risk of overfitting. **Bilinear CNNs** employ two networks to separate the localization and the extraction of important features. The networks are combined by calculating the outer product of both resulting feature vectors [23]. Several extensions have been proposed to improve the accuracy and efficiency of bilinear CNNs [9, 19, 43]. Multiple authors employ **multi-task learning** by learning an auxiliary task like predicting the viewpoint of the image that provides support for the main task of fine-grained classification. The auxiliary task is performed only during

training to improve the learned features [3] or also during inference to provide the network with additional information [24, 29]. **Hierarchical classification** exploits that fine-grained categories are usually defined on multiple layers, e.g. make, model and year of a car. This technique was explored by training multiple layers of the hierarchy in a round-robin manner [16] and by training cascaded classifiers [2]. **Metric learning** has also been applied to improve the features by minimizing intra-class variance and maximizing inter-class variance [17, 30, 42]. **Temporal classification** uses videos as input modality for fine-grained object recognition [1, 46, 18] instead of single images as done by most works. **Webly-supervised classification** gathers additional data from the web with image databases like Flickr providing images with additional meta information that can be used for defining labels [6, 39].

#### **2.2 Cross-domain classification**

Domain adaptation is usually employed if classification has to be done in a domain for which a lack of data exists. The lack of data can be in the form of missing images or missing annotations. Mostly an unsupervised scenario is considered which contains abundant but unlabaled data for the target domain. To approach a cross-domain setting, multiple methods have been proposed. We follow the taxonomy of Wang and Deng [33] for the categorization of the approaches. **Discrepancy-based domain adaptation** methods are based on a criterion during fine-tuning to increase the accuracy for the target domain. Proposed criteria are class-based [31, 45], statistic-based [48], architecturebased [22] or geometry-based [4]. **Adversarial-based domain adaptation** methods target a domain confusion of the trained network which disables the possibility of exploiting the domain of an image for the classification decision. This can be done by generative approaches which transform the appearance of a source sample such that it can not be distinguished from the distribution of target samples [25]. Non-generative approaches have also been explored by using domain adversarial training with a domain classifier that is preceded by a gradient reversal layer during training. This leads to features which are invariant in regard to distinguishing the domains. **Reconstruction-based domain adaptation** methods reconstruct samples from either domain to the other domain to create a

domain-invariant representation. This has been explored by using a combination of an encoder and a decoder [11] as well as using a Cycle-GAN [47] that keeps semantic information intact by using a cycle-consistency constraint [14].

### **2.3 Cross-domain fine-grained classification**

Some researchers have already addressed fine-grained classification in a crossdomain setting. Gebru et al. [10] exploit the hierarchical nature of fine-grained classification by adding an attribute consistency loss that enforces a matching of coarse-grained attributes like vehicle types to the fine-grained category. With the coarse-grained attribute prediction being a significantly easier task, it is more domain invariant and thus, supports stabilizing the fine-grained prediction due to the new consistency loss. Tzeng et al. [31] and Wang et al. [34] also exploit the attribute and coarse-grained labels inherent to fine-grained classification tasks to improve the domain adaptation. Wang et al. [35] extends adversarial domainlevel adaptation by a category-level domain alignment for semi-supervised domain adaptation. Additionally, a part-wise classification to optimize the fine-grained classification accuracy is introduced. Yu et al. [44] achieve a class confusion by training separate class labels for each domain in a pre-training phase and swapping the class labels in a fine-tuning phase with the target of achieving domain confusion while compared to domain adversarial training, keeping the class-separability of the features intact during the adaption process.

## **3 Methods**

In this section, the evaluated methods are described. They can be mainly divided by the type of classification. We evaluate a softmax classifier and a distance-based classifier. As feature extracting backbone, we use ResNet-50 [12] for both variants. On top of both variants, we evaluate the usage of domain adversarial training [8] to improve the domain invariance. While only common for distance-based classification, we evaluate a triplet loss [36] for both variants due to the reported advantages in regards to cross-domain classification [21].

#### **3.1 Softmax classifier**

The softmax classifier employs a fully-connected layer to predict as many logits as number of classes and afterwards applies a softmax activation layer to normalize the scores. On top of this output, a cross entropy loss is used to calculate an error measurement.

Additionally, we evaluate the use of a domain adversarial head and an auxiliary triplet loss to improve the domain invariance of the features. Both additions are applied directly on the features of the backbone.

### **3.2 Distance-based classifier**

For the distance-based classifier, during inference, we feed each preprocessed image into the backbone network and calculate the distance between the feature vector of the sample and a prototype feature vector for each class. We choose the class as final prediction for which the distance has the lowest value. The prototype is calculated as the mean of all training samples of a class from the source domain. We also evaluated the use of a medoid instead of a mean but the results indicated an advantage for the mean. Regarding the distance measure, we evaluated the euclidean norm and the negative cosine similarity with the results showing a clear advantage for the negative cosine similarity while the euclidean norm usually prevented the network from converging properly. Since the cosine similarity is originally a similarity instead of a distance measure, we use the negative of the cosine similarity as distance measure. The classification can be described by the following formulas:

$$\mathbf{p\_c} = \frac{1}{|X\_c|} \sum\_{x \in X\_c} f(x) \tag{3.1}$$

$$c(x) = \underset{c \in C}{\text{argmin}} -\frac{f(x) \cdot \mathbf{p\_c}}{||f(x)|| ||\mathbf{p\_c}||} \tag{3.2}$$

where **pc** is the feature prototype for the class *c*, *Xc* is the set of training images of a class *c* from the source domain, *f* is the backbone feature extractor, *c*(*x*) is the predicted class for an image *x* and *C* is the set of known classes.

During training, we apply a cross entropy loss with a softmax activation on top of a fully-connected layer. Since the cross entropy loss tends to learn features which are highly dependent on the domain, we use a triplet loss as additional loss function that regularizes the network in regards to the domains. Additionally, the triplet loss ensures that the chosen distance measure is appropriate for the features during inference. After training, the fully-connected layer is dropped and the extracted features are directly used as described above.

### **3.3 Domain adversarial training**

Ganin et al. [8] proposed a domain adversarial training method. It applies a simple domain classifier on top of the features extracted by the backbone and inserts a gradient reversal layer between the network and the domain classifier. The gradient reversal layer leads to learning features which are most inappropriate for a classification of the domain and thus, the features are expected to be invariant in regards to the domain. Therefore, the classification loss which is applied in parallel will focus on learning features which are inherent to the class instead of exploiting the domain.

For the domain classification head, we employ two hidden fully-connected layers with 1024 channels with each being followed by a ReLU activation and a batch normalization layer. A final fully-connected layer with a single output channel which is followed by a sigmoid activation predicts the domain. A binary cross entropy is applied as training loss for the domain classification.

The gradient reversal layer includes a gating that controls the influence of the reversed gradient of the domain classification loss onto the main network. We call this parameter *λ*. A *λ* of 1 means an unhindered influence while a *λ* of 0 means that the domain classification has no influence on the main network at all. A good choice of *λ* might depend on the current state of training and a pre-set value is probably not appropriate. Our results showed that the loss coupling of *λ* proposed by Wiedemer et al. [37] was superior to a pre-set value and an increasing schedule of *λ* as it was originally proposed for the domain adversarial training [8]. The loss coupling sets *λ* for each iteration based on the domain classification loss value of the previous iteration. The exact formula is *λi* = exp(−*Ld,i*−1) with *λi* being the set *λ* for the iteration *i* and *Ld,i*−1 being the domain classification loss for iteration *i* − 1. This ensures that the domain classification only has a strong influence on the main network if the loss is low meaning that the domain classifier is able to classify the domain adequately. In case of a high domain loss, the domain classifier is not able to classify the domains properly and will not provide a good domain adversarial loss.

### **3.4 Triplet loss**

A triplet loss [36] explicitly minimizes the distance of features of the same class while maximizing the distance of features of different classes with respect to a chosen distance measure. While the cross entropy loss also tends to show a similar behavior, it only enforces a linear separability of classes which can result in features of a single class still being spread in feature space. This can be particularly dramatic for cross-domain scenarios for which the distribution of images is different between training and inference. Thus, we apply a triplet loss as additional loss that directly minimizes the distance of features of the same class.

## **4 Experiments**

We execute quantitative evaluations to find a good baseline for cross-domain classification under a supervised partially zero-shot setting. First, the settings of the comparisons are described. Afterwards, the results are discussed. The comparisons include ablation studies for a softmax classifier, ablation studies for a distance-based classifier and a comparison between both approaches.

### **4.1 Settings**

The datasets used for the experiments are described first. Afterwards, the evaluation metrics and training details are reported.

#### **4.1.1 Dataset**

As dataset, we choose CompCars [40] which is one of the largest fine-grained vehicle classification datasets available and consists of a web-nature part (CompCars Web) and a surveillance-nature part (CompCars SV). The CompCars Web has a predefined split of 16.016 training images and 14.939 test images. The predefined split of the CompCars SV contains 31.148 training images and 13.333 test images.

While the CompCars Web is labeled according to the make, model and year of a specific car, the CompCars SV is only labeled up to the model of a car and lacks the year as annotation. Thus, we also only consider the model for all cars in CompCars Web. This results in a total of 431 classes for CompCars Web and a total of 281 classes for CompCars SV. We identify the intersection of both sets of classes and use only these for our experiments. Thus, we consider a total of 181 classes. Based on this set of classes, we create three different random splits of base and novel classes with the base classes containing 90% and the novel classes containing 10% of the classes. While during training, for the base classes abundant labeled images are available in both domains, we restrict the availability of data for the novel classes to the source domain of CompCars Web and no images from CompCars SV are available for the novel classes. For each experiment, a model is trained and evaluated on each split and the results are averaged.

#### **4.1.2 Evaluation metric**

We use the F1 score on the CompCars SV as main metric for our experiments. We report the class-wise F1 score averaged over the base and the novel classes separately. Since our focus is on adding new classes to the classification, we focus mainly on the F1 score of the novel classes. Due to images of all classes being included in the test set, base classes still influence the score of the novel classes and vice versa. This is sensible since a network only focused on the prediction of novel classes should still be able to distinguish them from the total of all base classes even when distinguishing the base classes might be of minor importance.

#### **4.1.3 Training details**

We choose SGD as optimizer with an initial learning rate of 0.04 and a learning rate reduction by 10× is applied after 2500 iterations. We apply a momentum of 0.9 and a weight decay of 10−4 . The training is running for 12000 iterations in total. A batch size of 512 per GPU with two GPUs is used. Each batch contains 256 Web and 256 SV images. We evaluate after every 1000 iterations and apply early-stopping by choosing the checkpoint with the highest F1 score for novel classes on the CompCars SV images. The weights are initialized from a model pre-trained on ImageNet. During training, for each image, a crop spanning an area between 8% and 100% of the original image is taken randomly and is resized to a size of 224×224 pixels afterwards. Additionally, a random horizontal flip is applied with 50% probability. Afterwards, the image is normalized using the mean and the standard deviation values of the pre-training on ImageNet. For experiments with a triplet loss, we employ hard negative mining [13] and a margin of 0.3 since preliminary experiments have shown good results for this value.

### **4.2 Inference details**

During evaluation, the images are resized such that the shorter side has 256 pixels while keeping the aspect ratio. Afterwards, a crop of size 224×224 pixels is taken from the center of the resized image. The normalization is applied similar to the training configuration.

#### **4.3 Softmax classification**

We evaluate a softmax classifier as the most common architecture for deeplearning-based classification. Since softmax classifiers tend to heavily exploit domains in the classification, we explore the use of domain adversarial training and an auxiliary triplet loss to improve the domain invariance of the network.

**Table 4.1**: Evaluation of different schedules for the *λ* parameter of the domain adversarial training. The results indicate a clear advantage for the coupled schedule when focusing on the important novel classes.

#### **4.3.1 Domain adversarial training**

Adversarial domain adaptation [8] is a widely applied approach for domain adaptation. In order to find a strong baseline, we evaluate different schedules of the *λ* parameter that controls the influence of the domain adversarial head onto the main network. Besides a constant value and a widely applied monotonically increasing schedule [8], the coupled schedule by Wiedemer et al. [37] is also evaluated. The set *λ* base value describes the constant value for the constant schedule, the maximum value for the increasing schedule and the highest possible value (in case of zero domain classification loss) for the coupled schedule. The results are shown in Table 4.1.

The adversarial training leads to a large improvement of the base F1 score but particularly of the novel F1 score with all evaluated schedules for *λ*. While the impact of the schedule for *λ* is negligible for the base F1 score, for the important novel F1 score, the best results are achieved with the coupled schedule and a *λ* base value of 0.1. Based on these results, we continue to use these settings for all further experiments involving adversarial domain adaptation. The adversarial training reduces the impact of the domain onto the features and thus, leads to features of novel classes in the target domain being closer to features of the same class in the source domain. Therefore, the samples of novel classes in the target

**Table 4.2**: Evaluation of a triplet loss as auxiliary loss for a softmax classifier. The results indicate that an auxiliary triplet loss can improve the domain invariance of a softmax classifier.

domain are classified more accurately which in turn leads to less confusion with base classes. Thus, also the base class accuracy is improved.

#### **4.3.2 Auxiliary triplet loss**

The triplet loss has shown to be more domain invariant than a pure cross entropy loss. Thus, we evaluate the impact of an auxiliary triplet loss in Table 4.2. The triplet loss uses the negative cosine similarity as distance measure. A training with euclidean norm as distance measure did not converge properly since the euclidean norm enforces a feature space that is not well suited for the cross entropy loss. Thus, results for the euclidean norm are not reported.

The results show a clear advantage of the triplet loss for the accuracy of the base as well as the novel classes. The increase is probably a result of the triplet loss forcing a distance of close to zero in feature space for all samples of a class and thus, reducing the possibility of a spread due to different domains. While this only applies for the base classes in training, it probably also reduces the distance of samples of the novel classes between both domains leading to the improvement in the novel class accuracy. This improvement then leads to an improvement in base class accuracy due to less confusion with novel classes occurring.

**Table 4.3**: Comparing distance measures for a distance-based classifier. The negative cosine similarity shows a strong advantage with the euclidean norm showing poor results due to the cross entropy loss not converging properly.

### **4.4 Distance-based classification**

While CNNs are mostly combined with a logit-based classification head, distancebased classification and metric learning provide a higher flexibility due to not limiting the model to a specific set of classes during training.

#### **4.4.1 Distance measure**

For the distance-based classification, the choice of the distance measure is a crucial parameter. Thus, we compare the use of an euclidean norm as well as negative cosine similarity. The respective distance measure is applied for the triplet loss as well as for the classification. The results of the comparison are shown in Table 4.3. They indicate a strong advantage of the negative cosine similarity while the training with the euclidean norm does not properly converge. Particularly, the training of the triplet loss with an euclidean norm leads to a non-decreasing cross entropy loss. The embedding induced by a triplet loss with an euclidean norm seems to be incompatible with a logit-based softmax classification and a cross entropy loss. Seemingly, the optimizer can not converge to a proper embedding which suits both losses.

#### **4.4.2 Prototype aggregation**

For the classification, we aggregate all training samples from the source domain to estimate a prototype for each class and choose the class whose prototype is the closest to the input samples in terms of feature distance. For the aggregation

**Table 4.4**: Comparison of the estimation methods for the class prototype. Using the mean of the train samples shows a significant advantage over using the medoid.

**Table 4.5**: Evaluation of applying domain adversarial training with distance-based classification. The results show that adversarial training can provide an advantage in combination with a distance-based classifier.

of the samples, we evaluate a mean of the features and a medoid of the features. The medoid is defined as the sample which has the smallest total distance to all other samples. The results are shown in Table 4.4. While the difference on the base classes is negligible, the mean aggregation shows a clear advantage over the medoid for the novel classes.

#### **4.4.3 Domain adversarial training**

While the triplet loss already provides a strong improvement in terms of domain invariance for distance-based classification, we evaluate if domain adversarial training can still lead to an improved accuracy. Therefore, we apply domain adversarial training with the best setting as in the previous ablation studies additional to the cross entropy loss and the triplet loss we commonly use for the distance-based classifier. The results are shown in Table 4.5 and indicate a slight increase in terms of base class accuracy and a high increase in terms of novel class accuracy.

**Table 4.6**: Comparison of softmax classifiers with and without domain regularization methods and distance-based classification. The results show the advantage of distance-based classification for the accuracy of the novel classes while the softmax classifier with domain adversarial training shows a slight advantage for the base class accuracy.

### **4.5 Comparison of softmax classification and distance-based classification**

We compare softmax-based classification methods with and without domain adaptation extensions to a distance-based classification method in Table 4.6. For the softmax-based classification, a domain adversarial training as well as an auxiliary triplet loss is evaluated to improve cross-domain classification accuracy.

While the softmax classifier with the adversarial training shows the highest accuracy for the base classes, the distance-based classifier combined with a domain adversarial training follows closely behind and has a significant advantage in terms of novel class accuracy compared to all evaluated distancebased classifiers. Without adversarial training, the softmax classifier shows a heavy drop in accuracy particularly of the novel classes. The triplet loss also provides a large benefit for the softmax classifier. However, it still shows a large accuracy gap when compared to the adversarial loss.

## **5 Conclusion**

In this work, different domain adaptation approaches were evaluated in a supervised partially zero-shot setting for fine-grained vehicle classification to employ web images as training data for classification on surveillance images. The results show the importance of domain adversarial training to achieve acceptable results with a softmax-based classifier. However, a distance-based classifier employing a combination of a cross entropy loss and a triplet loss still show competitive results which can still be improved by domain adversarial training. This combination showed the overall best results for the classification of the novel classes in our evaluation.

Evaluation of better backbones as modern vision transformers [26, 5] or state-ofthe-art convolutional network architectures [27] is up to future work. Other areas of future research are improvements directly targeting the supervised partially zero-shot setting which have not yet been evaluated for other settings.

## **References**

### **Long-term Action Anticipation: A Quick Survey**

*Zeyun Zhong*

Vision and Fusion Laboratory Institute for Anthropomatics Karlsruhe Institute of Technology (KIT), Germany<EMAIL_ADDRESS>
## **Abstract**

The ability to anticipate possible human actions in the distant future is of fundamental interest for a wide range of applications, including autonomous driving, surveillance, and human-robot interaction. Consequently, various methods have been presented for action anticipation in recent years, with deep learning-based approaches being particularly popular. In this work, we give a short overview of the recent advances of long-term action anticipation algorithms.

## **1 Introduction**

In the last years, we have seen a tremendous progress in the capabilities of computer systems to classify and segment activities in videos. These systems, however, analyze the past or in the case of real-time systems the present with a delay of a few milliseconds. For applications, where a moving system has to react or interact with humans, this is insufficient. For instance, to be able to offer a hand at the right time or to generate proactive dialog to provide more natural interactions, collaborative robots that work closely with humans have to anticipate the activities of a human in the future. Compared to human action recognition and early action recognition, where entire or part of action

segments are observable, action anticipation aims to predict future action without observing any part of it, as displayed in Figure 1.1.

As the anticipation results are just assumptions, this tends to be significantly more challenging than traditional action recognition, which performs well with todays well-honed discriminative models [7, 17]. Consistent with action recognition, anticipation approaches start with prediction on only one single video frame [28] and tend to use longer temporal context [24, 29] in recent years. Apart from using a long action history, many approaches attempt to leverage several modalities other than just the raw video frames, such as the motion information and objects contained in the scene, to further improve the predictive ability.

While many recent works anticipate activities only for a very short time horizon of a few seconds [9, 8], there is a parallel line of work [6] which addresses the problem of anticipating all activities that will be happening within a time horizon of up to several minutes, which is particularly interesting for robot systems that require certain time to react and plan the future tasks.

In spite of the enormous amount of research conducted in this area, the problem is still challenging due to the fundamental challenges inherent to the task such as the multi-modal distribution of future action candidates, especially for the scenario where we are going to predict far into the future (long-term anticipation). As action recognition is usually a fundamental sub-component of an anticipation system, the challenges of action recognition [14] are also included, such as the tremendous intra-class variance among the activities, huge spatio-temporal scale variation, target motion variations, etc. Moreover, low image resolution, object occlusion, illumination change and viewpoint change further aggravate these challenges.

Although classical learning approaches, such as Conditional Random Fields (CRFs) [15], Markov models [23], and other statistical methods [19, 22], have been widely used in the literature, we put our focus on deep learning techniques and how they have been extended or applied to daily-living action anticipation, leaving the classical approaches outside the scope of the present review. In this context, the terms action anticipation, action prediction, and action forecasting are used interchangeably.

**Figure 1.1**: The action anticipation task aims to anticipate future action(s) before it happens, whereas action recognition and early action recognition require the observation of complete and partial actions, respectively.

This survey is structured as follows. In Section 2, we describe both short-term and long-term anticipation tasks which are commonly used in the literature, so that the reader can better distinguish between them. In Section 3, we introduce the current approaches that address the long-term anticipation task and discuss their limitations. Finally, we conclude this survey in Section 4.

## **2 Problem Statement**

Based on the prediction time horizon, action anticipation approaches can be grouped into two categories: short-term anticipation approaches and long-term anticipation approaches. While short-term approaches predict a single action a

**Figure 2.1**: Category of the action anticipation task. While the short-term anticipation aims at predicting a single future action, long-term task aims to predict a sequence of the following actions.

few seconds into the future, long-term approaches predict a sequence of future actions with their durations up to several minutes into the future. In the following sections, we show the detailed task definition of both categories usually used in the literature.

#### **2.1 Short-term anticipation**

Most short-term anticipation approaches follow the setup defined in [28, 4, 5]. As illustrated in Figure 2.1(a), the task aims to predict a future action by observing a video segment of length *τo*. The observation segment is *τa* seconds preceding the action, i.e., from time *τs* − (*τa* + *τo*) to *τs* − *τa*, where *τa* denotes the "anticipation time", i.e., how many seconds in advance actions are to be anticipated. The anticipation time *τa* is usually fixed for each dataset, whereas the length of the observation segment is typically dependent on the individual method. Methods in this category typically use synchronous data to perform the anticipation task, meaning that the input to the model is a sequence of frames that have the same temporal spacing before the action [9, 8].

Some work [18, 21] attempts to predict the starting time of the next action as well. As this task involves the duration of each action, these approaches usually use asynchronous data as input to the model, containing a sequence of action categories and inter-arrival times. The inter-arrival time is defined as the difference between the starting time of last and the current action. With the predicted inter-arrival time, the starting time of the next action can be easily deduced.

### **2.2 Long-term anticipation**

There is a parallel line of research addressing the long-tern anticipation task, which is proposed in [6]. The goal is to anticipate the category and the duration of future actions for a given time horizon, which can take up to several minutes, as illustrated in Figure 2.1(b). Long-term approaches typically take a sequence of observed action categories and their durations to predict another sequence of actions and durations [6, 1, 31].

## **3 Long-term Anticipation Approaches**

### **3.1 Methods**

Farha et al. [6] first introduced the long-term action anticipation task and proposed two models to tackle the task. One is based on an RNN model, which outputs the remaining length of the current action, the next action class and its length, as shown in Figure 3.1. The long-term prediction is conducted recursively, i.e., observations are combined with the current prediction to produce the next prediction. Another method is based on a CNN model, which outputs a sequence of future actions in a form of a matrix in one single step. Considering the limitations of these two methods, i.e., the RNN model is time-consuming and suffers from error accumulation and the CNN model introduces many parameters when predicting long sequences, Ke et al. [12] proposed a method to explicitly address these issues. They chose to condition on a time variable representing the prediction horizon. Specifically, they transformed the prediction time horizon

**Figure 3.1**: Architecture of the RNN system [6]. The input is a sequence of (length, 1-hot class encoding)-tuples. The network predicts the remaining length of the last observed action and the label and length of the next action. Appending the predicted result to the original input, the next action segment can be predicted. Figure is taken from [6].

to a time representation, and concatenated it with the original inputs forming time-conditioned observations. Their model is therefore capable of anticipating a future action at arbitrary and variable time horizons in a one-shot fashion. Additionally, they introduced a time-conditioned skip connection between the last observed action and the initial anticipation based on the intuition that the last action of the observation is generally relevant to the future actions.

Inspired by [12], Gong et al. [10] proposed an encoder-decoder structure based on transformer architecture [27, 2], which effectively captures long-term relations over the whole sequence of actions. The encoder learns to capture fine-grained long-range temporal relations between the observed frames from the past, while the decoder learns a sequence of future action queries, capturing global relations between upcoming actions in the future along with the observed features from the encoder. Because of the proposed parallel decoding, the model is able to

make more accurate and faster inference without potential error accumulations caused by autoregressive decoding. However, the number of predictable future actions is also limited to the number of action queries used in the training process, which might need to be adapted, if the model is applied for other datasets.

Predicting future is inherently multi-modal. Given an observed video segment containing an ongoing action, multiple actions could be possible to be the next action following the observed one. This uncertainty becomes even larger if we are going to predict far into the future. Therefore, it may be beneficial to model the underlying uncertainty, allowing to capture different possible future actions. However, in most approaches, action prediction is taken as a classification problem and optimized under cross-entropy loss, suffering from overly high resemblance to dominant ground truth, while suppressing other reasonable possibilities [3]. Moreover, approaches that are optimized with mean square error tend to produce the mean of the modes [28, 20]. To this end, some approaches are proposed to tackle the uncertainty in the future predictions, which are described below.

Farha and Gall [1] introduced a framework that predicts all subsequent actions and corresponding durations in a stochastic manner. In their framework, an action model similar to the one proposed in [6] (shown in Figure 3.1) and a time model are trained to predict the probability distribution of the future action label and duration, respectively. While action labels are taken as classifications and optimized under cross-entropy (CE) loss, durations are taken as real-valued variables which are modeled with a Gaussian distribution and optimized with the negative log likelihood (NLL). At test time, future action label and its duration are sampled from the learned distributions. Long-term predictions are achieved by feeding the predicted action segment to the model recursively.

Zhao and Wildes [31] proposed Conditional Adversarial Generative Networks to address the underlying uncertainty when predicting future action sequence. More specifically, different from many works that operate with continuous time variable [6, 1, 12, 10], they treated both action labels and time as discrete data which are formated as one-hot vectors. These vectors are first projected to higher dimension continuous spaces and concatenated, and then fed to a *seq2seq* generator [26] to compute logits of future action labels and their corresponding time. To obtain differentiable sampling to generate future sequences with

both quality and diversity during training, the Gumbel-Softmax relaxation technique [11] that mimics one-hot vectors from categorical distributions and a normalized distance regularizer [30] that encourages diversity are adopted. A ConvNet classifier is used as the discriminator to allow to train the generator adversarially.

Mehrasa et al. [21] proposed using a recurrent variational auto-encoder (VAE [13]) to capture the distribution over the times and categories of action sequences. To overcome the problem that a fixed prior distribution of the latent variable (usually N (0*, I*) in VAE models) may ignore temporal dependencies present between actions, authors learned a prior that varies across time. At test time, a latent code is sampled from the learned prior distribution, based on which the probability distributions of the action class and the corresponding time are inferred.

### **3.2 Limitations**

Despite the impressive performance on the standard benchmarks [25, 16], current approaches have several limitations, which are described below.

**Limited representativity of the evaluation datasets.** The commonly used benchmark datasets for long-term anticipation, i.e., Breakfast [16] and 50Salads [25], contain only videos of a specific kitchen activity, which usually last several minutes. Since there is only one activity per video, i.e., either *preparing a breakfast* or *preparing a salad*, it is easier to predict the following actions than in the real-world scenarios, where a completely different action might occur next. Furthermore, since these videos are typically only several minutes long, the current setting may not be directly applicable for longer videos, especially for real-world applications. Moreover, these datasets do not contain any concurrent actions. However, actions in the real-world scenarios, such as *making a phone call* and *taking notes* may be performed simultaneously.

**Difficult deployment of methods that incorporate uncertainty.** Methods that incorporate uncertainty typically learn a joint distribution of all data samples. For evaluation, authors usually draw many samples from the learned distribution, and compute the average metric value of all drawn samples [1, 31], or select

the most frequent sample as the final result [21]. However, such an evaluation protocol requires multiple runs of the model, which is time-consuming and therefore difficult to deploy for real-time systems.

## **4 Conclusion**

In this survey, we gave a short overview of the current approaches that are proposed to tackle the long-term action anticipation task. We analyzed different methods from two perspectives: research question each individual method addresses and method description. In the end, we also described the limitations of the current approaches. In conclusion, long-term action anticipation is an interesting and relatively new research topic, which attracts increasing attention in the community, and benefits many intelligent decision-making systems. While great strides have been made, there is still large room for improvement in action anticipation using deep learning techniques.

## **References**

### **Karlsruher Schriftenreihe zur Anthropomatik (ISSN 1863-6489)**

Thomas Bader

**Band 9**

#### Philipp Woock **Umgebungskartenschätzung aus Sidescan-Sonardaten für ein autonomes Unterwasserfahrzeug.** ISBN 978-3-7315-0541-9 **Band 26**

Lehrstuhl für Interaktive Echtzeitsysteme Karlsruher Institut für Technologie

Fraunhofer-Institut für Optronik, Systemtechnik und Bildauswertung IOSB Karlsruhe

In 2022, the annual joint workshop of the Fraunhofer Institute of Optronics, System Technologies and Image Exploitation (IOSB) and the Vision and Fusion Laboratory (IES) of the Institute for Anthropomatics, Karlsruhe Institute of Technology (KIT) was hosted at a Schwarzwaldhaus near Triberg. For a week from the 31st of July to the 5th of August the doctoral students of both institutions presented extensive reports on the status of their research and discussed topics ranging from computer vision and optical metrology to network security, usage control and machine learning.

The results and ideas presented at the workshop are collected in this book in the form of detailed technical reports. This volume provides a comprehensive and up-to-date overview of the research program of the IES laboratory and the Fraunhofer IOSB.

ISSN 1863-6489 (Schriftenreihe) ISSN 2510-7259 (Tagungsband) ISBN 978-3-7315-1304-9

J. Beyerer, T. Zander (Eds.)

Proceedings of the 2022 Joint Workshop

Band 62


**2. Physical Models**

## *2.1. Turbulent Flow Model*

The "standard" *k* − *ε* low Reynolds number (LRN) eddy-viscosity turbulence model in STAR-CD was adopted to model the flow and mass transfer [25]. The mass transfer is modelled by solving the mass transport equation together with the hydrodynamic equations, where the turbulent Schmidt number, *Sct* = *μt*/*ρDt*, defines the ratio of turbulent momentum transport to turbulent diffusive transport. In the present study, *Sct* was assigned a value of 0.9. The model is founded on the *k* − *ε* model proposed by Launder and Spalding [26], in which modelling of fully turbulent bulk flow is achieved through the transportation equations of turbulent kinetic energy (TKE), *k*, and its dissipation rate, *ε*. Launder and Spalding model the near-wall region, where the viscous effect is dominant, by way of algebraic wall functions. However, this approach uses a velocity profile that is not applicable to recirculating flow, where there is flow separation and reversal, and does not take the detailed mass transfer characteristics in the diffusion boundary layer into account [27]. It is therefore necessary to use a turbulence model that directly solves the flow field and mass transfer deep inside the viscous sublayer to account for the changes due to the wall. The LRN model does so by solving the transport equations of *k* and *ε* everywhere, including the near-wall regions.

To extend the *k* − *ε* high Reynolds number model to model low Reynolds number conditions, Lien et al. [28] have proposed damping functions that modify the transport equations of *k* and *ε* in the near-wall region. One such is the semi-viscous near-wall effect *fu*, which modifies the eddy-viscosity term *μt*. An additional term for turbulence generation *Pk* , which vanishes as *Rey* approaches the order of 100, is added to ensure that the correct level of TKE dissipation is returned, and the destruction of TKE dissipation is modified with another damping factor *f*2. These are formulated as follows in STAR-CD:

$$P\_k^{\prime} = 1.33 \left[ 1 - 0.3 \varepsilon^{-R\_t^{\prime} 2} \right] \left[ P\_k + 2 \frac{\mu}{\mu\_t} \frac{k}{\mathcal{Y}^2} \right] \varepsilon^{-0.00375 Re\_{\mathcal{Y}}^2},\tag{2}$$

$$f\_{\mu} = \left[1 - e^{-0.0198 R c\_{y}}\right] \left(1 + \frac{5.29}{R c\_{y}}\right) \tag{3}$$

$$f\_2 = 1 - 0.3e^{-Rc\_l^2},\tag{4}$$

where the turbulent Reynolds numbers are *Rey* = *ρy* √*k*/*μ* and *Ret* = *ρk*2/*με*.


**Acknowledgement**

To CAPES for the fellowship to Marques-Marinho FD, to Lima AA and Reis IA for the important initial collaboration.

## **5. References**

<sup>\*</sup> Corresponding Author

[6] da Costa PQ, Rey LC, Coelho HL. Lack of Drug Preparations for Use in Children in Brazil. J. Pediatr. 2009;85(3): 229-35.

Cellulose and Its Derivatives Use in the Pharmaceutical Compounding Practice 159

[25] Wada M, Chanzy H, Nishiyama Y, Langan P. Cellulose IIII Crystal Structure and Hydrogen Bonding by Synchrotron X-ray and Neutron Fiber Diffraction.

[26] Ishikawa A, Okano T, Sugiyama J. Fine Structure and Tensile Properties of Ramie Fibres in the Crystalline Form of Cellulose I, II, Illl and IVI. Polymer 1997;38(2): 463-8. [27] United States Pharmacopeial Convention. The United States Pharmacopeia-National

[28] Béchard SR, Levy L, Clas SD. Thermal, Mechanical and Functional Properties of Cellulose Acetate Phthalate (CAP) Coatings Obtained from Neutralized Aqueous

[29] Prista LN, Alves AC, Morgado R, Lobo JS. Tecnologia Farmacêutica. Lisboa: Fundação

[30] Prista LN, Alves AC, Morgado R. Tecnologia Farmacêutica. Lisboa: Fundação Calouste

[31] Marques-marinho FD, Vianna-soares CD, Carmo VA, Campos LM. Avaliação da Qualidade de Pantoprazol Cápsulas Manipuladas Gastro-Resistentes. Lat. Am. J.

[32] dos Santos L, Guterres SS, Bergold AM. Preparação e Avaliação de Cápsulas Gastro-

[33] Ferreira AO, Holandino C. Pharmaceutical Development of Enteric-Release Hard Gelatin Capsules in the Compounding Setting. Int. J. Pharm. Compound. 2008;12(2):

[34] Webster KD, Al-achi A, Greenwood R. In Vitro Studies on the Release of Morphine Sulfate from Compounded Slow-Release Morphine-Sulfate Capsules. Int. J. Pharm.

[35] Bogner RH, Szwejkowski J, Houston A. Release of Morphine Sulfate from Compounded Slow Release Capsules: the Effect of Formulation on Release. Int. J. Pharm. Compound.

[36] Timmons ED, Timmons SP. Custom-Compounded Micronized Hormones in a Slow-

[37] Glowiak DL, Green JL, Bowman BJ. In Vitro Evaluation of Extemporaneously Compounded Slow-Release Capsules Containing Morphine Sulfate or Oxycodone

[38] Pinheiro VA, Kaneko TM, Velasco MV, Consiglieri VO. Development and In Vitro Evaluation of Extended-Release Theophylline Matrix Capsules. Braz. J. Pharm. Sci.

[39] Ojantakanen S, Marvola M, Hannula AM, Klinge E, Naukkarinnen T. Bioavailability of Ibuprofen from Hard Gelatin Capsules Containing Different Viscosity Grades of Hydroxypropylmethylcellulose and Sodium Carboxymethylcellulose. Eur. J. Pharm.

[40] Ojantakanen S. Effect of Viscosity Grade of Polymer Additive and Compression Force on Dissolution of Ibuprofen from Hard Gelatin Capsules. Acta Pharm. Fennica

Release Capsule Matrix. Int. J. Pharm. Compound. 2002;6(5): 378-9.

Hydrochloride. Int. J. Pharm. Compound. 2005;9(2): 157-64.

Resistentes de Diclofenaco de Sódio. Lat. Am. J. Pharm. 2007;26(3): 355-61.

FormularyRockville: United States Pharmacopeia; 2011. p. 1416-19.

Macromolecules 2004;37: 8548-55.

Solutions. Int. J. Pharm. 1995;114(2): 205-13.

Calouste Gulbekian; 2002. p. 562-7.

Gulbekian; 1995. p. 2027-8.

Pharm. 2009;28(6): 899-906.

Compound. 1999;3(5): 409-11.

163-9.

2001;5(5): 401-5.

2007;43(2): 253-61.

Sci. 1993;1: 109-14.

1992;101(33): 119-26.

[25] Wada M, Chanzy H, Nishiyama Y, Langan P. Cellulose IIII Crystal Structure and Hydrogen Bonding by Synchrotron X-ray and Neutron Fiber Diffraction. Macromolecules 2004;37: 8548-55.

158 Cellulose – Medical, Pharmaceutical and Electronic Applications

Brazil. J. Pediatr. 2009;85(3): 229-35.

Other? Pharm. World Sci. 2007;29: 526-33.

London: Pharmaceutical Press; 2009.

Berlin:Springer-Verlag. 1990; 20-39.

Carbohydr. Res. 1988;180: 345-50.

Concepts. Ann. Bot. 2005;96: 9-21.

Washington: American Pharmacists Association; 2008.

J. Pharm. 2005;300: 56-66.

2112-9.

1999;54: 1-14.

3101-6.

Sci. 2006;9(3): 398-426.

[6] da Costa PQ, Rey LC, Coelho HL. Lack of Drug Preparations for Use in Children in

[7] Nahata MC, Allen LV Jr. Extemporaneous Drug Formulations. Clin. Ther. 2008;30(11):

[8] Standing JF, Tuleu C. Paediatric Formulations- Getting to the Heart of the Problem. Int.

[9] Prot-labarthe S, Bussières JF, Brion F, Bourdon O. Comparison of Hospital Pharmacy Practice in France and Canada: Can Different Practice Perspectives Complement each

[10] Brion F, Nunn AJ, Rieutord A. Extemporaneous (magistral) preparation of oral medicines for children in European hospitals. Acta Paediatr. 2003;92: 486-490. [11] Allen LV Jr. The Art, Science, and Technology of Pharmaceutical Compounding.

[12] Rowe RC, Sheskey PJ, Quinn ME., editors. Handbook of Pharmaceutical Excipients.

[13] Pifferi G, Santoro P, Pedrani M. Quality and functionality of excipients. Il Farmaco

[14] Glass BD, Haywood A. Stability Considerations in Liquid Dosage Forms Extemporaneously Prepared from Commercially Available Products. J. Pharm. Pharm.

[16] Kamel S, Ali N, Jahangir K, Shah SM, El-Gendy AA. Pharmaceutical Significance of

[17] Nobles D, Romanovicz DK, Brown RM Jr. Cellulose in Cyanobacteria. Origin of

[18] Brown RM Jr. Algae as Tools in Studying the Biosynthesis of Cellulose Nature's Most Abundant Macromolecule. In Wiessner G, Robinson DG, Starr RC, editors. Experimental Phycology. Cell Walls and Surfaces, Reproduction, Photosynthesis.

[19] Kimura S, Itoh T. A New Cellulosic Structure, the Tunic Cord in the Ascidian

[20] Brown RM Jr. Cellulose Structure and Biosynthesis: What is in Store for the 21 st

[21] Klemm D, Heublein B, Fink HP, Bohn A. Cellulose: Fascinating Biopolymer and

[22] Weimer PJ, French AD, Calamari TA Jr. Differential Fermentation of Cellulose Allomorphs by Ruminal Cellulolytic Bacteria. Appl. Environ. Microbiol. 1991;57(11):

[23] Kuga S, Brown RM Jr. Silver Labeling of the Reducing Ends of Bacterial Cellulose.

[24] Saxena IM, Brown RM Jr. Cellulose Biosynthesis: Current Views and Evolving

[15] Edgar K J. Cellulose Esters in Drug Delivery. Cellulose 2007;14(1): 49-64.

Cellulose- A Review. eXPRESS Polymer Letters 2008;2(11): 758-78.

Vascular Plant Cellulose Synthase? Plant Physiol. 2001;127: 529-42.

Polyandrocarpa misakiensis. Protoplasma 1998;204: 94-102.

Sustainable Raw Material. Angew. Chem. Int. Ed. 2005;44: 3358-93.

Century? J. Polym. Sci. A Polym. Chem. 2004;42: 487-95.

	- [41] United States Pharmacopeial Convention. USP pharmacists´ pharmacopeia Rockville: United States Pharmacopeia; 2008. p. 333.

Cellulose and Its Derivatives Use in the Pharmaceutical Compounding Practice 161

[58] Nahata MC, Morosco RS, Trowbridge JM. Stability of Dapsone in Two Oral Liquid

[59] Johnson CE, Wagner DS, Bussard WE. Stability of Dolasetron in Two Oral Liquid

[60] Dentinger PJ, Swenson CF, Anaizi NH. Stability of Famotidine in an Extemporaneously

[61] Allen LV Jr, Erickson MA 3rd. Stability of Ketoconazole, Metolazone, Metronidazole, Procainamide Hydrochloride and Spironolactone in Extemporaneously Compounded

[62] Allen LV Jr, Erickson MA 3rd. Stability of Labetalol Hydrochloride, Metoprolol Tartrate, Verapamil Hydrochloride, and Spironolactone with Hydrochlorothiazide in Extemporaneously Compounded Oral Liquids. Am. J. Health Syst. Pharm. 1996;53(19):

[63] Nahata MC, Morosco RS, Hipple TF. Stability of Lamotrigine in Two Extemporaneously Prepared Oral Suspensions at 4 and 25 Degrees C. Am. J. Health Syst. Pharm.

[64] Nahata MC, Morosco RS, Leguire LE. Development of Two Stable Oral Suspensions of Levodopa-Carbidopa for Children with Amblyopia. J. Pediatr. Ophthalmol. Strabismus.

[65] VandenBussche HL, Johnson CE, Fontana EM, Meram JM. Stability of Levofloxacin in an Extemporaneously Compounded Oral Liquid. Am. J. Health Syst. Pharm. 1999;

[66] Hutchinson DJ, Johnson CE, Klein KC. Stability of Extemporaneously Prepared

[68] Zhang YP, Trissel LA, Fox JL. Naratriptan Hydrochloride in Extemporaneously

[69] Johnson CE, Price J, Hession JM. Stability of Norfloxacin in an Extemporaneously

[70] Haslam JL, Egodage KL, Chen Y, Rajewski RA, Stella V. Stability of Rifabutin in Two Extemporaneously Compounded Oral Liquids. Am. J. Health Syst. Pharm. 1999;56(4):

[71] Nahata MC, Morosco RS, Hipple TF. Stability of Spironolactone in an Extemporaneously Prepared Suspension at Two Temperatures. Ann. Pharmacother.

[72] Navid F, Christensen R, Minkin P, Stewart CF, Furman WL, Baker S. Stability of

[73] Jacobson PA, Johnson CE, West NJ, Foster JA. Stability of Tacrolimus in an Extemporaneously Compounded Oral Liquid. Am. J. Health Syst. Pharm. 1997;54(2):

Moxifloxacin Oral Suspensions. Am. J. Health Syst. Pharm. 2009;66(7): 665-7. [67] Ensom MH. Stability of Mycophenolate Mofetil in a 1:1 Mixture of Ora-Sweet and Ora-

Compounded Oral Suspensions. Int. J. Pharm. Compound. 2000;4(1): 69-71.

Prepared Oral Liquid. Am. J. Health Syst. Pharm. 2001;58: 577-9.

Sunitinib in Oral Suspension. Ann. Pharmacother. 2008;42(7): 962-6.

Compounded Oral Liquid. Am. J. Health Syst. Pharm. 2000;57(14): 1340-2.

Dosage Forms. Ann. Pharmacother. 2000;34(7): 848-50.

Vehicles. Am. J. Health Syst. Pharm. 2003;60(21): 2242-4.

Oral Liquids. Am. J. Health Syst. Pharm. 1996;53(17): 2073-8.

2304-9.

1999;56(3): 240-2.

2000;37(6): 333-7.

56(22): 2316-8.

333-6.

178-80.

1993; 27(10): 1198-9.

Plus. Can. J. Hosp. Pharm. 2002;55(1): 63-5.

[58] Nahata MC, Morosco RS, Trowbridge JM. Stability of Dapsone in Two Oral Liquid Dosage Forms. Ann. Pharmacother. 2000;34(7): 848-50.

160 Cellulose – Medical, Pharmaceutical and Electronic Applications

United States Pharmacopeia; 2008. p. 333.

Compound. 1997;6(5): 378-9.

Neurol. 1999;2(3): 195-7.

Pharm. 2000;57(12): 1141-3.

Syst. Pharm. 2006;63(3): 254-7.

Pharm. Compound. 2005;9(5): 402-6.

Health Syst. Pharm. 2003;60(1): 75-7.

Health Syst. Pharm. 1996;53(16): 1944-9.

Am. J. Health Syst. Pharm. 1995;52(14): 1558-60.

Temperatures. Ann. Pharmacother. 2003;37(4): 506-9.

1720-4.

[41] United States Pharmacopeial Convention. USP pharmacists´ pharmacopeia Rockville:

[42] Roy JJ, Besner JG. Stability of Clonazepam Suspension in HSC Vehicle. Int. J. Pharm.

[43] Nahata MC. Development of Two Stable Oral Suspensions for Gabapentin. Pediatr.

[44] Nahata MC, Morosco RS, Willhite, EA. Stability of Nifedipine in Two Oral Suspensions

[45] Metras JI, Swenson CF, Mcdermott MP. Stability of Procainamide Hydrochloride in an Extemporaneously Compounded Oral Liquid. Am. J. Health Syst. Pharm. 1992;49(7):

[46] Nahata MC, Morosco RS, Trowbridge JM. Stability of Propylthiouracil in Extemporaneously Prepared Oral Suspensions at 4 and 25 °C. Am. J. Health Syst.

[47] Nahata MC, Morosco RS, Peritore SP. Stability of Pyrazinamide in Two Suspensions.

[48] Nahata MC, Morosco RS, Hipple TF. Stability of Pyrimethamine in a Liquid Dosage Formulation Stored for Three Months. Am. J. Health Syst. Pharm. 1997;54(23): 2714-6. [49] Nahata MC, Morosco RS, Brady MT. Extemporaneous Sildenafil Citrate Oral Suspensions for the Treatment of Pulmonary Hypertension in Children. Am. J. Health

[50] Sidhom MB, Rivera N, Almoazen H, Taft DR, Kirschenbaum HL. Stability of Sotalol Hydrochloride in Extemporaneously Prepared Oral Suspension Formulations. Int. J.

[51] Nahata MC, Morosco RS. Stability of Sotalol in Two Liquid Formulations at Two

[52] Nahata MC, Morosco RS. Stability of Tiagabine in Two Oral Liquid Vehicles. Am. J.

[53] Alexander KS, Haribhakti RP, Parker GA. Stability of Acetazolamide in Suspension

[54] Allen LV Jr, Erickson MA 3rd. Stability of Acetazolamide, Allopurinol, Azathioprine, Clonazepam, and Flucytosine in Extemporaneously Compounded Oral Liquids. Am. J.

[55] Allen LV Jr, Erickson MA 3rd. Stability of Alprazolam, Chloroquine Phosphate, Cisapride, Enalapril Maleate, and Hydralazine Hydrochloride in Extemporaneously

[56] Allen LV Jr, Erickson MA 3rd. Stability of Baclofen, Captopril, Diltiazem Hydrochloride, Dipyridamole, and Flecainide Acetate in Extemporaneously

[57] Allen LV Jr, Erickson MA 3rd. Stability of Bethanechol Chloride, Pyrazinamide, Quinidine Sulfate, Rifampin, and Tetracycline Hydrochloride in Extemporaneously

Compounded from Tablets. Am. J. Hosp. Pharm. 1991;48(6): 1241-4.

Compounded Oral Liquids. Am. J. Health Syst. Pharm. 1998;55: 1915-20.

Compounded Oral Liquids. Am. J. Health Syst. Pharm. 1996;53(18): 2179-84.

Compounded Oral Liquids. Am. J. Health Syst. Pharm. 1998;55(17), 1804-9.

Stored at Two Temperatures. J. Am. Pharm. Assoc. 2002; 42(6): 865-7.

	- [74] Han J, Beeton A, Long PF, Wong I, Tuleu C. Physical and Microbiological Stability of an Extemporaneous Tacrolimus Suspension for Paediatric Use. J. Clin. Pharm. Ther. 2006;31(2): 167-72.

**Chapter 9**

© 2013 Kleinschek et al., licensee InTech. This is an open access chapter distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

© 2013 Kleinschek et al., licensee InTech. This is a paper distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

**Plasma Induced Hydrophilic Cellulose Wound**

The wound care industry is a highly diverse and competitive arena - including standard products such as dry bandages to sophisticated hydrogels and alginate dressings, and encompassing artificial skin and anti-infective used in wound care. A broad range of available modern dressings composed of layers possessing specific functionality has been placed on the market. Waddings and gauzes were replaced by new natural, synthetic and semi-synthetic biomaterials, which depending on their physical, chemical and technological properties are capable to absorb the exudate excess by maintaining the proper moisture environment on/in the wound. The use of textiles in medicine has a long tradition; specially

Cellulose and its derivatives are very often used as a functional part of different wound dressing material e.g. as hydrogels, fibre, non-woven. Cellulose micro-fibrillar structure control the accessibility of the surface, while the bulk polar groups (-OH groups) ensure an attraction of the water molecule to the cellulose fibre (hydrogen bonds). Generally different standard chemical pre-treatments (i.e. alkaline washing, bleaching, and slack-mercerization) are used to improve the sorption capacity [1-4], but besides the ecological pollution, they also worsen fibres mechanical properties to a large extent. On the subject of the latter, plasma treatments are known to be rather harmless surface activation procedures leading to enhancement of hydrophilic properties; the best results are obtained using highly nonequilibrium oxygen plasma. Reactive particles created in plasma readily interact with cellulose resulting in formation of surface functional groups, as well as etching. The properties of plasma treated materials depend on the processing parameters and are measured using surface sensitive techniques. Numerous authors worldwide addressed modification of cellulose and other suitable materials by gaseous plasma treatment [5-16].

Zdenka Persin, Miran Mozetic, Alenka Vesel, Tina Maver,

Uros Maver and Karin Stana Kleinschek

Additional information is available at the end of the chapter

**Dressing**

http://dx.doi.org/10.5772/55867

woven textiles are mostly used.

**1. Introduction**


**Vendor Parameters**

#### **Appendix B. Production Cycle Time and Inventory Level Calculations**

#### *Appendix B.1. Cycle Time Calculation*

The objective of the research is to minimize the total cost of production, and the formulation of the cycle time of the production system is prerequisite to calculate the total cost (TC). Cycle time is taken as a decision variable in the production model, which is dependent upon the production rates of manufacturing system. The production rate of the manufacturing firm and vendor are relying on the production rate of the machines (capital units). In order to meet the customer demand and no shortages in the production system, the production rates of the manufacturing system is di fferent in the manufacturing before outsourcing, during outsourcing and after outsourcing operations are

denoted as *Pja*, *Pjb*, and *Pjc* respectively. The total production cycle time of the item is the sum of the time utilized in production, outsourcing, reworking and delivery as expressed as in Equation (A1).

$$T\_j = t\_{1j} + t\_{2j} + t\_{3j} + t\_{4j} + t\_{5j} \tag{A1}$$

where, the individual time fractions of the total cycle time are mathematically formulated as given from Equations (A2) to (A6).

$$t\_{1\dot{j}} = \frac{Q\_{\dot{j}}}{P\_{j\dot{a}}} \tag{A2}$$

$$t\_{2j} = \frac{Q\_j}{P\_{jb}}\tag{A3}$$

$$t\_{3j} = \frac{Q\_j \alpha\_j}{P\_{jb}} \tag{A4}$$

$$t\_{4j} = \frac{Q\_j}{P\_{j\bar{c}}} \tag{A5}$$

$$t\_{\mathfrak{H}j} = \frac{I\_{m\mathbf{x}\mathbf{x}}(j\mathbf{c})}{D\_j} \tag{A6}$$

To find the value of *Imax*(*jc*) from the inventory diagram, the formula of slope in the *Area*10,11,12 can be expressed as in Equation (A7).

$$
tan\theta\_4 = P\_{j\text{c}} - D\_j \Rightarrow \frac{I\_{\text{max}(j\text{c})}}{t\_{4j}}\tag{A7}$$

$$I\_{\text{max}(j\varepsilon)} = (P\_{j\varepsilon} - D\_j) \frac{Q\_j}{P\_{j\varepsilon}} = Q\_j (1 - \frac{D\_j}{P\_{j\varepsilon}}) \tag{A8}$$

By putting the value of *Imax*(*jc*), the simplified form of Equation (A6) can be expressed as in Equation (A10).

$$t\_{5j} = \frac{Q\_j(1 - \frac{D\_j}{P\_{j\epsilon}})}{D\_j} \tag{A9}$$

$$t\_{\mathbb{S}j} = Q\_j(\frac{1}{D\_j} - \frac{1}{P\_{j\mathbb{C}}}) \tag{A10}$$

After calculating all the time fractions (*t*1*j* to *t*5*j*) for the production of automobile parts, the mathematical form of the total cycle time can be given as in Equation (A11).

$$T\_j = \frac{Q\_j}{P\_{ja}} + \frac{Q\_j}{P\_{jb}} + \frac{Q\_j \alpha\_j}{P\_{jb}} + \frac{Q\_j}{P\_{jc}} + Q\_j(\frac{1}{D\_j} - \frac{1}{P\_{jc}}) \tag{A11}$$

#### *Appendix B.2. Average Inventory Costs*

The inventory is obtained by calculating the area under the curve of the production, storage, and buffer quantity in complete manufacturing process. As followed the inventory diagram given as in Figure 2, the inventories can be divided into the areas i.e., *Area*123, *Area*456, *Area*5678, *Area*10,11,12, and *Area*11,12,13. These areas are formulated separately from Equation (A12) to Equation (A26) respectively.

$$Area\_{123} = \frac{1}{2} t\_{1\circ} I\_{\text{max}(\text{ja})} \tag{A12}$$

$$
tau\Theta\_1 = \frac{I\_{\max(ja)}}{t\_{1j}} P\_{ja} \times t\_{1j} = I\_{\max(ja)} I\_{\max(ja)} = Q\_j \tag{A13}
$$

By putting *Imax*(*ja*), Equation (A12) can be written as in Equation (A14).

$$Area\_{123} = \frac{Q\_j^2}{2P\_{j\bar{\mu}}} \tag{A14}$$

Similarly, the area of triangle *456* can be given as in Equation (A15)

$$Area\_{456} = \frac{1}{2} I\_{\max(\text{jib})} t\_{2\text{j}} \tag{A15}$$

The maximum inventory of the vendor *Imax*(*jbb*) is calculated by the slope formula of *Area*456 (Equation (A16)), which is then simplified to express in the form of Equation (A17)

$$
tan\Theta\_2 = \frac{I\_{\max(jlb)}}{t\_{2j}}\tag{A16}
$$

$$P\_{jb}(1 - \alpha\_{j}) \times \frac{Q\_{j}}{P\_{jb}} = I\_{\max(jbb)} I\_{\max(jbb)} = Q\_{j}(1 - \alpha\_{j}) \tag{A17}$$

After substituting, the *Area*456 can be written as in Equation (A18)

$$Area\_{456} = \frac{1}{2} Q\_j (1 - \alpha\_j) \frac{Q\_j}{P\_{jb}} Area\_{456} = \frac{Q\_j^2 (1 - \alpha\_j)}{2P\_{jb}} \tag{A18}$$

The simplified form of area under the curve of the rectangular region *5678* is formulated and expressed as in Equation (A19)

$$Area\_{5678} = I\_{\max(\text{jib})} t\_{\text{j}} = Q\_{\dot{f}} (1 - \alpha\_{\dot{f}}) \frac{Q\_{\dot{f}} \alpha\_{\dot{f}}}{P\_{\dot{j}\dot{b}}} = \frac{\alpha\_{\dot{f}} (1 - \alpha\_{\dot{f}}) Q\_{\dot{f}}^2}{P\_{\dot{j}\dot{b}}} \tag{A19}$$

The triangular area of the inventory storage *Area*689 by reworking operations during outsourcing operation is expressed as in Equation (A20).

$$Area\_{689} = \frac{1}{2} I\_{\max(j\text{ha})} t\_{3j} \tag{A20}$$

where, to find the maximum inventory *Imax*(*jba*) of the reworking operation in the production system, the slope formula is given as in Equation (A21),

$$
tan\theta\_3 = \frac{I\_{\text{max}(j\text{ha})}}{t\_{3j}} P\_{j\text{b}}(1 - a\_j \beta\_j) \frac{Q\_j a\_j}{P\_{j\text{b}}} = I\_{\text{max}(j\text{ha})} I\_{\text{max}(j\text{ha})} = a\_j Q\_j (1 - a\_j \beta\_j) \tag{A21}
$$

By substitution, Equation (A20) can be written in the form of Equation (A22).

$$Area\_{689} = \frac{1}{2} \frac{Q\_{\dot{\jmath}} \alpha\_{\dot{\jmath}}}{P\_{\dot{\jmath}b}} \alpha\_{\dot{\jmath}} Q\_{\dot{\jmath}} (1 - \alpha\_{\dot{\jmath}} \beta\_{\dot{\jmath}}) = \frac{\alpha\_{\dot{\jmath}}^2 Q\_{\dot{\jmath}}^2 (1 - \alpha\_{\dot{\jmath}} \beta\_{\dot{\jmath}})}{2P\_{\dot{\jmath}b}} \tag{A22}$$

Likewise, the *Area*10,11,12 and *Area*11,12,13 can be formulated and expressed in simplified form as in Equations (A25) and (A26) respectively, where the maximum inventory is *Imax*(*jc*).

$$Area\_{10,11,12} = \frac{1}{2} I\_{\max(jc)} t\_{4j} \tag{A23}$$

$$
tan\theta\_4 = P\_{j\text{c}} - D\_j \Rightarrow \frac{I\_{\text{max}(j\text{c})}}{t\_{4j}} I\_{\text{max}(j\text{c})} = (P\_{j\text{c}} - D\_j) \frac{Q\_j}{P\_{j\text{c}}} \tag{A24}
$$

$$Area\_{10,11,12} = \frac{1}{2} Q\_j (1 - \frac{D\_j}{P\_{j\text{c}}}) \frac{Q\_j}{P\_{j\text{c}}} \tag{A25}$$

$$Area\_{11,12,13} = \frac{1}{2} I\_{\max(\text{jc})} t\_{5\text{j}} = \frac{1}{2} \frac{I\_{\max(\text{jc})}^2}{D\_j} = \frac{Q\_j^2}{2D\_j} (1 - \frac{D\_j}{P\_{j\text{c}}})^2 \tag{A26}$$