# Leveraging Visemes for Better Visual Speech Representation and Lip Reading

1st Javad Peymanfard  Iran University of Science and Technology
<EMAIL_ADDRESS>2nd Vahid Saeedi  Iran University of Science
and Technology
<EMAIL_ADDRESS>3rd Mohammad Reza Mohammadi  Iran University
of Science and Technology
<EMAIL_ADDRESS>4th Hossein Zeinali  Amirkabir University of
Technology
<EMAIL_ADDRESS>5th Nasser Mozayani  Iran University of Science and
Technology
<EMAIL_ADDRESS>
###### Abstract

Lip reading is a challenging task that has many potential applications in
speech recognition, human-computer interaction, and security systems. However,
existing lip reading systems often suffer from low accuracy due to the
limitations of video features. In this paper, we propose a novel approach that
leverages visemes, which are groups of phonetically similar lip shapes, to
extract more discriminative and robust video features for lip reading. We
evaluate our approach on various tasks, including word-level and sentence-
level lip reading, and audio-visual speech recognition using the Arman-AV
dataset, a large-scale Persian corpus. Our experimental results show that our
viseme based approach consistently outperforms the state-of-the-art methods in
all these tasks. The proposed method reduces the lip-reading word error rate
(WER) by 9.1% relative to the best previous method.

###### Index Terms:

lip reading, visual speech recognition, audio-visual speech recognition,
visual speech representation

##  I Introduction

Lip reading is a challenging task that requires deep learning models to
achieve high accuracy. These models have surpassed traditional methods such as
Hidden Markov models, but they need large-scale labeled datasets that cover
the application domain. In lip reading, some methods directly extract the text
from the input video with a single model, while others use a two-step approach
[1, 2]. The two-step lip reading consists of extracting visemes from lip
movements in the first step, and converting visemes into characters in the
second step. Visemes are the visual equivalent of phonemes, but they do not
have a one-to-one correspondence with phonemes. For example, the words "back"
and "pack" have different phonemes, but the same lip movements, because the
phonemes /b/ and /p/ correspond to one viseme. This issue makes lip reading,
also known as visual speech recognition (VSR), more difficult than normal
speech recognition.

Lip reading problems can be divided into two levels: word level and sentence
level. At the word level, the problem is a classification task, where each
visual speech sample belongs to a class labeled by the spoken word. However,
as mentioned earlier, some different phonemes have the same visemes, which
creates ambiguity and limits lip reading accuracy. For example, if several
words correspond to one sequence of visemes, and the model predicts the most
frequent word, the accuracy will reach the highest possible level (assuming
that the model makes no mistake).

In sentence-level lip reading, we can use text processing methods such as
language modeling to select the best candidate word (among the words that
share the same viseme sequence) that fits the phrase context. Paper [2]
proposes a two-step method that converts visemes into characters using a
sequence-to-sequence model with an attention mechanism.

Speech recognition can be improved by using a combination of audio and visual
inputs. AV-HuBERT [3] is a well-known model in the field of audio-visual
speech recognition, which is trained using a self-supervised learning method.
AV-HuBERT recognizes the sentence spoken in the video at the character level
by capturing both the video and the sound. The model is trained in two stages:
first, it learns a general representation of the problem space that is not
specific to any language or task; second, it fine-tunes the representation for
the target language and task.

In this research, we used the pre-trained AV-HuBERT model and fine-tuned it
with a novel approach. For the audio part, we followed the same procedure as
AV-HuBERT; but for the visual part, we trained the model with the
corresponding visemes. With this idea, we were able to achieve higher accuracy
than the original fine-tuning process. We evaluate our method on both single-
modal (VSR) and multi-modal audio-visual speech recognition (AVSR) tasks and
show that it outperforms the original AV-HuBERT method in both cases. We also
demonstrate that our method can recognize the spoken words from the lip
movements of the speaker without using any audio data.

The rest of the paper is organized as follows. First, in the related work
section, we review the existing research in the lip reading field and follow
it by describing our approach for fine-tuning AV-HuBERT with visemes in the
proposed method section. Then, we compare our method with similar works on
various tasks in the experiments section. Finally, in conclusion, we summarize
the main contributions and findings of this paper.

##  II Related Works

Lip reading problems can be divided into two levels: word-level and sentence-
level. In the word-level category, methods consider the problem as a
classification task, where the input videos are padded to achieve the same
number of frames. The model takes the video frames as input and predicts the
corresponding word class as output. On the other hand, in the sentence-level
category, the video samples contain multiple words, and the model outputs the
predicted sentence as a sequence of words.

Lip reading methods can be classified into two types: one-step and two-step.
The one-step methods predict the text directly from the input video, while the
two-step methods use an intermediate representation of visemes. Paper [1]
proposed a two-step lip reading method using Hidden Markov Models (HMM). It
uses two separate HMM models: one to extract visemes from the input video, and
another to convert visemes into characters.

The authors of [4] proposed a deep learning model for lip reading called WLAS.
This model is the first one developed for in-the-wild conditions and is used
for sentence-level problems. The WLAS model takes both video and audio inputs.
It consists of three components: Watch (image sequence encoder), Listen (audio
sequence encoder), and Spell (decoder). The input data is fed to the Watch and
Listen components in separate time units. The Watch consists of a VGG [5]
feature extraction network and a long short-term memory (LSTM) network to
preserve the semantic relevance of the input sequence. Listen is similar to
Watch, except that the input of the LSTM is the Mel-frequency cepstral
coefficient (MFCC) features of the audio. The encoded features are then passed
to Spell which consists of an LSTM module, two separate attention modules (one
for image and one for sound), and a multilayer perceptron (MLP) module. The
output of the MLP goes to Softmax, and a character is predicted for the
current time unit. This model is trained on the LRS3 dataset [6] and achieves
better performance than its previous works.

The authors of [7] introduced a hybrid Connectionist temporal classification
(CTC) with attention architecture for decoding the audio-visual encoded
embeddings. The encoder architecture consists of BiLSTM layers that take
convolutional neural network (CNN) features for the visual part and Mel-
spectrogram features for the audio part. The decoder architecture consists of
CTC loss and attention modules. The paper explores two ways of combining audio
and visual features: early fusion and late fusion. In early fusion, the
encoder outputs one audio-visual embedding; in late fusion, the encoder
outputs two separate embeddings for audio and image sequences. The paper shows
that early fusion performs better than late one.

The paper [8] compares two different loss functions for sentence-level audio-
visual speech recognition: CTC loss and sequence-to-sequence loss. It proposes
two different decoder architectures based on self-attention transformers, one
for each loss function. The encoder architecture is the same for both models.
It shows that the sequence-to-sequence model achieves better results than the
CTC model for both audio-visual and visual speech recognition tasks.

AV-HuBERT [3] is one of the state-of-the-art methods for lip reading, which is
an end-to-end method for audio-visual speech recognition. This model takes
both the audio and visual features of the speaker as input and outputs the
corresponding text. AV-HuBERT is based on BERT [9] and trains in a self-
supervised way. In this model, the input video is split into several equal
time intervals; then, the audio signal is processed by the MFCC network, and
the video sequence is processed by the ResNet network with 3D convolution. The
features of the audio and video hidden layers are combined and fed to the
Transformer model. The model is first pre-trained with a large amount of
unlabeled data, then fine-tuned with labeled samples. This improves the
performance of the model significantly. For pre-training, similar to BERT,
some input units are masked and the model is trained to predict the cluster of
each masked unit. After that, the model is finetuned in a supervised way with
the labeled part of the dataset.

The paper [10] proposed a knowledge distillation method for training lip-
reading models. This method uses an ASR model as a teacher and a VSR model as
a student. The teacher is a pre-trained model, and the student learns from the
teacher's outputs. In this paper, the cross-entropy loss function is used to
measure the error between the outputs of the VSR and ASR models, and the CTC
loss function is used to measure the error between the text outputs of the VSR
and ASR models.

In [2], a deep learning model was proposed for lip reading that uses a two-
step approach. This model uses a 3D convolutional network to extract visemes
from the lip movements in the video. Then, it uses a sequence-to-sequence
model composed of a two-layer gated recurrent unit (GRU) network with an
attention mechanism to convert the viseme sequence into a character sequence.
Since some visemes can correspond to multiple phonemes, the attention
mechanism can help to recognize the correct phoneme by relating the current
and previous positions of the sequence.

There are various datasets available for lip reading. The LRW dataset [11] is
one of the first datasets collected in natural conditions and on a large
scale. It is a word-level dataset in English and consists of clips from UK BBC
TV programs. The LRW-1000 dataset [12] is another large-scale dataset for the
Chinese language. The paper [13] proposed a word-level dataset for the Persian
language, which contains 30 hours of clips from the Aparat
website111https://www.aparat.com/ and more than 500 words. According to [13],
the dataset is collected under different audio and light conditions to cover
various situations in real conditions.

##  III Proposed method

###  III-A Challenging Issue

AV-HuBERT is a general model for audio and video representation. It uses a
self-supervised technique to train the network with unlabeled data. This stage
is called pre-training. At the end of this stage, the model can produce a
suitable representation of the audio and visual information in a video. This
information can include both the speaker and speech features. After this
stage, the model needs to be fine-tuned for a specific task. For example, for
recognition problems, such as visual speech recognition or lip reading, the
information about the speaker is not relevant, and the speech in the video is
the target. The AV-HuBERT paper conducts experiments on the largest available
dataset for lip reading, called LRS3. These experiments involve fine-tuning
the pre-trained model with different proportions of LRS3 labeled data.

###  III-B Main Idea

Since the viseme is the smallest meaningful unit in lip reading, our proposed
technique is to use visemes to fine-tune AV-HuBERT. When we train the network
for character-level lip reading, it learns not only the various lip movements
of different speakers but also the language information from the dataset. We
think that when we train a video representation model for lip reading, using a
loss function based on character prediction, the representation implicitly
contains language information from the dataset. Therefore, our proposed method
is to train the network using the sequence transformation of characters to
visemes and a loss function based on that. This method is expected to produce
representations that have better information about lip movements, as well as
higher generalization on other datasets, especially for samples with different
languages.

In [2], it has been shown that the conversion of visemes to characters can be
done with high accuracy by using a large amount of textual data. Therefore,
the main challenge for lip reading is to convert video to visemes, which can
only be done by using labeled video data. In our proposed technique, we first
obtain the phoneme sequence for each video using a semi-automatic technique
and then transform it into a viseme sequence using a phoneme-to-viseme
mapping. Then, we use these data to fine-tune the AV-HuBERT model. We expect
that the video representation obtained by the model will be more suitable for
lip reading than when the fine-tuning is done at the character level.

To compare these approaches, we can apply the fine-tuned models to lip reading
problems in another language and compare the resulting accuracies. We have two
models: one fine-tuned at the character level and the other at the viseme
level. In general, we expect that a better video representation for lip
reading will lead to a lower error rate. Therefore, by freezing the part of
the model that is related to the representation and training the other layers
of the lip reading network, we can obtain a more accurate model that uses
better embedding. We can also use these two models for video representation in
audio-visual speech recognition and compare their performance.

##  IV Experiment and Results

###  IV-A Metrics

In this section, we evaluate our proposed method, fine-tuning AV-HuBERT with
visemes, by comparing it with the conventional fine-tuning method that uses
the loss based on the final characters. All the experiments are done on
Persian datasets for both word and sentence levels. We use different metrics
for the evaluation: accuracy for the word-level task, and two error rates for
the sentence-level tasks.

Accuracy is a metric that is used for classification problems. It measures the
percentage of correct predictions out of all predictions. Since word-level lip
reading is a classification task, we use accuracy to evaluate our method in
this problem. We also calculate top-3, top-5, and top-10 accuracy for this
problem. Top-N means that the model output is considered correct if the
expected class is among the N highest probabilities.

In sentence-level problems, the input expressions can have different lengths
and varieties. Therefore, we use Character Error Rate (CER) and Word Error
Rate (WER) metrics that measure the error rate by comparing the ground truth
and predicted word sequences. The CER metric algorithm transforms the
predicted sequence into the ground-truth one by using the minimum number of
deletions, insertions, and substitutions of characters. The CER value is
computed by (1).

$CER=((i_{c}+d_{c}+s_{c})/n_{c})*100$ (1)

WER is another metric for comparing two sequences. WER counts the minimum
number of word changes needed to obtain the ground-truth expression. In the
same way as CER, it uses insertion, deletion, and substitution of words. WER
is computed by (2).

$WER=((i_{w}+d_{w}+s_{w})/n_{w})*100$ (2)

###  IV-B Datasets

In this section, we give a brief overview of the datasets that we use to
train, fine-tune and evaluate the speech recognition model with our method.

LRS2 [4] is a lip-reading dataset collected from BBC channel. It is a public
and large-scale in-the-wild dataset that has been used as a benchmark in
several researches [14, 15, 16]. We use this dataset to fine-tune the AV-
HuBERT model. Note that the AV-HuBERT-base model has trained with the LRS3 [6]
and VoxCeleb2 [17] datasets.

Arman-AV[18] is a Persian dataset for audio-visual speech recognition. It is a
large-scale in-the-wild dataset that contains over 220 hours of speech
presented by 1760 celebrities. The dataset is collected from programs on the
Aparat website. It also contains the speaker name as metadata, which makes it
suitable for the speaker recognition task. We used the training set of Arman-
AV for fine-tuning the model. We also evaluated all our methods on the test
portion of this dataset.

Common Voice [19] is an audio-only speech dataset. It is a multi-lingual
dataset that contains many languages (English, Chinese, Persian, etc). Mozilla
created this dataset by collecting speech data using crowdsourcing. The used
version of the dataset contains 2383 hours of speech in English and 352 hours
in Persian. We used this dataset to train the audio part of the AVSR model.

###  IV-C Word-level Visual Speech Recognition

One of the challenges in lip reading is doing it at the word-level because we
cannot use any cue from the surrounding words. In this task, the goal is to
recognize the word uttered by the speaker from a short video clip (usually 100
milliseconds to 2 seconds) by extracting relevant visual information from the
video frames. The words belong to a predefined vocabulary. This task can be
considered as a video classification problem. There are suitable datasets for
this task in different languages such as English, Chinese (Mandarin), and
Persian. In this part, we evaluate and compare the quality of the
representations obtained by our proposed method using the Persian dataset. The
reason for not using English data, which is the most common data in this task,
is to ensure a fair comparison of the two methods. In fact, in this part, we
also examine and analyze the generalization ability of the representations
learned by the two methods.

As shown in Table II, the model trained with our proposed method achieves
higher accuracy on the test data. Other metrics for evaluating the quality of
word-level lip reading, such as top-k accuracy, also indicate that the
performance of our proposed model is better than the other models. These
experiments demonstrate that the representation learned by the proposed model
is more suitable for word-level lip reading.

Figure 1: The architecture used for the speech recognition task.

###  IV-D Sentence-level Visual Speech Recognition

The architecture of our method is shown in Figure 1. A sequence of frames
containing the speaker's lip is given to the single visual HuBERT modal, which
outputs the visual feature vector. The visual features are fed into a decoder,
consisting of a linear layer and a BiLSTM module, to obtain the hidden state
vectors. The hidden states are passed through the decoder part, composed of
two LSTM modules and an attention module, to output the probability vectors
that are fed to a softmax layer to get the output visemes.

We evaluated our method on the Arman-AV dataset [18] for sentence-level speech
recognition. The results in Table I show that fine-tuning the visual speech
recognition model with visemes can reduce the uncertainty of the model,
especially when different characters have similar lip movements. In fact, by
using our proposed model, we obtained more suitable visual features for the
lip reading task. We believe this is because the trained model has learned
less language-specific information from the dataset (here English) and
therefore has been able to extract more general features for other languages
(Persian). According to Table I both CER and WER are significantly decreased
by the proposed method.

TABLE I: Sentence-level Visual Speech Recognition Error Rates in Percentage | CER | WER
---|---|---
Fine-tuned by Characters | 49.54 | 77.22
Fine-tuned by Visemes | 44.08 | 70.19

###  IV-E Audio-visual Speech Recognition

In this section, we tackled the problem of audio-visual speech recognition. As
we explained, this problem involves recognizing the speaker's speech using
both audio and visual features. In this experiment, we used the model proposed
in [18]. This model takes two separate audio and visual inputs and extracts
the corresponding features using the trained AV-HuBERT model in the first
step. We used the same pre-trained model for the audio part in both
experiments. To ensure a fair comparison of the two trained models, we used an
independent model for the audio part to isolate the effect of the two models.
For the visual part, we used our proposed model and the model trained at the
character level.

The first two rows of Table III show the comparison results of our proposed
strategy with the baseline that uses characters in the fine-tuning stage. It
is evident that our proposed method outperforms the baseline in both metrics.
We also compared the results for two common decoding methods: greedy and beam
search. As expected, the beam search achieves better results based on both CER
and WER.

TABLE II: Word-level Visual Speech Recognition Accuracy in Percentage | Accuracy | Top3 Acc. | Top5 Acc. | Top10 Acc.
---|---|---|---|---
Baseline Model [13] | 24.79 | 38.79 | 45.47 | 55.03
Fine-tuned by Characters | 23.66 | 38.14 | 47.23 | 54.31
Fine-tuned by Visemes | 25.90 | 39.64 | 47.72 | 56.21
TABLE III: Audio-Visual Speech Recognition Error Rates in Percentage. Greedy and Beam indicate the used decoding strategies. The external dataset also indicates the situation in that we used an external audio dataset in the training (see Section IV-F). | CER | CER | WER | WER
---|---|---|---|---
Decoding Method | Greedy | Beam | Greedy | Beam
Fine-tuned by Characters | 14.56 | 13.84 | 46.63 | 45.12
Fine-tuned by Visemes | 13.56 | 12.87 | 44.18 | 42.63
Fine-tuned by Characters (External dataset) | 8.84 | 8.66 | 28.71 | 28.19
Fine-tuned by Visemes (External dataset) | 8.11 | 7.95 | 25.88 | 25.37

###  IV-F Audio-visual Speech Recognition using External Audio Dataset

In this experiment, we used Common Voice as an external dataset in addition to
Arman-AV. The Common Voice dataset contains over 352 hours of speech. The aim
of this experiment is to examine the effect of the features extracted by our
method on the model that extracts high-quality speech features. For this
purpose, we trained the speech recognition model with the conformer-small
network [20] using both datasets. Then, we used the encoder part of the
trained network to extract speech features for each sample in the Arman-AV
dataset. Finally, we trained the audio-visual speech recognition model using
the model described in the previous section.

The next two rows of Table III show the comparison results when an external
audio dataset was used in the training. By comparing these results with the
results of the previous section, we can see that using audio-only external
datasets can greatly reduce errors. Also, here the proposed viseme-based
strategy consistently outperforms the baseline.

(a) The proposed method (b) AV-HuBERT baseline model

Figure 2: 2D representation of Persian phonemes using a visual representation
model by averaging embeddings for each phoneme obtained from phoneme-level
labeled data.

##  V Analysis

In this section, we aim to compare the two models. To do so, we extract and
analyze the embedding of each phoneme. We use the ARMAN-AV dataset for this
purpose, which has phoneme-level transcription. For each phoneme, we compute
the average visual embedding. We then analyze these embeddings using two
methods. The first method involves mapping the 768-dimensional vector for each
model to a two-dimensional space using t-SNE (Figure 2). The second method
involves clustering the embeddings and comparing the quality of the obtained
clusters.

The embeddings of the phonemes are compared using a clustering evaluation
metric called Silhouette. In this comparison, we first assign a cluster to
each phoneme. Each phoneme belongs to a cluster that has similar lip movements
to other phonemes in the same cluster. In fact, we group the phonemes of each
viseme into a cluster. The proposed model improves the clustering quality for
phonemes that have a high degree of visual similarity to each other, as shown
in Table IV. The results in this table show that our proposed model achieves
higher-quality clustering of the video features for the lip-reading problem.
This means that the phonemes that have similar lip movements are closer to
each other in the feature space. This improvement is remarkable considering
that the model was trained on English data, but the analysis was performed on
Persian data.

TABLE IV: Silhouette Score Comparison | Silhouette
---|---
Pre-trained AV-HuBERT | 33.79 %
The Proposed Method | 36.21 %

##  VI Conclusion

Lip reading is a challenging task that has received significant attention in
recent years due to its potential applications in various fields. However,
developing an accurate lip-reading system is still a difficult task due to the
scarcity of data and the similarity of lip movements for some phonemes (these
similar phonemes are called visemes). AV-HuBERT is a self-supervised method
that can learn audio and visual features. We proposed a method to fine-tune
the AV-HuBERT model using visemes to enhance the visual feature extraction. We
evaluated our proposed method on the Arman-AV dataset, using different tasks
such as word and sentence level lip reading, and audio-visual speech
recognition. In all of them, our proposed method outperformed the baseline AV-
HuBERT model. As future work, we plan to extend our approach to a multilingual
setting.

## References

  * [1] H. L. Bear and R. Harvey, ``Decoding visemes: Improving machine lip-reading,'' _ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings_ , vol. 2016-May, pp. 2009–2013, 5 2016.
  * [2] J. Peymanfard, M. R. Mohammadi, H. Zeinali, and N. Mozayani, ``Lip reading using external viseme decoding,'' _Iranian Conference on Machine Vision and Image Processing, MVIP_ , vol. 2022-February, 2022.
  * [3] B. Shi, W.-N. Hsu, K. Lakhotia, and A. Mohamed, ``Learning audio-visual speech representation by masked multimodal cluster prediction,'' 1 2022. [Online]. Available: https://arxiv.org/abs/2201.02184v2
  * [4] J. S. Chung, A. Senior, O. Vinyals, and A. Zisserman, ``Lip reading sentences in the wild,'' _Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017_ , vol. 2017-January, pp. 3444–3450, 11 2017\.
  * [5] K. Simonyan and A. Zisserman, ``Very deep convolutional networks for large-scale image recognition,'' _3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings_ , 9 2014. [Online]. Available: https://arxiv.org/abs/1409.1556v6
  * [6] T. Afouras, J. S. Chung, and A. Zisserman, ``Lrs3-ted: a large-scale dataset for visual speech recognition,'' 9 2018. [Online]. Available: https://arxiv.org/abs/1809.00496v2
  * [7] S. Petridis, T. Stafylakis, P. Ma, G. Tzimiropoulos, and M. Pantic, ``Audio-visual speech recognition with a hybrid ctc/attention architecture,'' in _2018 IEEE Spoken Language Technology Workshop (SLT)_ , 2018, pp. 513–520.
  * [8] T. Afouras, J. S. Chung, A. Senior, O. Vinyals, and A. Zisserman, ``Deep audio-visual speech recognition,'' _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 44, no. 12, pp. 8717–8727, 2022.
  * [9] J. Devlin, M. W. Chang, K. Lee, and K. Toutanova, ``Bert: Pre-training of deep bidirectional transformers for language understanding,'' _NAACL HLT 2019 \- 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - Proceedings of the Conference_ , vol. 1, pp. 4171–4186, 10 2018. [Online]. Available: https://arxiv.org/abs/1810.04805v2
  * [10] T. Afouras, J. S. Chung, and A. Zisserman, ``Asr is all you need: Cross-modal distillation for lip reading,'' _ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings_ , vol. 2020-May, pp. 2143–2147, 5 2020.
  * [11] J. S. Chung and A. Zisserman, ``Lip reading in the wild,'' _Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)_ , vol. 10112 LNCS, pp. 87–103, 2017. [Online]. Available: https://link.springer.com/chapter/10.1007/978-3-319-54184-6_6
  * [12] S. Yang, Y. Zhang, D. Feng, M. Yang, C. Wang, J. Xiao, K. Long, S. Shan, and X. Chen, ``Lrw-1000: A naturally-distributed large-scale benchmark for lip reading in the wild,'' _Proceedings - 14th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2019_ , 5 2019.
  * [13] J. Peymanfard, A. Lashini, S. Heydarian, H. Zeinali, and N. Mozayani, ``Word-level persian lipreading dataset,'' _2022 12th International Conference on Computer and Knowledge Engineering, ICCKE 2022_ , pp. 225–230, 2022\.
  * [14] J. Yu, S.-X. Zhang, J. Wu, S. Ghorbani, B. Wu, S. Kang, S. Liu, X. Liu, H. Meng, and D. Yu, ``Audio-visual recognition of overlapped speech for the lrs2 dataset,'' in _ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 2020, pp. 6984–6988.
  * [15] P. Ma, S. Petridis, and M. Pantic, ``End-to-end audio-visual speech recognition with conformers,'' in _ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 2021, pp. 7613–7617.
  * [16] A. Haliassos, P. Ma, R. Mira, S. Petridis, and M. Pantic, ``Jointly learning visual and auditory speech representations from raw data,'' _arXiv preprint arXiv:2212.06246_ , 2022.
  * [17] J. S. Chung, A. Nagrani, and A. Zisserman, ``Voxceleb2: Deep speaker recognition,'' _arXiv preprint arXiv:1806.05622_ , 2018.
  * [18] J. Peymanfard, S. Heydarian, A. Lashini, H. Zeinali, M. R. Mohammadi, and N. Mozayani, ``A multi-purpose audio-visual corpus for multi-modal persian speech recognition: the arman-av dataset,'' _arXiv preprint arXiv:2301.10180_ , 2023.
  * [19] R. Ardila, M. Branson, K. Davis, M. Henretty, M. Kohler, J. Meyer, R. Morais, L. Saunders, F. M. Tyers, and G. Weber, ``Common voice: A massively-multilingual speech corpus,'' _arXiv preprint arXiv:1912.06670_ , 2019.
  * [20] A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, Y. Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y. Wu _et al._ , ``Conformer: Convolution-augmented transformer for speech recognition,'' _arXiv preprint arXiv:2005.08100_ , 2020.


# IBM Employee Attrition Analysis

Shenghuan Yang  Jiangxi University of Finance and Economics  and  Md Tariqul
Islam  Syracuse University

###### Abstract.

In this paper, we analyzed the dataset IBM Employee Attrition to find the main
reasons why employees choose to resign. Firstly, we utilized the correlation
matrix to see some features that were not significantly correlated with other
attributes and removed them from our dataset. Secondly, we selected important
features by exploiting Random Forest, finding monthlyincome, age, and the
number of companies worked significantly impacted employee attrition. Next, we
also classified people into two clusters by using K-means Clustering.Finally,
we performed binary logistic regression quantitative analysis: the attrition
of people who traveled frequently was 2.4 times higher than that of people who
rarely traveled. And we also found that employees who work in Human Resource
have a higher tendency to leave.

##  1\. Introduction

Employee attrition is defined as the natural process by which employees leave
the workforce – for example, through resignation for personal reasons or
retirement – and are not immediately replaced(ref5, ). Employee turnover is
regarded as the key issue for all organizations these days, because of its
adverse effects on workplace productivity, and accomplishing organizational
objectives on time (ref6, ). In order for an organization to continually have
a higher competitive advantage over its competition, it should make it a duty
to minimize employee attrition Therefore, for the better development of
corporation, it is essential for the leader of companies to know the main
reasons why their employees choose to leave the company,then take relevant
measures to improve their company’s productivity, overall workflow and
business performance.

Objectives. In this paper, we aim to select the main causes that contribute to
an employee’s decision to leave a company, and to be able to predict whether a
particular employee will leave the company by utilizing machine learning
models.

Contributions. Following are the main contributions of this paper:

  * •

We select the main factors affecting the employee attrition by using Random
Forest, and classify which types of people are more likely to quit by
utilizing the K-means Clustering

  * •

We represent a given reality in terms of a numerical value to compare the
employee attrition in different categories by utilizing quantitative analysis.

The rest of the paper is described below. We present some related work in
Section 2. We interpret data and introduce methodologies in Section 3. We
process the original data set and remove some variables which are not very
correlated with other features in section 4. We implement our machine learning
model in Section 5. Finally, we draw a conclusion in section 6.

##  2\. Related Work

Alao et al.(ref111, ) used decision tree models and rule-sets to develop a
predictive model that was used to predict new cases of employee attrition.
Alduayj et al.(ref222, )utilized support victor machine (SVM) with several
kernel functions, random forest and Knearest neighbour (KNN) to predict
employee attrition based on their features and found quadratic SVM scored the
highest results. Frye et al.(ref333, )applied Principal Component Analysis and
classification methods K-Nearest Neighbors and Random Forest, finding that
Logistic Regression predicts employee quits with the highest accuracy. Yadavet
et al.(ref444, ) provided a framework for predicting the employee churn by
analyzing the employee’s precise behaviors and attributes using classification
techniques. Srivastava et al.(ref555, )provided a framework for predicting the
employee churn by analyzing the employee’s precise behaviors and attributes
using classification techniques. El-Rayes et al.(ref666, ) presented a
framework for predicting the employee attrition with respect to voluntary
termination employing predictive analytics. Setiawan et al.(ref777, ) found
that eleven variables that have a significant impact on employee attrition.

##  3\. Data and Methodology

Employee attrition is the internal data of the company, which is difficult to
obtain, and some data has a certain degree of confidentiality, therefore our
paper used the data set disclosed by kaggle. The sample size of the data set
is 1471, there are 34 feature variables,mainly divided into three types of
variables: personal basic information, work experience, attendance rate.This
paper explored the relationship between employee’s characteristics and
employee attrition, found Whether characteristics have a great influence on
employee attrition. In addition, we used machine learning algorithms to select
important features that influenced the employee attrition, and predicted the
it. In this paper, we exploited three machine learning algorithms: Decision
Tree, and Logistic Regression and k-means clustering.

###  3.1. Random Forest

Random forest is an learning method used for classification, regression and
other tasks, which combines multiple decision trees to select the best result.
Random forest corrects the habit of decision trees that rely too much on the
training set and improves the accuracy of the model.First, there are randomly
selected sub-data sets that are replaced from the original data(ref7, ). The
elements between the sub-datasets may have the same elements, and the content
of each sub-data set is 1 different. Then use the sub-dataset to construct the
sub-decision tree, each decision tree will output a result, you can vote
through the output result of the sub-decision tree, and finally get the output
result. As shown in Figure 1, the data set extracts 4 sub-datasets to
construct 4 sub-decision trees, 3 trees voted as A, one sub-decision tree
voted as B, and the final output is A.

Figure 1. Decision-making process of random forest

Random forest is one of the most popular machine learning algorithms. Random
forest provides a unique combination of prediction accuracy and model
interpretability among popular machine learning methods. The random sampling
and ensemble strategies utilized in RF enable it to achieve accurate
predictions as well as better generalizations (ref8, ). In addition, it has a
strong explanatory nature. It can directly calculate the importance of each
variable. In other words, it is easy to calculate how much each variable
contributes to decision-making. In our research, we build Random Forest model
based on Employee Attrition Features.There are 34 employee attributes in the
data set, we select randomly k(k¡34) employee attributes to build a decision
tree, and create 100 random sub-samples of our dataset with replacement. Each
node of each decision tree will be split according to predictor variables so
that child nodes are more “pure” (i.e., Gender) in terms of the outcome
variable. If the feature of the next child node has appeared in the parent
node, the second child node will stop splitting. In the end, each decision
tree outputs a result, and the result with the most votes is the final result.

###  3.2. Binary Logistic Regression

Logistic regression analysis can be used to estimate the probability of an
event, and it can also analyze the influencing factors of the problem. In
medical research, Logistic regression is often used to analyze the risk
factors of a certain disease(ref99, ). Such as analyzing whether age, smoking,
drinking, and diet are risk factors for diabetes. In questionnaire research,
Logistic regression is often used to analyze non-scale questions, such as
taking the basic background information of the sample as X and purchasing
intention as Y to analyze whether gender, age, and families’ socioeconomic
conditions affect purchase intention. Among them, binary logistic regression
analysis is used most frequently.

(1)
$logit_{(}p)=\ln\dfrac{{p(y=1)}}{1-p(y=1)}={\beta}_{0}+{\beta_{1}}{x_{1}}+...+{\beta_{n}}{x_{n}}$

Here, employee attrition as the dependent variable y, 0 represents No, 1
represents Yes. Gender, EducationLevel, OverTime, JobLevel and so on as
predictor variables, n=34, ${x_{1}}$,${x_{2}}$,…, ${x_{34}}$. ${\beta_{0}}$,
${\beta_{1}}$,…, ${\beta_{n}}$ are coefficients, which represent the influence
of the variable on the dependent variable. Whether it is positive or negative,
it needs to be explained in conjunction with the corresponding regression
coefficient value. If the regression coefficient value is greater than 0, it
means a positive influence; otherwise, it means a negative influence for the
dependent variable.

###  3.3. k-means Clustering

The K-means Clustering algorithm is the most commonly used clustering
algorithm. The main idea is: given K values and K initial cluster centers,
assign each point to the nearest cluster center, after all the points are
allocated, the center point of the cluster is recalculated based on all the
points in the cluster (take the average value), updates the center point of
the cluster Steps until the center point of the cluster changes very little
(ref10, ). The objective function is:

(2) $J=\sum_{j=1}^{k}\sum_{i=1}^{n}{||x^{(j)}_{i}-c_{j}||^{2}}$

Where $||x^{(j)}_{i}-c_{j}||$ is the Euclidean distance between $x^{(j)}_{i}$
and $c_{j}$, n is the number of data points in $j^{th}$ cluster, k is the
number of cluster centers. Algorithmic steps for k-means clustering:
$X={x_{1},x_{2},x_{3},...,x_{n}}$ is the set of data points, and
$c={c_{1},c_{2},...,c_{k}}$ is the set of centers.

  1. (1)

Randomly select ’k’ cluster centers

  2. (2)

Calculate the distance between each data point and cluster centers

  3. (3)

Assign the data point to the cluster center whose distance from the cluster
center is minimum of all the cluster centers

  4. (4)

Recalculate the new cluster center using:

(3) $c_{j}=\dfrac{1}{n}\sum_{i=1}^{n}x_{i}$

  5. (5)

Recalculate the distance between each data point and new obtained cluster
centers

  6. (6)

If no data point was reassigned then stop, otherwise repeat from step (3)

The process of K-means Clustering is shown in Figure 2.

Figure 2.  Clustering process of K-means Clustering

##  4\. data process

In our studies, there are 34 variables, however, some features just have one
data level that do not make sense for our research, such as EmployeeCount,
Over18 and StandardHours, and employee number does not have meaning in
analyzing resulting so we also deleted these features. In addition, some
features are not very correlated with other attributes, we need to remove them
from our dataset to improve computing efficiency. We built correlation matrix,
which is a table showing correlation coefficients between variables. Each cell
in the table shows the correlation between two variables.

Figure 3. Correlation Matrix

We can see from Figure  3, daily rate, hourly rate and monthly rate are barely
correlated with other attributes, while Monthly Income and job level (0.95),
Job level and total working years(0.78), Monthly Income and total working
years(0.77) are highly correlated. Therefore we delete daily rate, hourly rate
and monthly rate from dataset.

##  5\. Model Construction

In this part, we build machine learning models to select important features
that influence the employee attrition and classify the features to help us
understand the main reason why people left the IBM company.And we will
construct the model to predict the employee attrition according to the given
data.

###  5.1. Feature Selection

We all may have faced this problem of identifying the related features from a
set of data and removing the irrelevant or less important features with do not
contribute much to our target variable in order to achieve better accuracy for
our model and.Therefore, it is better for us to select important features that
influence the employee attrition, and ignore the variables that are not
significant in explaining such behavior.

Figure 4.  Important Features

From Figure  4, Monthly Income, Age, DisatnceFromHome are top 3 important
features to indicate whether the employee has a tendency to leave, while
marital status married, and female aged 40-50 are less likely to go. Indeed,
income is the main cause that people choose to leave the company, which is
highly related to people’s life quality. More affluent people have more
disposable income and can more easily afford expensive service (such as
medical care) and a healthy lifestyle—benefits.So people want to have more
money that they intend to leave the current company and find other companies
that offer high salaries. And also young people are more likely to want to try
different jobs and finally find a job that suits them when they just graduated
from universities. Whereas older people prefer a stable life because they
already have own families and children, so it is not easy to leave the job. In
addition, the distance between the company and home is also the main reason
why people quit the current job, they would waste much time commuting. There’s
mass of social science and public health research on the negative effects of
commuting on personal and societal well-being. Longer commutes are linked with
increased rates of obesity, high cholesterol, high blood pressure, back and
neck pain, divorce, depression and death.

Here, we need to train and test dataset to predict the employee attrition
according to the features. We splited the data set into two sets: a training
set and a testing set (80% for training, and 20% for testing). To better
reflect the real situation and avoid the influence of extreme values, we
predicted the employee attrition for 100 times by utilizing RandomForest
Classifier, and recorded the average accuracy of the test set. Table  1 shows
the average accuracy is 0.84561, which means our model fits well.

Table 1. Prediction Accuracy of Random Forest  Times |  Accuracy
---|---
1 |  0.85714
2 |  0.83673
3 |  0.84353
4 |  0.84693
… |  …
100 |  0.85374
Average Accuracy |  0.84561

###  5.2. Classification of people

To distinguish which types of people are more likely to resign, we use K-means
clustering to divide the dataset into two categories. The first type is prone
to leave, and the other is less likely to quit.

Table 2. Clustering Cluster Type | 0 | 1
---|---|---
Age | 44.215152 | 34.813158
Attrition | 0.1 | 0.178947
DistanceFromHome | 9.072727 | 9.227193
Education | 3.039394 | 2.876316
EnvironmentSatisfaction | 2.693939 | 2.729825
JobInvolvement | 2.690909 | 2.741228
JobLevel | 3.684848 | 1.594737
JobSatisfaction | 2.709091 | 2.734211
MonthlyIncome | 14060.49394 | 4315.215789
NumCompaniesWorked | 3.342424 | 2.505263
OverTime | 0.290909 | 0.280702
… | … | …
… | … | …
PercentSalaryHike | 15.066667 | 15.250877
PerformanceRating | 3.148485 | 3.155263
RelationshipSatisfaction | 2.781818 | 2.692105
StockOptionLevel | 0.80303 | 0.791228
TotalWorkingYears | 21.072727 | 8.444737

We can see from Table 2 (complete form in appendix table 4),cluster set 0
represents low attrition, cluster set 1 means high attrition, and older
people, high job level, high job satisfaction,high monthly income, more number
of companies worked and so forth are less likely to leave.These finding are in
line with people’s behavior in the real world and previous accounts in feature
selection of RandomForest. In last section, we illustrated the relationship
between age, monthly income, distance from the company and home, this section,
we also find that the number of companies employees worked are related to the
probability of leaving the corporation. People who have worked in 3-4
companies are less likely to quit because by this time, they have roughly
found the direction of employment, and people who have more than four
companies indicate that they are unstable and often change jobs. In addition,
people who are in the higher the job level, enjoy the higher salary and
respect , so they are less likely to leave. While those employees in lower-
level positions are often not satisfied with the status quo and want to seek
better job chance. What is more, job satisfaction is also the main cause
influencing the employee attrition rate.Intention to stay on the job is
clearly correlated with job satisfaction in such aspects as educational system
and environment, income and welfare, leadership and administration.(ref4, ).

###  5.3. Logistics Regression

In this section, we exploited the binary logistics regression to predict the
relationship between predictors (employee characteristics) and a predicted
variable (employee attrition) where the dependent variable is binary (NO:0,
YES:1). And we will compare the differences between each category to help us
understand which type of person is more likely to quit (complete regression
table can be seen in appendix table 5).

Table 3. Regression Result | OR(95% CI) | P-value
---|---|---
Travel$\\_$Rarely | 1 |
NonTravel | 0.361 |
Travel_Frequently | 2.411 | ¡0.05
Male | 1 |
Female | 0.659 | ¡0.05
Sales Representative | 1 |
Healthcare | 0.160 |
Human Resource | 4.060 |
Laboratory | 0.556 |
Manager | 0.347 |
Manufacturing | 0.200 |
Research | 0.200 |
Sales Executive | 0.484 | ¡0.05
Single | 1 |
Divorced | 0.304 |
Married | 0.427 | ¡0.05
OverTime | 0.138 | ¡0.05

From Table 3, we can see, taking people who rarely travel as the standard 1,
employees who travel frequently are 0.361 times more than those to leave the
company whereas people who never travel are more than twice. And Women are
0.659 times more than men to go. As for JobRole, taking Sales Representative
as the standard 1, employee who who work about Research Science, Laboratory,
Manufacturing, Healthcare are not likely to quit, while HumanResource
Department has high employee attrition. Then single people are more likely to
leave their jobs than married and divorced people.Finally,not surprised,
people who usually work overtime would like to leave the company. To evaluate
the logistics regression model, we need to test the accuracy of model in
Python, the test result is 0.8843, which means our model fitting is well.

##  6\. conclusion

According the above model results, we can know that our finding are in line
with people’s behavior in the real world and previous studies other scholar
did. We utilized Random Forest and K-means Clustering to select important
features that had obvious impact on the employee attrition. Firstly, according
to Random Forest results, monthlyincome, age, the number of companies worked
are the main reasons why people choose to resign. Then we found older people,
high job level, high job satisfaction,high monthly income, more number of
companies worked, these kinds of people are not likely to go based on the
clustering result of K-means Clustering. However, different people have
various intention, we need to do further and detailed research to find
qualitative findings by using qualitative analysis. So we exploited the binary
logistics regression to compare the difference between people. Our study found
that females’ attrition was 0.659 times than that of males, married and
divorced people were 0.427 and 0.304 times than people who were single,
respectively. Besides, the attrition of people who traveled frequently was 2.4
times higher than that of people who rarely traveled. And we also found that
employees who work in Human Resource have a higher tendency to leave. Finally,
there are other interesting findings in our study:in terms of number of
companies worked, people who worked in 2 - 4 companies are less likely to
leave, the female attrition rate is less than male after working for six
companies , and people who earned Doctor’s Degree are almost always having the
lowest attrition rate.

To evaluate the model performance, we trained and tested the dataset to
predict the employee attrition, split it into two parts(80% for training, 20%
for testing ), and recorded the test set’s accuracy. Random Forest and
Logistics Regression accuracy were 0.8456 and 0.8843, respectively, which
meant Logistics Regression fitted better and was more suitable for prediction
in our dataset.

We also want to make some suggestions to the company through this research,
hoping that they will care more about their employees and improve their job
satisfaction. Simultaneously, they must pay more attention to human resources
employees because they have very low job satisfaction. Besides, the company
should allow employees to have enough time to rest and spend time with their
families.There is a general belief that employees who take regular breaks are
more productive.

## References

  * (1) Budhwar P S, Bhatnagar J. Talent management strategy of employee engagement in Indian ITES employees: key to retention[J]. Employee relations, 2007.
  * (2) Jain, Rachna, and Anand Nayyar. Predicting employee attrition using xgboost machine learning approach. 2018 International Conference on System Modeling $\&$ Advancement in Research Trends (SMART). IEEE, 2018.
  * (3) Alao, D. A. B. A., and A. B. Adeyemo. ”Analyzing employee attrition using decision tree algorithms.” Computing, Information Systems, Development Informatics and Allied Research Journal 4.1 (2013): 17-28.
  * (4) Alduay j, Sarah S., and Kashif Rajpoot. ”Predicting employee attrition using machine learning.” 2018 International Conference on Innovations in Information Technology (IIT). IEEE, 2018.
  * (5) Frye, Alex, et al. ”Employee Attrition: What Makes an Employee Quit?.” SMU Data Science Review 1.1 (2018): 9.
  * (6) Yadav, Sandeep, Aman Jain, and Deepti Singh. ”Early Prediction of Employee Attrition using Data Mining Techniques.” 2018 IEEE 8th International Advance Computing Conference (IACC). IEEE, 2018.
  * (7) Srivastava, Devesh Kumar, and Priyanka Nair. ”Employee attrition analysis using predictive techniques.” International Conference on Information and Communication Technology for Intelligent Systems. Springer, Cham, 2017.
  * (8) El-Rayes, Nesreen, et al. ”Predicting employee attrition using tree-based models.” International Journal of Organizational Analysis (2020).
  * (9) Setiawan, I., et al. ”HR analytics: Employee attrition analysis using logistic regression.” IOP Conference Series: Materials Science and Engineering. Vol. 830. No. 3. IOP Publishing, 2020.
  * (10) Pal M. Random forest classifier for remote sensing classification[J]. International journal of remote sensing, 2005, 26(1): 217-222.
  * (11) Qi, Yanjun. ”Random forest for bioinformatics.” Ensemble machine learning. Springer, Boston, MA, 2012. 307-323.
  * (12) Meng X H, Huang Y X, Rao D P, et al. Comparison of three data mining models for predicting diabetes or prediabetes by risk factors[J]. The Kaohsiung journal of medical sciences, 2013, 29(2): 93-99.
  * (13) Pham D T, Dimov S S, Nguyen C D. Selection of K in K-means clustering[J]. Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science, 2005, 219(1): 103-119.
  * (14) Weiqi, Chen. ”The structure of secondary school teacher job satisfaction and its relationship with attrition and work enthusiasm.” Chinese Education$\&$ Society 40.5 (2007): 17-31.

Table 4. Clustering Result Cluster Type | 0 | 1
---|---|---
Age | 44.215152 | 34.813158
Attrition | 0.1 | 0.178947
DistanceFromHome | 9.072727 | 9.227193
Education | 3.039394 | 2.876316
EnvironmentSatisfaction | 2.693939 | 2.729825
JobInvolvement | 2.690909 | 2.741228
JobLevel | 3.684848 | 1.594737
JobSatisfaction | 2.709091 | 2.734211
MonthlyIncome | 14060.49394 | 4315.215789
NumCompaniesWorked | 3.342424 | 2.505263
OverTime | 0.290909 | 0.280702
PercentSalaryHike | 15.066667 | 15.250877
PerformanceRating | 3.148485 | 3.155263
RelationshipSatisfaction | 2.781818 | 2.692105
StockOptionLevel | 0.80303 | 0.791228
TotalWorkingYears | 21.072727 | 8.444737
TrainingTimesLastYear | 2.79697 | 2.8
WorkLifeBalance | 2.781818 | 2.755263
YearsAtCompany | 11.927273 | 5.584211
YearsInCurrentRole | 6.133333 | 3.67807
YearsSinceLastPromotion | 4.109091 | 1.631579
YearsWithCurrManager | 5.821212 | 3.631579
Male | 0.581818 | 0.605263
BusinessTravelNon-Travel | 0.087879 | 0.10614
BusinessTravelTravelFrequently | 0.187879 | 0.188596
BusinessTravelTravelRarely | 0.724242 | 0.705263
DepartmentHuman Resources | 0.045455 | 0.042105
DepartmentResearch $\&$ Development | 0.642424 | 0.657018
DepartmentSales | 0.312121 | 0.300877
EducationFieldHuman Resources | 0.021212 | 0.017544
EducationFieldLife Sciences | 0.39697 | 0.416667
EducationFieldMarketing | 0.130303 | 0.101754
EducationFieldMedical | 0.324242 | 0.313158
EducationFieldOther | 0.045455 | 0.058772
EducationFieldTechnical Degree | 0.081818 | 0.092105
JobRoleHealthcare Representative | 0.112121 | 0.082456
JobRoleHuman Resources | 0.012121 | 0.042105
JobRoleLaboratory Technician | 0 | 0.227193
JobRoleManager | 0.309091 | 0
JobRoleManufacturing Director | 0.121212 | 0.092105
JobRoleResearch Director | 0.242424 | 0
JobRoleResearch Scientist | 0.00303 | 0.255263
JobRoleSales Executive | 0.2 | 0.22807
JobRoleSales Representative | 0 | 0.072807
MaritalStatusDivorced | 0.251515 | 0.214035
MaritalStatusMarried | 0.506061 | 0.44386
MaritalStatusSingle | 0.242424 | 0.342105
Table 5. Logistics Regression Result | 95% C.I.for EXP(B)
---|---
| B | S.E. | Wald | df | Sig. | Exp(B) | Lower | Upper
Age | -0.031 | 0.014 | 5.285 | 1 | $0.02^{**}$ | 0.969 | 0.944 | 0.995
TravelRarely |  |  | 27.561 | 2 | $0.00^{**}$ |  |  |
NonTravel | -1.020 | 0.381 | 7.175 | 1 | $0.02^{**}$ | 0.361 | 0.171 | 0.761
TravelFrequently | 0.880 | 0.211 | 17.323 | 1 | $0.04^{**}$ | 2.411 | 1.593 | 3.649
DailyRate | 0.000 | 0.000 | 1.645 | 1 | 0.200 | 1.000 | 0.999 | 1.000
Sales Department |  |  | 0.010 | 2 | 0.995 |  |  |
HumanResource Department | -18.223 | 10790.424 | 0.000 | 1 | 0.999 | 0.000 | 0.000 |
ResearchDepartment | 0.104 | 1.043 | 0.010 | 1 | 0.921 | 1.109 | 0.144 | 8.566
DistanceFromHome | 0.046 | 0.011 | 17.837 | 1 | $0.00^{**}$ | 1.047 | 1.025 | 1.069
Education | 0.007 | 0.088 | 0.006 | 1 | 0.937 | 1.007 | 0.848 | 1.196
Human Re |  |  | 13.256 | 5 | $0.02^{**}$ |  |  |
Life Science | -0.189 | 0.823 | 0.053 | 1 | 0.818 | 0.828 | 0.165 | 4.152
Marketing | -0.926 | 0.304 | 9.279 | 1 | $0.00^{**}$ | 0.396 | 0.218 | 0.719
Medical | -0.525 | 0.394 | 1.777 | 1 | 0.182 | 0.591 | 0.273 | 1.280
Other | -1.036 | 0.312 | 11.029 | 1 | $0.00^{**}$ | 0.355 | 0.193 | 0.654
Technical | -0.983 | 0.475 | 4.276 | 1 | $0.00^{**}$ | 0.374 | 0.147 | 0.950
EmployeeNumber | 0.000 | 0.000 | 0.888 | 1 | 0.346 | 1.000 | 1.000 | 1.000
EnvironmentSatisfaction | -0.431 | 0.083 | 26.988 | 1 | $0.00^{**}$ | 0.650 | 0.553 | 0.765
Gender(1) | -0.417 | 0.184 | 5.098 | 1 | $0.00^{**}$ | 0.659 | 0.459 | 0.947
HourlyRate | 0.001 | 0.004 | 0.087 | 1 | 0.768 | 1.001 | 0.993 | 1.010
JobInvolvement | -0.528 | 0.123 | 18.417 | 1 | $0.00^{**}$ | 0.590 | 0.463 | 0.751
JobLevel | -0.128 | 0.317 | 0.162 | 1 | 0.688 | 0.880 | 0.472 | 1.640
Sales Representative |  |  | 20.855 | 7 | $0.00^{**}$ |  |  |
Healthcare | -1.835 | 1.164 | 2.484 | 1 | 0.115 | 0.160 | 0.016 | 1.563
Human Resource | 17.601 | 10790.424 | 0.000 | 1 | 0.999 | 44066714.479 | 0.000 |
Laboratory | -0.588 | 1.090 | 0.291 | 1 | 0.590 | 0.556 | 0.066 | 4.702
Manager | -1.058 | 0.985 | 1.154 | 1 | 0.283 | 0.347 | 0.050 | 2.392
Manufacturing | -1.609 | 1.162 | 1.919 | 1 | 0.166 | 0.200 | 0.021 | 1.950
Research | -1.611 | 1.097 | 2.156 | 1 | 0.142 | 0.200 | 0.023 | 1.715
Sales Executive | -0.726 | 0.385 | 3.555 | 1 | 0.059 | 0.484 | 0.228 | 1.029
JobSatisfaction | -0.414 | 0.081 | 25.891 | 1 | $0.00^{**}$ | 0.661 | 0.563 | 0.775
Single |  |  | 14.274 | 2 | $0.00^{**}$ |  |  |
Divorced | -1.191 | 0.346 | 11.885 | 1 | $0.00^{**}$ | 0.304 | 0.154 | 0.598
Married | -0.852 | 0.251 | 11.525 | 1 | $0.00^{**}$ | 0.427 | 0.261 | 0.698
MonthlyIncome | 0.000 | 0.000 | 0.292 | 1 | 0.589 | 1.000 | 1.000 | 1.000
MonthlyRate | 0.000 | 0.000 | 0.138 | 1 | 0.710 | 1.000 | 1.000 | 1.000
NumCompaniesWorked | 0.196 | 0.039 | 25.654 | 1 | $0.00^{**}$ | 1.216 | 1.128 | 1.312
OverTime(1) | -1.984 | 0.194 | 104.913 | 1 | $0.00^{**}$ | 0.138 | 0.094 | 0.201
PercentSalaryHike | -0.021 | 0.039 | 0.287 | 1 | 0.592 | 0.979 | 0.907 | 1.058
PerformanceRating | 0.096 | 0.398 | 0.058 | 1 | 0.810 | 1.100 | 0.504 | 2.402
RelationshipSatisfaction | -0.270 | 0.083 | 10.591 | 1 | $0.00^{**}$ | 0.763 | 0.649 | 0.898
StockOptionLevel | -0.186 | 0.158 | 1.387 | 1 | 0.239 | 0.830 | 0.609 | 1.131
TotalWorkingYears | -0.057 | 0.029 | 3.772 | 1 | 0.052 | 0.945 | 0.892 | 1.001
TrainingTimesLastYear | -0.193 | 0.073 | 6.976 | 1 | $0.00^{**}$ | 0.825 | 0.715 | 0.951
WorkLifeBalance | -0.377 | 0.124 | 9.298 | 1 | $0.00^{**}$ | 0.686 | 0.538 | 0.874
YearsAtCompany | 0.098 | 0.039 | 6.348 | 1 | $0.00^{**}$ | 1.103 | 1.022 | 1.191
YearsInCurrentRole | -0.148 | 0.045 | 10.701 | 1 | $0.00^{**}$ | 0.862 | 0.789 | 0.942
YearsSinceLastPromotion | 0.175 | 0.042 | 16.934 | 1 | $0.00^{**}$ | 1.191 | 1.096 | 1.294
YearsWithCurrManager | -0.138 | 0.047 | 8.575 | 1 | $0.00^{**}$ | 0.871 | 0.794 | 0.955
Constant | 9.308 | 1.383 | 45.275 | 1 | $0.00^{**}$ | 11022.507 |  |
**significant at p ¡ 0.05 |  |  |  |  |  |  |  |