\begin{table}[t]
\centering
\setlength{\tabcolsep}{11mm}{
\begin{tabular}{@{}cc@{}}
\toprule
Error Types          & Ratio \\ \midrule
\bf{Under-spcified}       & \bf{9.0}\% \\
Over-specified       & 5.5\% \\
Self-reference       & 3.0\% \\
Wrong part-of-speech & 1.0\% \\
Opposite             & 1.0\% \\ \bottomrule
\end{tabular}}
\caption{Ratio of each error type of the definitions generated in \citet{huang2021definition}.}
\label{pic:error ratio}
\end{table}
\section{Introduction} \label{introduction}
When readers find some expressions unfamiliar during reading a text, machines can help. The task of Definition Generation (DG) aims to generate a textual definition for a given word or phrase (the target), according to a surrounding context (the local context)~\cite{ni-wang-2017-learning}. In addition to assisting readers in comprehending expressions, the task of DG is also useful for generating definition when building dictionaries. 
Recently, pre-trained encoder-decoder models have achieved great successes on this task~\cite{huang2021definition,kong2022multitasking}. Despite their successes, the definitions produced by these pre-trained models often contain several types of errors~\cite{noraset2017definition,huang2021definition}. According to Table~\ref{pic:error ratio}, ``under-specific problem'' is the most frequent error that the generated definition conforms to the general semantics but loses certain parts of meaning of the target word. As presented in Table~\ref{under-specific case}, the definition produced by T5 model is under-specific as it omits the meaning of \emph{great} in the word ``double'' under the context ``\emph{ate a double portion}''. The under-specific problem harms the accuracy of the generated definitions and in turn limits the applications of definition generation techniques in many scenarios. 

\section{Related Work}
\subsection{Definition Generation}
The task of Definition Generation is firstly proposed by~\citet{noraset2017definition}. They used word embedding to generate its corresponding definition, and utilize definition generation as an auxiliary task for reverse dictionary and word embedding training. Some later works explore more application scenarios and model architectures for definition generation. \citet{ni-wang-2017-learning} propose a dual-encoder model to generate the proper definition of the given word under a specific context, and use it for explaining emerging words on the Internet. \citet{gadetsky2018conditional} use both local and global information of the words in their model for word disambiguation. Following them, \citet{ishiwatari2019learning} design gate mechanisms to fuse multi-source information of the word and context. Furthermore, some works attempt to utilize other information of the target word. \citet{washio2019bridging} build relation of defined and defining words using word pair embedding \cite{joshi2018pair2vec}. Different from former works that using distributed representations of target words, \citet{yang2019incorporating} introduce target words' concepts in HowNet~\cite{dong2003hownet} as fine-grained knowledge in Chinese definition modeling. Also, there exist literature works based on refined methods to learn the target words. Both \citet{li2020explicit} and \citet{reid2020vcdm} decompose the meaning of the target word into a group of latent variables and rely on variational inference for estimation. 

Recently, pre-trained encoder-decoder models have been used in definition generation and achieved great success. \citet{bevilacqua2020generationary} use special tokens to mark the target word in the context and feed them into a BART model~\cite{lewis2019bart}. \citet{huang2021definition} fine-tune a T5 model and re-rank all the candidate results from the T5 model to obtain definitions in a proper specificity. \citet{kong2022multitasking} design a MASS model based on multi-task framework to generate simple definition in an unsupervised manner. Despite of their promising performances on definition generation, the under-specific problem has been less investigated. Although \citet{huang2021definition} design a scoring mechanism that measures definitions' specificity, we argue that the fundamental reason of the under-specific problem lies in the lack of fine-grained semantic learning in pre-trained encoder-decoder models, which we leverage contrastive learning to address in this work.

\subsection{Task Formulation}
\label{Task Formulation}
Given a word or phrase $W = \{w_i,...,w_j\}$ and its surrounding context $C = \{w_0,...,W_k\}(0<i<j<k)$, the task of definition generation is to generate the definition $D = \{d_0,...d_T\}$ to explain the meaning of $W$ under $C$. This process can be formulated as:

For evaluation, we follow previous works and acquire three popular datasets, which are ensembled by \citet{ishiwatari2019learning}\footnote{\url{http://www.tkl.iis.u-tokyo.ac.jp/~ishiwatari/naacl_data.zip}}.
Each entry in a dataset consists of three elements: (1) a target word or phrase, (2) the corresponding definition, and (3) one usage example of the target as a local context. If a target has multiple definitions and examples, we treat them as different entries. For fair comparison, each dataset is split into $\textit{train}$, $\textit{dev}$ and $\textit{test}$ sets according to \citet{ishiwatari2019learning}. The statistics of these datasets are shown in Table~\ref{Statistics of datasets}.

\subsection{Manual Evaluation}
To adequately evaluate the generated definitions, we also adopt three kinds of manual metrics: (1) Acccuracy (Acc.) ~\cite{ishiwatari2019learning} learning measures the semantic similarity between the generated definitions and the target words; (2) Fluency (Flu.) evaluates the readability of the generated definitions without considering the semantic alignment; (3) Under-specified (Under-spec.)~\cite{noraset2017definition}  calculates the ratio of under-specific definitions in the generated cases, which is curated to assess the model's capabilities in addressing the under-specific problem. The lower the ratio is, the better the model is in capturing fine-grained semantics during definition generation. Note that both Acc. and Flu. metrics are likert-scale of \{0,1,2,3,4,5\}.

According to Table~\ref{manual-evaluation}, the definitions generated by the proposed method T5-Contrast are better than those by other two models in terms of all the three metrics. Notably, the under-specific ratio significantly drops from 7.6\% (T5-Base) to 4.8\% (Ours). The manual evaluation results imply that the definitions produced by our method are more accurate, fluent, and fine-grained as compared to other pre-trained models.

\subsection{Case Study}
For better understanding, we show some example definitions generated by these compared models in Table~\ref{tab:case study}. It is obvious that T5-Base produces an under-specific definition ``\emph{a positive criticism}'' for the target word ``\emph{praise}'' in the context \emph{he always appreciated praise for this work}. The generated definition roughly expresses the positive meaning of the target word \emph{appreciate}, but fails to provide the accurate meaning of \emph{approval and commendation} in \emph{praise}. In this case, this example definition by T5-Base is under-specific. As for T5-Reranking, it generates the word ``\emph{goodwill}'', which is a multi-sense word where the one sense is ``a kindly feeling of support'' and the other sense is ``the favor or advantage of a business''. As such, this definition by T5-Reranking is also inaccurate to describe the word \emph{praise}. On the contrary, the definition generated by our model that ``\emph{an expression of admiration or approval}'' is more specific, which shows the effectiveness of our proposed method to remedy the under-specific problem. Due to the space limit, we give more sampled examples in Appendix \ref{Additonal Case Study}. 
{"value": "9.0", "type": "Other"} {"value": "5.5", "type": "Other"} {"value": "3.0", "type": "Other"} {"value": "1.0", "type": "Other"} {"value": "1.0", "type": "Other"}