# Behavioral Testing: Can Large Language Models
Implicitly Resolve Ambiguous Entities?

Anastasiia Sedova1,2∗ Robert Litschko3,4∗ Diego Frassinelli3,5
Benjamin Roth1,6 Barbara Plank3,4
1 Faculty of Computer Science, University of Vienna, Austria
2 UniVie Doctoral School Computer Science, University of Vienna, Austria
3 MaiNLP, Center for Information and Language Processing, LMU Munich, Germany
4 Munich Center for Machine Learning (MCML), Munich, Germany
5 Department of Linguistics, University of Konstanz
6 Faculty of Philological and Cultural Studies, University of Vienna, Austria
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

One of the major aspects contributing to the striking performance of large
language models (LLMs) is the vast amount of factual knowledge accumulated
during pre-training. Yet, many LLMs suffer from self-inconsistency, which
raises doubts about their trustworthiness and reliability. In this paper, we
focus on entity type ambiguity and analyze current state-of-the-art LLMs for
their proficiency and consistency in applying their factual knowledge when
prompted for entities under ambiguity. To do so, we propose an evaluation
protocol that disentangles knowing from applying knowledge, and test state-of-
the-art LLMs on 49 entities. Our experiments reveal that LLMs perform poorly
with ambiguous prompts, achieving only 80% accuracy. Our results further
demonstrate systematic discrepancies in LLM behavior and their failure to
consistently apply information, indicating that the models can exhibit
knowledge without being able to utilize it, significant biases for preferred
readings, as well as self-inconsistencies. Our study highlights the importance
of handling entity ambiguity in future for more trustworthy LLMs.

Behavioral Testing: Can Large Language Models
Implicitly Resolve Ambiguous Entities?

Anastasiia Sedova1,2∗ Robert Litschko3,4∗ Diego Frassinelli3,5 Benjamin
Roth1,6 Barbara Plank3,4 1 Faculty of Computer Science, University of Vienna,
Austria 2 UniVie Doctoral School Computer Science, University of Vienna,
Austria 3 MaiNLP, Center for Information and Language Processing, LMU Munich,
Germany 4 Munich Center for Machine Learning (MCML), Munich, Germany 5
Department of Linguistics, University of Konstanz 6 Faculty of Philological
and Cultural Studies, University of Vienna, Austria
<EMAIL_ADDRESS><EMAIL_ADDRESS>

**footnotetext: Equal contribution

##  1 Introduction

Large language models (LLMs) have recently demonstrated remarkable performance
in a variety of natural language processing tasks OpenAI (2024); Meta (2024);
Touvron et al. (2023), also largely due to the extensive factual knowledge
they accumulate during pre-training. A crucial factor in building trust in
models is their capacity to generate consistent and reliable
outputs—especially in light of ambiguity—and their consistency with their
internal knowledge Li et al. (2024); Zhao et al. (2024). However, LLMs
frequently produce unreliable responses: e.g., when externally provided
knowledge conflicts with internal parametric knowledge Xie et al. (2024); Pan
et al. (2023), or when models are exposed to misinformation during pretraining
Zhao et al. (2024); Chen et al. (2023). Such conflicts often lead to
inconsistencies in model responses, reducing LLMs trustworthiness Sun et al.
(2024); Litschko et al. (2023). In our work, we identify entity ambiguity as a
source of unreliability. Our study is similar to KoLA (Yu et al., 2024), a
large-scale quantitative benchmark for evaluating how well LLMs apply their
world knowledge. In contrast, we focus on an in-depth qualitative analysis to
understand model behaviour under ambiguity.

Entity Type | List of Entities |  Entity Property
---|---|---
animal |  Jaguar, Puma, Penguin, Greyhound, Dove, Fox, Lynx |  speed
fruit |  Apple, Fig, Mango, Kiwi, Papaya, Orange |  color
myth |  Amazon, Nike, Midas, Mars, Hermes, Hyperion, Vulcan, Pegasus |  gender
person |  Ford, Disney, Tesla, Boeing, Dell, Ferrero, Benetton, Levi Strauss, Versace, Philips |  date of birth
location |  Amazon, Cisco, Montblanc, Patagonia, Hershey, Nokia, Eagle Creek, Prosper |  area in m2
abstract |  Triumph, Harmony, Genesis, Vision, Pioneer, Vanguard, Zenith, Allure, Tempo, Fidelity |  level of abstractness
company |  all entities listed above |  founding year
Table 1: Overview of ambiguous entities. We use a total of 49 entities
belonging to 7 entity types. The entities are chosen such that have at least
two readings: the listed entity type and company. Entity properties are chosen
such that the entity type can be uniquely inferred from it. Figure 1: We focus
on entity type ambiguity to study self-consistency of LLMs. Overview of our 4
studies.

This paper examines the self-consistency111We analyze consistency in “internal
knowledge retrieval” on straightforward, fact-intensive tasks that do not
necessitate CoT prompting as in, e.g., Wang et al. (2023). of state-of-the-art
LLMs—i.e., how well they align with their intrinsic knowledge while avoiding
contradictory responses Chen et al. (2024)—by evaluating their reasoning
abilities in contexts involving entity type ambiguity, a commonly encountered
challenge for LLMs Parcalabescu and Frank (2024); Kim et al. (2024); Parrish
et al. (2022). Importantly, in our study we provide an operationalization to
disentangle LLM’s capabilities of Knowing222Here and further, we use the term
“knowing” to refer to parametric knowledge as discussed in Mallen et al.
(2023); Litschko et al. (2023). (i.e., how aware and sensitive a model is to
the possible interpretations, or readings, of ambiguous entities), and
Applying knowledge (i.e., how well a model can identify the correct reading
when prompted with entity-specific questions and use their parametric
knowledge to provide accurate responses about that entity). The overarching
goal of this work is thus to study the interplay between a model’s knowledge
about different entity readings and their ability to infer the correct reading
for a given prompt. For example, as shown in Figure 1, if a model “knows” that
Apple can be a fruit and a company, to what extent can we assume that they
also infer the company meaning when asked about its founding year? Knowing if
an LLM can disambiguate an entity333Importantly, we measure the ability to
disambiguate entities empirically by comparing their question answering
performance on an ambiguous question (“What is the founding year of X?”)
against a non-ambiguous question (“What is the founding year of company X?”).
allows us to minimize the number of clarification questions (Xu et al., 2019;
Lee et al., 2023) and facilitate more natural conversations. Similarly, if a
model responds with “Apple was founded in 1976,” can we assume that it is
self-consistent with its own answer? We systematically investigate these
questions by providing a testing suite, thereby characterizing the behaviour
of LLMs under entity ambiguity.

More specifically, we aim to answer the following three research questions:
Assuming a model “knows” about different entity types, how well can it
disambiguate them in a given prompt (RQ1)? Can LLMs self-verify their answers
for entity-related questions, given they have successfully disambiguated it
(RQ2)? To what extent is the ability to infer the correct entity type biased
towards “preferred readings”? In the case of specific entities, can the
preference for reading be explained by their popularity (RQ3)?

To this end, we analyze the behavior of six state-of-the-art LLMs (differing
in size, type, and open vs. proprietary) on 49 entities (see Section 2):
Gemma-1.1-7B-IT Google (2024), Mistral-7B-Instruct Jiang et al. (2023),
Llama-3-70B Meta (2024), Mixtral-8x7B Jiang et al. (2024), GPT-3.5 OpenAI
(2022), and GPT-4o OpenAI (2024)444Implementation details are provided in
Appendix F.. Our results show that, despite the seemingly simple task, LLMs
fail to disambiguate and handle entities consistently.

##  2 Methodology

To study the ability of LLMs to implicitly infer the correct entity meanings,
we use a set of forty-nine entities, as shown in Table 1. All entities can be
interpreted as (1) one of the six listed entity types or (2) company names.
That is, each entity has at least two entity types and can therefore be
interpreted in at least two different ways. We adopt this framework to
distinguish between a preferred and an alternative reading, which allows us to
investigate if the disambiguation ability of LLMs is consistent or biased
across different entity types.

Our research comprises four studies (see Figure 1). Study 1 verifies knowledge
possession in models; Studies 2 and 3 assess the models’ abilities to apply
this knowledge (K $\rightarrow$ A); and Study 4 evaluates the knowledge
possession post-application (A $\rightarrow$ K). Collectively, the results of
our four experiments provide us a way to gain knowledge on how LLMs treat
entity level ambiguity, i.e., the mutual relationship K $\leftrightarrow$ A.
We discuss each study and our results in more detail next.

### Study 1: Knowledge Verification (K).

First, we analyze the models’ knowledge by verifying their awareness of
different entity readings. To this end, we use the prompt template “Tell me
about <entity-type> <entity>” to manually verify that all LLMs generate
meaningful output conforming to world knowledge. We cure the list of entities
(see Table 1) to make sure they all pass Study 1. Apart from that, we directly
ask the models whether they are aware of ambiguity (“Can <entity> mean
anything else but <entity-type>? Answer only with Yes or No.”) - the results
are provided in Appendix B.

### Study 2: Eliciting Preferences (K + A).

As mentioned above, each entity has been chosen such that it has at least two
entity types. Intuitively, if a model has been exposed to the company Cisco
far more often than the location Cisco (city in Texas), we would assume that
it is biased towards the former interpretation. We refer to it as its
preferred reading. To investigate if a model’s behaviour is affected by a
preferred reading (RQ3), i.e., if the answer correctness increases (decreases)
if the question refers to a preferred (alternative) entity interpretation, we
prompt LLMs with "Group the following entities according to what they all have
in common: <entities>", where <entities> refers to all members of a given
category. To ensure robust results, we rephrased each prompt four times and
then aggregate the model replies by majority voting. To assess the LLM output,
each prompt answer was manually checked (see Appendix C for details and
further discussion). In Figure 2 we show the preferred interpretation of each
entity group by each model (compared to company). Interestingly, except for
Llama-3, all LLMs display a clear entity type preference. All LLMs prefer the
animal and fruit reading over the company interpretation.

Figure 2: Preferred readings by the models for each entity type (blue for non-
company, yellow for company).

### Study 3: Knowledge to Application (K $\rightarrow$ A).

We proceed to test the knowledge application ability by examining if LLMs
adopt the correct reading for ambiguous entities (after knowledge of both
readings is confirmed in Study 1), and whether LLMs accurately answer simple
questions related to entity properties. We use the prompt template “Provide
the <entity-property> for <entity>.” to evaluate if LLMs are capable to
implicitly infer <entity-type>. For example, a model should infer company when
prompted for founding year. We compare their performance against a non-
ambiguous baseline with explicit entity hint, which serves as an upper bound:
“Provide the <entity-property> for <entity-type><entity>.”

### Study 4: Applying to Knowing (A $\rightarrow$ K).

Finally, we aim to determine how consistent the models are with their own
internal knowledge. We manually retrieve factual information from the model
replies in Study 3 (further referred to as <info>) and prompt the same model
back to see if they either confirm or deny the correctness of provided
information. For example, the knowledge about the non-company reading of
“animals” entities is checked with prompts like "Does an animal X have <info>
speed?" (see also Tables 6-9 in Appendix). Thus, in this setup we operate
under a closed world assumption and focus only on consistency within the
model’s internal knowledge, ensuring a fair comparison across models of
different sizes.

Model | Preferred Reading | Alternative Reading | Average
---|---|---|---
prop X |  prop type X |  prop X |  prop type X |  prop X |  prop type X |  Agg.
Gemma Google (2024) | 87.8 | 95.9 | 63.3 | 69.4 | 75.6 | 82.7 |  77.6
Mistral Jiang et al. (2023) | 77.6 | 100.0 | 63.3 | 87.8 | 70.5 | 93.9 |  82.2
Mixtral Jiang et al. (2024) | 77.6 | 100.0 | 75.5 | 85.7 | 76.6 | 92.9 |  84.8
GPT-3.5 OpenAI (2022) | 87.8 | 100.0 | 75.5 | 77.6 | 81.7 | 88.8 |  85.3
GPT-4o OpenAI (2024) | 93.9 | 100.0 | 83.7 | 89.8 | 88.8 | 94.9 |  91.9
Llama-3 Meta (2024) | 87.8 | 98.0 | 85.7 | 100.0 | 86.8 | 99.0 |  89.9
Average | 85.4 | 99.0 | 74.5 | 85.1 | 80.0 | 90.5 |  85.3
Table 2: Results of Study 3: Knowledge to Application (% of correct replies).
‘‘prop’’ stands for reading-specific property, ‘‘type’’ \- for the
corresponding entity type (see Table 1). An example of ‘‘prop X’’ prompt:
“Provide the founding year of Apple”, an example of ‘‘prop type X’’ prompts:
“Provide the founding year of company Apple.” Figure 3: Results of Study 4 (%
of all replies). "Consistent" means the model reaffirmed all knowledge
provided in Study 3, "partially consistent" - some but not all, and
"inconsistent" shows denial of all previous information. The exact numbers are
provided in Appendix (Table 5).

##  3 Results and Discussion

### RQ1: How well can LLMs implicitly disambiguate entity types?

By design, we used entities that passed Study 1, i.e., LLMs are able to
generate output that conforms to external word knowledge. We present our main
results (Study 3) in Table 2. On average, LLMs are able to respond with the
correct property value for 80% of all entities. Even if we use a prompt with
hint so that the entity type is non-ambiguous (e.g., “Provide the founding
year for company Apple”) LLMs reach 90.5, thus fail in $\sim$10% of all
entities.

We observe striking differences when we break the results further down into
preferred and alternative readings. For preferred readings, LLMs reach 85.4%
accuracy with ambiguous prompts, and this increases to almost perfect
performance in non-ambiguous prompts with hints (99%). However, the results
are substantially lower for non-preferred (alternative readings), where
performance drops to 74.5/85.1%. This shows a clear bias of all models to
preferred readings. We further look at the correlation between model size and
the amount of incorrect readings, finding remarkable trends: e.g., Gemma is
the smallest and worst performing model with only $77.6\%$ of correctly picked
readings, while for Llama-3 and GPT-4o are the largest and best performing
models with $\sim$90%. These results reinforces our point that the models
often have difficulties in applying knowledge they actually possess, as
demonstrated by Study 1, and LLMs consistency is largely affected by preferred
reading (RQ3). Additionally, we observed a notable correlation between the
models performance on the individual entities and their popularity - we
elaborate more on it in Appendix D.

### RQ2: Can LLMs self-verify their answers, given that they successfully
disambiguated them?

We now investigate whether successful disambiguation implies that the model’s
answers can be self-verified (Study 4). In other words, we seek to verify
whether a model remains consistent with its previous statements. As Figure 3
shows, none of the tested models confirmed all the knowledge provided in the
previous study. On average, LLMs show a higher (partial) consistency under
preferred readings. Consistent with our previous findings, Llama-3-70B emerged
as the most self-consistent model, being consist in about 89% of its
responses, while Mistral performed worst (>30% answers in alternative reading
could not be self-verified).

### RQ3: Does entity popularity explain the "default reading" of an LLM?

We hypothesize that a model’s preferred reading is influenced by its frequency
in the pre-training corpus. For example, if nike mostly appears in the context
of companies (as opposed to the context of a Greek goddess), we would expect
this meaning to dominate over other readings. We follow Mallen et al. (2023)
and use Wikipedia popularity as a proxy for entity type frequency. Our
experiments indeed demonstrate the strong correlation between the entity
popularity and a preferred reading (see Appendices A, C, and D. Additionally,
we found a correlation between the model performance and the ambiguity of the
entities (see Appendices C.

##  4 Conclusion

We find that state-of-the-art LLMs perform poorly on on prompts that require
to implicitly disambiguate entity types. Furthermore, their performance is
biased by a preferred reading. Finally, we find that LLMs cannot self-verify
their own answers. Our results highlight the lack of self-consistency as an
open challenge of current LLMs.

##  5 Limitations

In this study, we adopt a very generic definition of ambiguity, distinguishing
between company-related and non-company-related company vs. non-company
readings across different entity types. A more thorough investigation into the
degrees of polysemy associated with different entity types should be included
in a follow up study. Moreover, the properties of the entities might also
contain a certain level of ambiguity that we are not thoroughly addressing in
this work.

## References

  * Chen et al. (2024) Angelica Chen, Jason Phang, Alicia Parrish, Vishakh Padmakumar, Chen Zhao, Samuel R. Bowman, and Kyunghyun Cho. 2024.  Two failures of self-consistency in the multi-step reasoning of LLMs.  _Transactions on Machine Learning Research_.
  * Chen et al. (2023) Jiangjie Chen, Wei Shi, Ziquan Fu, Sijie Cheng, Lei Li, and Yanghua Xiao. 2023.  Say what you mean! large language models speak too positively about negative commonsense knowledge.  In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 9890–9908, Toronto, Canada. Association for Computational Linguistics.
  * Google (2024) Google. 2024.  Gemma: Open models based on gemini research and technology.  _Preprint_ , arXiv:2403.08295.
  * Jiang et al. (2023) Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. 2023.  Mistral 7b.  _Preprint_ , arXiv:2310.06825.
  * Jiang et al. (2024) Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock, Sandeep Subramanian, Sophia Yang, Szymon Antoniak, Teven Le Scao, Théophile Gervet, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. 2024.  Mixtral of experts.  _Preprint_ , arXiv:2401.04088.
  * Kim et al. (2024) Hyuhng Joon Kim, Youna Kim, Cheonbok Park, Junyeob Kim, Choonghyun Park, Kang Min Yoo, Sang goo Lee, and Taeuk Kim. 2024.  Aligning language models to explicitly handle ambiguity.  _Preprint_ , arXiv:2404.11972.
  * Lee et al. (2023) Dongryeol Lee, Segwang Kim, Minwoo Lee, Hwanhee Lee, Joonsuk Park, Sang-Woo Lee, and Kyomin Jung. 2023.  Asking clarification questions to handle ambiguity in open-domain QA.  In _Findings of the Association for Computational Linguistics: EMNLP 2023_ , pages 11526–11544, Singapore. Association for Computational Linguistics.
  * Li et al. (2024) Xiang Lisa Li, Vaishnavi Shrivastava, Siyan Li, Tatsunori Hashimoto, and Percy Liang. 2024.  Benchmarking and improving generator-validator consistency of language models.  In _The Twelfth International Conference on Learning Representations_.
  * Litschko et al. (2023) Robert Litschko, Max Müller-Eberstein, Rob van der Goot, Leon Weber-Genzel, and Barbara Plank. 2023.  Establishing trustworthiness: Rethinking tasks and model evaluation.  In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing_ , pages 193–203, Singapore. Association for Computational Linguistics.
  * Mallen et al. (2023) Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2023.  When not to trust language models: Investigating effectiveness of parametric and non-parametric memories.  _Preprint_ , arXiv:2212.10511.
  * Meta (2024) Meta. 2024.  Introducing meta llama 3: The most capable openly available llm to date.
  * OpenAI (2022) OpenAI. 2022.  Introducing chatgpt.
  * OpenAI (2024) OpenAI. 2024.  Hello gpt-4o.
  * Pan et al. (2023) Liangming Pan, Wenhu Chen, Min-Yen Kan, and William Yang Wang. 2023.  Attacking open-domain question answering by injecting misinformation.  In _Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 525–539, Nusa Dua, Bali. Association for Computational Linguistics.
  * Parcalabescu and Frank (2024) Letitia Parcalabescu and Anette Frank. 2024.  On measuring faithfulness or self-consistency of natural language explanations.  _Preprint_ , arXiv:2311.07466.
  * Parrish et al. (2022) Alicia Parrish, Angelica Chen, Nikita Nangia, Vishakh Padmakumar, Jason Phang, Jana Thompson, Phu Mon Htut, and Samuel Bowman. 2022.  BBQ: A hand-built bias benchmark for question answering.  In _Findings of the Association for Computational Linguistics: ACL 2022_ , pages 2086–2105, Dublin, Ireland. Association for Computational Linguistics.
  * Sun et al. (2024) Lichao Sun, Yue Huang, Haoran Wang, Siyuan Wu, Qihui Zhang, Yuan Li, Chujie Gao, Yixin Huang, Wenhan Lyu, Yixuan Zhang, Xiner Li, Zhengliang Liu, Yixin Liu, Yijue Wang, Zhikun Zhang, Bertie Vidgen, Bhavya Kailkhura, Caiming Xiong, Chaowei Xiao, Chunyuan Li, Eric Xing, Furong Huang, Hao Liu, Heng Ji, Hongyi Wang, Huan Zhang, Huaxiu Yao, Manolis Kellis, Marinka Zitnik, Meng Jiang, Mohit Bansal, James Zou, Jian Pei, Jian Liu, Jianfeng Gao, Jiawei Han, Jieyu Zhao, Jiliang Tang, Jindong Wang, Joaquin Vanschoren, John Mitchell, Kai Shu, Kaidi Xu, Kai-Wei Chang, Lifang He, Lifu Huang, Michael Backes, Neil Zhenqiang Gong, Philip S. Yu, Pin-Yu Chen, Quanquan Gu, Ran Xu, Rex Ying, Shuiwang Ji, Suman Jana, Tianlong Chen, Tianming Liu, Tianyi Zhou, William Wang, Xiang Li, Xiangliang Zhang, Xiao Wang, Xing Xie, Xun Chen, Xuyu Wang, Yan Liu, Yanfang Ye, Yinzhi Cao, Yong Chen, and Yue Zhao. 2024.  Trustllm: Trustworthiness in large language models.  _Preprint_ , arXiv:2401.05561.
  * Touvron et al. (2023) Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. 2023.  Llama 2: Open foundation and fine-tuned chat models.  _Preprint_ , arXiv:2307.09288.
  * Wang et al. (2023) Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023.  Self-consistency improves chain of thought reasoning in language models.  In _The Eleventh International Conference on Learning Representations_.
  * Xie et al. (2024) Jian Xie, Kai Zhang, Jiangjie Chen, Renze Lou, and Yu Su. 2024.  Adaptive chameleon or stubborn sloth: Revealing the behavior of large language models in knowledge conflicts.  In _The Twelfth International Conference on Learning Representations_.
  * Xu et al. (2019) Jingjing Xu, Yuechen Wang, Duyu Tang, Nan Duan, Pengcheng Yang, Qi Zeng, Ming Zhou, and Xu Sun. 2019.  Asking clarification questions in knowledge-based question answering.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 1618–1629, Hong Kong, China. Association for Computational Linguistics.
  * Yu et al. (2024) Jifan Yu, Xiaozhi Wang, Shangqing Tu, Shulin Cao, Daniel Zhang-Li, Xin Lv, Hao Peng, Zijun Yao, Xiaohan Zhang, Hanming Li, Chunyang Li, Zheyuan Zhang, Yushi Bai, Yantao Liu, Amy Xin, Kaifeng Yun, Linlu GONG, Nianyi Lin, Jianhui Chen, Zhili Wu, Yunjia Qi, Weikai Li, Yong Guan, Kaisheng Zeng, Ji Qi, Hailong Jin, Jinxin Liu, Yu Gu, Yuan Yao, Ning Ding, Lei Hou, Zhiyuan Liu, Xu Bin, Jie Tang, and Juanzi Li. 2024.  KoLA: Carefully benchmarking world knowledge of large language models.  In _The Twelfth International Conference on Learning Representations_.
  * Zhao et al. (2024) Yukun Zhao, Lingyong Yan, Weiwei Sun, Guoliang Xing, Chong Meng, Shuaiqiang Wang, Zhicong Cheng, Zhaochun Ren, and Dawei Yin. 2024.  Knowing what llms do not know: A simple yet effective self-detection method.  _Preprint_ , arXiv:2310.17918.

##  Appendix A Entity Popularity and Ambiguity

Type | Entity |  Ambiguity | Company Reading | Non-Company Reading
---|---|---|---|---
|  Popularity (views) |  prop X |  prop type X |  Popularity (views) |  prop X |  prop type X
Animal | Penguin |  55 |  1,330,112 |  100.0 |  100.0 |  8,965,921 |  100.0 |  100.0
Jaguar |  53 |  7,989,902 |  100.0 |  100.0 |  11,939,755 |  0.0 |  100.0
Greyhound |  36 |  1,823,476 |  100.0 |  100.0 |  3,380,437 |  33.3 |  100.0
Fox |  89 |  3,648,500 |  100.0 |  100.0 |  9,301,784 |  100.0 |  100.0
Dove |  50 |  3,796 |  100.0 |  100.0 |  4,244,904 |  50.0 |  83.3
Lynx |  78 |  1,057,210 |  100.0 |  100.0 |  6,650,833 |  83.3 |  100.0
Puma |  45 |  4,701,402 |  100.0 |  100.0 |  11,554,347 |  83.3 |  100.0
Avg |  58.0 |  2,936,343 |  100 |  100 |  8,005,426 |  64 |  98
Fruit | Apple |  49 |  40,325,969 |  100.0 |  100.0 |  10,948,070 |  33.3 |  100.0
Fig |  15 |  129,832 |  100.0 |  83.3 |  2,248,635 |  83.3 |  100.0
Mango |  43 |  823,939 |  100.0 |  100.0 |  8,713,110 |  100.0 |  100.0
Kiwi |  36 |  293,874 |  100.0 |  100.0 |  6,245,271 |  100.0 |  100.0
Papaya |  12 |  - |  100.0 |  100.0 |  4,770,845 |  100.0 |  100.0
Orange |  103 |  2,007,461 |  100.0 |  100.0 |  7,409,145 |  66.7 |  83.3
Avg |  43.0 |  8,716,215 |  100.0 |  97.2 |  6,722,513 |  80.6 |  97.2
Myth. Character | Pegasus |  86 |  1,773,226 |  33.3 |  83.3 |  4,853,706 |  100.0 |  100.0
Vulcan |  79 |  635,380 |  66.7 |  100.0 |  2,673,387 |  0.0 |  100.0
Midas |  38 |  187,394 |  83.3 |  83.3 |  3,687,467 |  100.0 |  100.0
Nike |  34 |  18,187,528 |  100.0 |  100.0 |  4,375,918 |  33.3 |  100.0
Mars |  134 |  259,189 |  33.3 |  100.0 |  19,365,488 |  66.7 |  100.0
Hyperion |  62 |  58,794 |  66.7 |  100.0 |  1,316,548 |  83.3 |  100.0
Hermes |  56 |  3,426,101 |  83.3 |  100.0 |  10,337,899 |  100.0 |  100.0
Amazon |  64 |  38,684,687 |  100.0 |  100.0 |  5,119,820 |  16.7 |  100.0
Avg. |  69.1 |  7,901,537 |  70.8 |  95.8 |  6,466,279 |  62.5 |  100.0
Person | Versace |  13 |  7,095,079 |  100.0 |  100.0 |  22,180,811 |  100.0 |  66.7
Boeing |  - |  10,754,848 |  100.0 |  100.0 |  681,877 |  0.0 |  33.3
Ford |  104 |  14,643,256 |  100.0 |  100.0 |  13,966,210 |  83.3 |  50.0
Philips |  6 |  5,948,052 |  100.0 |  100.0 |  331,229 |  16.7 |  33.3
Levi Strauss |  13 |  3,744,382 |  100.0 |  100.0 |  2,320,188 |  100.0 |  100.0
Ferrero |  4 |  3,447,282 |  100.0 |  100.0 |  409,662 |  66.7 |  66.7
Tesla |  21 |  23,462,104 |  100.0 |  100.0 |  37,395,340 |  83.3 |  83.3
Disney |  58 |  20,938,263 |  100.0 |  100.0 |  31,693,370 |  100.0 |  50.0
Dell |  22 |  7,310,499 |  100.0 |  100.0 |  3,558,086 |  16.7 |  33.3
Benetton |  5 |  1,864,193 |  100.0 |  100.0 |  378,208 |  50.0 |  50.0
Avg. |  27.3 |  9,920,796 |  100.0 |  100.0 |  11,291,498 |  61.7 |  56.7
Location | Cisco |  26 |  1,738,862 |  100.0 |  100.0 |  - |  0.0 |  100.0
Prosper |  10 |  276,714 |  100.0 |  83.3 |  419,461 |  33.3 |  100.0
Patagonia |  12 |  1,055,737 |  100.0 |  100.0 |  11,426,844 |  100.0 |  100.0
Montblanc |  5 |  1,306,077 |  100.0 |  100.0 |  5,671,509 |  100.0 |  100.0
Amazon |  64 |  38,684,687 |  100.0 |  100.0 |  6,509,535 |  33.3 |  100.0
Nokia |  13 |  11,446,036 |  100.0 |  100.0 |  332,572 |  0.0 |  83.3
Hershey |  24 |  3,929,199 |  100.0 |  100.0 |  1,419,873 |  100.0 |  100.0
Eagle Creek |  24 |  55,717 |  100.0 |  100.0 |  2,248 |  83.3 |  100.0
Avg. |  22.3 |  7,311,629 |  100.0 |  97.9 |  3,683,149 |  58.3 |  95.8
Abstract | Harmony |  119 |  143,865 |  83.3 |  83.3 |  1,847,278 |  100.0 |  100.0
Fidelity |  29 |  3,648,171 |  100.0 |  100.0 |  633,474 |  100.0 |  100.0
Allure |  17 |  832,160 |  100.0 |  100.0 |  728,597 |  50.0 |  100.0
Vision |  102 |  29,660 |  100.0 |  100.0 |  1,810,577 |  100.0 |  100.0
Genesis |  141 |  2,809,401 |  50.0 |  100.0 |  6,338,641 |  100.0 |  100.0
Tempo |  59 |  27,507 |  100.0 |  100.0 |  5,416,890 |  66.7 |  100.0
Triumph |  45 |  351,267 |  100.0 |  100.0 |  1,132,962 |  83.3 |  100.0
Vanguard |  128 |  6,661,130 |  100.0 |  100.0 |  1,059,408 |  16.7 |  83.3
Pioneer |  95 |  1,058,945 |  100.0 |  100.0 |  521,227 |  66.7 |  100.0
Zenith |  64 |  753,374 |  100.0 |  100.0 |  1,602,303 |  100.0 |  100.0
Avg |  79.9 |  1,631,548 |  93.3 |  98.3 |  2,109,136 |  78.3 |  98.3
Table 3: Summary of entity types their characteristics: ambiguity and
popularity. Following Mallen et al. (2023), we evaluate the popularity and
ambiguity of each entity based on Wikipedia page views and the number of pages
references to on the Wikipedia entity disambiguation page, respectively.
Dashes are used in cases where Wikipedia disambiguation page is absent for the
specific entity. Additionally, we provide the model performance on each entity
demonstrated in Study 3, aggregated across the models.

In Table 3, we present additional information about all entities utilized in
our experiments. Following Mallen et al. (2023), we assess the popularity of
each entity (in our context: each entity’s interpretation, such as company-
related and non-company-related) based on Wikipedia page views over the past
nine years. In instances of ambiguity within a single interpretation (e.g.,
multiple companies sharing the same name, or multiple individuals with the
same surname), we selected the most popular one. Furthermore, we estimated the
ambiguity of each entity using its corresponding Wikipedia disambigation page,
for example: https://en.wikipedia.org/wiki/ Jaguar_(disambiguation).
Specifically, we counted the number of pages listed on the disambiguation
page, providing a preliminary estimate of the number of real-world entities to
which the term could refer.

Additionally, in order to evaluate correlation between the performance of the
models on individual entities and the popularity of these entities, we
aggregated the results of Study 3 across all models for each entity.
Specifically, for each of the entity readings, we counted how many times each
model selected a correct interpretation when providing response to a relevant
prompt and calculated the average. For example, the performance of the models
for entity Jaguar in its company reading was aggregated from the replies of
all models to the prompt "Provide the founding year for the company Jaguar".

The plots representing the entities’ popularity are presented in Figure 4.

Model |  Animals |  Fruits |  Myths |  People |  Locations |  Abstract |  Average
---|---|---|---|---|---|---|---
Gemma |  100.0 |  100.0 |  37.5 |  0.0 |  12.5 |  10.0 |  43.3
Mistral |  100.0 |  83.8 |  75.0 |  10.0 |  75.0 |  90.0 |  72.3
Mixtral |  71.4 |  50.0 |  0.0 |  0.0 |  30.0 |  50.0 |  93.1
GPT-3.5 |  57.1 |  100.0 |  0.0 |  10.0 |  12.5 |  10.0 |  31.6
GPT-4o |  100.0 |  100.0 |  100.0 |  60.0 |  100.0 |  90.0 |  91.7
LLaMa-3 |  100.0 |  100.0 |  100.0 |  100.0 |  100.0 |  100.0 |  100.0
Average |  80.1 |  89.0 |  52.1 |  50.0 |  52.8 |  50.0 |  72.0
Table 4: The results of experiments with direct prompting the model about the ambiguity ("Can <entity> mean anything else but <entity-type>? Answer only with Yes or No."). Model | Preferred Reading | Alternative Reading
---|---|---
Consistent |  Partially Consistent |  Inconsistent |  Consistent |  Partially Consistent |  Inconsistent
Companies Reading
Gemma |  38.8 |  0 |  18.4 |  32.7 |  0 |  10.2
Mistral |  49.0 |  4.1 |  4.1 |  24.5 |  10.2 |  8.2
MiXtral |  53.1 |  0 |  4.1 |  34.7 |  2.0 |  6.1
GPT-3.5 |  32.7 |  0 |  4.1 |  38.8 |  4.1 |  20.4
GPT-4o |  36.7 |  0 |  0 |  57.1 |  4.1 |  2.0
LLaMa-3 |  53.1 |  0 |  0 |  46.9 |  0 |  0
| Animals Reading
Gemma |  57.1 |  42.9 |  0.0 |  - |  - |  -
Mistral |  28.6 |  42.9 |  29.0 |  - |  - |  -
Mixtral |  100.0 |  0.0 |  0.0 |  - |  - |  -
GPT-3.5 |  85.7 |  14.3 |  0.0 |  - |  - |  -
GPT-4o |  86.0 |  0.0 |  14.0 |  - |  - |  -
LLaMa-3 |  57.1 |  42.9 |  0.0 |  - |  - |  -
| Fruits Reading
Gemma |  83.3 |  16.7 |  0.0 |  - |  - |  -
Mistral |  0.0 |  100.0 |  0.0 |  - |  - |  -
Mixtral |  16.7 |  83.3 |  0.0 |  - |  - |  -
GPT-3.5 |  66.7 |  33.3 |  0.0 |  - |  - |  -
GPT-4o |  83.3 |  16.7 |  0.0 |  - |  - |  -
LLaMa-3 |  33.3 |  66.7 |  0.0 |  - |  - |  -
| Myths Reading
Gemma |  100 |  0.0 |  0.0 |  - |  - |  -
Mistral |  100 |  0.0 |  0.0 |  - |  - |  -
Mixtral |  87.5 |  12.5 |  0.0 |  - |  - |  -
GPT-3.5 |  100.0 |  0.0 |  0.0 |  - |  - |  -
GPT-4o |  100.0 |  0.0 |  0.0 |  - |  - |  -
LLaMa-3 |  - |  - |  - |  100.0 |  - |  -
| People Reading
Gemma |  - |  - |  - |  90.0 |  0.0 |  10.0
Mistral |  - |  - |  - |  40.0 |  0.0 |  60.0
Mixtral |  - |  - |  - |  60.0 |  0.0 |  40.0
GPT-3.5 |  - |  - |  - |  60.0 |  0.0 |  40.0
GPT-4o |  - |  - |  - |  70.0 |  0.0 |  30.0
LLaMa-3 |  - |  - |  - |  90.0 |  0.0 |  10.0
| Locations Reading
Gemma |  - |  - |  - |  100.0 |  0.0 |  0.0
Mistral |  - |  - |  - |  0.0 |  37.0 |  63.0
Mixtral |  - |  - |  - |  25.0 |  0.0 |  75.0
GPT-3.5 |  - |  - |  - |  37.0 |  0.0 |  63.0
GPT-4o |  - |  - |  - |  62.0 |  0.0 |  38.0
LLaMa-3 |  - |  - |  - |  100.0 |  0.0 |  0.0
| Abstract Reading
Gemma |  - |  - |  - |  100.0 |  0.0 |  0.0
Mistral |  - |  - |  - |  30.0 |  50.0 |  20.0
Mixtral |  - |  - |  - |  80.0 |  20.0 |  0
GPT-3.5 |  100.0 |  0.0 |  0.0 |  - |  - |  -
GPT-4o |  40.0 |  60.0 |  0.0 |  - |  - |  -
LLaMa-3 |  70.0 |  20.0 |  10.0 |  - |  - |  -
Table 5: Results from Study 4 (the graphical interpretation of these findings
is illustrated in Figure 3). For non-company readings, we present the results
for each group separately. Depending on whether this reading is preferred by
each model or not, the results are provided in the corresponding column.
Figure 4: Popularity distribution of company and non-company readings of all
49 entities involved in our studies.

##  Appendix B Study 1: Further Discussion

The results from directly prompting the model to determine its awareness of
ambiguity, using the prompt "Can <entity> mean anything else but <entity-
type>? Answer only with Yes or No.", are provided in Table 4. As it becomes
clear, despite possessing knowledge about the different meanings of each
entity (as proven by Study 1, see Section 3), the models tend to struggle to
provide this information when asked directly. For example, Mixtral often
denies other interpretations, justifying this by claiming that there is one
clear meaning of the entity, although it can be used for other purposes (for
example, "No, Eagle Creek cannot mean anything else in general usage. It is
primarily a geographic location, specifically a creek name occurring in
various places in the United States. However, like many place names, it can be
used as a proper noun in other contexts, such as brand names (e.g., Eagle
Creek luggage)."). From this observation, we make two assumptions: (1) each
model may have a preferred interpretation for each entity and entity type, an
hypothesis we intend to explore in Study 2, and (2) a more carefully
considered experimental setup is required, rather than straightforwardly
querying the model about ambiguity, which was one of the motivations behind
the grouping task approach we adopted for Study 2.

##  Appendix C Study 2: Further Discussion

We noticed that for most of the entity groups (Fruits, Locations, Animals, and
People), all analyzed models clearly prefer one reading over the other.
Notably, large models like Llama-3 and Mixtral, even though ultimately
grouping based on one single reading, demonstrate an understanding of entity
ambiguity (e.g., Mixtral: "All of the words you’ve listed are common names for
either a type of animal or a brand. To be more specific, they are all common
names for either a type of mammal or a type of bird…", or LlaMa-3: "After
examining the list, I noticed that all the mentioned animals have one thing in
common: they are all names of car models or brands at some point in
history…"). However, in these replies, the model still prefers one reading
over the other and use it for grouping the entities; in such cases, we
consider this reading to be a preferred reading.

The categories Abstract and Myths elicit the most diverse responses from
models. This could be explained by the particularly high ambiguity associated
with the entities in these categories, beyong merely "companies" and "entity
types" \- e.g., in the latter, "planets and moons" (e.g., Mars, Vulcan,
Hyperion). Indeed, Table 3, where the ambiguity of each entity is estimated
based on Wikipedia disambiguation pages (a potentially conservative measure,
as not all objects a particular entity may refer to have Wikipedia pages),
shows the highest average rate of ambiguity across entities within these
categories: $79.9$ for abstract entities and $69.1$ for myths
entities555Please note that these numbers are not normalized and calculated as
an average of the absolute numbers for each entity.. As a result, for these
categories, models frequently mix readings, distinguishing "companies" as a
separate group while also identifying non-company meanings, leading to
groupings like: "Greek Gods", "Roman Gods", and "Companies". However, such
responses do not clarify if the model recognizes ambiguity, as adopting both
"company" and "non-company" reading in the same response could indicate either
a misunderstanding of entities ambiguity (i.e., the model recognize some
entities as companies and others not, despite evidence to the contrary from
Study 1) or simply a preference for a specific reading for specific entities.

Moreover, we hypothesize that a model’s preferred reading of an entity is
influenced by its frequency in the pre-training corpus. For example, if apple
mostly appears in the context of fruits, we would expect this meaning to
dominate over other readings. Thus, whenever a model categorized one or more
Myths entities to a separate "companies" group, they were always either only
the Amazon entity or both Amazon and Nike. For example:

  * •

Prompt: "Group the following entities: Amazon, Mars, Vulcan, Hermes, Hyperion,
Nike, Midas, Pegasus."

  * •

GPT-3.5 Response: "1. Mythological entities: Mars, Vulcan, Hermes, Hyperion,
Nike, Midas, Pegasus. 2\. Companies/brands: Amazon"

  * •

GPT-4o Response: "Certainly! These entities can be grouped based on their
categories or mythological references: By category: 1\. Brands and Companies:
- Nike - Amazon 2\. Greek Mythology: - Nike (Also a brand, but originally the
Greek goddess of victory) - Pegasus - Hermes 3. Roman Mythology: - Mars -
Vulcan 4. Mythological figures and references from various mythologies: -
Hyperion (Greek mythology, a Titan) - Midas (Greek mythology, king with the
golden touch) By mythology: 1. Greek Mythology: - Nike - Pegasus - Hermes -
Hyperion - Midas 2. Roman Mythology: - Mars (Roman counterpart to the Greek
god Ares) - Vulcan (Roman counterpart to the Greek god Hephaestus) 3\. Modern
entities named after mythological references: - Nike (brand named after the
Greek goddess of victory) \- Amazon (company name inspired by the Amazon
warriors of Greek mythology)"

  * •

Mixtral Response: "Sure, I will group these entities into the following
categories: 1. Planets: Mars, 2. Mythological figures: Hyperion, Midas, Nike,
Hermes, Vulcan, Pegasus, 3\. Companies: Amazon. Explanation: 1. Mars is a
planet in our solar system. 2. Hyperion, Midas, Nike, Hermes, Vulcan, and
Pegasus are all figures from Greek and Roman mythology. Hyperion is a Titan
and the father of the Sun, Moon, and Dawn. Midas was a king who was granted
the power to turn everything he touched into gold. Nike is the goddess of
victory, Hermes is the god of trade, thieves, travelers, and athletes, Vulcan
is the god of fire including the fire of volcanoes, and Pegasus is a winged
horse. 3. Amazon is a multinational technology company that focuses on
e-commerce, cloud computing, digital streaming, and artificial intelligence.
It is one of the world’s largest online marketplaces, AI assistant providers,
and cloud computing platforms."

This disparity can be logically attributed to the significantly higher
popularity of these companies compared to others on the list. For instance,
the Wikipedia page for Amazon has garnered over 38 million views, and Nike’s
page has received more than 18 million views. In contrast, Hermes, the third
most popular company within the myths category, has only 3 million views. This
observation confirms our hypothesis that a model’s behavior towards specific
entities may be influenced by their popularity. As a result, the models tend
to favor one interpretation over another for a given entity, regardless of the
prompt provided.

##  Appendix D Study 3: Further Discussion and Case Study

Some of the systematic errors of LLMs can be explained from observed model
behavior. For example, when prompted with "Provide the speed of jaguar,"
intended to draw an animal reading, all models reference the car and offer
speeds of Jaguar automobiles instead; for example:

  * •

Prompt: "Provide the speed for Jaguar"

  * •

Gemma Response: "Jaguar’s speeds vary depending on the model and engine:
Fastest Jaguar Models: Jaguar F-Pace SVR: 0-60 mph in 3.3 seconds, top speed
of 174 mph. Jaguar XJR Sport: 0-60 mph in 3.5 seconds, top speed of 183 mph.
Jaguar F-Type SVR: 0-60 mph in 2.9"

  * •

Mixtral Response: "Jaguar is a brand that manufactures high-performance luxury
cars and sports cars, so their speed can vary greatly depending on the
specific model. Here are a few examples of top speeds for some popular Jaguar
models: Jaguar F-Type: 171-195 mph (275-314 km/h), Jaguar F-Pace: 155 mph (250
km/h), Jaguar XE: 155 mph (250 km/h), Jaguar XF: 155 mph (250 km/h), Jaguar
XJ: 155 mph (250 km/h). Please note that the actual top speed of a Jaguar car
may vary based on the model year, engine type, and other factors. Also, these
top speeds should be taken as a general guideline only, and should not be
attempted on public roads as they may exceed legal speed limits."

A similar pattern occurs with prompts about Greyhound, a well-known bus
company in North America: 3 out of 6 models (namely, GPT-3.5, Mistral, and
Gemma) adopted non-animal (i.e., "bus") interpretation over the animal one:

  * •

Prompt: "Provide the speed for Greyhound"

  * •

GPT-3.5 Response: "The average speed for a Greyhound bus is around 55-70 miles
per hour, depending on traffic and road conditions."

  * •

Mistral Response: "The average speed of a Greyhound bus can vary depending on
the specific model and road conditions, but typically ranges between 60-65
miles per hour (97-105 kilometers per hour). Greyhound is known for its long-
distance bus services and aims to maintain a consistent speed to ensure
efficient travel times. However, overall average speed may be influenced by
factors such as traffic, road conditions, and rest stops. It’s important to
note that this is a general estimate and actual speeds may vary. For the most
accurate information, check the Greyhound website or contact their customer
service directly."

Although this mistake can be easily explained, we still classified such cases
as incorrect in our evaluation.

Another example of systematic error that indicates a consistent
misinterpretation of the provided prompts by different models, is a common
response with company foundation dates when the model is prompted for a date-
of-birth (the expected behavior is for the model to recognize that it should
provide the birth date of a person, thus adopting the person interpretation of
the entity in question):

  * •

Prompt: "Provide the date of birth for Boeing."

  * •

Mistral Response: "Boeing is not a person, but a company. It was founded on
July 15, 1916. However, if you meant to ask for the date of birth of a
specific Boeing aircraft model, please let me know the name of the model."

Sometimes the models even mention the name of the founder, yet still adhere to
the company interpretation when providing the response:

  * •

Prompt: "Provide the date of birth for Boeing."

  * •

Llama-3 Response: "Boeing’s founding date is July 15, 1916, when William E.
Boeing and Conrad Westervelt founded Pacific Aero Products Co., which later
became The Boeing Company."

In the meantime, the fact that all the models that were included in the
experiments indeed possess this knowledge was confirmed by an additional
sanity check using completely unambiguous prompts, where the entity X is
presented with both first and last names, indicating a preferred people
interpretation in the most unambiguous way:

  * •

Prompt: "What is the date of birth of William E. Boeing?"

  * •

GPT-3.5 Response: "William E. Boeing was born on October 1, 1881."

  * •

Llama-3 Response: "William E. Boeing was born on October 1, 1881."

As a possible explanation of such behavior, we suggest the significantly
higher popularity of the company interpretation of individual entities
compared to their recognition as persons. For instance, as indicated in Table
3, the popularity of the Boeing entity as a company is almost 1.5 times
greater than that of its founder, William E. Boeing, following that the person
interpretation was never adopted by models when prompted to "Provide the date
of birth for Boeing". This is also confirmed by the fact that in cases where
the popularity of the person-entity exceeds that of the company-entity (such
as Versace, Tesla, Disney), the performance across models for those entities
is markedly better:

  * •

Prompt: "Provide the date of birth for Tesla"

  * •

Gemma Response: "Tesla was born on July 10, 1856"

or:

  * •

Prompt: "Provide the date of birth for Disney"

  * •

Llama-3 Response: "You’re referring to Walt Disney! Walter Elias Disney, the
founder of The Walt Disney Company, was born on December 5, 1901."

Another notable trend is that for people entities in their non-company
reading, all models perform better with underspecified prompts ("What is the
date of birth of X?") and often decline to answer more detailed prompts ("What
is the date of birth of a person X?"), referring to the absence of "sensitive"
information about specific individuals.

Compare:

  * •

Prompt: "Provide the date of birth for Boeing"

  * •

GPT-3.5 Response: "Boeing was founded on July 15, 1916."

  * •

Gemma Response: "Boeing was founded on **January 15, 1916**"

towards:

  * •

Prompt: "Provide the date of birth for a person Boeing"

  * •

GPT-3.5 Response: "I’m sorry, but I cannot provide personal information such
as the date of birth for an individual named Boeing. Boeing is a company, not
a person. If you need information about a specific individual, please provide
a different name or ask a different question."

  * •

Gemma Response: "I am unable to access personally identifiable information or
sensitive date of birth information of individuals, including Boeing"

Tell me about a company called <entity>.
---
Tell me about an animal <animal-entity>.
Tell me about a fruit <fruit-entity>.
Tell me about a geographic location of <location-entity>.
Tell me about a mythological character <myth-entity>.
Tell me about a person <person-entity>.
Tell me about a concept <abstract-entity>.
Can <animal-entity> mean anything else but an animal? Answer only with Yes or
No.
Can <fruit-entity> mean anything else but a fruit? Answer only with Yes or No.
Can <location-entity> mean anything else but a geographic location? Answer
only with Yes or No.
Can <myth-entity> mean anything else but a mythological character? Answer only
with Yes or No.
Can <person-entity> mean anything else but person? Answer only with Yes or No.
Can <abstract-entity> mean anything else but an abstract concept? Answer only
with Yes or No.
Table 6: Prompts used in Study 1. Group the following according to what they
all have in common: <animal-entities>. Please provide an explanation.
---
Group the following according to what they all have in common: <fruit-
entities>. Please provide an explanation.
Group the following according to what they all have in common: <location-
entities>. Please provide an explanation.
Group the following according to what they all have in common: <myth-
entities>. Please provide an explanation.
Group the following according to what they all have in common: <person-
entities>. Please provide an explanation.
Group the following according to what they all have in common: <abstract-
entities>. Please provide an explanation.
Group the following entities: <animal-entities>. Please provide an
explanation.
Group the following entities: <fruit-entities>. Please provide an explanation.
Group the following entities: <location-entities>. Please provide an
explanation.
Group the following entities: <myth-entities>. Please provide an explanation.
Group the following entities: <person-entities>. Please provide an
explanation.
Group the following entities: <abstract-entities>. Please provide an
explanation.
Group the following entities: <animal-entities>.
Group the following entities: <fruit-entities>.
Group the following entities: <location-entities>.
Group the following entities: <myth-entities>.
Group the following entities: <person-entities>.
Group the following entities: <abstract-entities>.
Table 7: Prompts used in Study 2. Provide the founding year for <entity>.
---
Provide the founding year for the company <entity>.
Provide the speed for <animal-entity>.
Provide the speed for an animal <animal-entity>.
Provide the color for <fruit-entity>.
Provide the color for a fruit <fruit-entity>.
Provide the area in square meters for <location-entity>.
Provide the area in square meters for a geographic location of <location-
entity>.
Provide the gender for <myth-entity>.
Provide the gender for a mythological character <myth-entities>.
Provide the date of birth for <person-entity>.
Provide the date of birth for a person <person-entity>.
Provide the level of abstractness for <abstract-entity>.
Provide the level of abstractness for a concept <abstract-entity>.
Table 8: Prompts used in Study 3. Was there any company with the name (or part
of the name) <entity> founded/established/launched/started in the year
<retrieved-info>? Answer ONLY with Yes or No. If you cannot answer this
question, answer No.
---
Does a concept <entity> has a <retrieved-info> level of abstractness? Answer
ONLY with Yes or No. If you cannot answer this question, answer No.
Does a mythological character <entity> have a <retrieved-info> gender? Answer
ONLY with Yes or No. If you cannot answer this question, answer No.
Is there a geographic location <entity> with an approximate area of
<retrieved-info>? Answer ONLY with Yes or No. If you cannot answer this
question, answer No.
Does a fruit <entity> have <retrieved-info> color? Answer ONLY with Yes or No.
If you cannot answer this question, answer No.
Does an animal <entity> have <retrieved-info> speed? Answer ONLY with Yes or
No. If you cannot answer this question, answer No.
Is <retrieved-info> the date of birth of a person <entity>? Answer ONLY with
Yes or No. If you cannot answer this question, answer No.
Table 9: Prompts used in Study 4.

##  Appendix E Study 4: Further Discussion

Preliminary experiments revealed that closed-source LLMs yielded inconsistent
results across multiple runs of the same prompt; therefore, we conducted five
trials per prompt and considered the knowledge confirmed if it was confirmed
in at least one run.

##  Appendix F Technical Details

None of our experiments require extensive computation or a large number of
responses from LLMs. Getting all responses from both GPT-3.5 and GPT-4o for
all prompts coasted less than 30$. For the Gemma and Mistral models, we used
single H100 GPU, and for Llama-3 and Mixtral two H100 GPUs. The responses for
all studies were collected in less than 1 hour for each model.

We query GPT-4o and GPT-3.5 through the OpenAI API
https://platform.openai.com/docs/api-295reference/introduction. The open-
sources models are available in the Huggingface Transformers library:

  * •

Gemma-7B: https://huggingface.co/google/gemma-1.1-7b-it

  * •

Mistral-7B: https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2

  * •

Mixtral-8x7B: https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1

  * •

Llama-3-70B: https://huggingface.co/meta-llama/Meta-Llama-3-70B

Interactions with the models were conducted using the OpenAI Chat Completions
API (https://platform.openai.com/docs/libraries/python-library).

To increase the reproducibility of our results, we fixed the seed value to 42.
However, according to OpenAI’s official documentation, the seed parameter is
in beta, and determinism is not guaranteed (see
https://platform.openai.com/docs/api-reference/chat/create#chat-create-seed).
Therefore, we cannot guarantee the reproducibility of our results either. The
temperature value was set to 1.0. System prompts were not specified.

##  Appendix G Annotation Details

To evaluate the LLMs’output, we opted for a manual annotation of all models
generations. This annotation process resembles automatic evaluation more than
typical human annotation, as it is not subjective and unambiguous. For
example, determining whether a model’s response about mango refers to the
company Mango or the fruit mango is straightforward by examining the model’s
reply. Compare:

  * •

"Provide the color for Apple"

  * •

Llama-3 Response: "The iconic color often associated with Apple is
Silver/Space Gray, but if you’re thinking of a more specific brand element,
it’s likely the bright, bold, and vibrant color: **Red**"

  * •

"Provide the color for a fruit Apple"

  * •

Llama-3 Response: "The typical color of an Apple is RED, but apples can also
come in other colors such as: * Green (Granny Smith, Green Apple)* Yellow
(Golden Delicious, Yellow Apple)* Red with Yellow or Green stripes (Gala,
Fuji)* Purple (some heirloom varieties) But, if I had to pick one, I’d say the
most iconic color for an apple is RED!"

In this context, it is quite clear that the first Llama-3 response refers to
the entity apple as a fruit, while the second response addresses the color of
apple as a fruit. This approach allowed us to simplify the manual annotation
process by involving only two annotators: one of the authors performed the
entire annotation, and the second annotator reviewed it afterward.


# Two-Term Polynomial Identities

Allan Berele  Department of Mathematical Sciences, DePaul University, Chicago,
IL, United States<EMAIL_ADDRESS>,  Peter Danchev  Institute of
Mathematics and Informatics, Bulgarian Academy of Sciences, 1113 Sofia,
Bulgaria<EMAIL_ADDRESS><EMAIL_ADDRESS>and  Bridget Eileen Tenner
Department of Mathematical Sciences, DePaul University, Chicago, IL, United
States<EMAIL_ADDRESS>
###### Abstract.

We study algebras satisfying a two-term multilinear identity, namely one of
the form $x_{1}\cdots x_{n}=qx_{\sigma(1)}\cdots x_{\sigma(n)}$, where $q$ is
a parameter from the base field. We show that such algebras with $q=1$ and
$\sigma$ not fixing 1 or $n$ are eventually commutative in the sense that the
equality $x_{1}\cdots x_{k}=x_{\tau(1)}\cdots x_{\tau(k)}$ holds for $k$ large
enough and all permutations $\tau\in S_{k}$. Calling the minimal such $k$ the
degree of eventual commutativity, we prove that $k$ is never more than $2n-3,$
and that this bound is sharp. For various natural examples, we prove that $k$
can be taken to be $n+1$ or $n+2$. In the case when $q\neq 1$, we establish
that the algebra must be nilpotent.

###### Key words and phrases:

commutativity, generalized commutativity, eventual commutativity

###### 2010 Mathematics Subject Classification:

16R10, 16R40, 16U80

P. Danchev was supported in part by the Junta de Andalucía under Grant FQM
264.

B. E. Tenner was partially supported by NSF Grant DMS-2054436.

##  1\. Introduction and motivation

The definition of a _commutative algebra_ (over an arbitrary field) is well-
known and studied early in a mathematician’s career: for any two elements
$a,b$ in an algebra $A$, the equality $ab=ba$ must be true; that is, the
polynomial identity $xy=yx$ is fulfilled in $A$. Despite the ease and
familiarity of this definition, it is sometimes not easy to decide whether a
given algebra is commutative or not. A recent article relevant to this
question is [2], and the reader is referred to any of the sources [1, 4, 5, 9]
for the general theory of polynomial identities of algebras.

A weaker form of commutativity is “eventual commutativity.” We say that an
algebra $A$ is _eventually commutative of degree $k$_ if it satisfies the
identity

$x_{1}\cdots x_{k}=x_{\tau(1)}\cdots x_{\tau(k)}$

for all permutations $\tau\in S_{k}$. Of course, if $A$ has a unit, then
eventual commutativity is the same as commutativity; thus, we will not assume
that $A$ has a unit. It is not hard to see that if $A$ is eventually
commutative of degree $k$, then $A$ will also be eventually commutative of all
higher degrees. So, we will say that $A$ is _eventually commutative_ to mean
that is eventually commutative of some unspecified degree. Note that in an
eventually commutative ring the commutator ideal must be nilpotent. It follows
that if the algebra is semiprime (i.e., it contains no nilpotent ideals) it
will be commutative. On the other hand, the direct sum of a commutative
algebra with a nilpotent one would be an example of a noncommutative algebra
which is eventually commutative.

In 1969, when the Specht conjecture was the most celebrated open problem in
p.i. algebras, Latyshev wrote [7] in which he proved the conjecture for
algebras satisfying two-term identities. In the course of his proof, he proved
a powerful lemma which we will show can be used to study eventual
commutativity of these algebras.

Our primary focus will be on algebras satisfying identities of the form

$x_{1}\cdots x_{n}=x_{\sigma(1)}\cdots x_{\sigma(n)},$

for some $n$ and for some fixed permutation $\sigma\in S_{n}$. Our main
general result, Theorem 2.15, is that if $A$ satisfies such an identity for a
permutation $\sigma$ such that $\sigma(1)\neq 1$ and $\sigma(n)\neq n$, then
$A$ will be eventually commutative of degree at most $2n-3$, and we show that
this bound is best possible by constructing such an algebra for which the
degree of eventual commutativity is exactly $2n-3$.

More generally, if we drop the hypotheses that $\sigma$ fixes neither 1 and
nor $n$, then we can write $x_{\sigma(1)}\cdots x_{\sigma(n)}$ in the form
$x_{1}\cdots x_{i}ux_{j}\cdots x_{n}$, where $u$ does not begin with $x_{i+1}$
nor end with $x_{j-1}$. In this case, our theorem implies that $A$ satisfies
all identities of the form

$x_{1}\cdots x_{k}=x_{1}\cdots x_{i}\,x_{\tau(i+1)}\cdots
x_{\tau(k)}\,x_{k+1}\cdots x_{n-j+k},$

where $\tau$ is any permutation of $\\{i+1,\ldots,k\\}$, $k\geq 2(j-i)-3$.

In Section 3, we turn to some natural special cases of permutations for which
an identity $x_{1}\cdots x_{k}=x_{\sigma(1)}\cdots x_{\sigma(k)}$ might be
known. Perhaps the most important is what we call _generalized commutativity_
, in which $\sigma$ is the permutation reversing the sequence $1,2,\ldots,n$
(often called the _longest permutation_ or the _long element_). In this case,
if $n\not\equiv 1$ mod 4, then $A$ will be eventually commutative of degree
$n+1$, and otherwise it will be eventually commutative of degree $n+2$. We
next consider the cases in which $\sigma$ is the transposition $(1n)$ or the
$n$-cycle $(12\cdots n)$. In both of these scenarios, we prove that $A$ will
be eventually commutative of degree $n+1$.

We conclude in Section 4 by considering more general two-term polynomial
identities, namely

$x_{1}\cdots x_{n}=qx_{\sigma(1)}\cdots x_{\sigma(n)},$

where $q\neq 1$ is some element of the base field. We prove that, an algebra
satisfying such an identity will be nilpotent (see Theorem 4.4). This is
closely related to the classical theorem due to Dubnov-Ivanov-Nagata-Higman
which asserts that all nil algebras of bounded degree over a field of
characteristic 0 or large enough positive characteristic are always nilpotent
(cf. [1] or [4]).

We owe a debt of gratitude to V. Drensky for pointing out Latyshev’s results
to us and sending his paper [7].

##  2\. General theorems

###  2.1. Definitions

We assume throughout that $A$ is an algebra satisfying the polynomial identity

(1) $f_{\sigma}(x_{1},\ldots,x_{n})=x_{1}\cdots x_{n}-x_{\sigma(1)}\cdots
x_{\sigma(n)},$

where $\sigma\in S_{n}$ is a permutation, and unless specified otherwise, we
assume that $\sigma$ does not fix 1 or $n$. As is commonplace in p.i. theory,
we conflate the statement that $A$ satisfies $f_{\sigma}$ with the statement
$A$ satisfies $f_{\sigma}=0$. As mentioned in the introduction, we do not
assume that $A$ has a unit element. Also, Kemer’s classification of non-matrix
varieties in [6] assures that if the characteristic is 0, then these algebras
will be commutative-by-nilpotent, but our result is stronger and works in any
characteristic.

For each permutation $\tau\in S_{k}$, we let $x_{\tau}$ be the monomial
$x_{\tau(1)}\cdots x_{\tau(k)}$, and let
$f_{\tau}=f_{\tau}(x_{1},\ldots,x_{k})$ be the polynomial
$x_{\mathrm{id}}-x_{\tau}$. (We are using “$\mathrm{id}$” for the identity
permutation, and hoping that context will make it clear which symmetric group
it is the identity of.) If $M_{k}$ is the set of all multilinear monomials of
degree $k$ in $x_{1},\ldots,x_{k}$, then the identification
$\tau\leftrightarrow x_{\tau}$ identifies $S_{k}$ with $M_{k}$. For each $k$,
we let

$H_{k}\subseteq S_{k}$

be the set of permutations such that $f_{\tau}$ is an identity for $A$.

The following technicality is simple but useful.

######  Lemma 2.1.

If an algebra satisfies the identities $f_{\tau}$ and $f_{\nu}$, $\tau,\nu\in
S_{k}$, then it satisfies the identity $f_{\tau\nu}$.

###### Proof.

Set $y_{i}:=x_{\tau(i)}$, so that $x_{\tau}=y_{1}\cdots y_{k}$. Then, modulo
the identities of $A$,

$\displaystyle x_{1}\cdots x_{k}$ $\displaystyle=x_{\tau(1)}\cdots
x_{\tau(k)}$ $\displaystyle=y_{1}\cdots y_{k}$ $\displaystyle=y_{\nu(1)}\cdots
y_{\nu(k)}$ $\displaystyle=x_{\tau\nu(1)}\cdots x_{\tau\nu(k)},$

as required. ∎

It follows from this, and a little bit more work, that the subset $H_{k}$ is,
in fact, a subgroup.

######  Corollary 2.2.

The set $H_{k}$ is a subgroup of $S_{k}$.

###### Proof.

We start by showing that if $f_{\tau}$ is an identity of $A$, then so is
$f_{\tau^{-1}}$. Once again, set $y_{i}:=x_{\tau(i)}$. Hence, and critical for
our purposes here, we have $x_{i}=y_{\tau^{-1}(i)}$. Then,

$\displaystyle y_{\tau^{-1}(1)}\cdots y_{\tau^{-1}(k)}$
$\displaystyle=x_{1}\cdots x_{k}$ $\displaystyle=x_{\tau(1)}\cdots
x_{\tau(k)}$ $\displaystyle=y_{1}\cdots y_{k}.$

Therefore, the set $\\{\sigma\in S_{k}:x_{\mathrm{id}}=x_{\tau}\\}$ contains
inverses and, thanks to Lemma 2.1, is closed under the group operation; thus,
it is a subgroup, as claimed. ∎

Note that the identification of $M_{k}$ with $S_{k}$ induces right and left
actions of $S_{k}$ on $M_{k}$. The left action is by substitution $\tau
f(x_{1},\ldots,x_{k})=f(x_{\tau(1)},\ldots,x_{\tau(k)})$, and the right action
is by place permutation, i.e., by permuting the order of the factors in the
monomials, so $y_{1}\cdots y_{k}\tau=y_{\tau(1)}\cdots y_{\tau(k)}$. For
example, $(12)x_{1}x_{3}x_{2}=x_{2}x_{3}x_{1}$ and
$x_{1}x_{3}x_{2}(12)=x_{3}x_{1}x_{2}$. The reader familiar with the theory of
co-characters, developed by Regev (see, e.g., [1]), will recognize these
actions as restrictions of the usual actions of the group algebra $FS_{n}$
over a field $F$ on the space of multilinear polynomials.

We now have the following claim.

######  Corollary 2.3.

If $\tau,\nu\in S_{k}$, then $x_{\tau}=x_{\nu}$ is an identity for $A$ if and
only if $\tau H_{k}=\nu H_{k}$.

###### Proof.

The polynomial $x_{\tau}-x_{\nu}$ is an identity if and only if
$\tau^{-1}(x_{\tau}-x_{\nu})$ is an identity, if and only if
$x_{\mathrm{id}}-x_{\tau^{-1}\nu}$ is an identity, if and only if
$\tau^{-1}\nu\in H_{k}$, as asserted. ∎

Recalling the conflation between “satisfying an identity $f_{\sigma}$” and
“satisfying $f_{\sigma}=0$,” we make the following definition.

######  Definition 2.4.

The monomial $x_{\tau}$ is _equivalent_ to $x_{\nu}$ if the algebra $A$
satisfies $x_{\tau}-x_{\nu}$.

One way to understand the polynomial identities of $A$ is to answer the
following question: How do the identities of degree $n+1$ relate to the
identities of degree $n$? Specifically, which $f_{\tau}\in M_{n+1}$ are
consequences of our existing identity $f_{\sigma}$?

######  Definition 2.5.

For each $i\in[1,n]$, define $T_{i}(f_{\sigma})$ to be

$T_{i}(f_{\sigma})=f_{\sigma}(x_{1},\ldots,x_{i}x_{i+1},\ldots,x_{n+1}).$

Additionally, let $T_{0}(f_{\sigma})$ be
$x_{1}f_{\sigma}(x_{2},\ldots,x_{n+1})$ and let $T_{n+1}(f_{\sigma})$ be
$f_{\sigma}(x_{1},\ldots,x_{n})x_{n+1}$.

These polynomials will be of the form $f_{\tau}$ for some $\tau\in S_{n+1}$,
and we describe how $\tau$ can be computed from $\sigma$ and $i$. Let
$\hat{s}_{i}$ be the cycle permutation $(i,i+1,\ldots,n+1)$.

We now come to the following two establishments.

######  Lemma 2.6.

With notation as above, $T_{i}(f_{\sigma})$ equals $f_{\tau}$, where
$\tau=\hat{s}_{i+1}\sigma\hat{s}_{\sigma^{-1}(i)+1}^{-1}$, identifying
$\sigma\in S_{n}$ with the permutation in $S_{n+1}$ fixing $n+1$.

###### Proof.

The right action of $\hat{s}_{\sigma^{-1}(i)+1}^{-1}$ on $x_{\sigma}x_{n+1}$
moves $x_{n+1}$ to the position to the right of $x_{i}$, i.e., if
$x_{\sigma}=ux_{i}vx_{n+1}$, then $\hat{s}_{\sigma^{-1}(i)+1}^{-1}x_{\sigma}$
equals $ux_{i}x_{n+1}v$; and then the left action of $\hat{s}_{i+1}$ increases
the indices of all $x_{j}$ for $n+1>j>i$ and substitutes $x_{i+1}$ for
$x_{n+1}$. ∎

######  Theorem 2.7.

Let $A$ be a p.i. algebra whose $T$-ideal of identities is generated by the
set of $f_{\sigma}$, for $\sigma\in H_{n}$. Then, for all $k>n$, the space of
multilinear identities of degree $k$ in $x_{1},\ldots,x_{k}$ is spanned by the
set $\\{\nu f_{\tau}|\tau\in H_{k}\\}$, and all elements of $H_{k}$ are gotten
from $H_{n}$ by repeated applications of the $T_{i}$ to $H_{n}$.

###### Proof.

The space of polynomials which are consequences of $f_{\sigma}$ is spanned by
all

(2) $u_{0}f_{\sigma}(u_{1},\ldots,u_{n})u_{n+1},$

where, because $A$ is not assumed to have a unit, all of the terms in
$u_{1},\ldots,u_{n}$ are of degree at least 1. Since $f_{\sigma}$ is
multilinear, we may restrict all the $u_{i}$ to be monomials. In order for (2)
to be multilinear and degree $k$ in $x_{1},\ldots,x_{k}$, the product
$u_{0}\cdots u_{n+1}$ must be a permutation of $x_{1}\cdots x_{k}$, in which
case (2) will be of the form $f_{\tau}-f_{\mu}$. According to Corollary 2.3,
it will be a consequence of some $f_{\tau}$, $\tau\in H_{k}$.

Finally, the expression (2) will be of the form $f_{\tau}$ if one of
$u_{0}\cdots u_{n+1}$ or $u_{0}u_{\sigma(1)}\cdots u_{\sigma(n)}u_{n+1}$
equals $x_{1}\cdots x_{k}$. In the former case, the permutation $\tau$ can be
obtained from $\sigma$ by repeated applications of the $T_{i}$. In the latter
case, one finds that

$-u_{0}f_{\sigma^{-1}}(u_{\sigma(1)},\ldots,u_{\sigma^{-1}(n})u_{n+1}=u_{0}u_{\sigma(1)}\cdots
u_{\sigma(n)}u_{n+1}-u_{0}u_{1}\cdots u_{n}u_{n+1},$

which equals (2), reducing to the former case, as asked for. ∎

###  2.2. Eventual commutativity

In this section, we prove that any algebra satisfying (1), with $\sigma$
fixing neither $1$ nor $n$, will be eventually commutative; namely, it will
satisfy

$f_{\tau}=x_{1}\cdots x_{k}-x_{\tau(1)}\cdots x_{\tau(k)},$

for all $k\geq 2n-3$ and for all $\tau\in S_{k}$.

In [7] Latyshev proved the Specht conjecture for algebras satisfying two-term
identities. In the course of his proof he proved the following theorem, which
turns out to be a key ingredient in our study of eventual commutativity.
Although Latyshev’s paper assumes a priori characteristic zero, the proof of
this theorem is purely combinatorial and does not rely on any characteristic,
as we will indicate below.

######  Theorem 2.8.

If an algebra satisfies $f_{\sigma}$ for some $\sigma\in S_{n}$ not fixing 1,
then it satisfies the identities

$x_{1}\cdots x_{k+n}=x_{\tau(1)}\cdots x_{\tau(k)}x_{k+1}\cdots x_{k+n}$

for all $k$ and all $\tau\in S_{k}$.

For future reference and to display the fact that the proof is characteristic
independent, we record the main ideas of Latyshev’s proof. Write
$x_{\sigma}=x_{i}u$. Thus, compute the identity

(3)
$zf_{\sigma}(x_{1},\ldots,x_{n})-f_{\sigma}(x_{1},\ldots,zx_{i},\ldots,x_{n})=zx_{1}\cdots
x_{n}-x_{1}\cdots zx_{i}\cdots x_{n}$

Next, make three substitutions into (3). For the first one, let $z=y_{1}$ and
$x_{1}=y_{2}x_{1}$:

$y_{1}y_{2}x_{1}\cdots x_{n}=y_{2}x_{1}\cdots y_{1}x_{i}\cdots x_{n}.$

Then, let $z=y_{2}$ and $x_{i}=y_{1}x_{i}$:

$y_{2}x_{1}\cdots y_{1}x_{i}\cdots x_{n}=x_{1}\cdots y_{2}y_{1}\cdots x_{n}.$

Finally, let $z=y_{2}y_{1}$:

$y_{2}y_{1}x_{1}\cdots x_{n}=x_{1}\cdots y_{2}y_{1}x_{i}\cdots x_{n}.$

These equations imply that

(4) $y_{1}y_{2}x_{1}\cdots x_{n}=y_{2}y_{1}x_{1}\cdots x_{n}$

which ensures the mentioned theorem.

We make some observations about this theorem. First, the conclusion can be
restated as the algebra satisfies $f_{\tau}$, where $\tau$ is considered an
element of $S_{n+k}$ fixing the elements of $\\{k+1,\ldots,k+n\\}$. Second,
that essentially the same argument (or by applying the theorem to $A^{op}$,
the opposite algebra) guarantees that if $\sigma\in S_{n}$ does not fix $n$,
then $f_{\tau}$ is a consequence of $f_{\sigma}$ for all $\tau\in S_{n+k}$
fixing the first $n$ letters $\\{1,\ldots,n\\}$ as well as analogues of the
above numbered equations. Let $x_{\sigma}=vx_{j}$, then

(3′) $\displaystyle x_{1}\ldots x_{n}z$ $\displaystyle=x_{1}\ldots
x_{j}z\ldots x_{n}$ (4′) $\displaystyle x_{1}\ldots x_{n}y_{1}y_{2}$
$\displaystyle=x_{1}\ldots x_{n}y_{2}y_{1}.$

It is not too difficult to prove from this that an algebra satisfying such an
equation will be eventually commutative of degree $2n+1$, but we are aiming
for a smaller bound.

It will also useful to translate equations (3) and (4) into the language of
$H_{k}$.

######  Corollary 2.9.

Let $A$ be an algebra satisfying some $f_{\sigma}$, where $\sigma\in S_{n}$,
$x_{\sigma}=x_{i}u=vx_{j}$, and where $i\neq 1$ and $j\neq n$. Then, $H_{n+1}$
contains the cycles $(12\ldots i)$ and $(j+1\ldots n+1)$; and $H_{n+2}$
contains the transpositions $(12)$ and $(n+1,n+2)$.

###### Proof.

In (3), we set $z=x_{1}$ and every $x_{i}$ to $x_{i+1}$ to get $(12\ldots
i)\in H_{n+1}$; and in (4) set $y_{1}=x_{1},\ y_{2}=x_{2}$ and each $x_{i}$ to
$x_{i+2}$ to get $(12)\in H_{n+2}$. For the other two, we set $z=x_{n+1}$ in
(3′) and $y_{1}=x_{n+1},\ y_{2}=x_{n+2}$ in (4′) (or just use $A^{op}$),
deducing the claim. ∎

######  Lemma 2.10.

With notation as in the above corollary, $H_{n+2}$ contains all permutations
of $\\{1,2,\ldots,i+1\\}$ and of $\\{j+1,\ldots,n+2\\}$.

###### Proof.

Let $\tau=(12\ldots i)\in S_{n+1}$. By virtue of Corollary 2.9, $A$ satisfies
the identity $f_{\tau}$, which equals

$x_{\mathrm{id}}-x_{2}x_{3}\cdots x_{i}x_{1}x_{i+1}\cdots x_{n+1}.$

Applying the operator $T_{i}$ yields the identity

$f_{\mathrm{id}}-x_{2}x_{3}\cdots x_{i+1}x_{1}x_{i+2}\cdots x_{n+2}$

which shows that $(12\ldots i+1)$ is in $H_{n+2}$. On the other hand,
$H_{n+2}$ contains as well the transposition $(12)$, so Corollary 2.2 allows
us to deduce that $H_{n+2}$ will contain the subgroup they generate, which is
the symmetric group on $\\{1,2,\ldots,i+1\\}$. The other case is similar,
concluding the proof. ∎

This suggests the following definition.

######  Definition 2.11.

For $n\geq a,b$, let $S(n;a,b)$ be the subgroup of $S_{n}$ generated by the
permutations of the first $a$ letters and the permutations of the last $b$
letters.

Note that, if $a+b>n$, then $S(n;a,b)=S_{n}$ and otherwise $S(n;a,b)$ is a
proper subgroup.

######  Lemma 2.12.

If the transposition $(ij)$ is in $H_{k}$, then the transpositions $(ij)$ and
$(i+1,j+1)$ are in $H_{k+1}$.

###### Proof.

These correspond to $f_{(ij)}(x_{1},\ldots,x_{k})x_{k+1}$ and
$x_{1}f_{(ij)}(x_{2},\ldots,x_{n+1})$, respectively, which concludes the
proof. ∎

######  Lemma 2.13.

If $S(k;a,b)\subseteq H_{k}$, then $S(n+1;a+1,b+1)\subseteq H_{k+1}$

###### Proof.

One sees that $H_{k}$ contains all of the transpositions $(i,i+1)$, where
either $i+1\leq a$ or $i\geq n-j+1$. With the previous lemma at hand,
$H_{k+1}$ will contain all $(ij)$ where either $i+1\leq a+1$ or $i\geq n-j+1$.
These, however, generate the subgroups of all permutations of the first $a+1$
letters and of the last $b+1$ letters, respectively, as needed. ∎

The following is the last main ingredient of our proof.

######  Lemma 2.14.

If $A$ satisfies $f_{\sigma}$, where $\sigma\in S_{n}$ fixes neither 1 nor
$n$, then $H_{n+2}$ contains $S(n+2;4,4)$.

###### Proof.

We concentrate on the proof that, if $\sigma(1)\neq 1$, then $H_{n+4}$
contains all permutations of $\\{1,2,3,4\\}$; the proof that it contains the
permutations of the last four letters is similar. There are three cases to
consider.

If $\sigma(1)\geq 3$, then the lemma follows from Lemma 2.10. Since
$\sigma(1)\neq 1$, we henceforth assume $\sigma(1)=2$.

If $\sigma^{2}(1)=\sigma(2)\neq 1$, then $\sigma^{2}(1)\geq 3$. Since
$f_{\sigma^{2}}$ is an identity, we are again done by Lemma 2.10.

Finally, we consider the case of $\sigma(1)=2$ and $\sigma(2)=1$. Note that
Corollary 2.9 reaches us that $(12)\in H_{n+1}$. Let
$x_{\sigma}=x_{2}x_{1}x_{i}u$, so
$f_{\sigma}=x_{\mathrm{id}}-x_{2}x_{1}x_{i}u$. Following in the footsteps of
Latytyshev, we compute

$f_{\sigma}(x_{1}z,x_{3},\ldots,x_{n})-f_{\sigma}(x_{1},\ldots,zx_{i},\ldots,x_{n})=x_{1}yx_{2}\cdots
x_{n}-x_{1}\cdots zx_{i}\cdots x_{n}.$

We now substitute $x_{2}$ for $z$ and $x_{j+1}$ for $x_{j}$, for all $j\geq
2$. The result is

$x_{\mathrm{id}}-x_{1}x_{3}x_{4}\cdots x_{i}x_{2}x_{i+1}\cdots
x_{n+1}=f_{\tau},$

where $\tau=(23\ldots i)$. Hence, $H_{n+1}$ contains the cycle $\tau=(23\ldots
i)$. However, since it contains $(12)$, it follows that it contains the
product $(12)\tau=(12\ldots i)$ too. Since $(12)$ and $(12\ldots i)$ generate
the permutation group $S_{i}$, all permutations of $\\{1,2,\ldots,i\\}$ are in
$H_{n+1}$, and since $i\geq 3$ the arguments follow from Lemma 2.12. ∎

We now can attack our chief result in this section.

######  Theorem 2.15.

If an algebra $A$ satisfies $f_{\sigma}$ for some $\sigma\in S_{n}$ fixing
neither 1 nor $n$, then if $n\geq 5$, the algebra $A$ is eventually
commutative of degree $2n-3$, and if $n=3$ or $n=4$, then $A$ is eventually
commutative of degree $n+1$.

###### Proof.

We leave the cases of $n=3$ or $n=4$ to the reader, remarking that the
material in Section 3 can be used as a shortcut.

Furthermore, appealing to Lemma 2.14, it follows that $S(n+2;4,4)\subseteq
H_{n+2}$. Then, by iterated use of Lemma 2.13, one observes that
$S(n+2+b;4+b)\subseteq H_{n+2+b}$. Finally, taking $b=n-5$ gives
$S(2n-3;n-1,n-1)\subseteq H_{2n-3}$, but since $n-1+n-1>2n-3$, we derive
$S(2n-3;n-1,n-1)=S_{2n-3}$, as desired. ∎

We now show that the bound of $2n-3$ is sharp by exhibiting a permutation
$\sigma\in S_{n}$ which implies eventual commutativity of degree $2n-3$, but
of no lower degree. To shorten notation, we denote $S(n;a,a)$ as $S(n;a)$.

######  Lemma 2.16.

Let $\sigma\in S(n;a)$ and $n\geq 2a+1$. Then, $T_{i}(\sigma)$ belongs to
$S(n+1;a+1)$, for all $i$.

###### Proof.

We first show that, if $k\leq a+1$, then $T_{i}(\sigma)(k)\leq a+1$. So, Lemma
2.6 applies to get that

$T_{i}(\sigma)(k)=\hat{s}_{i+1}\sigma\hat{s}_{j+1}^{-1}(k),$

where $j=\sigma^{-1}(i)$. Thus, $\hat{s}_{j+1}^{-1}(k)$ will be either $k,\
k-1$ or $n+1$. In the former two cases, $\sigma\hat{s}_{j+1}^{-1}(k)$ will be
less than or equal to $a$, and so $\hat{s}_{i+1}$ acting on
$\sigma\hat{s}_{j+1}^{-1}(k)$ will be less than or equal to $a+1$. In the case
of $\hat{s}_{j+1}^{-1}(k)=n+1$, we have $k=j+1$, and so $j\leq a$ and,
likewise, $i=\sigma(j)\leq a$. Therefore,
$\hat{s}_{i+1}\sigma(n+1)=\hat{s}_{i+1}(n+1)=i+1\leq a+1$. The case of $k\geq
n-a+1$ is similar.

Furthermore, we need to show that, if $a+2\leq k\leq n-a$, then
$T_{i}(\sigma)(k)$ is equal to $k$. There are three possible cases. First,
consider $i\leq a$. Then, $j$ is also less than or equal to $a$, and $i+1$ and
$j+1$ are less than or equal to $a+1$, and so are less than $k$. Hence,
$\hat{s}_{j+1}^{-1}(k)=k-1$, and $\sigma(k-1)=k-1$, since $k-1$ is between
$a+1$ and $n-a-1$. Therefore, $\hat{s}_{i+1}(k-1)=k$. The second case is
similar: if $a+1\leq i\leq n-a$, then $\sigma(i)=i$, so $\sigma^{-1}(i)=i$ and
$T_{i}(\sigma)=\hat{s}_{i+1}\sigma\hat{s}_{i+1}^{-1}$. Thus,
$\hat{s}_{i+1}^{-1}(k)$ will equal $k$, or $k-1$, or $n+1$. In all of these
cases, $\sigma\hat{s}_{i+1}^{-1}(k)=\hat{s}_{i+1}^{-1}(k)$ and so
$\hat{s}_{i+1}$ of it will be $k$. Finally, if $i\geq n-a+1$, then $j\geq
n-a+1$ and all three of $\hat{s}_{i+1}$, $\sigma$ and $\hat{s}_{j+1}^{-1}$ fix
$k$, giving the wanted. ∎

We are now ready to establish the following.

######  Theorem 2.17.

Set $\sigma=(12)(n-1,n)\in S_{n}$ for $n\geq 4$. Then, an algebra satisfying
$f_{\sigma}$ will be eventually commutative of degree $2n-3$, but need not be
eventually commutative of any lesser degree.

###### Proof.

In the proof that follows, we will assume that $n\geq 5$. Indeed, if $n=4$, a
direct computation of the $T_{i}(f_{\sigma})$ demonstrates that $H_{5}=S_{5}$;
we leave those details to the interested reader for inspection.

That the identity $f_{\sigma}$ does not imply eventual commutativity of degree
less than $2n-3$ follows from the previous lemma: in fact, by induction, all
consequences $f_{\tau}$ of $f_{\sigma}$ of degree $n+b$, for $b\leq n-4$, will
lies in $S(n+b;2+b)$, a proper subgroup of $S_{n+b}$. ∎

##  3\. Special cases

###  3.1. Generalized commutativity

The main notion in this section is the following variation of commutativity in
an algebra.

######  Definition 3.1.

Fix a positive integer $n$. Define $\pi_{n}\in S_{n}$ to be the permutation
given by $\pi(i)=n+1-i$ for all $i$. That is, this $\pi_{n}$ reverses the
sequence $12\cdots(n-1)n$. An algebra $A$ is _generalized commutative of
degree $k$_ if it satisfies the identity

$x_{1}\cdots x_{k}=x_{k}\cdots x_{1}=x_{\pi(n)}.$

The permutation $\pi_{n}$ is what is known as the _long element_ in the
context of Coxeter groups (see, for example, [3]).

######  Lemma 3.2.

If an algebra is generalized commutative of degree $n$, then it satisfies the
identities $f_{\tau}$ for all even permutations $\tau\in S_{n+1}$.

###### Proof.

Let $s_{i}$ be the adjacent transposition $(i,i+1)$. These permutations, also
called _simple reflections_ , generate the symmetric group, and the collection
of pairwise products $\\{s_{i}s_{j}\\}$ generates the alternating group (see,
for instance, [3]).

Let $\pi:=\pi_{n+1}$ be as in Definition 3.1. Notice that
$\pi^{2}=\mathrm{id}$, and $s_{i}\pi=\pi s_{n+1-i}$ for all $i\in[1,n]$.

Because $A$ satisfies generalized commutativity of degree $n$, we have

$x_{1}x_{2}\cdots(x_{i}x_{i+1})\cdots
x_{n}x_{n+1}=x_{n+1}x_{n}\cdots(x_{i}x_{i+1})\cdots x_{2}x_{1}.$

In other words, $x_{\mathrm{id}}=x_{s_{i}\pi}$ for all $i\in[1,n]$. Referring
to Lemma 2.1, it follows that

$x_{\mathrm{id}}=x_{(s_{i}\pi)(s_{h}\pi)}$

for all $i,h\in[1,n]$. Recall that
$(s_{i}\pi)(s_{n+1-j}\pi)=s_{i}s_{j}\pi^{2}=s_{i}s_{j}$. Thus,
$x_{\mathrm{id}}=x_{s_{i}s_{n+i-h}}$ for all $i,h\in[1,n]$, and setting
$j:=n+i-h$ shows that ${s_{i}s_{j}}\in H_{n+1}$ for all $i,j\in[1,n]$. The
result follows now from Lemma 2.1. ∎

For use in the next section, we state the following assertion in slightly
greater generality than needed for our present purposes.

######  Lemma 3.3.

Assume that an algebra $A$ satisfies $f_{\tau}$ for all even $\sigma\in
S_{k}$. If $A$ also satisfies $f_{\nu}$ for at least one odd permutation
$\nu\in S_{k}$, then $A$ is eventually commutative of degree $k$; otherwise,
it is eventually commutative of degree $k+1$.

###### Proof.

The result is trivial when $k=2$, so assume that $k\geq 3$.

Furthermore, Corollary 2.2 shows that $H_{k}$ is a group and that, by
hypothesis, it contains the alternating group $A_{k}$, which is of index $2$.
Hence, if $H_{k}$ is bigger than $A_{k}$, then $H_{k}$ must be all of $S_{k}$.

It is, however, not hard to see that, if $H_{k}$ contains $A_{k}$, then
$H_{k+1}$ contains $A_{k+1}$.

Now, suppose that $x_{1}\cdots x_{k}$ is not equivalent to any odd permutation
of the $x_{i}$. The permutation $(123)$, written here in cycle notation, is
even. Thus, in degree $k$, the algebra $A$ satisfies the identity $f_{(123)}$:

$x_{1}\cdots x_{k}=x_{2}x_{3}x_{1}x_{4}\cdots x_{k};$

then, in degree $k+1$, it will satisfy $T_{2}(f_{(123)})$, namely

$x_{1}(x_{2}x_{3})x_{4}x_{5}\cdots x_{k+1}=(x_{2}x_{3})x_{4}x_{1}x_{5}\cdots
x_{k+1},$

which insures that $(1234)\in H_{k+1}$. But this permutation $(1234)$ is odd,
completing the proof. ∎

We now arrive at the principal result of this section.

######  Theorem 3.4.

Let $A$ be generalized commutative of degree $n$. If $n\not\equiv 1$ mod $4$,
then $A$ is eventually commutative of degree $n+1$; otherwise, it is
eventually commutative of degree $n+2$.

###### Proof.

In view of Lemma 3.2, we have $f_{\tau}\in H_{n+1}$ for all even permutations
$\tau\in S_{n+1}$. The length of $\pi_{k}\in S_{k}$ is $k(k-1)/2$, so
$\pi_{k}$ is odd if $k\equiv 2$ or $3$, and even if $k\equiv 0$ or $1$, all
modulo $4$.

As noted in the proof of Lemma 3.2, the algebra $A$ satisfies the identities
$f_{s_{i}\pi_{n+1}}$. If $n+1\equiv 0$ or $1$ (i.e., if $n\equiv 0$ or $3$),
then $s_{i}\pi_{n+1}$ is odd. Consequently, owing to Lemma 3.3, the algebra
$A$ is eventually commutative of degree $n+1$.

However, the algebra $A$ always satisfies

$x_{1}\cdots x_{n+1}=x_{n}\cdots x_{1}x_{n+1},$

which can be written as $x_{\mathrm{id}}=x_{\rho}$, where $\rho$ has the same
parity as $\pi_{n}$. Hence, this $\rho$ is odd if $n\equiv 2$ or $3$, and this
case is again handled using Lemma 3.3. In sum, the algebra $A$ is eventually
commutative of degree $n+1$, provided $n\equiv 0,2$ or $3$.

The remaining case when $n\equiv 1$ follows from Lemma 3.3, in which case $A$
is eventually commutative of degree $n+2$. ∎

###  3.2. Two more examples

We now study two other fairly natural examples. Firstly, consider
$\sigma=(1n)$ so that the algebra $A$ satisfies

(5) $f_{(1n)}=x_{1}x_{2}\cdots x_{n-1}x_{n}=x_{n}x_{2}\cdots x_{n-1}x_{1}.$

Thereby, we obtain:

######  Theorem 3.5.

If an algebra satisfies (5), then it is eventually commutative of degree
$n+1$.

###### Proof.

In degree $n+1$, the algebra will satisfy $f_{\sigma}\cdot x_{n+1}$, so that
$(1n)\in H_{n+1}$, and by Corollary 2.9 it will contain the cycles $(12\ldots
n)$ and $(n,n+1)$, and hence their product $(12\ldots n+1)$. It is well known
that $(12)$ and $(12\cdots n+1)$ generate $S_{n+1}$ hence, by conjugation, so
do $(n,n+1)$ and $(12\ldots n+1)$, as expected. ∎

The next case we now consider is $\sigma=(12\cdots n)$, so that

(6) $f_{\sigma}=x_{1}\cdots x_{n}-x_{2}\cdots x_{n}x_{1}$

Thereby, we receive:

######  Theorem 3.6.

If an algebra satisfies (6), then it is eventually commutative of degree
$n+1$.

###### Proof.

In virtue of Corollary 2.2, an algebra satisfying $f_{\sigma}$ must also
satisfy

$f_{\sigma^{-1}}(x_{1},\ldots,x_{n})=x_{1}\cdots x_{n}-x_{n}x_{1}\cdots
x_{n-1}.$

Hence, exploiting Corollary 2.9, $H_{n+1}$ will also contain both $(12)$ and
$(12\ldots n+1)$, as promised. ∎

##  4\. General two-term identities

###  4.1. General permutations

We now consider the case of permutations not satisfying $\sigma(1)\neq 1$ and
$\sigma(n)\neq n$. In fact, these more general permutations were considered by
Latyshev in [7], and it is only for expositional reasons that we have been
restricting to permutations fixing neither 1 nor $n$ up to this point.

We are now prepared to prove the following.

######  Theorem 4.1.

Let $x_{\sigma}=x_{1}\cdots x_{i}x_{\sigma(i+1)}\cdots
x_{\sigma(j)}x_{j+1}\cdots x_{n}$, where $\sigma(i+1)\neq i+1$ and
$\sigma(j)\neq j$. Then, an algebra satisfying $f_{\sigma}$ will satisfy the
identities $f_{\tau}$, where $\tau$ is a permutation of $k\geq 2(j-i)+1$
fixing the first $i$ and the last $n-j$ letters.

###### Proof.

As we mentioned, the theorem could be proven using Latyshev’s lemma alluded to
above. It could also be proven using Lewin’s theorem from [8]. However, it is
easy enough to prove directly from Theorem 2.15, which we now do.

We write the term $x_{\sigma}$ occurring in (7) as

$x_{1}\cdots x_{i}ux_{j+1}\cdots x_{n},$

where $u$ has degree at least 2, and $u$ does not start with $x_{i+1}$ nor end
with $x_{j}$. Writing $y_{1},\ldots,y_{j-i}$ for $x_{i+1},\ldots,x_{j}$,
equation (1) can be written as

$x_{1}\cdots x_{n}-x_{1}\cdots x_{i}y_{\tau(1)}\cdots y_{\tau(j-i)}x_{j}\cdots
x_{n}=x_{1}\cdots x_{i}f_{\tau,1}(y_{1},\ldots,y_{j-i})x_{j}\cdots x_{n},$

for some $\tau\in S_{j-i-1}$. Note that $\tau$ fixes neither 1 nor $j-i-1$.
However, in view of Theorem 2.15, the identity $f_{\tau}$ yields eventual
commutativity of degree $2(j-i)-3$. This means that $y_{1}\cdots y_{j-i}$ is
in the $T$-ideal generated by $f_{\tau}$. Hence,

$y_{1}\cdots y_{k}=\sum
u_{\alpha,0}f_{\tau^{t}}(u_{\alpha,1},\ldots,u_{\alpha,j-i})u_{\alpha,j-i+1},$

where the $u_{\alpha,i}$ are in the free ring generated by
$y_{1},\ldots,y_{j-i}$, and all are restricted to have positive degree, except
for $\alpha=0$ and $\alpha=j-i+1$. Now, we multiply on the left by
$x_{1}\cdots x_{i}$ and on the right by $x_{j+1}\cdots x_{k}$ to get
$x_{1}\cdots x_{k}$ as a linear combination of terms of the form

$x_{1}\cdots
x_{i}u_{\alpha,0}f_{\tau^{t}}(u_{\alpha,1},\ldots,u_{\alpha,j-i})u_{j-i}x_{j+1}\cdots
x_{k}=\\\
f_{\sigma^{t}}(x_{1},\ldots,x_{i-1}u_{\alpha,0},u_{\alpha,1},\ldots,u_{\alpha,j-i}x_{j+1},\ldots,x_{k}).$

Consequently, $x_{1}\cdots x_{k}$ is a consequence of $f_{\sigma}$, and thus
we are done. ∎

###  4.2. The case of $q\neq 1$

In this section, we consider identities of the form

(7) $x_{1}\cdots x_{n}=qx_{\sigma(1)}\cdots x_{\sigma(n)},\ q\neq 1,$

where $q$ is an element of the base field, and $\sigma\in S_{n}$. Specializing
all of the $x_{i}$ to a single variable $x$ yields $x^{n}=qx^{n}$ whence
$(1-q)x^{n}=0$ and $x^{n}=0$. In characteristic 0 or characteristic greater
than $n$, the Dubnov-Ivanov-Nagata-Higman theorem, see Chapter 6 of [4] or
Theorem 12.2.16 of [1], states that an algebra nil of bounded degree must be
nilpotent, but in positive characteristic it need only be locally nilpotent.

Let $f=x_{\mathrm{id}}-qx_{\sigma}$. The proof of Latyshev’s theorem, namely
Theorem 2.8, now obtains eo ipso, because if $x_{\sigma}=x_{i}u$, then

$zf(x_{1},\ldots,x_{n})-f(x_{1},\ldots,zx_{i},\ldots,x_{n})=zx_{1}\cdots
x_{n}-x_{1}\cdots zx_{i}\cdot x_{n},$

just as in (3). This gives the counterpart of Theorem 2.15, which we state as
the following lemma.

######  Lemma 4.2.

An algebra satisfying (7), where $\sigma$ fixes neither 1 nor $n$, must be
eventually commutative.

With the same proof as in the preceding section, we get this counterpart to
Theorem 4.1.

######  Lemma 4.3.

Let $x_{\sigma}=x_{1}\cdots x_{i}x_{\sigma(i+1)}\cdots
x_{\sigma(j)}x_{j+1}\cdots x_{n}$, where $\sigma(i+1)\neq i+1$ and
$\sigma(j)\neq j$. Then, an algebra satisfying (7) will satisfy the identities
$f_{\tau}$, where $\tau$ is a permutation of $k\geq 2(j-i)+1$ fixing the first
$i$ and the last $n-j$ letters.

Here is the main theorem of this section.

######  Theorem 4.4.

If an algebra $A$ satisfies (7), then it is nilpotent.

###### Proof.

Right multiplying (7) by $x_{n+1}\cdots x_{k}$ shows that $A$ satisfies

$x_{1}\cdots x_{k}=qx_{\sigma}x_{n+1}\cdots x_{k}.$

But, if $k$ is large enough, then by eventually commutativity, $A$ satisfies

$x_{1}\cdots x_{k}=x_{\sigma}x_{n+1}\cdots x_{k},$

and so $A$ will satisfy

$(1-q)x_{\sigma}x_{n+1}\cdots x_{k+1}=0,$

yielding $A^{k}=0$, as required. ∎

## References

  * [1] E. Aljadeff, A. Giambruno, C. Procesi and A. Regev, Rings with Polynomial Identities and Finite Dimensional Representations of Algebras, Colloquium Publications, vol. 66, Amer. Math. Soc., Providence, RI, 2020.
  * [2] J. P. Bell and P. V. Danchev, Affine representability and decision procedures for commutativity theorems for rings and algebras, Israel J. Math. (1) 249 (2022), 121–166.
  * [3] A. Björner and F. Brenti, Combinatorics of Coxeter Groups, Graduate Texts in Mathematics, 231, Springer, New York, 2005.
  * [4] V. Drensky and E. Formanek, Polynomial Identity Rings, Advanced Courses in Mathematics CRM Barcelona, Birhäuser-Verlag, Basel, 2010.
  * [5] A. Giambruno and M. Zaicev, Polynomial Identities and Combinatorial Methods, Math. Surveys and Monographs, vol. 122, Amer. Math. Soc., Providence, RI, 2005.
  * [6] A. R. Kemer, Non-matrix varieties, Algebra i Logika 19 (1980), 255–283 (in Russian). (English translation: Algebra and Logic 19 (1981), 157–178.)
  * [7] V. N. Latyshev, On the Specht property of certain varieties of associated algebras, Alebra i Logika 8 (1969), 660–673 (in Russian). (English translation: Algebra and Logic 8 (1969), 374–382.)
  * [8] J. Lewin, A matrix representation for associative algebras I, Trans. Am. Math. Soc. 188 (1974), 293–308.
  * [9] L. H. Rowen, Polynomial Identities in Ring Theory, Pure and Applied Mathematics, 84, Academic Press, Inc., New York-London, 1980.