---
language: 
  - en
tags:
- conversational
- dialogue
- response generation
license: apache-2.0
datasets:
- allenai/soda
- allenai/prosocial-dialog
---

# Model Card for 🧑🏻‍🚀COSMO

🧑🏻‍🚀COSMO is a conversation agent with greater generalizability on both in- and out-of-domain chitchat datasets (e.g., DailyDialog, BlendedSkillTalk). It is trained on two datasets: SODA and ProsocialDialog. COSMO is especially aiming to model natural human conversations. It can accept situation descriptions as well as instructions on what role it should play in the situation.

## Model Description
- **Repository:** [Code](https://github.com/skywalker023/sodaverse)
- **Paper:** [SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualization](https://arxiv.org/abs/2212.10465)
- **Point of Contact:** [Hyunwoo Kim](mailto:hyunwook@allenai.org)

## Model Training

🧑🏻‍🚀COSMO is trained on our two recent datasets: 🥤[SODA](https://huggingface.co/datasets/allenai/soda) and [Prosocial Dialog](https://huggingface.co/datasets/allenai/prosocial-dialog).
The backbone model of COSMO is the [lm-adapted T5](https://huggingface.co/google/t5-xl-lm-adapt).

### How to use

```python
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("allenai/cosmo-xl")
model = AutoModelForSeq2SeqLM.from_pretrained("allenai/cosmo-xl")
# model.to('cuda')

def set_input(narrative, instruction, dialogue_history):
    input_text = " <turn> ".join(dialogue_history)

    if instruction != "":
        input_text = instruction + " <sep> " + input_text

    if narrative != "":
        input_text = narrative + " <sep> " + input_text
    

    return input_text

def generate(narrative, instruction, dialogue_history):
    """
    narrative: the description of situation/context with the characters included (e.g., "David goes to an amusement park")
    instruction: the perspective/speaker instruction (e.g., "Imagine you are David and speak to his friend Sarah").
    dialogue history: the previous utterances in the dialogue in a list
    """

    input_text = set_input(narrative, instruction, dialogue_history) 

    inputs = tokenizer([input_text], return_tensors="pt")
    # inputs = inputs.to('cuda')
    outputs = model.generate(inputs["input_ids"], max_new_tokens=128, temperature=1.0, top_p=.95, do_sample=True)
    response = tokenizer.decode(outputs[0], skip_special_tokens=True, clean_up_tokenization_spaces=False)

    return response

situation = "Cosmo had a really fun time participating in the EMNLP conference at Abu Dhabi."
instruction = "You are Cosmo and you are talking to a friend." # You can also leave the instruction empty

dialogue = [
    "Hey, how was your trip to Abu Dhabi?"
]

response = generate(situation, instruction, dialogue)
print(response)
```

### Further Details, Social Impacts, Bias, and Limitations
Please refer to our [paper](https://arxiv.org/abs/2212.10465).
Significant research has explored bias and fairness issues with language models (see, e.g., [Sheng et al. 2021](https://aclanthology.org/2021.acl-long.330.pdf) and [Bender et al. 2021](https://dl.acm.org/doi/pdf/10.1145/3442188.3445922)). Predictions generated by the model may include disturbing and harmful stereotypes across protected classes; identity characteristics; and sensitive, social, and occupational groups.

## Additional Information

For a brief summary of our paper, please see this [tweet](https://twitter.com/hyunw__kim/status/1605400305126248448).

### Citation

Please cite our work if you find the resources in this repository useful:
```
@article{kim2022soda,
    title={SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualization},
    author={Hyunwoo Kim and Jack Hessel and Liwei Jiang and Peter West and Ximing Lu and Youngjae Yu and Pei Zhou and Ronan Le Bras and Malihe Alikhani and Gunhee Kim and Maarten Sap and Yejin Choi},
    journal={ArXiv},
    year={2022},
    volume={abs/2212.10465}
}
```