README.md

---
datasets:
- McAuley-Lab/Amazon-Reviews-2023
language:
- en
library_name: pytorch
pipeline_tag: text-generation
base_model: openai-community/gpt2-medium
---

# GPT-2 Medium - Review

## Model Details

**Model Description:** This model is a checkpoint of GPT-2 Medium the **355M parameter** version of GPT-2, a transformer-based language model created and released by OpenAI. The model is a further pretrained model on a causal language modeling (CLM) objective with English Amazon Product Reviews from the Fashion category.

- **Developed by:** Students at University of Konstanz
- **Model Type:** Transformer-based language model
- **Language(s):** English
- **Base Model:** [GPT2-medium](https://huggingface.co/openai-community/gpt2-medium)
- **Resources for more information:** [GitHub Repo](https://github.com/TomSOWI/DLSS-24-Synthetic-Product-Reviews-Generation)


## How to Get Started with the Model 

Use the code below to get started with the model. You can use this model directly with a pipeline for text generation. Since the generation relies on some randomness, we
set a seed for reproducibility:

```python
>>> from transformers import pipeline, set_seed
>>> generator = pipeline('text-generation', model='TomData/GPT2-review')
>>> set_seed(42)
>>> generator("Hello, I'm a language model,", max_length=30, num_return_sequences=5)
```


Here is how to use this model to get the features of a given text in PyTorch:

```python
tokenizer = AutoTokenizer.from_pretrained("TomData/GPT2-review")
model = AutoModelForCausalLM.from_pretrained("TomData/GPT2-review")
text = "Replace me by any text you'd like."
encoded_input = tokenizer(text, return_tensors='pt')
output = model(**encoded_input)
```


and in TensorFlow:

```python
tokenizer = AutoTokenizer.from_pretrained("TomData/GPT2-review")
model = AutoModelForCausalLM.from_pretrained("TomData/GPT2-review")
text = "Replace me by any text you'd like."
encoded_input = tokenizer(text, return_tensors='tf')
output = model(encoded_input)
```

## Uses

This model is further pretrained to generate artificial product reviews. This can be usefull for:
- Market research
- Product analysis
- Customer preferences
- Fashion trends
- Research


## Training


The model is further pretrained on the [Amazion Review Dataset](https://huggingface.co/datasets/McAuley-Lab/Amazon-Reviews-2023) from McAuley-Lab.
For training only the reviews related to the Amazon Fashion category are used. See:

```python
dataset = load_dataset("McAuley-Lab/Amazon-Reviews-2023", "raw_review_Amazon_Fashion", trust_remote_code=True)
```