Hugging Face's logo

nnat03
/
trump-mistral-adapter

Safetensors
English
mistral
lora
adapter
fine-tuned
politics
conversational
Model card Files Community
trump-mistral-adapter
File size: 3,913 Bytes 
a93697f
200f782
 
 
 
 
 
 
 
 
 
 
 
 
a93697f
 
200f782
a93697f
200f782
a93697f
 
 
200f782
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
 
---
language:
- en
tags:
- mistral
- lora
- adapter
- fine-tuned
- politics
- conversational
license: mit
datasets:
- pookie3000/trump-interviews
- bananabot/TrumpSpeeches
---

# Trump Mistral Adapter

This is a LoRA adapter for the [Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2) model, fine-tuned to emulate Donald Trump's distinctive speaking style, discourse patterns, and policy positions.

## Model Details

- **Base Model**: [mistralai/Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2)
- **Model Type**: LoRA adapter (Low-Rank Adaptation)
- **LoRA Rank**: 16
- **Language**: English
- **Training Focus**: Emulation of Donald Trump's communication style and response patterns

## Intended Use

This model is designed for:
- Educational and research purposes related to political discourse and communication styles
- Interactive simulations for understanding political rhetoric
- Creative applications exploring political communication

## Training Data

This adapter was fine-tuned on two key datasets:
- [Trump interviews dataset](https://huggingface.co/datasets/pookie3000/trump-interviews)
- [Trump speeches dataset](https://huggingface.co/datasets/bananabot/TrumpSpeeches)

These datasets were processed into an instruction format:

## Training Procedure

- **Framework**: Hugging Face Transformers and PEFT
- **Optimization**: 4-bit quantization for memory efficiency
- **LoRA Configuration**:
  - `r=16`
  - `lora_alpha=64`
  - `lora_dropout=0.05`
  - Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
- **Training Parameters**:
  - Batch size: 4
  - Gradient accumulation steps: 4
  - Learning rate: 2e-4
  - Epochs: 3
  - Learning rate scheduler: cosine
  - Optimizer: paged_adamw_8bit
  - BF16 precision

## Limitations and Biases

- The model is designed to mimic a speaking style and may not always provide factually accurate information
- While it emulates Trump's rhetoric, it does not represent his actual views or statements
- The model may reproduce biases present in the training data
- Not suitable for production applications requiring factual accuracy without RAG enhancement

## Usage

This adapter should be applied to the Mistral-7B-Instruct-v0.2 base model:

```python
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
import torch

# Load base model with 4-bit quantization
base_model_id = "mistralai/Mistral-7B-Instruct-v0.2"
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True,
)

# Load model and tokenizer
model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    quantization_config=bnb_config,
    device_map="auto",
    torch_dtype=torch.float16
)
tokenizer = AutoTokenizer.from_pretrained(base_model_id)

# Apply the adapter
model = PeftModel.from_pretrained(model, "nnat03/trump-mistral-adapter")

# Generate a response
prompt = "What's your plan for border security?"
input_text = f"<s>[INST] {prompt} [/INST]"
inputs = tokenizer(input_text, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_length=512, temperature=0.7, do_sample=True)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response.split("[/INST]")[-1].strip())
```

## Citation and Acknowledgments

If you use this model in your research, please cite:
@misc{nnat03-trump-mistral-adapter,
author = {nnat03},
title = {Trump Mistral Adapter},
year = {2023},
publisher = {Hugging Face},
howpublished = {\url{https://huggingface.co/nnat03/trump-mistral-adapter}}
}


## Ethical Considerations

This model is created for educational and research purposes. It attempts to mimic the speaking style of a public figure but does not represent their actual views or statements. Use responsibly.