Create README_EN.md
Browse filesEnglish translation!
- README_EN.md +140 -0
README_EN.md
ADDED
@@ -0,0 +1,140 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
license: cc-by-sa-4.0
|
3 |
+
language:
|
4 |
+
- ru
|
5 |
+
- en
|
6 |
+
pipeline_tag: text-generation
|
7 |
+
library_name: transformers
|
8 |
+
---
|
9 |
+
Strela is a powerful language model designed to provide high speed and quality responses on weak devices.
|
10 |
+
Strela is recommended for the following purposes:
|
11 |
+
* Chat bot for dialogue
|
12 |
+
* Story writer
|
13 |
+
* Song writer
|
14 |
+
* Translation of Russian and English languages
|
15 |
+
* When it is ineffective to use heavier models
|
16 |
+
|
17 |
+
## Description from strela itself
|
18 |
+
I am a computer program designed to process and analyze natural language.
|
19 |
+
I have the ability to understand, analyze, and process natural language, allowing me to communicate with people through various communication channels. My main goal is to help people solve tasks and provide information based on requests.
|
20 |
+
I can be used for various purposes: from automatic text generation, translation from one language to another, or even creating your own verses and songs.
|
21 |
+
## Using the model online
|
22 |
+
You can try it out [here](https://huggingface.co/spaces/gai-labs/chat-with-strela-q4_k_m).
|
23 |
+
## Using the model for in-app chat
|
24 |
+
Recommended is [GTP4ALL](https://gpt4all.io/index.html), it supports GGUF, so you need to download the [special model in GGUF format](https://huggingface.co/gai-labs/strela-GGUF).
|
25 |
+
## Using the model for Unity chat
|
26 |
+
Recommended is [LLM for Unity](https://assetstore.unity.com/packages/tools/ai-ml-integration/llm-for-unity-273604), it supports GGUF, so you need to download the [special model in GGUF format](https://huggingface.co/gai-labs/strela-GGUF).
|
27 |
+
## Using the quantized model for chat in Python | Recommended
|
28 |
+
You should install [gpt4all](https://docs.gpt4all.io/gpt4all_python.html)
|
29 |
+
```
|
30 |
+
pip install gpt4all
|
31 |
+
```
|
32 |
+
Then, download the [GGUF version of the model](https://huggingface.co/gai-labs/strela-GGUF) and move the file to your script's directory
|
33 |
+
```py
|
34 |
+
# Library Imports
|
35 |
+
import os
|
36 |
+
from gpt4all import GPT4All
|
37 |
+
|
38 |
+
# Initializing the model from the strela-q4_k_m.gguf file in the current directory
|
39 |
+
model = GPT4All(model_name='strela-q4_k_m.gguf', model_path=os.getcwd())
|
40 |
+
|
41 |
+
# Callback function to stop generation if Arrow generates the '#' symbol, which marks the beginning of roles declaration
|
42 |
+
def stop_on_token_callback(token_id, token_string):
|
43 |
+
if '#' in token_string:
|
44 |
+
return False
|
45 |
+
else:
|
46 |
+
return True
|
47 |
+
|
48 |
+
# System prompt
|
49 |
+
system_template = """### System:
|
50 |
+
You are an AI assistant who gives a helpful response to whatever humans ask of you.
|
51 |
+
"""
|
52 |
+
|
53 |
+
# Human and AI prompt
|
54 |
+
prompt_template = """
|
55 |
+
### Human:
|
56 |
+
{0}
|
57 |
+
### Assistant:
|
58 |
+
"""
|
59 |
+
|
60 |
+
# Chat session
|
61 |
+
with model.chat_session(system_template, prompt_template):
|
62 |
+
print("To exit, enter 'Exit'")
|
63 |
+
while True:
|
64 |
+
print('')
|
65 |
+
user_input = input(">>> ")
|
66 |
+
if user_input.lower() != "exit":
|
67 |
+
|
68 |
+
# Streaming generation
|
69 |
+
for token in model.generate(user_input, streaming=True, callback=stop_on_token_callback):
|
70 |
+
print(token, end='')
|
71 |
+
else:
|
72 |
+
break
|
73 |
+
```
|
74 |
+
```
|
75 |
+
To exit, enter 'Exit'
|
76 |
+
|
77 |
+
>>> Hello
|
78 |
+
Hello! How can I help you today?
|
79 |
+
>>>
|
80 |
+
```
|
81 |
+
## Using the full model for chat in Python
|
82 |
+
```py
|
83 |
+
# Library Imports
|
84 |
+
from transformers import AutoTokenizer, AutoModelForCausalLM
|
85 |
+
|
86 |
+
# Loading the model
|
87 |
+
tokenizer = AutoTokenizer.from_pretrained("gai-labs/strela")
|
88 |
+
model = AutoModelForCausalLM.from_pretrained("gai-labs/strela")
|
89 |
+
|
90 |
+
# System prompt
|
91 |
+
system_prompt = "You are an AI assistant who gives a helpful response to whatever humans ask of you."
|
92 |
+
|
93 |
+
# Your prompt
|
94 |
+
prompt = "Hello!"
|
95 |
+
|
96 |
+
# Chat template
|
97 |
+
chat = f"""### System:
|
98 |
+
{system_prompt}
|
99 |
+
### Human:
|
100 |
+
{prompt}
|
101 |
+
### Assistant:
|
102 |
+
"""
|
103 |
+
|
104 |
+
# Generation
|
105 |
+
model_inputs = tokenizer([chat], return_tensors="pt")
|
106 |
+
generated_ids = model.generate(**model_inputs, max_new_tokens=64) # Adjust the maximum token count for generation
|
107 |
+
output = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
|
108 |
+
|
109 |
+
# Cleaning the generated output from the chat template
|
110 |
+
output = output.replace(chat, "")
|
111 |
+
|
112 |
+
# Output of the generation results
|
113 |
+
print(output)
|
114 |
+
```
|
115 |
+
```
|
116 |
+
Hello! How can I help?
|
117 |
+
```
|
118 |
+
## Using the model for text generation in Python
|
119 |
+
```py
|
120 |
+
# Library Imports
|
121 |
+
from transformers import AutoTokenizer, AutoModelForCausalLM
|
122 |
+
|
123 |
+
# Loading the model
|
124 |
+
tokenizer = AutoTokenizer.from_pretrained("gai-labs/strela")
|
125 |
+
model = AutoModelForCausalLM.from_pretrained("gai-labs/strela")
|
126 |
+
|
127 |
+
# Prompt
|
128 |
+
prompt = "AI - "
|
129 |
+
|
130 |
+
# Generation
|
131 |
+
model_inputs = tokenizer([prompt], return_tensors="pt")
|
132 |
+
generated_ids = model.generate(**model_inputs, max_new_tokens=64) # Adjust the maximum token count for generation
|
133 |
+
output = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
|
134 |
+
|
135 |
+
# Output of the generation results
|
136 |
+
print(output)
|
137 |
+
```
|
138 |
+
```
|
139 |
+
AI - is a field of computer science and technology that deals with creating machines capable of "understanding" humans or performing tasks with logic similar to that of humans.
|
140 |
+
```
|