Blue-kod commited on
Commit
a3a789c
·
verified ·
1 Parent(s): 77c66dc

Create README_EN.md

Browse files

English translation!

Files changed (1) hide show
  1. README_EN.md +140 -0
README_EN.md ADDED
@@ -0,0 +1,140 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-sa-4.0
3
+ language:
4
+ - ru
5
+ - en
6
+ pipeline_tag: text-generation
7
+ library_name: transformers
8
+ ---
9
+ Strela is a powerful language model designed to provide high speed and quality responses on weak devices.
10
+ Strela is recommended for the following purposes:
11
+ * Chat bot for dialogue
12
+ * Story writer
13
+ * Song writer
14
+ * Translation of Russian and English languages
15
+ * When it is ineffective to use heavier models
16
+
17
+ ## Description from strela itself
18
+ I am a computer program designed to process and analyze natural language.
19
+ I have the ability to understand, analyze, and process natural language, allowing me to communicate with people through various communication channels. My main goal is to help people solve tasks and provide information based on requests.
20
+ I can be used for various purposes: from automatic text generation, translation from one language to another, or even creating your own verses and songs.
21
+ ## Using the model online
22
+ You can try it out [here](https://huggingface.co/spaces/gai-labs/chat-with-strela-q4_k_m).
23
+ ## Using the model for in-app chat
24
+ Recommended is [GTP4ALL](https://gpt4all.io/index.html), it supports GGUF, so you need to download the [special model in GGUF format](https://huggingface.co/gai-labs/strela-GGUF).
25
+ ## Using the model for Unity chat
26
+ Recommended is [LLM for Unity](https://assetstore.unity.com/packages/tools/ai-ml-integration/llm-for-unity-273604), it supports GGUF, so you need to download the [special model in GGUF format](https://huggingface.co/gai-labs/strela-GGUF).
27
+ ## Using the quantized model for chat in Python | Recommended
28
+ You should install [gpt4all](https://docs.gpt4all.io/gpt4all_python.html)
29
+ ```
30
+ pip install gpt4all
31
+ ```
32
+ Then, download the [GGUF version of the model](https://huggingface.co/gai-labs/strela-GGUF) and move the file to your script's directory
33
+ ```py
34
+ # Library Imports
35
+ import os
36
+ from gpt4all import GPT4All
37
+
38
+ # Initializing the model from the strela-q4_k_m.gguf file in the current directory
39
+ model = GPT4All(model_name='strela-q4_k_m.gguf', model_path=os.getcwd())
40
+
41
+ # Callback function to stop generation if Arrow generates the '#' symbol, which marks the beginning of roles declaration
42
+ def stop_on_token_callback(token_id, token_string):
43
+ if '#' in token_string:
44
+ return False
45
+ else:
46
+ return True
47
+
48
+ # System prompt
49
+ system_template = """### System:
50
+ You are an AI assistant who gives a helpful response to whatever humans ask of you.
51
+ """
52
+
53
+ # Human and AI prompt
54
+ prompt_template = """
55
+ ### Human:
56
+ {0}
57
+ ### Assistant:
58
+ """
59
+
60
+ # Chat session
61
+ with model.chat_session(system_template, prompt_template):
62
+ print("To exit, enter 'Exit'")
63
+ while True:
64
+ print('')
65
+ user_input = input(">>> ")
66
+ if user_input.lower() != "exit":
67
+
68
+ # Streaming generation
69
+ for token in model.generate(user_input, streaming=True, callback=stop_on_token_callback):
70
+ print(token, end='')
71
+ else:
72
+ break
73
+ ```
74
+ ```
75
+ To exit, enter 'Exit'
76
+
77
+ >>> Hello
78
+ Hello! How can I help you today?
79
+ >>>
80
+ ```
81
+ ## Using the full model for chat in Python
82
+ ```py
83
+ # Library Imports
84
+ from transformers import AutoTokenizer, AutoModelForCausalLM
85
+
86
+ # Loading the model
87
+ tokenizer = AutoTokenizer.from_pretrained("gai-labs/strela")
88
+ model = AutoModelForCausalLM.from_pretrained("gai-labs/strela")
89
+
90
+ # System prompt
91
+ system_prompt = "You are an AI assistant who gives a helpful response to whatever humans ask of you."
92
+
93
+ # Your prompt
94
+ prompt = "Hello!"
95
+
96
+ # Chat template
97
+ chat = f"""### System:
98
+ {system_prompt}
99
+ ### Human:
100
+ {prompt}
101
+ ### Assistant:
102
+ """
103
+
104
+ # Generation
105
+ model_inputs = tokenizer([chat], return_tensors="pt")
106
+ generated_ids = model.generate(**model_inputs, max_new_tokens=64) # Adjust the maximum token count for generation
107
+ output = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
108
+
109
+ # Cleaning the generated output from the chat template
110
+ output = output.replace(chat, "")
111
+
112
+ # Output of the generation results
113
+ print(output)
114
+ ```
115
+ ```
116
+ Hello! How can I help?
117
+ ```
118
+ ## Using the model for text generation in Python
119
+ ```py
120
+ # Library Imports
121
+ from transformers import AutoTokenizer, AutoModelForCausalLM
122
+
123
+ # Loading the model
124
+ tokenizer = AutoTokenizer.from_pretrained("gai-labs/strela")
125
+ model = AutoModelForCausalLM.from_pretrained("gai-labs/strela")
126
+
127
+ # Prompt
128
+ prompt = "AI - "
129
+
130
+ # Generation
131
+ model_inputs = tokenizer([prompt], return_tensors="pt")
132
+ generated_ids = model.generate(**model_inputs, max_new_tokens=64) # Adjust the maximum token count for generation
133
+ output = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
134
+
135
+ # Output of the generation results
136
+ print(output)
137
+ ```
138
+ ```
139
+ AI - is a field of computer science and technology that deals with creating machines capable of "understanding" humans or performing tasks with logic similar to that of humans.
140
+ ```