initial commit

README.md

@@ -109,55 +109,6 @@ generated_text = tokenizer.decode(outputs[0].tolist())
 print(generated_text)
 ```
 
-### Generation Strategies
-
-The model supports multiple generation strategies optimized for different use cases:
-
-#### 1. Focused Generation (Low Temperature)
-```python
-# Best for factual, coherent content
-outputs = model.generate(
-    input_ids,
-    max_new_tokens=128,
-    do_sample=True,
-    temperature=0.3,
-    top_p=0.8,
-    top_k=20,
-    repetition_penalty=1.3,
-    no_repeat_ngram_size=5,
-)
-```
-
-#### 2. Balanced Generation (Medium Temperature)
-```python
-# Good balance of creativity and coherence
-outputs = model.generate(
-    input_ids,
-    max_new_tokens=128,
-    do_sample=True,
-    temperature=0.7,
-    top_p=0.9,
-    top_k=40,
-    repetition_penalty=1.2,
-    no_repeat_ngram_size=4,
-)
-```
-
-#### 3. Creative Generation (High Temperature)
-```python
-# Most creative and diverse outputs
-outputs = model.generate(
-    input_ids,
-    max_new_tokens=128,
-    do_sample=True,
-    temperature=0.9,
-    top_p=0.95,
-    top_k=50,
-    repetition_penalty=1.15,
-    no_repeat_ngram_size=3,
-)
-```
-
 ## Performance
 
 

config.json

@@ -0,0 +1,25 @@
+{
+  "architectures": [
+    "SSLLMForCausalLM"
+  ],
+  "attention_dropout": 0.1,
+  "bos_token_id": 100256,
+  "d_ff": 2560,
+  "d_model": 768,
+  "dropout_rate": 0.1,
+  "eos_token_id": 100257,
+  "hidden_size": 768,
+  "intermediate_size": 2560,
+  "max_position_embeddings": 1024,
+  "max_seq_len": 1024,
+  "model_type": "ssllm",
+  "num_attention_heads": 12,
+  "num_heads": 12,
+  "num_hidden_layers": 10,
+  "num_layers": 10,
+  "pad_token_id": 100257,
+  "stochastic_depth_rate": 0.1,
+  "torch_dtype": "float32",
+  "transformers_version": "4.52.4",
+  "vocab_size": 100277
+}

generation_config.json

@@ -0,0 +1,11 @@
+{
+  "bos_token_id": 100256,
+  "eos_token_id": 100257,
+  "pad_token_id": 100257,
+  "max_length": 1024,
+  "do_sample": true,
+  "temperature": 0.7,
+  "top_p": 0.9,
+  "repetition_penalty": 1.1,
+  "no_repeat_ngram_size": 3
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5c2a35c0354a62e08a22386cd7ed6844fb44cf6a840a252d3250d6d54ebe85a6
+size 871704100

tokenizer_config.json

@@ -0,0 +1,11 @@
+{
+  "tokenizer_class": "tiktoken",
+  "model_name": "cl100k_base",
+  "vocab_size": 100277,
+  "bos_token": "",
+  "eos_token": "",
+  "pad_token": "",
+  "unk_token": "",
+  "mask_token": "",
+  "additional_special_tokens": []
+}

tokenizer_info.txt

@@ -0,0 +1,5 @@
+Tokenizer: cl100k_base (tiktoken)
+Vocabulary size: 100277
+BOS token ID: 100256
+EOS token ID: 100257
+PAD token ID: 100257