Duplicate from mattshumer/Reflection-Llama-3.1-70B

Co-authored-by: Matt Shumer <[email protected]>

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,65 @@
+---
+license: llama3.1
+base_model: meta-llama/Meta-Llama-3.1-70B-Instruct
+pipeline_tag: text-generation
+library_name: transformers
+---
+# Reflection Llama-3.1 70B
+
+| IMPORTANT UPDATE – There was an issue with the model when we first uploaded it. If you tried it and didn't have good results, please, try again, we think we've fixed the issue.
+
+**Reflection Llama-3.1 70B is (currently) the world's top open-source LLM, trained with a new technique called Reflection-Tuning that teaches a LLM to detect mistakes in its reasoning and correct course.**
+
+The model was trained on synthetic data generated by [Glaive](https://glaive.ai). If you're training a model, Glaive is incredible — use them.
+
+You can [try the model here](https://reflection-playground-production.up.railway.app/).
+
+## Benchmarks
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/60518f3731c5be7f3dd5ebc3/zNs-ZFs0SbnomH7mikiOU.png)
+
+All benchmarks tested have been checked for contamination by running [LMSys's LLM Decontaminator](https://github.com/lm-sys/llm-decontaminator). When benchmarking, we isolate the `<output>` and benchmark on solely that section.
+
+Trained from Llama 3.1 70B Instruct, you can sample from Reflection Llama-3.1 70B using the same code, pipelines, etc. as any other Llama model. It even uses the stock Llama 3.1 chat template format (though, we've trained in a few new special tokens to aid in reasoning and reflection).
+
+During sampling, the model will start by outputting reasoning inside `<thinking>` and `</thinking>` tags, and then once it is satisfied with its reasoning, it will output the final answer inside `<output>` and `</output>` tags. Each of these tags are special tokens, trained into the model.
+
+This enables the model to separate its internal thoughts and reasoning from its final answer, improving the experience for the user.
+
+Inside the `<thinking>` section, the model may output one or more `<reflection>` tags, which signals the model has caught an error in its reasoning and will attempt to correct it before providing a final answer.
+
+## System Prompt
+
+The system prompt used for training this model is:
+
+```
+You are a world-class AI system, capable of complex reasoning and reflection. Reason through the query inside <thinking> tags, and then provide your final response inside <output> tags. If you detect that you made a mistake in your reasoning at any point, correct yourself inside <reflection> tags.
+```
+
+We recommend using this exact system prompt to get the best results from Reflection Llama-3.1 70B. You may also want to experiment combining this system prompt with your own custom instructions to customize the behavior of the model.
+
+## Chat Format
+
+As mentioned above, the model uses the standard Llama 3.1 chat format. Here’s an example:
+
+```
+<|begin_of_text|><|start_header_id|>system<|end_header_id|>
+
+You are a world-class AI system, capable of complex reasoning and reflection. Reason through the query inside <thinking> tags, and then provide your final response inside <output> tags. If you detect that you made a mistake in your reasoning at any point, correct yourself inside <reflection> tags.<|eot_id|><|start_header_id|>user<|end_header_id|>
+
+what is 2+2?<|eot_id|><|start_header_id|>assistant<|end_header_id|>
+```
+
+## Tips for Performance
+
+- We are initially recommending a `temperature` of `.7` and a `top_p` of `.95`.
+- For increased accuracy, append `Think carefully.` at the end of your messages.
+
+## Dataset / Report
+
+Both the dataset and a brief report detailing how we trained this model will be released next week, alongside our Reflection 405B model that we expect will be the top-performing LLM in the world, including closed-source models.
+
+---
+
+Thanks to Jason Kuperberg and Josh Bickett from the [HyperWrite](https://hyperwriteai.com) team for reviewing drafts of the report we'll be releasing next week.
+
+Also, we know right now the model is split into a ton of files. We'll condense this soon to make the model easier to download and work with!

config.json

@@ -0,0 +1,28 @@
+{
+  "_name_or_path": "meta-llama/Meta-Llama-3-70B-Instruct",
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 128000,
+  "eos_token_id": 128009,
+  "hidden_act": "silu",
+  "hidden_size": 8192,
+  "initializer_range": 0.02,
+  "intermediate_size": 28672,
+  "max_position_embeddings": 8192,
+  "model_type": "llama",
+  "num_attention_heads": 64,
+  "num_hidden_layers": 80,
+  "num_key_value_heads": 8,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "rope_theta": 500000.0,
+  "tie_word_embeddings": false,
+  "torch_dtype": "float32",
+  "transformers_version": "4.40.0",
+  "use_cache": true,
+  "vocab_size": 128262
+}

generation_config.json

@@ -0,0 +1,12 @@
+{
+  "bos_token_id": 128000,
+  "do_sample": true,
+  "eos_token_id": [
+    128001,
+    128009
+  ],
+  "max_length": 4096,
+  "temperature": 0.6,
+  "top_p": 0.9,
+  "transformers_version": "4.40.0"
+}

model-00001-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8f809c805d669903d3de4fa7e016c6f477a2abde4feea8cdd24beaea30c894aa
+size 4202889352

model-00002-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:44821dc7a19e2ff5c15461bdb3cf45648b2d167dca2e103e491fad5ad8559c87
+size 1543504480

model-00003-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:242634933a6eb95c58e27345c378c5f80286d832b300533182b6adbf87ac85d5
+size 1879114208

model-00004-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fbf99f1ca533a697d6bb27d2ffff50c3e9120b37d94e08d256204d84006e3208
+size 1543504480

model-00005-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:caa075ce0ccc45b44afe182d0d064a34766f3bc15d2ffbc6ae0295c0c3b3bcc5
+size 1879114208

model-00006-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d3fad36e66abf09860c6c026adeeee0c527922a474e1b419056cb910acb9a2ec
+size 1543504480

model-00007-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e9257bdb60a4825e167b474a76021fb2dc58b5ef929f1840c4de93b761dc7bde
+size 1879114208

model-00008-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1e117cbf79b7c8142417444d64cdf162edf4887e59039df1ab5430c608a7d361
+size 1543504480

model-00009-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ea6e795e1aea9889541f39faed08af077399068c5470159eab35563d0169cbc9
+size 1879114208

model-00010-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:22ccaa07394e718f5891b3d4a85c2bd369e2df86e3f42ba6a5e70103adbe215e
+size 1543504480

model-00011-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:720378d2fadef6bb733ffc4266842358f8f540c215252cdc5a2c0077b7ac5c26
+size 1879114208

model-00012-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:65cfa32187488131cd3d99be28965df1be5793107eec125a40354ec11bdd8788
+size 1543504480

model-00013-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8ee493198ac58b82ba9db612979723927c5d1b505af7bec4de3eaca5a34abc63
+size 1879114208

model-00014-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5b2b105825f41a1393a248396ce1a932f02b65a4e2d352afa6bd83d8ee57d4be
+size 1543504480

model-00015-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:96d31dd63129e54cdf34db5263dcf0dab36a27a745c815f6cb3bde75a78d819f
+size 1879114208

model-00016-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:231a84636210ff2726d2d50ce065356eea18cacbcb3cebe5a85432405280edd4
+size 1543504480

model-00017-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c6d115675f5b0d3ebca0e580f3f177eb68bffb71e664ba26fb01cea253e6e87d
+size 1879114208

model-00018-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c07583b2fab739fb973c264a507fba881945bc51264b8b24656a6d809b82bc79
+size 1543504480

model-00019-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:df6d5867db9f1b6381fb7b5a0028811f947f81cbc77afdac565ad72ee3184c8f
+size 1879114208

model-00020-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3596fd7261b56ec897b4c0d957f031cc2663c67ca1c4a36b8949bb81616cf026
+size 1543504480

model-00021-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b8d78241036d4ae35651bf463d00fe2ba08b5072521ec45f33a4a1cc6cd4b8f8
+size 1879114208

model-00022-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f6e2f9cb78e01da982ef2a89cb239566b4d03a374e9506ca1b86822cf0681465
+size 1543504488

model-00023-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a1fa97677beaabf459a6e6888cb3cfdbb3cd78a9a860ed3982dbe67f3de1f4d3
+size 1879114208

model-00024-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d7e9c7808ff0f707d21d831c7be8f7908a36207e60efb19581a4317ee34477d0
+size 1543504488

model-00025-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:66414420b04c67d2397ac9f8bc1a49882ea5e9853d2adbd61b8e7484ba7bb79c
+size 1879114208

model-00026-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:339035476299b3c936c22c9406a607033e6da9380ab69b0f9ef1b7d0a2a6af2b
+size 1543504488

model-00027-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c307f06094239e7e37a04353d508a4e516414ac5cd76ec4f26a26f6a2b484d7a
+size 1879114208

model-00028-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5adeb0fb794a775ea83aeaa0c32ded964ef8f910aaff510eebc9a6d6ced47f5a
+size 1543504488

model-00029-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2a5febe564290a3c570da54342128efbdc9cb2b646ef384597252e04d3736848
+size 1879114208

model-00030-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fa88625c23c929f03970a6380883c3cbfc6c24cf58bf1351fb9c08fe67f71a98
+size 1543504488

model-00031-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:666b233e63540fd20f34d5ffb12a4cc9457cdc765ecc2dff22aa71970d6e962d
+size 1879114208

model-00032-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cb2eccbe44efe9736f4982c4ebc1f358c416de80df47c4127df1cc0677113892
+size 1543504488

model-00033-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6bae3cb6e0101d00e20f8618a9bf6948aa350506877708ac419c3ed31d983913
+size 1879114208

model-00034-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e019f5ddd347087db938ba4880a5f8cf014d248ab8bdca436028d57d9586af3f
+size 1543504488

model-00035-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:91009dff3feb99e10c98ae735f382e2015d06e0e74e4f3f69acdf6ebea138b8f
+size 1879114208

model-00036-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:82ce2b573295f4b7bc6ee25f290bd5f6fd7142a43f45c6698d8ad8ff72e3ed9b
+size 1543504488

model-00037-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8b909066f1bbac069f56ba563b28cfa2e29a39ffed6493ed8ecee3fa05045e81
+size 1879114208

model-00038-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bb0f55c0c9adf45844cbed1255d7d7403bd60255694896c3892b08f42c506977
+size 1543504488

model-00039-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ddb0b3997fe468f87b386b21ed1b206bfc051d5b240cb3b26a80935e9c3be92f
+size 1879114208

model-00040-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a77e4e295e3033964e52dfced378067a39d5a74fd354fedda545a3a1f333a1c1
+size 1543504488

model-00041-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e7249b344a6e9cca7dc11b32fe424d5b9e1f913f069ab9299217d303327f00a9
+size 1879114208

model-00042-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:79c19a7a99701cd40eed3be151fdae45a6980eb8e33116f52c3ff1b903a33271
+size 1543504488

model-00043-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7839554a2797b0035ec3877edce3c52d2e15ac5d1b4a5ee7d317e92a9bc8c67e
+size 1879114208

model-00044-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a972bb2f458242fc42c1c16e16d1e32af020bcdad0178de58db852c6f13d4fd3
+size 1543504488

model-00045-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:da63d6468d174ec50efb7c483f5c84b47eb1a6a6a90247ae85d3484ddb598d36
+size 1879114208

model-00046-of-00162.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:65ccff7133fd4937b82d04a6be1d9a1c05ff86e15d752710153d354d0fc0c166
+size 1543504488