k4yt3x commited on
Commit
bf74e71
·
verified ·
1 Parent(s): 7c28292

Upload folder using huggingface_hub

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. .gitattributes +1 -0
  2. README.md +57 -0
  3. config.json +40 -0
  4. mergekit_config.yml +12 -0
  5. model-00001-of-00062.safetensors +3 -0
  6. model-00002-of-00062.safetensors +3 -0
  7. model-00003-of-00062.safetensors +3 -0
  8. model-00004-of-00062.safetensors +3 -0
  9. model-00005-of-00062.safetensors +3 -0
  10. model-00006-of-00062.safetensors +3 -0
  11. model-00007-of-00062.safetensors +3 -0
  12. model-00008-of-00062.safetensors +3 -0
  13. model-00009-of-00062.safetensors +3 -0
  14. model-00010-of-00062.safetensors +3 -0
  15. model-00011-of-00062.safetensors +3 -0
  16. model-00012-of-00062.safetensors +3 -0
  17. model-00013-of-00062.safetensors +3 -0
  18. model-00014-of-00062.safetensors +3 -0
  19. model-00015-of-00062.safetensors +3 -0
  20. model-00016-of-00062.safetensors +3 -0
  21. model-00017-of-00062.safetensors +3 -0
  22. model-00018-of-00062.safetensors +3 -0
  23. model-00019-of-00062.safetensors +3 -0
  24. model-00020-of-00062.safetensors +3 -0
  25. model-00021-of-00062.safetensors +3 -0
  26. model-00022-of-00062.safetensors +3 -0
  27. model-00023-of-00062.safetensors +3 -0
  28. model-00024-of-00062.safetensors +3 -0
  29. model-00025-of-00062.safetensors +3 -0
  30. model-00026-of-00062.safetensors +3 -0
  31. model-00027-of-00062.safetensors +3 -0
  32. model-00028-of-00062.safetensors +3 -0
  33. model-00029-of-00062.safetensors +3 -0
  34. model-00030-of-00062.safetensors +3 -0
  35. model-00031-of-00062.safetensors +3 -0
  36. model-00032-of-00062.safetensors +3 -0
  37. model-00033-of-00062.safetensors +3 -0
  38. model-00034-of-00062.safetensors +3 -0
  39. model-00035-of-00062.safetensors +3 -0
  40. model-00036-of-00062.safetensors +3 -0
  41. model-00037-of-00062.safetensors +3 -0
  42. model-00038-of-00062.safetensors +3 -0
  43. model-00039-of-00062.safetensors +3 -0
  44. model-00040-of-00062.safetensors +3 -0
  45. model-00041-of-00062.safetensors +3 -0
  46. model-00042-of-00062.safetensors +3 -0
  47. model-00043-of-00062.safetensors +3 -0
  48. model-00044-of-00062.safetensors +3 -0
  49. model-00045-of-00062.safetensors +3 -0
  50. model-00046-of-00062.safetensors +3 -0
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ tokenizer.json filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,57 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - SicariusSicariiStuff/Negative_LLAMA_70B
4
+ - Sao10K/L3.1-70B-Hanami-x1
5
+ - Saxo/Linkbricks-Horizon-AI-Japanese-Superb-V4-70B
6
+ - Sao10K/L3.3-70B-Euryale-v2.3
7
+ - EVA-UNIT-01/EVA-LLaMA-3.33-70B-v0.1
8
+ - Sao10K/70B-L3.3-Cirrus-x1
9
+ - k4yt3x/Cornerstone-0.1-LLaMA-70B
10
+ - LatitudeGames/Wayfarer-Large-70B-Llama-3.3
11
+ - TheDrummer/Anubis-70B-v1
12
+ library_name: transformers
13
+ tags:
14
+ - mergekit
15
+ - merge
16
+
17
+ ---
18
+ # Arynia-LLaMA-70B
19
+
20
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
21
+
22
+ ## Merge Details
23
+ ### Merge Method
24
+
25
+ This model was merged using the [SCE](https://arxiv.org/abs/2408.07990) merge method using [k4yt3x/Cornerstone-0.1-LLaMA-70B](https://huggingface.co/k4yt3x/Cornerstone-0.1-LLaMA-70B) as a base.
26
+
27
+ ### Models Merged
28
+
29
+ The following models were included in the merge:
30
+ * [SicariusSicariiStuff/Negative_LLAMA_70B](https://huggingface.co/SicariusSicariiStuff/Negative_LLAMA_70B)
31
+ * [Sao10K/L3.1-70B-Hanami-x1](https://huggingface.co/Sao10K/L3.1-70B-Hanami-x1)
32
+ * [Saxo/Linkbricks-Horizon-AI-Japanese-Superb-V4-70B](https://huggingface.co/Saxo/Linkbricks-Horizon-AI-Japanese-Superb-V4-70B)
33
+ * [Sao10K/L3.3-70B-Euryale-v2.3](https://huggingface.co/Sao10K/L3.3-70B-Euryale-v2.3)
34
+ * [EVA-UNIT-01/EVA-LLaMA-3.33-70B-v0.1](https://huggingface.co/EVA-UNIT-01/EVA-LLaMA-3.33-70B-v0.1)
35
+ * [Sao10K/70B-L3.3-Cirrus-x1](https://huggingface.co/Sao10K/70B-L3.3-Cirrus-x1)
36
+ * [LatitudeGames/Wayfarer-Large-70B-Llama-3.3](https://huggingface.co/LatitudeGames/Wayfarer-Large-70B-Llama-3.3)
37
+ * [TheDrummer/Anubis-70B-v1](https://huggingface.co/TheDrummer/Anubis-70B-v1)
38
+
39
+ ### Configuration
40
+
41
+ The following YAML configuration was used to produce this model:
42
+
43
+ ```yaml
44
+ base_model: k4yt3x/Cornerstone-0.1-LLaMA-70B
45
+ merge_method: sce
46
+ dtype: float32
47
+ models:
48
+ - model: SicariusSicariiStuff/Negative_LLAMA_70B
49
+ - model: LatitudeGames/Wayfarer-Large-70B-Llama-3.3
50
+ - model: TheDrummer/Anubis-70B-v1
51
+ - model: EVA-UNIT-01/EVA-LLaMA-3.33-70B-v0.1
52
+ - model: Sao10K/L3.3-70B-Euryale-v2.3
53
+ - model: Sao10K/70B-L3.3-Cirrus-x1
54
+ - model: Sao10K/L3.1-70B-Hanami-x1
55
+ - model: Saxo/Linkbricks-Horizon-AI-Japanese-Superb-V4-70B
56
+
57
+ ```
config.json ADDED
@@ -0,0 +1,40 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "k4yt3x/Cornerstone-0.1-LLaMA-70B",
3
+ "architectures": [
4
+ "LlamaForCausalLM"
5
+ ],
6
+ "attention_bias": false,
7
+ "attention_dropout": 0.0,
8
+ "bos_token_id": 128000,
9
+ "eos_token_id": [
10
+ 128001,
11
+ 128008,
12
+ 128009
13
+ ],
14
+ "head_dim": 128,
15
+ "hidden_act": "silu",
16
+ "hidden_size": 8192,
17
+ "initializer_range": 0.02,
18
+ "intermediate_size": 28672,
19
+ "max_position_embeddings": 131072,
20
+ "mlp_bias": false,
21
+ "model_type": "llama",
22
+ "num_attention_heads": 64,
23
+ "num_hidden_layers": 80,
24
+ "num_key_value_heads": 8,
25
+ "pretraining_tp": 1,
26
+ "rms_norm_eps": 1e-05,
27
+ "rope_scaling": {
28
+ "factor": 8.0,
29
+ "high_freq_factor": 4.0,
30
+ "low_freq_factor": 1.0,
31
+ "original_max_position_embeddings": 8192,
32
+ "rope_type": "llama3"
33
+ },
34
+ "rope_theta": 500000.0,
35
+ "tie_word_embeddings": false,
36
+ "torch_dtype": "float32",
37
+ "transformers_version": "4.49.0",
38
+ "use_cache": true,
39
+ "vocab_size": 128256
40
+ }
mergekit_config.yml ADDED
@@ -0,0 +1,12 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ base_model: k4yt3x/Cornerstone-0.1-LLaMA-70B
2
+ merge_method: sce
3
+ dtype: float32
4
+ models:
5
+ - model: SicariusSicariiStuff/Negative_LLAMA_70B
6
+ - model: LatitudeGames/Wayfarer-Large-70B-Llama-3.3
7
+ - model: TheDrummer/Anubis-70B-v1
8
+ - model: EVA-UNIT-01/EVA-LLaMA-3.33-70B-v0.1
9
+ - model: Sao10K/L3.3-70B-Euryale-v2.3
10
+ - model: Sao10K/70B-L3.3-Cirrus-x1
11
+ - model: Sao10K/L3.1-70B-Hanami-x1
12
+ - model: Saxo/Linkbricks-Horizon-AI-Japanese-Superb-V4-70B
model-00001-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:72053b6e14fe8d87552e15879be2c60f0d4bf293fcc25138816618287db09c69
3
+ size 4202692736
model-00002-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b148588055bf9495eb7d8a257033ba16616cb897e1cd85b5fde8de0b89ba2855
3
+ size 4202725624
model-00003-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e43ec1b876b0ccf42c2c9a38beda49ce1ccf6d153da301fa3af627e9745bc7a3
3
+ size 4362142872
model-00004-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b40f1d277f454ebaccf2092e0d278a9807cf9283aac94d5c0740dc4c0f64b82f
3
+ size 4362142880
model-00005-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3eaeeee8c0390d0a155fb28c86b685891e03f792b096957bc59e7c724ebeb6ac
3
+ size 4966188888
model-00006-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:300beebc620dc3304ab5aac199d19e2186acdb66f5d84f22c419cb10d4151a07
3
+ size 4362142888
model-00007-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b81ec53ca6eda95995b6712e7b8cf07f4a068d6d718bacc63295cee48b7cce65
3
+ size 4362142888
model-00008-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f85c265f7ec51a9db6db97b50cab61529a2f250441da29ec227a9adb74f10bbe
3
+ size 4966188888
model-00009-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a7b103248e46fe6da58582dd4858d0bb347f38f147e75f39881cd2250c387194
3
+ size 4362142888
model-00010-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:de248968d452ff1ba0fb11d57f8aef9a10d861f2a71d9dbc4979de2ee12d9ca2
3
+ size 4362142888
model-00011-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a293194ef1d916b48165957f7843dd5241b046426acf88c8d9b579bcb12522ad
3
+ size 4966188888
model-00012-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:822f221f37e4a33798fa3433767ab40bd4d5b10d5d6f970ebca3a35cb322c2c8
3
+ size 4362142880
model-00013-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e605ca6dacc43c495c66c3927358b36c2148212f3cbfe2d1980e501b9ebc2ed7
3
+ size 4362142888
model-00014-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0d7a9ce54baf8e89488589f0c0baa51c6c0e2dc4c25aa5cb42ce3a81839e33d1
3
+ size 4966188888
model-00015-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5d587bcc331068f5d9825806d2e1753e5f22242f712549fbc8801fcdce99c8e2
3
+ size 4362142888
model-00016-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c432feb678bf9bc46cdeae5da4a9fe86fe051ab759f1d67dc7f8a9086e35497f
3
+ size 4362142888
model-00017-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:45ebf4e31b3d634bec5b5cdf50f3cb14875f46c72586ae24b7079fba90880ba3
3
+ size 4966188888
model-00018-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:51bf7700e0c37f15c0ea2eddc78de71a28ef37e6ec47cc717586c0072f550a1d
3
+ size 4362142888
model-00019-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:727179fe450574d7bbb9e1d43745a16cf5cbf8e2e6bfc2bf628a4d1b4263181e
3
+ size 4362142888
model-00020-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e4f164037242c2543e434165d10707b220f2316d860bd56428a19d544426189b
3
+ size 4966188880
model-00021-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:76d2369abae6546f1c207e02b38073a8ea2954d30348bc7c920179f93e4cc3b4
3
+ size 4362142888
model-00022-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9fe82bb02f0d314134821903d0d21c2ab14b24e38b8e65f11da5256e7faf4171
3
+ size 4362142888
model-00023-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7ba37e0830275a613a843f453f2fbbb4a70c838db6ed33e3887edd6769b3e474
3
+ size 4966188888
model-00024-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0ff5602e0bf284e358078a192b9914ef78b3e0b02a85bb44426102abf9a0a9b0
3
+ size 4362142888
model-00025-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2106ffb3362292cbcb104595e46c359fcd83544ef2e2b1fe487c78ca8246d6d2
3
+ size 4362142888
model-00026-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:65c57275fc052ddea2a60c8df32b8f1bc2b12760f7aa578dfd11eb02029d5944
3
+ size 4966188888
model-00027-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:73215698979764347a20800c25e2b652701e517c5d6887bf544a406f650a07f5
3
+ size 4362142888
model-00028-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1cf651d14e3858e8890029972b8fe628ec35e9e3f073899f8e903591c2d76e9c
3
+ size 4362142880
model-00029-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:62e43f414cd15b0067c3ab537bc5796e490feba694e4292546931c2d2b64f2ce
3
+ size 4966188888
model-00030-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0d0962d5962487cb349a97dffa83b275dfd6f8f116489002ad239c4692e33aad
3
+ size 4362142888
model-00031-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9ebd9a0559075316da1846a6d645e776946ae25734d84bc077f05115b26aaeb0
3
+ size 4362142888
model-00032-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:db579140ea8806d9cb93caa9868154a6ca53074935c7b7b0a0d3ce70d690495f
3
+ size 4966188888
model-00033-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f53fe6dc5d921ba189af2fbf319122b9c3227508e0acedfe9c18bf2c37032a60
3
+ size 4362142888
model-00034-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ce9d19d07837aeafa5412f7a3a2637c1ce02a21a4d0e9872f46a8fd8ed08bb22
3
+ size 4362142888
model-00035-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:101d28d169308f45cc5819c77abb694070039ecc2252d8e029bf742a4321d053
3
+ size 4966188888
model-00036-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1c4e9429e4354ed648eaac4647511c7bd364a2c24322375db3ff96055c0b6861
3
+ size 4362142880
model-00037-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cd09a64b94df8ccf9d4c3faf2ce883a214fa3ce845a8a0a8eab8b9cf9bf679f3
3
+ size 4362142880
model-00038-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6c786bd49e8f46b0c1285e8d2bbd156f2e92361b42f9a2025a7c34a08ac0f1d9
3
+ size 4966188888
model-00039-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6323012f82f8c8cd270a8f7b8e346e642be4e447dbcf0c1829e8b3f5ec85d7ec
3
+ size 4362142888
model-00040-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:74b8a845e1467fb20c698518531ed4ee4ca07dddb5455a29e91946ce2293d2ef
3
+ size 4362142888
model-00041-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8dd41327aef156aeb55c9f62f832423382af0ba6c17b2505c376a20ed3cb9706
3
+ size 4966188888
model-00042-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:192c87e57f47a317894629dc6b7f866b155cf7458026b55d5671ca81eedf6fd6
3
+ size 4362142888
model-00043-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:08f9ea3051b5207692ced7eb515d7bdf78a108c5a82f6bc976b824cb4153f693
3
+ size 4362142888
model-00044-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fe1f4a41c697641599b1d493c9ddef450c302c93c30212992d7386b3e57023db
3
+ size 4966188888
model-00045-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dc5e05713de0f93236b5eae36778f7abd58a01c7ab6e0731f793da02de46b073
3
+ size 4362142880
model-00046-of-00062.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e4cbdecc4956cd07077b44d061350bb76f0b619aebad223c5731fd7e5d28a504
3
+ size 4362142888