Hugging Face's logo

schnapper79
/
lumikabra-123B_v0.3

Text Generation
Transformers
Safetensors
mistral
mergekit
lumikabra-123B
conversational
text-generation-inference
Model card Files Community
schnapper79 commited on
Commit
dc8852a
·
verified ·
1 Parent(s): 8142f00

Upload folder using huggingface_hub

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. README.md +50 -0
  2. config.json +27 -0
  3. mergekit_config.yml +20 -0
  4. model-00001-of-00051.safetensors +3 -0
  5. model-00002-of-00051.safetensors +3 -0
  6. model-00003-of-00051.safetensors +3 -0
  7. model-00004-of-00051.safetensors +3 -0
  8. model-00005-of-00051.safetensors +3 -0
  9. model-00006-of-00051.safetensors +3 -0
  10. model-00007-of-00051.safetensors +3 -0
  11. model-00008-of-00051.safetensors +3 -0
  12. model-00009-of-00051.safetensors +3 -0
  13. model-00010-of-00051.safetensors +3 -0
  14. model-00011-of-00051.safetensors +3 -0
  15. model-00012-of-00051.safetensors +3 -0
  16. model-00013-of-00051.safetensors +3 -0
  17. model-00014-of-00051.safetensors +3 -0
  18. model-00015-of-00051.safetensors +3 -0
  19. model-00016-of-00051.safetensors +3 -0
  20. model-00017-of-00051.safetensors +3 -0
  21. model-00018-of-00051.safetensors +3 -0
  22. model-00019-of-00051.safetensors +3 -0
  23. model-00020-of-00051.safetensors +3 -0
  24. model-00021-of-00051.safetensors +3 -0
  25. model-00022-of-00051.safetensors +3 -0
  26. model-00023-of-00051.safetensors +3 -0
  27. model-00024-of-00051.safetensors +3 -0
  28. model-00025-of-00051.safetensors +3 -0
  29. model-00026-of-00051.safetensors +3 -0
  30. model-00027-of-00051.safetensors +3 -0
  31. model-00028-of-00051.safetensors +3 -0
  32. model-00029-of-00051.safetensors +3 -0
  33. model-00030-of-00051.safetensors +3 -0
  34. model-00031-of-00051.safetensors +3 -0
  35. model-00032-of-00051.safetensors +3 -0
  36. model-00033-of-00051.safetensors +3 -0
  37. model-00034-of-00051.safetensors +3 -0
  38. model-00035-of-00051.safetensors +3 -0
  39. model-00036-of-00051.safetensors +3 -0
  40. model-00037-of-00051.safetensors +3 -0
  41. model-00038-of-00051.safetensors +3 -0
  42. model-00039-of-00051.safetensors +3 -0
  43. model-00040-of-00051.safetensors +3 -0
  44. model-00041-of-00051.safetensors +3 -0
  45. model-00042-of-00051.safetensors +3 -0
  46. model-00043-of-00051.safetensors +3 -0
  47. model-00044-of-00051.safetensors +3 -0
  48. model-00045-of-00051.safetensors +3 -0
  49. model-00046-of-00051.safetensors +3 -0
  50. model-00047-of-00051.safetensors +3 -0
README.md ADDED
@@ -0,0 +1,50 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: []
3
+ library_name: transformers
4
+ tags:
5
+ - mergekit
6
+ - merge
7
+
8
+ ---
9
+ # lumikabra3
10
+
11
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
12
+
13
+ ## Merge Details
14
+ ### Merge Method
15
+
16
+ This model was merged using the della_linear merge method using /workspace/text-generation-webui/models/mistralai_Mistral-Large-Instruct-2407 as a base.
17
+
18
+ ### Models Merged
19
+
20
+ The following models were included in the merge:
21
+ * /workspace/text-generation-webui/models/FluffyKaeloky_Luminum-v0.1-123B
22
+ * /workspace/text-generation-webui/models/anthracite-org_magnum-v2-123b
23
+ * /workspace/text-generation-webui/models/migtissera_Tess-3-Mistral-Large-2-123B
24
+
25
+ ### Configuration
26
+
27
+ The following YAML configuration was used to produce this model:
28
+
29
+ ```yaml
30
+ models:
31
+ - model: /workspace/text-generation-webui/models/anthracite-org_magnum-v2-123b
32
+ parameters:
33
+ weight: 0.34
34
+ density: 0.5
35
+ - model: /workspace/text-generation-webui/models/FluffyKaeloky_Luminum-v0.1-123B
36
+ parameters:
37
+ weight: 0.24
38
+ density: 0.8
39
+ - model: /workspace/text-generation-webui/models/migtissera_Tess-3-Mistral-Large-2-123B
40
+ parameters:
41
+ weight: 0.34
42
+ density: 0.9
43
+ merge_method: della_linear
44
+ base_model: /workspace/text-generation-webui/models/mistralai_Mistral-Large-Instruct-2407
45
+ parameters:
46
+ epsilon: 0.05
47
+ lambda: 1
48
+ int8_mask: true
49
+ dtype: bfloat16
50
+ ```
config.json ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "/workspace/text-generation-webui/models/mistralai_Mistral-Large-Instruct-2407",
3
+ "architectures": [
4
+ "MistralForCausalLM"
5
+ ],
6
+ "attention_dropout": 0.0,
7
+ "bos_token_id": 1,
8
+ "eos_token_id": 2,
9
+ "head_dim": 128,
10
+ "hidden_act": "silu",
11
+ "hidden_size": 12288,
12
+ "initializer_range": 0.02,
13
+ "intermediate_size": 28672,
14
+ "max_position_embeddings": 131072,
15
+ "model_type": "mistral",
16
+ "num_attention_heads": 96,
17
+ "num_hidden_layers": 88,
18
+ "num_key_value_heads": 8,
19
+ "rms_norm_eps": 1e-05,
20
+ "rope_theta": 1000000.0,
21
+ "sliding_window": null,
22
+ "tie_word_embeddings": false,
23
+ "torch_dtype": "bfloat16",
24
+ "transformers_version": "4.44.2",
25
+ "use_cache": true,
26
+ "vocab_size": 32768
27
+ }
mergekit_config.yml ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ models:
2
+ - model: /workspace/text-generation-webui/models/anthracite-org_magnum-v2-123b
3
+ parameters:
4
+ weight: 0.34
5
+ density: 0.5
6
+ - model: /workspace/text-generation-webui/models/FluffyKaeloky_Luminum-v0.1-123B
7
+ parameters:
8
+ weight: 0.24
9
+ density: 0.8
10
+ - model: /workspace/text-generation-webui/models/migtissera_Tess-3-Mistral-Large-2-123B
11
+ parameters:
12
+ weight: 0.34
13
+ density: 0.9
14
+ merge_method: della_linear
15
+ base_model: /workspace/text-generation-webui/models/mistralai_Mistral-Large-Instruct-2407
16
+ parameters:
17
+ epsilon: 0.05
18
+ lambda: 1
19
+ int8_mask: true
20
+ dtype: bfloat16
model-00001-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c15fbee97fd6a1053a0242de4b69fcd18d83486aaefb9f9b206e2e66cd7e5132
3
+ size 4378928504
model-00002-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:95a1be3519207a56f51deb85a788013c6081794d2af03b58a60c96bfe93735e5
3
+ size 4907411088
model-00003-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:561bef5b2fdd3d814639c55b761d2f5d6f248acbf06279820b79295d13e6f7e1
3
+ size 4806747904
model-00004-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f3aea6320a7254f1e10b0b1f68d20429a0bb7068882d1e90de125192762d3546
3
+ size 4831938544
model-00005-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:58324e621dec2ae32a2d9cc18d91f29e044d5dc258c9e7f572bab47f64dd6aed
3
+ size 4831938552
model-00006-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8b8320b63a92bba5db3d3855737f5f8eda778d79763328c00012b3fcbac26d85
3
+ size 4907411096
model-00007-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d631e6ec8e4fa0ffbd82424dc5eaf44c81cf43cec97a48176e8b7b2fcd4cc1ff
3
+ size 4806747904
model-00008-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2b81ab0543586c472e1754b38245e2310407d5c53dfbe3aef2ed6b6b8d6fd984
3
+ size 4831938536
model-00009-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ab564ca576b2a2237185e79524eae24040ac8956b55801ae256d11b1483445a0
3
+ size 4831938552
model-00010-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8f32c420ef3ac13c59689bc95f3a93d8533aed11b987c822d06d88478ebfc618
3
+ size 4907411096
model-00011-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a40abf307b49623a8f777b73ba77773f733dbf12dc1e29450eb75b5b8001cda1
3
+ size 4806747904
model-00012-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4cb414b87601cd240ab710b1468f06ae910a153696ddab2dc102ca787ec21725
3
+ size 4831938544
model-00013-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:67e3b51aa769a708ceb086cdba81ac8f5c848391c1fdb7e0d3ceb7bd782d0888
3
+ size 4831938552
model-00014-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:eccfa9342a4a45359d1d749b4b1d15d009f53762a6dd30495048bd2359b7d526
3
+ size 4907411088
model-00015-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:813be14c337568e15fe17b9f0a920c3a90459a55e093b4eedf816c6172927862
3
+ size 4806747904
model-00016-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:84e57fa61a96b41ff35bba86b36748c26fc5e0f62b40d900583d1baf6883038e
3
+ size 4831938544
model-00017-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8536af7afc038284dab8c453c7ad67e9999b4a5d511760281708a058689e7ff2
3
+ size 4831938552
model-00018-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8c41048c39d0bc43cbc3b983ba38a8f56b48f7fc98b00387f2f31e57218f38d9
3
+ size 4907411096
model-00019-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:10ece47ef4b368a87d0e088d07e1a919224c936e0afeb0b09dc3612723a3bb66
3
+ size 4806747904
model-00020-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fa8111138c5b720c9838cd24cb7d1abfd1a6588652ab3cc1255e9d62382365be
3
+ size 4831938544
model-00021-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f8eb36130e201bd0365c287b77f4f6190eeec774bfbd63dd8cff97527cf449ef
3
+ size 4831938544
model-00022-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c02abdbc3837cee535ea9f7cafb27a72e3db6aefc5708b29694e6ba76559a8f5
3
+ size 4907411096
model-00023-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6ad023636db1234af6591dbc2be4300eb58cb7200075fd392bdd819c448dfff2
3
+ size 4806747904
model-00024-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1f27d02e0c0e060a40a75508fed209417c01265ca284cf9298e0f51b4cbba846
3
+ size 4831938544
model-00025-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c58b0bd369dc62d6a84d279c4a765537063fe7e52c0040aeb8d82b000ef2976a
3
+ size 4831938552
model-00026-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d21331bfae538b9f1d6b294537c2f3e77612095d425b3604ed45d811be7321d0
3
+ size 4907411096
model-00027-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:03988c435c3696d260a44517723b9b38afb569b776d812477fbc082151895cfc
3
+ size 4806747896
model-00028-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e1a8447cb9dad2eb918d46703e5f8a3218753b4642627a09c30fbff2708ec09d
3
+ size 4831938544
model-00029-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fb9d757c47412c03ad7fbbea98ff5fe52a3632f0af396cf2c09c42b01c1a7e89
3
+ size 4831938552
model-00030-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3e639b79a74839a29ee5551f6b7295482841a819242ddc0fbf911872047f77ab
3
+ size 4907411096
model-00031-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c1e093f85034700eb5adfe4b32198c6d020b9bc0775c1347ea5bab0d85f2c3b5
3
+ size 4806747904
model-00032-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e17d5ebf37be367495d513c482d91d9c5f4c789b83a86c154500e917f502d3bb
3
+ size 4831938544
model-00033-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:664ec553f890d093fa834630d167dfc05ec788dfbd670774ce152b48d0801768
3
+ size 4831938544
model-00034-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6f818c381a5b076ebe82c3ecf2364cd4fd032c4688a10d0508c0572c691e4755
3
+ size 4907411096
model-00035-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:736aa5d6a867a64edce65e3fcacb3184d73209893566ba93248faf3b0e9a35e0
3
+ size 4806747904
model-00036-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:56b0081033979992a06baf60ca4f6553ad03d23785ce23ada5f2e8bd5baf6ba7
3
+ size 4831938544
model-00037-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:64a36f3ee43297636e18a9e61fedcc68b9d3bf29cee29b754090ba70cce83d08
3
+ size 4831938552
model-00038-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:66c4c7801b8ba997b433881908ec2b420980e9350d4239f48d0faead6bf1d8f9
3
+ size 4907411096
model-00039-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:073197b6cf58155698df6713d36f812f5541e3a4db59e733e501eceed1dd7c68
3
+ size 4806747904
model-00040-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e2339be74e057df7ccdf8325768af4a0cad7d644d7844fff76f861bf463e58fa
3
+ size 4831938544
model-00041-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3ce31a743e612e513c65954cf9af7cbb6353f7c94e05fcc2fdd9cc89b218e2b5
3
+ size 4831938552
model-00042-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7a3933b476b3c1961c8584c6a260bb95666f347363858b0bee8d7fe79cd54422
3
+ size 4907411096
model-00043-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8e8728b37e1f2ee580638871ae7baefa8a680c5c38c921d81894e95514c3ee6c
3
+ size 4806747904
model-00044-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4e37c407a8edefc3b24258f447b7432ef5a46ccb9369d15d3f452638c0d79041
3
+ size 4831938544
model-00045-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:29ba71b5f197c960f001f7cdb6d9161114d689dd3140a56d22dd09812510e253
3
+ size 4831938552
model-00046-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8307469faa21f2e57c6fab3259dba9c59e462f4d780bc265219d37b868122bc0
3
+ size 4907411088
model-00047-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3d7b14651dbd0c9f6e804299364dc1a3ab8ae6273d6dffae80fec353cc1aa935
3
+ size 4806747904