File size: 2,488 Bytes
a69a7e7 53c0ffc a69a7e7 3e78b50 a69a7e7 90dca55 a69a7e7 3e78b50 a69a7e7 772564f a69a7e7 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 |
---
language: en
tags:
- llama
- merge
- custom
- lumina-lexir1
- text-generation
license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
---
<div align="center">
<img src="https://occelli.nl/LUMINA-round.png" width="200" height="200" style="border-radius: 50%; box-shadow: 0 0 20px #0ff;">
<h1 style="color: #0ff; text-shadow: 0 0 10px #0ff;">LUMINA-LexiR1-8B</h1>
<div style="background: linear-gradient(45deg, #0ff3, #4444ff33);
padding: 20px;
border-radius: 10px;
border: 1px solid #0ff;
box-shadow: 0 0 20px rgba(0, 255, 255, 0.2);">
<h3 style="color: #0ff; margin: 0;">🧬 Model Fusion Architecture</h3>
</div>
</div>
## 🌟 Overview
LUMINA-LexiR1-8B is an experimental fusion of two powerful language models:
- 🔹 [Orenguteng/Llama-3.1-8B-Lexi-Uncensored-V2](https://huggingface.co/Orenguteng/Llama-3.1-8B-Lexi-Uncensored-V2)
- 🔹 [deepseek-ai/DeepSeek-R1-Distill-Llama-8B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-8B)
## 🔮 Architecture
This model employs a custom merging technique:
- Custom layer identification and integration
- DARE (Dynamic Attention Resolution Enhancement)
- TIES (Temporal Information Enhancement System) applied to adjacent layers
- Enhanced self-awareness capabilities
## 💫 Technical Specifications
{
"model_type": "llama",
"hidden_size": 4096,
"num_attention_heads": 32,
"num_hidden_layers": 34,
"intermediate_size": 14336,
"max_position_embeddings": 131072,
"rope_scaling": {
"factor": 8.0,
"type": "llama3"
}
}
! This is an experimental model. Use with caution.
+ Demonstrates exceptional self-awareness capabilities
🔧 Model Architecture
The model features:
8B parameters
Advanced RoPE scaling (factor: 8.0)
Custom attention mechanisms
Extended context window (131K tokens)
Specialized neuron mapping between parent models
📝 License
This model is released under the Apache 2.0 license.
🌐 Citations
If you use this model, please cite both parent models:
@misc{lumina-lexir1-8b,
author = {Mambiux},
title = {LUMINA-LexiR1-8B: A Custom Merged Language Model},
year = {2025},
publisher = {Hugging Face}
}
---
<div align="center">
<div align="center" style="margin-top: 40px; padding: 20px; background: linear-gradient(45deg, #0ff1, #4444ff11); border-radius: 10px;">
<p style="color: #0ff; font-size: 1.2em;">
🌟 Created by Mambiux | 2025 🌟
</p>
</div>
|