Update README.md
Browse files
    	
        README.md
    CHANGED
    
    | 
         @@ -50,7 +50,7 @@ This model was fully fine-tuned with BF16 on first 20k rows of `FreedomIntellige 
     | 
|
| 50 | 
         | 
| 51 | 
         
             
             - **Base Model**: Qwen3-0.6B, pre-trained by the Qwen team, Experts, pre-trained by the Suayptalha team.
         
     | 
| 52 | 
         
             
             - **Conversion**: The model copies embeddings, self-attention, and normalization weights from Qwen3-0.6B, replacing MLP layers with MoE layers (3 experts). Gating weights are randomly initialized.
         
     | 
| 53 | 
         
            -
             - **Fine-Tuning**: Not fine-tuned; users are recommended to fine-tune for specific tasks to optimize expert routing.
         
     | 
| 54 | 
         | 
| 55 | 
         
             
            ## Usage
         
     | 
| 56 | 
         | 
| 
         | 
|
| 50 | 
         | 
| 51 | 
         
             
             - **Base Model**: Qwen3-0.6B, pre-trained by the Qwen team, Experts, pre-trained by the Suayptalha team.
         
     | 
| 52 | 
         
             
             - **Conversion**: The model copies embeddings, self-attention, and normalization weights from Qwen3-0.6B, replacing MLP layers with MoE layers (3 experts). Gating weights are randomly initialized.
         
     | 
| 53 | 
         
            +
             - **Fine-Tuning**: Not fine-tuned; users are recommended to fine-tune for specific tasks to optimize expert routing. The fine-tuned version is already available and can be referred to as [huihui-ai/Huihui-MoE-1B-A0.6B-SFT](https://huggingface.co/huihui-ai/Huihui-MoE-1B-A0.6B-SFT).
         
     | 
| 54 | 
         | 
| 55 | 
         
             
            ## Usage
         
     | 
| 56 | 
         |