OpenMOSE
/

HRWKV7-Reka-Flash3-Preview

Text Generation

linear-attention

knowledge-distillation

Model card Files Files and versions

OpenMOSE commited on Jul 5

Commit

157ff0e

·

verified ·

1 Parent(s): 2ed9c45

Update README.md

Files changed (1) hide show

README.md +1 -0

README.md CHANGED Viewed

@@ -23,6 +23,7 @@ HRWKV7-Reka-Flash3-Preview is an experimental hybrid architecture model that com
 - **Hidden Dimension:** 6144
 - **Training Context Window:** 4096 tokens
 - **Inference Context Window** 32768
 ## Technical Innovation

 - **Hidden Dimension:** 6144
 - **Training Context Window:** 4096 tokens
 - **Inference Context Window** 32768
+- **Training Strategy** Following RADLADS method based knowledge distillation
 ## Technical Innovation