sajjadhadi/Disease-Diagnosis-DeepSeek-R1-Distill-Llama-8B

Disclaimer

Important Notice: This model is a research tool for disease diagnosis and is NOT INTENDED for clinical or medical use. It is designed for educational and experimental purposes only. The model's outputs should NOT be used to make medical decisions, diagnose conditions, or guide treatment. Always consult a qualified healthcare professional for medical advice.

The developers and contributors of this model are not responsible for any misuse or consequences arising from its application in medical contexts. Use this model responsibly and in compliance with ethical guidelines.

---

Model Description

This model is a fine-tuned version of DeepSeek-R1-Distill-Llama-8B, adapted for disease diagnosis research. It leverages LoRA (Low-Rank Adaptation) to efficiently fine-tune the base model on a specialized dataset. The model is designed to analyze symptom descriptions and provide diagnostic suggestions in a structured format.

Key Features:

• Base Model: deepseek-ai/DeepSeek-R1-Distill-Llama-8B
• Fine-Tuning Method: LoRA (Low-Rank Adaptation)
• Training Framework: PEFT (Parameter-Efficient Fine-Tuning)
• Intended Use: Research and educational applications in medical diagnosis.

---

Intended Uses & Limitations

Intended Uses:

• Research: Study of AI applications in medical diagnosis.
• Education: Simulation of diagnostic scenarios for training purposes.
• Prototyping: Development of AI-assisted diagnostic tools (non-clinical).

Limitations:

• Not for Clinical Use: This model is not validated for real-world medical applications.
• Data Dependency: The model's performance depends on the quality and scope of its training data.
• Ethical Concerns: The model may generate incomplete or inaccurate suggestions. Always verify outputs with medical professionals.

---

Training and Evaluation Data

The model was fine-tuned on a dataset containing symptom-disease mappings. The dataset includes:

• Symptom Descriptions: Textual descriptions of patient symptoms.
• Disease Labels: Corresponding disease classifications based on symptoms.

The dataset was preprocessed and tokenized to ensure compatibility with the base model's architecture. Specific details about the dataset size and composition are not disclosed.

---

Training Procedure

Training Hyperparameters:

Parameter	Value
Learning Rate	1e-4
Batch Size	64
Evaluation Batch Size	8
Optimizer	Paged AdamW 32-bit
Scheduler	Cosine with 3% Warmup
Epochs	1
Seed	42

Technical Stack:

• PEFT: 0.14.0
• Transformers: 4.49.0
• PyTorch: 2.6.0+cu124
• Datasets: 3.3.2
• Tokenizers: 0.21.0

---

Ethical Considerations

Responsible Use:

• Transparency: Users should be aware of the model's limitations and intended use cases.
• Bias Mitigation: The model may inherit biases from its training data. Careful evaluation is required.
• Privacy: No real patient data was used in training.

Prohibited Uses:

• Clinical diagnosis or treatment decisions.
• Self-diagnosis tools for patients.
• Applications that could harm individuals or communities.

---

Acknowledgments

This model was developed using the DeepSeek-R1-Distill-Llama-8B base model and fine-tuned with the PEFT library. Special thanks to the open-source community for their contributions to AI research.

---

Note: This model is a work in progress. Further evaluation and documentation will be provided in future updates.