README.md

---
license: mit
license_name: deepseek
license_link: LICENSE
pipeline_tag: image-text-to-text
library_name: mlx
base_model:
  - deepseek-ai/Janus-Pro-7B
tags:
  - chat
---

This model is derived from https://huggingface.co/deepseek-ai/Janus-Pro-7B and the main modifications are as follows

- bin files are updated to safetensors
- Add chat_template

`4bit` mainly refers to quantifying the LLM part to 4 bits.

## Quick Start

In Macos (Apple silicon), use [mlx](https://github.com/ml-explore/mlx) framework https://github.com/wnma3mz/tLLM

```bash
tllm.server --model_path $MODEL_PATH --hostname localhost --is_local --client_size 1
```

`$MODEL_PATH` like `wnma3mz/Janus-Pro-1B-4bit`