Spaces:
Running
Running
Commit
Β·
7e3ac1d
1
Parent(s):
97b162d
docs: Fix hardware selection guidance in SMOLTRACE documentation
Browse files- Changed example from gpu-h200 to gpu-a10 for 8B model (more appropriate and cost-effective)
- Removed incorrect --hardware CLI flag documentation (this flag doesn't exist in SMOLTRACE)
- Clarified that hardware selection happens in HuggingFace Jobs YAML configuration
- Added hardware selection guide: cpu-basic for API models, gpu-a10 for 7B-13B, gpu-h200 for 70B+
- Updated best practices section to show correct YAML-based hardware configuration
- screens/documentation.py +22 -17
screens/documentation.py
CHANGED
|
@@ -966,16 +966,11 @@ GPU metrics and performance data:
|
|
| 966 |
|
| 967 |
### Hardware Selection
|
| 968 |
|
| 969 |
-
|
| 970 |
-
# Automatic (default)
|
| 971 |
-
# API models β CPU
|
| 972 |
-
# Local models β GPU if available
|
| 973 |
|
| 974 |
-
|
| 975 |
-
|
| 976 |
-
|
| 977 |
-
--hardware gpu_h200
|
| 978 |
-
```
|
| 979 |
|
| 980 |
### OpenTelemetry Options
|
| 981 |
|
|
@@ -994,7 +989,7 @@ SMOLTRACE works seamlessly with HuggingFace Jobs:
|
|
| 994 |
```yaml
|
| 995 |
# job.yaml
|
| 996 |
name: SMOLTRACE Evaluation
|
| 997 |
-
hardware: gpu-h200
|
| 998 |
environment:
|
| 999 |
MODEL: meta-llama/Llama-3.1-8B
|
| 1000 |
HF_TOKEN: ${{ secrets.HF_TOKEN }}
|
|
@@ -1010,10 +1005,15 @@ command: |
|
|
| 1010 |
--leaderboard-repo huggingface/smolagents-leaderboard
|
| 1011 |
```
|
| 1012 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1013 |
**Benefits:**
|
| 1014 |
-
- π° **H200 GPUs**: 2x faster evaluation
|
| 1015 |
- π **Automatic Upload**: Results β HuggingFace datasets
|
| 1016 |
- π **Reproducible**: Same environment every time
|
|
|
|
| 1017 |
|
| 1018 |
---
|
| 1019 |
|
|
@@ -1053,13 +1053,18 @@ smoltrace-eval --model gpt-4 --num-tests 10
|
|
| 1053 |
smoltrace-eval --model gpt-4 --num-tests 1000
|
| 1054 |
```
|
| 1055 |
|
| 1056 |
-
### 2.
|
| 1057 |
-
|
| 1058 |
-
|
| 1059 |
-
|
|
|
|
|
|
|
| 1060 |
|
| 1061 |
-
#
|
| 1062 |
-
|
|
|
|
|
|
|
|
|
|
| 1063 |
```
|
| 1064 |
|
| 1065 |
### 3. Enable Full Observability
|
|
|
|
| 966 |
|
| 967 |
### Hardware Selection
|
| 968 |
|
| 969 |
+
Hardware is selected in HuggingFace Jobs configuration (`hardware:` field in job.yaml), not via CLI flags.
|
|
|
|
|
|
|
|
|
|
| 970 |
|
| 971 |
+
SMOLTRACE automatically detects available resources:
|
| 972 |
+
- API models (via litellm) β Uses CPU
|
| 973 |
+
- Local models (via transformers) β Uses available GPU if present
|
|
|
|
|
|
|
| 974 |
|
| 975 |
### OpenTelemetry Options
|
| 976 |
|
|
|
|
| 989 |
```yaml
|
| 990 |
# job.yaml
|
| 991 |
name: SMOLTRACE Evaluation
|
| 992 |
+
hardware: gpu-a10 # Use gpu-h200 for 70B+ models
|
| 993 |
environment:
|
| 994 |
MODEL: meta-llama/Llama-3.1-8B
|
| 995 |
HF_TOKEN: ${{ secrets.HF_TOKEN }}
|
|
|
|
| 1005 |
--leaderboard-repo huggingface/smolagents-leaderboard
|
| 1006 |
```
|
| 1007 |
|
| 1008 |
+
**Hardware Selection:**
|
| 1009 |
+
- π§ **gpu-a10**: Perfect for 7B-13B models (cost-effective)
|
| 1010 |
+
- π **gpu-h200**: Use for 70B+ models (high performance)
|
| 1011 |
+
- π» **cpu-basic**: API models (OpenAI, Anthropic via LiteLLM)
|
| 1012 |
+
|
| 1013 |
**Benefits:**
|
|
|
|
| 1014 |
- π **Automatic Upload**: Results β HuggingFace datasets
|
| 1015 |
- π **Reproducible**: Same environment every time
|
| 1016 |
+
- β‘ **Optimized Compute**: Right hardware for your model size
|
| 1017 |
|
| 1018 |
---
|
| 1019 |
|
|
|
|
| 1053 |
smoltrace-eval --model gpt-4 --num-tests 1000
|
| 1054 |
```
|
| 1055 |
|
| 1056 |
+
### 2. Choose Appropriate Hardware in HF Jobs
|
| 1057 |
+
Hardware selection happens in your HuggingFace Jobs configuration:
|
| 1058 |
+
|
| 1059 |
+
```yaml
|
| 1060 |
+
# For API models (OpenAI, Anthropic, etc.)
|
| 1061 |
+
hardware: cpu-basic
|
| 1062 |
|
| 1063 |
+
# For 7B-13B local models
|
| 1064 |
+
hardware: gpu-a10
|
| 1065 |
+
|
| 1066 |
+
# For 70B+ local models
|
| 1067 |
+
hardware: gpu-h200
|
| 1068 |
```
|
| 1069 |
|
| 1070 |
### 3. Enable Full Observability
|