kshitijthakkar commited on
Commit
7e3ac1d
Β·
1 Parent(s): 97b162d

docs: Fix hardware selection guidance in SMOLTRACE documentation

Browse files

- Changed example from gpu-h200 to gpu-a10 for 8B model (more appropriate and cost-effective)
- Removed incorrect --hardware CLI flag documentation (this flag doesn't exist in SMOLTRACE)
- Clarified that hardware selection happens in HuggingFace Jobs YAML configuration
- Added hardware selection guide: cpu-basic for API models, gpu-a10 for 7B-13B, gpu-h200 for 70B+
- Updated best practices section to show correct YAML-based hardware configuration

Files changed (1) hide show
  1. screens/documentation.py +22 -17
screens/documentation.py CHANGED
@@ -966,16 +966,11 @@ GPU metrics and performance data:
966
 
967
  ### Hardware Selection
968
 
969
- ```bash
970
- # Automatic (default)
971
- # API models β†’ CPU
972
- # Local models β†’ GPU if available
973
 
974
- # Manual override
975
- --hardware cpu
976
- --hardware gpu_a10
977
- --hardware gpu_h200
978
- ```
979
 
980
  ### OpenTelemetry Options
981
 
@@ -994,7 +989,7 @@ SMOLTRACE works seamlessly with HuggingFace Jobs:
994
  ```yaml
995
  # job.yaml
996
  name: SMOLTRACE Evaluation
997
- hardware: gpu-h200
998
  environment:
999
  MODEL: meta-llama/Llama-3.1-8B
1000
  HF_TOKEN: ${{ secrets.HF_TOKEN }}
@@ -1010,10 +1005,15 @@ command: |
1010
  --leaderboard-repo huggingface/smolagents-leaderboard
1011
  ```
1012
 
 
 
 
 
 
1013
  **Benefits:**
1014
- - πŸ’° **H200 GPUs**: 2x faster evaluation
1015
  - πŸ“Š **Automatic Upload**: Results β†’ HuggingFace datasets
1016
  - πŸ”„ **Reproducible**: Same environment every time
 
1017
 
1018
  ---
1019
 
@@ -1053,13 +1053,18 @@ smoltrace-eval --model gpt-4 --num-tests 10
1053
  smoltrace-eval --model gpt-4 --num-tests 1000
1054
  ```
1055
 
1056
- ### 2. Use Appropriate Hardware
1057
- ```bash
1058
- # API models β†’ CPU (no GPU needed)
1059
- smoltrace-eval --model openai/gpt-4 --hardware cpu
 
 
1060
 
1061
- # Local models β†’ GPU (faster)
1062
- smoltrace-eval --model meta-llama/Llama-3.1-8B --hardware gpu_h200
 
 
 
1063
  ```
1064
 
1065
  ### 3. Enable Full Observability
 
966
 
967
  ### Hardware Selection
968
 
969
+ Hardware is selected in HuggingFace Jobs configuration (`hardware:` field in job.yaml), not via CLI flags.
 
 
 
970
 
971
+ SMOLTRACE automatically detects available resources:
972
+ - API models (via litellm) β†’ Uses CPU
973
+ - Local models (via transformers) β†’ Uses available GPU if present
 
 
974
 
975
  ### OpenTelemetry Options
976
 
 
989
  ```yaml
990
  # job.yaml
991
  name: SMOLTRACE Evaluation
992
+ hardware: gpu-a10 # Use gpu-h200 for 70B+ models
993
  environment:
994
  MODEL: meta-llama/Llama-3.1-8B
995
  HF_TOKEN: ${{ secrets.HF_TOKEN }}
 
1005
  --leaderboard-repo huggingface/smolagents-leaderboard
1006
  ```
1007
 
1008
+ **Hardware Selection:**
1009
+ - πŸ”§ **gpu-a10**: Perfect for 7B-13B models (cost-effective)
1010
+ - πŸš€ **gpu-h200**: Use for 70B+ models (high performance)
1011
+ - πŸ’» **cpu-basic**: API models (OpenAI, Anthropic via LiteLLM)
1012
+
1013
  **Benefits:**
 
1014
  - πŸ“Š **Automatic Upload**: Results β†’ HuggingFace datasets
1015
  - πŸ”„ **Reproducible**: Same environment every time
1016
+ - ⚑ **Optimized Compute**: Right hardware for your model size
1017
 
1018
  ---
1019
 
 
1053
  smoltrace-eval --model gpt-4 --num-tests 1000
1054
  ```
1055
 
1056
+ ### 2. Choose Appropriate Hardware in HF Jobs
1057
+ Hardware selection happens in your HuggingFace Jobs configuration:
1058
+
1059
+ ```yaml
1060
+ # For API models (OpenAI, Anthropic, etc.)
1061
+ hardware: cpu-basic
1062
 
1063
+ # For 7B-13B local models
1064
+ hardware: gpu-a10
1065
+
1066
+ # For 70B+ local models
1067
+ hardware: gpu-h200
1068
  ```
1069
 
1070
  ### 3. Enable Full Observability