Simontwice commited on
Commit
5c536d2
·
verified ·
1 Parent(s): 2591375

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -5
README.md CHANGED
@@ -43,13 +43,13 @@ mistral-chat $HOME/mistral_models/mamba-codestral-7B-v0.1 --instruct --max_toke
43
 
44
  ## Evaluation
45
  We evaluate Codestral Mamba, Codestral and open-weight models of the similar size on industry-standard benchmarks.
46
- | Benchmarks | HumanEval | MBPP | Spider | CruxE | HumanEval C++ | HumanEvalJava |HumanEvalJS |HumanEval Bash
47
- | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
48
  | CodeGemma 1.1 7B | 61.0% | 67.7% | 46.3% | 50.4% | 49.1% | 41.8% | 52.2% | 9.4% |
49
  | CodeLlama 7B | 31.1% | 48.2% | 29.3% | 50.1% | 31.7% | 29.7% | 31.7% | 11.4% |
50
- | DeepSeek v1.5 7B | 65.9% | 70.8% | 61.2% | 55.5% | 59.0% | 62.7% | 60.9% | 33.5% |
51
- | Mamba 7B | 75.0% | 68.5% | 58.8% | 57.8% | 59.8% | 57.0% | 61.5% | 31.1% |
52
- | CodeLlama 34B | 43.3% | 75.1% | 50.8% | 55.2% | 51.6% | 57.0% | 59.0% | 29.7% |
53
 
54
  ## The Mistral AI Team
55
 
 
43
 
44
  ## Evaluation
45
  We evaluate Codestral Mamba, Codestral and open-weight models of the similar size on industry-standard benchmarks.
46
+ | Benchmarks | HumanEval | MBPP | Spider | CruxE | HumanEval C++ | HumanEvalJava |HumanEvalJS |HumanEval Bash |
47
+ | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
48
  | CodeGemma 1.1 7B | 61.0% | 67.7% | 46.3% | 50.4% | 49.1% | 41.8% | 52.2% | 9.4% |
49
  | CodeLlama 7B | 31.1% | 48.2% | 29.3% | 50.1% | 31.7% | 29.7% | 31.7% | 11.4% |
50
+ | DeepSeek v1.5 7B | 65.9% | **70.8%** | **61.2%** | 55.5% | 59.0% | **62.7%** | 60.9% | **33.5%** |
51
+ | **Codestral Mamba (7B)** | **75.0%** | 68.5% | 58.8% | **57.8%** | **59.8%** | 57.0% | **61.5%** | 31.1% |
52
+
53
 
54
  ## The Mistral AI Team
55