Hugging Face's logo

Spaces:
Shitqq
/
Openness-leaderboard
Running

App Files Community
2
Openness-leaderboard
File size: 3,138 Bytes 
ea432a6
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
4b2d619
ea432a6
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
 
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Openness Leaderboard</title>
    <style>
        table {
            width: 50%;
            border-collapse: collapse;
            margin: 20px 0;
        }
        th, td {
            border: 1px solid #ddd;
            padding: 8px;
            text-align: center;
        }
        th {
            background-color: #f2f2f2;
        }
    </style>
</head>
<body>

<h2>Openness Leaderboard</h2>

<table>
    <thead>
        <tr>
            <th>Rank</th>
            <th>Model</th>
            <th>weights</th>
            <th>data</th>
            <th>open source license</th>
            <th>Organization</th>
        </tr>
    </thead>
    <tbody>
        <tr>
            <td>1</td>
            <td>olmo</td>
            <td>✅</td>
            <td>✅</td>
            <td>✅</td>
            <td>AllenAI (AI2)</td>
        </tr>
        <tr>
            <td>2</td>
            <td>phi 3.5</td>
            <td>✅</td>
            <td>❌</td>
            <td>✅</td>
            <td>Microsoft</td>
        </tr>
        <tr>
            <td>3</td>
            <td>yi 1.5</td>
            <td>✅</td>
            <td>❌</td>
            <td>✅</td>
            <td>01.ai</td>
        </tr>
        <tr>
            <td>4</td>
            <td>mixtral8x22b</td>
            <td>✅</td>
            <td>❌</td>
            <td>✅</td>
            <td>Mistral</td>
        </tr>
        <tr>
            <td>5</td>
            <td>qwen2 72b</td>
            <td>✅</td>
            <td>❌</td>
            <td>❌</td>
            <td>Alibaba</td>
        </tr>
        <tr>
            <td>6</td>
            <td>llama 3.1</td>
            <td>✅</td>
            <td>❌</td>
            <td>❌</td>
            <td>Meta</td>
        </tr>
        <tr>
            <td>7</td>
            <td>deepseek 2.5</td>
            <td>✅</td>
            <td>❌</td>
            <td>❌</td>
            <td>Deepseek</td>
        </tr>
        <tr>
            <td>8</td>
            <td>jamba 1.5</td>
            <td>✅</td>
            <td>❌</td>
            <td>❌</td>
            <td>AI21 Labs</td>
        </tr>
    </tbody>
</table>
<p><strong>🚨</strong>: This is an oversimplified leaderboard, there are so much more other than the weights, training data and license for a large language model such as the code to train the model, the model architecture, data preprocessing code etc. Please refer to the <a href="https://arxiv.org/abs/2403.13784" target="_blank">MOF paper</a> to learn more. Future updates will include a more detailed leaderboard with more models and components added. The purpose of this leaderboard is to let the world know that there are certain models claiming to be "open" despite falling short of openness standards (also known as open washing) due to restricted access to training data, training code, the use of custom licenses etc. Transparency and accurate representation of AI models openness are crucial.</p>
</body>
</html>