|
<head> <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.0.0-beta3/css/all.min.css"> </head> <style> body { font-family: "Helvetica Neue", Arial, sans-serif; background: radial-gradient(circle, #ffb347, #ffa92d, #ff9f14, #ff9500, #f08b00); color: #fff; line-height: 1.6; } .container { max-width: 800px; margin: 0 auto; padding: 40px; background-color: rgba(255, 255, 255, 0.1); border-radius: 10px; box-shadow: 0 0 20px rgba(0, 0, 0, 0.2); backdrop-filter: blur(10px); } .header { text-align: center; margin-bottom: 40px; } .title { font-size: 48px; font-weight: bold; text-transform: uppercase; letter-spacing: 2px; color: #fff; text-shadow: 2px 2px 4px rgba(0, 0, 0, 0.5); margin-bottom: 10px; } .subtitle { font-size: 24px; font-style: italic; color: #e6f7ff; text-shadow: 1px 1px 2px rgba(0, 0, 0, 0.3); margin-bottom: 20px; } .gif { text-align: center; margin-bottom: 40px; } .gif img { max-width: 100%; height: auto; border-radius: 10px; box-shadow: 0 0 20px rgba(0, 0, 0, 0.3); } .info-section { margin-bottom: 40px; } .section-title { font-size: 32px; font-weight: bold; color: #e6f7ff; text-shadow: 2px 2px 4px rgba(0, 0, 0, 0.5); margin-bottom: 20px; position: relative; padding-left: 30px; } .section-title::before { content: ""; position: absolute; left: 0; top: 50%; transform: translateY(-50%); width: 20px; height: 20px; background-color: #e6f7ff; border-radius: 50%; box-shadow: 0 0 10px rgba(0, 0, 0, 0.3); } .info-item { background-color: rgba(255, 255, 255, 0.1); padding: 20px; border-radius: 10px; box-shadow: 0 0 10px rgba(0, 0, 0, 0.2); margin-bottom: 20px; } .info-item h3 { font-size: 24px; font-weight: bold; color: #e6f7ff; text-shadow: 1px 1px 2px rgba(0, 0, 0, 0.3); margin-bottom: 10px; } .info-item p { font-size: 18px; color: #fff; text-shadow: 1px 1px 2px rgba(0, 0, 0, 0.3); line-height: 1.4; } .info-item pre { background-color: rgba(0, 0, 0, 0.2); padding: 20px; border-radius: 10px; font-family: monospace; font-size: 16px; color: #fff; text-shadow: 1px 1px 2px rgba(0, 0, 0, 0.3); overflow-x: auto; } .info-item a { color: #e6f7ff; text-decoration: none; border-bottom: 1px dashed #e6f7ff; transition: border-bottom 0.3s ease; } .info-item a:hover { border-bottom: 1px solid #e6f7ff; } .info-item table { width: 100%; border-collapse: collapse; box-shadow: 0 0 10px rgba(0, 0, 0, 0.2); } .info-item th, .info-item td { padding: 10px; text-align: left; border: 1px solid rgba(255, 255, 255, 0.2); } .info-item th { background-color: rgba(0, 0, 0, 0.2); font-weight: bold; color: #fff; text-shadow: 1px 1px 2px rgba(0, 0, 0, 0.3); } .info-item td { color: #e6f7ff; text-shadow: 1px 1px 2px rgba(0, 0, 0, 0.3); } </style> <div class="container"> <div class="header"> <h1 class="title">Finch 7B Merge</h1> <p class="subtitle">A SLERP merge of two powerful 7B language models</p> </div> <div class="gif"> <img src="https://i.imgur.com/Da14544.gif" alt="Finch GIF"> </div> <div class="info-section"> <h2 class="section-title">Description</h2> <div class="info-item"> <p>Finch is a 7B language model created by merging <a href="https://huggingface.co/macadeliccc/WestLake-7B-v2-laser-truthy-dpo">macadeliccc/WestLake-7B-v2-laser-truthy-dpo</a> and <a href="https://huggingface.co/SanjiWatsuki/Kunoichi-DPO-v2-7B">SanjiWatsuki/Kunoichi-DPO-v2-7B</a> using the SLERP method.</p> </div> </div> <div class="info-section"> <h2 class="section-title">Quantized Models</h2> <div class="info-item"> <p>Quantized versions of Finch are available:</p> <ul> <li><a href="https://huggingface.co/antiven0m/finch-6bpw-exl2">6bpw EXL2 Quant</a></li> <li><a href="https://huggingface.co/antiven0m/finch-gguf">GGUF Quants</a></li> </ul> </div> </div> <div class="info-section"> <h2 class="section-title">Recommended Settings</h2> <div class="info-item"> <p>For best results, use the <b>ChatML</b> format with the following sampler settings:</p> <pre>Temperature: 1.2 Min P: 0.2 Smoothing Factor: 0.2</pre> </div> </div> <div class="info-section"> <h2 class="section-title">Mergekit Configuration</h2> <div class="info-item"> <pre>base_model: macadeliccc/WestLake-7B-v2-laser-truthy-dpo dtype: float16 merge_method: slerp parameters: t: - filter: self_attn value: [0.0, 0.5, 0.3, 0.7, 1.0] - filter: mlp value: [1.0, 0.5, 0.7, 0.3, 0.0] - value: 0.5 slices: - sources: - layer_range: [0, 32] model: macadeliccc/WestLake-7B-v2-laser-truthy-dpo - layer_range: [0, 32] model: SanjiWatsuki/Kunoichi-DPO-v2-7B</pre> </div> </div> <div class="info-section"> <h2 class="section-title">Evaluation Results</h2> <div class="info-item"> <p>Finch's performance on the Open LLM Leaderboard:</p> <table> <tr><th>Metric</th><th>Value</th></tr> <tr><td>Avg.</td><td>73.78</td></tr> <tr><td>AI2 Reasoning Challenge (25-Shot)</td><td>71.59</td></tr> <tr><td>HellaSwag (10-Shot)</td><td>87.87</td></tr> <tr><td>MMLU (5-Shot)</td><td>64.81</td></tr> <tr><td>TruthfulQA (0-shot)</td><td>67.96</td></tr> <tr><td>Winogrande (5-shot)</td><td>84.14</td></tr> <tr><td>GSM8k (5-shot)</td><td>66.34</td></tr> </table> <p>Detailed results: <a href="https://huggingface.co/datasets/open-llm-leaderboard/details_antiven0m__finch">https://huggingface.co/datasets/open-llm-leaderboard/details_antiven0m__finch</a></p> </div> </div> </div> |
