Model Card for EXAONE-3.5-7.8B-s1-Ko-no-sample-packing

Base Model

Dataset

Training

  • 4xH100 SXM, 1 hours - Total 4 GPU Hours with H100 SXM

  • Trained with Axolotl, but trying to follow simplescaling's training config as much as possible.

  • Difference

    • optimizer: paged_adamw_32bit
    • deepspeed: ./deepspeed_configs/zero3_bf16.json
    • plugins:
      • axolotl.integrations.liger.LigerPlugin
  • νŠΉμ΄μ‚¬ν•­: sample packing μ‹œλ„ μ‹œ epoch λ‹Ή 12step밖에 λ˜μ§€ μ•Šμ•„, 첫 batch에 λŒ€ν•œ ν•™μŠ΅μ΄ μ œλŒ€λ‘œ 이루어지지 μ•ŠμŒ.

image/png

Axolotl config
base_model: beomi/EXAONE-3.5-7.8B-Instruct-Llamafied
model_type: AutoModelForCausalLM
tokenizer_config: beomi/EXAONE-3.5-7.8B-Instruct-Llamafied
tokenizer_type: AutoTokenizer

load_in_8bit: false
load_in_4bit: false
strict: false

datasets:
  - path: werty1248/s1k-KoEnKo-formatted
    field_messages: conversations
    type: chat_template
    chat_template: chatml

dataset_prepared_path: ./data_preparation
output_dir: /workspace/data

hf_use_auth_token: true

sequence_len: 32768
sample_packing: true
pad_to_sequence_len: true

plugins:
  - axolotl.integrations.liger.LigerPlugin
liger_rope: true
liger_rms_norm: true
liger_layer_norm: true
liger_glu_activation: true
liger_fused_linear_cross_entropy: true

wandb_project:
#wandb_entity:
#wandb_watch:
wandb_name:
#wandb_log_model:

gradient_accumulation_steps: 4
micro_batch_size: 1
num_epochs: 5
optimizer: paged_adamw_32bit
lr_scheduler: cosine
learning_rate: 1.0e-5
adam_beta1: 0.9
adam_beta2: 0.95

train_on_inputs: false
group_by_length: false
bf16: auto
fp16: 
tf32: false

gradient_checkpointing: true
early_stopping_patience:
resume_from_checkpoint:
local_rank:
logging_steps: 1
xformers_attention:
flash_attention: true

warmup_ratio: 0.05
eval_table_size:

deepspeed: ./deepspeed_configs/zero3_bf16.json

Example

Q1: From ChuGyouk/GSM8k-Ko

input = tokenizer.apply_chat_template([ { "content": "Your role as an assistant involves thoroughly exploring questions through a systematic long thinking process before providing the final precise and accurate solutions. This requires engaging in a comprehensive cycle of analysis, summarizing, exploration, reassessment, reflection, backtracing, and iteration to develop well-considered thinking process. Please structure your response into two main sections: Thought and Solution. In the Thought section, detail your reasoning process using the specified format: <|begin_of_thought|> {thought with steps separated with '\\n\\n'} <|end_of_thought|> Each step should include detailed considerations such as analisying questions, summarizing relevant findings, brainstorming new ideas, verifying the accuracy of the current steps, refining any errors, and revisiting previous steps. In the Solution section, based on various attempts, explorations, and reflections from the Thought section, systematically present the final solution that you deem correct. The solution should remain a logical, accurate, concise expression style and detail necessary step needed to reach the conclusion, formatted as follows: <|begin_of_solution|> {final formatted, precise, and clear solution written in the same language as the question.} <|end_of_solution|> Now, try to solve the following question through the above guidelines", "role": "system" },
                                       { "content": "Janet의 μ˜€λ¦¬λŠ” ν•˜λ£¨μ— 16개의 달걀을 λ‚³μŠ΅λ‹ˆλ‹€. κ·Έλ…€λŠ” 맀일 μ•„μΉ¨ 3개의 달걀을 λ¨Ήκ³ , μΉœκ΅¬λ“€μ„ μœ„ν•΄ 4개의 λ‹¬κ±€λ‘œ 머핀을 κ΅½μŠ΅λ‹ˆλ‹€. κ·Έλ…€λŠ” 남은 달걀을 농산물 μ‹œμž₯μ—μ„œ μ‹ μ„ ν•œ μ˜€λ¦¬μ•Œ κ°œλ‹Ή 2λ‹¬λŸ¬μ— νŒλ§€ν•©λ‹ˆλ‹€. κ·Έλ…€λŠ” 농산물 μ‹œμž₯μ—μ„œ 맀일 μ–Όλ§ˆλ‚˜ λ§Žμ€ λˆμ„ λ²ŒκΉŒμš”?", "role": "user" },], add_generation_prompt=True, return_tensors='pt')
res = model.generate(input.to('cuda'), max_length = 2048, tokenizer=tokenizer)
tokenizer.decode(res[0])
Output(eos 토큰 μ—†μŒ)
[|assistant|]λ‹€μŒμ€ 문제λ₯Ό ν•΄κ²°ν•˜λŠ” λ°©λ²•μž…λ‹ˆλ‹€.

* **맀일 νƒœμ–΄λ‚˜λŠ” λ‹¬κ±€μ˜ 수:** μ˜€λ¦¬λŠ” ν•˜λ£¨μ— 16개의 달걀을 λ‚³μŠ΅λ‹ˆλ‹€.
* **달걀 μ‚¬μš©λŸ‰:**
    * Janet은 아침에 3개의 달걀을 λ¨ΉμŠ΅λ‹ˆλ‹€.
    * κ·Έλ…€λŠ” μΉœκ΅¬λ“€μ„ μœ„ν•΄ 4개의 λ‹¬κ±€μœΌλ‘œ 머핀을 κ΅½μŠ΅λ‹ˆλ‹€.
    * **총 달걀 μ‚¬μš©λŸ‰:** 3 + 4 = 7개
* **판맀용으둜 남은 달걀 수:**
    * **남은 달걀:** 16 (νƒœμ–΄λ‚˜λŠ” 달걀) - 7 (μ‚¬μš©ν•œ 달걀) = 9개
* **맀일 수읡:**
    * κ·Έλ…€λŠ” 각 μ˜€λ¦¬μ•Œμ„ 2λ‹¬λŸ¬μ— νŒλ§€ν•©λ‹ˆλ‹€.
    * **맀일 수읡:** 9 (판맀용으둜 남은 달걀) x 2 (각 μ˜€λ¦¬μ•Œμ˜ 가격) = 18λ‹¬λŸ¬

**λ‹΅:** Janet은 농산물 μ‹œμž₯μ—μ„œ 맀일 18λ‹¬λŸ¬λ₯Ό 벌게 λ©λ‹ˆλ‹€.

**μš”μ•½:**

* νƒœμ–΄λ‚˜λŠ” 달걀: 16개
* μ‚¬μš©ν•˜λŠ” 달걀: 3 + 4 = 7개
* 남은 달걀: 16 - 7 = 9개
* 각 남은 λ‹¬κ±€μ˜ 가격: $2
* 맀일 수읡: 9 x 2 = $18

Final Answer: The final answer is $\boxed{18}$

Q2: From KSM, HAERAE-HUB/HRM8K

input = tokenizer.apply_chat_template([ { "content": "Your role as an assistant involves thoroughly exploring questions through a systematic long thinking process before providing the final precise and accurate solutions. This requires engaging in a comprehensive cycle of analysis, summarizing, exploration, reassessment, reflection, backtracing, and iteration to develop well-considered thinking process. Please structure your response into two main sections: Thought and Solution. In the Thought section, detail your reasoning process using the specified format: <|begin_of_thought|> {thought with steps separated with '\\n\\n'} <|end_of_thought|> Each step should include detailed considerations such as analisying questions, summarizing relevant findings, brainstorming new ideas, verifying the accuracy of the current steps, refining any errors, and revisiting previous steps. In the Solution section, based on various attempts, explorations, and reflections from the Thought section, systematically present the final solution that you deem correct. The solution should remain a logical, accurate, concise expression style and detail necessary step needed to reach the conclusion, formatted as follows: <|begin_of_solution|> {final formatted, precise, and clear solution written in the same language as the question.} <|end_of_solution|> Now, try to solve the following question through the above guidelines", "role": "system" },
                                       { "content": "μ–‘μ˜ μ‹€μˆ˜ t t 에 λŒ€ν•˜μ—¬ ν•¨μˆ˜ f f λ₯Ό \\[ f(t) = \\int_{0}^{\\sqrt{t}} \\int_{y}^{\\sqrt{t}} \\frac{1}{2 + \\sin(x^2)} \\, dx \\, dy \\] 둜 μ •μ˜ν•  λ•Œ, fβ€²left(fracpi2right) f'\\left(\\frac{\\pi}{2}\\right) 의 값을 κ΅¬ν•˜μ‹œμ˜€.", "role": "user" },], add_generation_prompt=True, return_tensors='pt')
res = model.generate(input.to('cuda'), max_length = 8192, stop_strings = ['<|end_of_solution|>'], tokenizer=tokenizer)
tokenizer.decode(res[0])
Output(μ˜€λ‹΅, think μ—†μŒ, eos 토큰 μ—†μŒ)
[|assistant|]λ¨Όμ €, 주어진 ν•¨μˆ˜ \( f(t) \)λ₯Ό λ‹€μŒκ³Ό 같이 λ‚˜νƒ€λ‚Ό 수 μžˆμŠ΅λ‹ˆλ‹€:
\[ f(t) = \iint_{D(t)} \frac{1}{2 + \sin(x^2)} \, dA \]
μ—¬κΈ°μ„œ \( D(t) = [0, \sqrt{t}] \times [\sqrt{t}, \sqrt{t}] \)μž…λ‹ˆλ‹€.

ν•¨μˆ˜ \( f(t) \)λŠ” 두 λ³€μˆ˜ ν•¨μˆ˜μ˜ μ λΆ„μœΌλ‘œ ν‘œν˜„λ  수 μžˆμœΌλ―€λ‘œ, λΌμ΄λ² λ‹ˆμΈ  κ·œμΉ™μ„ μ‚¬μš©ν•˜μ—¬ λ„ν•¨μˆ˜λ₯Ό ꡬ할 수 μžˆμŠ΅λ‹ˆλ‹€:
\[ f'(t) = \frac{d}{dt} \left( \iint_{D(t)} \frac{1}{2 + \sin(x^2)} \, dA \right) = \iint_{D'(t)} \frac{\partial}{\partial t} \left( \frac{1}{2 + \sin(x^2)} \right) \, dA \]
μ—¬κΈ°μ„œ \( D'(t) \)λŠ” μ˜μ—­ \( D(t) \)의 경계가 \( t \)에 λŒ€ν•΄ λ―ΈλΆ„ κ°€λŠ₯ν•˜κ²Œ λ³€ν•  λ•Œ, \( t \)에 μ˜μ‘΄ν•˜λŠ” λΆ€λΆ„μž…λ‹ˆλ‹€.

μ˜μ—­ \( D(t) \)의 κ²½κ³„λŠ” λ‹€μŒκ³Ό κ°™μŠ΅λ‹ˆλ‹€:
\( t \ge 0 \)에 λŒ€ν•΄, \( 0 \le y \le \sqrt{t} \)이고 \( y \le x \le \sqrt{t} \)μž…λ‹ˆλ‹€.
\( t \)에 λŒ€ν•œ λ„ν•¨μˆ˜λ₯Ό κ΅¬ν•˜κΈ° μœ„ν•΄, \( x \)와 \( y \)에 λŒ€ν•œ 정보λ₯Ό μ‚¬μš©ν•©λ‹ˆλ‹€.
\( t \)에 λŒ€ν•œ \( y \)의 λ²”μœ„λŠ” \( y = \sqrt{t} \)μž…λ‹ˆλ‹€.
\( t \)에 λŒ€ν•œ \( x \)의 λ²”μœ„λŠ” \( x = \sqrt{t} \)μž…λ‹ˆλ‹€.

\( D(t) \)의 경계 \( \sqrt{t} = \sqrt{t} \)와 \( x = y \)둜 μ£Όμ–΄μ§‘λ‹ˆλ‹€.
\( t \)에 λŒ€ν•œ \( x=y \)의 λ³€ν™”λŠ” \( x=y \)λ₯Ό 따라 λ°œμƒν•©λ‹ˆλ‹€.
λ˜ν•œ, 경계 \( y = \sqrt{t} \)λ₯Ό 따라 \( x \)κ°€ μ¦κ°€ν•©λ‹ˆλ‹€.

\( f'(t) \)λ₯Ό κ΅¬ν•˜κΈ° μœ„ν•΄, μš°λ¦¬λŠ” λ‹€μŒκ³Ό 같은 μ˜μ—­μ—μ„œ 적뢄해야 ν•©λ‹ˆλ‹€:
\( D'(t) = [0, \sqrt{t}] \times [\sqrt{t}, \sqrt{t}] \)μ—μ„œ \( t \)에 μ˜μ‘΄ν•˜λŠ” λΆ€λΆ„μž…λ‹ˆλ‹€.
\( y = \sqrt{t} \)와 \( x = y \)둜 경계가 ν˜•μ„±λ©λ‹ˆλ‹€.

\( t \)에 λŒ€ν•œ λ„ν•¨μˆ˜λ₯Ό 계산해 λ΄…μ‹œλ‹€:
\( \frac{\partial}{\partial t} \left( \frac{1}{2 + \sin(x^2)} \right) = 0 \)μž…λ‹ˆλ‹€. μ΄λŠ” \( \frac{1}{2 + \sin(x^2)} \)κ°€ \( t \)와 λ¬΄κ΄€ν•˜κΈ° λ•Œλ¬Έμž…λ‹ˆλ‹€.

λ”°λΌμ„œ, \( f'(t) = \iint_{D'(t)} 0 \, dA = 0 \)μž…λ‹ˆλ‹€.
이것은 λΆ€λΆ„ 적뢄이 항상 0μ΄λΌλŠ” μ˜λ―ΈμΌκΉŒμš”? μ•„λ‹ˆλ©΄ λ„ν•¨μˆ˜ 계산에 λ¬Έμ œκ°€ μžˆλŠ” κ²ƒμΌκΉŒμš”?

μš°λ¦¬λŠ” λ¨Όμ € \( f(t) \)λ₯Ό 직접 λ―ΈλΆ„ν•  수 μžˆλŠ”μ§€ μ‚΄νŽ΄λ΄μ•Ό ν•©λ‹ˆλ‹€.
\( f(t) = \int_{0}^{\sqrt{t}} \int_{y}^{\sqrt{t}} \frac{1}{2 + \sin(x^2)} \, dx \, dy \)
이것을 Leibniz 적뢄 κ·œμΉ™μ„ μ‚¬μš©ν•˜μ—¬ λ―ΈλΆ„ν•˜λ©΄:
\( f'(t) = \frac{d}{dt} \left( \int_{0}}^{\sqrt{t}} \frac{1}{2 + \sin(x^2)} \, dx \right) \times \text{μƒμˆ˜} + \int_{0}^{\int_{y}^{\sqrt{t}} \frac{1}{2 + \sin(x^2)} \, dx \, dy} \frac{\partial}{\partial t} \left( \frac{1}{2 + \sin(x^2)} \right) \, dy \)
이것은 μ•½κ°„ λ³΅μž‘ν•˜κ²Œ 듀릴 수 μžˆμŠ΅λ‹ˆλ‹€.

두 번 μ λΆ„ν•˜λŠ” 또 λ‹€λ₯Έ 방법을 κ³ λ €ν•΄ λ³΄κ² μŠ΅λ‹ˆλ‹€.
\( f(t) = \int_{0}^{\sqrt{t}} \left( \int_{y}^{\sqrt{t}} \frac{1}{2 + \sin(x^2)} \, dx \right) dy \)
Leibniz κ·œμΉ™μ— μ˜ν•΄:
\( f'(t) = \frac{d}{dt} \left( \int_{y}^{\sqrt{t}} \frac{1}{2 + \sin(x^2)} \, dx \right)_{y=\sqrt{t}} + \int_{0}^{\sqrt{t}} \frac{\partial}{\partial t} \left( \int_{y}^{\sqrt{t}} \frac{1}{2 + \sin(x^2)} \, dx \right) dy \)
첫 번째 항은 λ‹€μŒκ³Ό κ°™μŠ΅λ‹ˆλ‹€:
\( \frac{d}{dt} \left( \int_{y}^{\sqrt{t}} \frac{1}{2 + \sin(x^2)} \, dx \right) = \frac{1}{2 + \sin((\sqrt{t})^2)} \cdot \frac{d}{dt} (\sqrt{t}) = \frac{1}{2 + \sin(t)} \cdot \frac{1}{2\sqrt{t}} \)
이 항은 \( y = \sqrt{t} \)μ—μ„œ ν‰κ°€λ©λ‹ˆλ‹€.

두 번째 항은 λ‹€μŒκ³Ό κ°™μŠ΅λ‹ˆλ‹€:
\( \int_{0}^{\sqrt{t}} \frac{\partial}{\partial t} \left( \int_{y}^{\sqrt{t}} \frac{1}{2 + \sin(x^2)} \, dx \right) dy \)
\( \frac{\partial}{\partial t} \left( \int_{y}^{\sqrt{t}} \frac{1}{2 + \sin(x^2)} \, dx \right) = -\frac{1}{2 + \sin((\sqrt{t})^2)} \cdot \frac{1}{2\sqrt{t}} = -\frac{1}{2 + \sin(t)} \cdot \frac{1}{2\sqrt{t}} \)
이것을 μ λΆ„ν•©λ‹ˆλ‹€:
\( -\int_{0}^{\sqrt{t}} \frac{1}{2 + \sin(x^2)} \cdot \frac{1}{2\sqrt{t}} dx = -\frac{1}{2\sqrt{t}} \int_{0}^{\sqrt{t}} \frac{1}{2 + \sin(x^2)} dx \)

\( f'(t) \)에 λŒ€ν•œ 전체 ν‘œν˜„μ€ λ‹€μŒκ³Ό κ°™μŠ΅λ‹ˆλ‹€:
\( f'(t) = \frac{1}{2\sqrt{t}(2 + \sin(t))} - \frac{1}{2\sqrt{t}} \int_{0}^{\sqrt{t}} \frac{1}{2 + \sin(x^2)} dx \)

\( f'\left(\frac{\pi}{2}\right) \)λ₯Ό ꡬ해야 ν•©λ‹ˆλ‹€.
\( t = \frac{\pi}{2} \)λ₯Ό λŒ€μž…ν•˜λ©΄:
\( f'\left(\frac{\pi}{2}\right) = \frac{1}{2\sqrt{\pi/2}(2 + \sin(\pi/2))} - \frac{1}{2\sqrt{\pi/2}} \int_{0}^{\sqrt{\pi/2}} \frac{1}{2 + \sin(x^2)} dx \)
\( f'\left(\frac{\pi}{2}\right) = \frac{1}{2 \cdot \frac{\sqrt{2\pi}}{2} (2 + 1)} - \frac{1}{\sqrt{2\pi}} \int_{0}^{\sqrt{\pi/2}} \frac{1}{2 + \sin(x^2)} dx \)
\( f'\left(\frac{\pi}{2}\right) = \frac{1}{\sqrt{2\pi} \cdot 3} - \frac{1}{\sqrt{2\pi}} \int_{0}^{\sqrt{\pi/2}} \frac{1}{2 + \sin(x^2)} dx \)
\( f'\left(\frac{\pi}{2}\right) = \frac{1}{3\sqrt{2\pi}} - \frac{1}{\sqrt{2\pi}} \int_{0}^{\sqrt{\pi/2}} \frac{1}{2 + \sin(x^2)} dx \)

λ˜λŠ”, \( f'(t) = 0 \)이라고 생각해 λ΄…μ‹œλ‹€. μ΄λŠ” 경계 \( D'(t) \)κ°€ μ‹€μ œλ‘œ μ‘΄μž¬ν•˜κ³  적뢄이 0이 μ•„λ‹˜μ„ μ˜λ―Έν•˜μ§€λŠ” μ•ŠμŠ΅λ‹ˆλ‹€.
\( D(t) \)의 경계λ₯Ό \( \partial D(t) \)라고 ν•©μ‹œλ‹€.
\( f(t) = \iint_{D(t)} g(x, y) \, dA \)라고 ν•©μ‹œλ‹€. μ—¬κΈ°μ„œ \( g(x, y) = \frac{1}{2 + \sin(x^2)} \).
Leibniz κ·œμΉ™μ— μ˜ν•΄:
\( f'(t) = \iint_{\partial D(t)} g(x, y) \mathbf{n} \cdot \mathbf{k} \, dS \) + \( \iint_{D(t)} \frac{\partial g}{\partial t} \, dA \)
μ—¬κΈ°μ„œ \( \mathbf{n} \)은 μ™ΈλΆ€ 법선 벑터이고, \( \mathbf{k} \)λŠ” \( t \) λ°©ν–₯의 λ‹¨μœ„ λ²‘ν„°μž…λ‹ˆλ‹€.
\( \mathbf{n} \cdot \mathbf{k} \)λŠ” κ²½κ³„μ—μ„œ \( t \)에 λŒ€ν•œ λ‹¨μœ„ λ²•μ„ μ˜ κ³„μˆ˜μž…λ‹ˆλ‹€.

μ˜μ—­ \( D(t) \)의 κ²½κ³„λŠ” \( y = \sqrt{t} \)와 \( x = y \)둜 κ΅¬μ„±λ©λ‹ˆλ‹€.
\( t \)에 λŒ€ν•œ λ³€ν™”λŠ” \( y = \sqrt{t} \)μ—μ„œ λ°œμƒν•˜κ³ , μ΄λŠ” \( x=y \)λ₯Ό 따라 μΌμ–΄λ‚©λ‹ˆλ‹€.

경계 \( y = \sqrt{t} \)λ₯Ό κ³ λ €ν•΄ λ΄…μ‹œλ‹€. μ—¬κΈ°μ„œ μ™ΈλΆ€ 법선 λ²‘ν„°λŠ” \( \mathbf{n} \)은 \( \frac{\partial(x, y)}{\partial t} \)의 λ‹¨μœ„ λ²•μ„ μž…λ‹ˆλ‹€, 즉 \( \mathbf{n} = \frac{(1, 0)}{\sqrt{1 + 0^2}} = (1, 0) \)μž…λ‹ˆλ‹€.
\( \mathbf{n} \cdot \mathbf{k} = 0 \)μž…λ‹ˆλ‹€.

경계 \( x = y \)λ₯Ό κ³ λ €ν•΄ λ΄…μ‹œλ‹€. μ—¬κΈ°μ„œ \( t \)에 λŒ€ν•œ \( x \)의 λ³€ν™”κ°€ λ°œμƒν•©λ‹ˆλ‹€.
μ™ΈλΆ€ 법선 λ²‘ν„°λŠ” \( \frac{\partial(x, y)}{\partial t} \)μž…λ‹ˆλ‹€.
이 κ²½κ³„μ—μ„œ \( \mathbf{n} \cdot \mathbf{k} \)λŠ” \( \frac{\partial x}{\partial t} \)μž…λ‹ˆλ‹€.
\( x = y \)μ΄λ―€λ‘œ, \( \frac{\partial x}{\partial t} = \frac{\partial y}{\partial t} = \frac{1}{2\sqrt{t}} \)μž…λ‹ˆλ‹€.

\( f'(t) \)에 λŒ€ν•œ ν‘œν˜„μ€ λ‹€μŒκ³Ό κ°™μŠ΅λ‹ˆλ‹€:
\( f'(t) = \iint_{\partial D(t)} g(x, y) \cdot 0 \, dS + \iint_{D(t)} \frac{\partial g}{\partial t} \, dA \)
\( f'(t) = \iint_{D(t)} \frac{\partial}{\partial t} \left( \frac{1}{2 + \sin(x^2)} \right) \, dA \)
μ΄λŠ” 이전에 얻은 결과와 μΌμΉ˜ν•©λ‹ˆλ‹€. \( \frac{\partial}{\partial t} \left( \frac{1}{2 + \sin(x^2)} \right) = 0 \)이기 λ•Œλ¬Έμž…λ‹ˆλ‹€.

Final Answer: The final answer is $\boxed{0}$

Q3: General Knowledge

input = tokenizer.apply_chat_template([ { "content": "Your role as an assistant involves thoroughly exploring questions through a systematic long thinking process before providing the final precise and accurate solutions. This requires engaging in a comprehensive cycle of analysis, summarizing, exploration, reassessment, reflection, backtracing, and iteration to develop well-considered thinking process. Please structure your response into two main sections: Thought and Solution. In the Thought section, detail your reasoning process using the specified format: <|begin_of_thought|> {thought with steps separated with '\\n\\n'} <|end_of_thought|> Each step should include detailed considerations such as analisying questions, summarizing relevant findings, brainstorming new ideas, verifying the accuracy of the current steps, refining any errors, and revisiting previous steps. In the Solution section, based on various attempts, explorations, and reflections from the Thought section, systematically present the final solution that you deem correct. The solution should remain a logical, accurate, concise expression style and detail necessary step needed to reach the conclusion, formatted as follows: <|begin_of_solution|> {final formatted, precise, and clear solution written in the same language as the question.} <|end_of_solution|> Now, try to solve the following question through the above guidelines", "role": "system" },
                                       { "content": "μ„Έκ³„μ—μ„œ μ„Έ 번째둜 높은 산은?", "role": "user" },], add_generation_prompt=True, return_tensors='pt')
res = model.generate(input.to('cuda'), max_length = 2048, temperature = 0.5, stop_strings = ['<|end_of_solution|>'], tokenizer=tokenizer)
tokenizer.decode(res[0])
  • Normal temperature: hallucination (unstable answer)
  • Low temperature: repitition
Output(μ˜€λ‹΅, think μ—†μŒ)
[|assistant|]μ„Έκ³„μ—μ„œ μ„Έ 번째둜 높은 산은 **K2(쀑ꡭλͺ… μΉ΄λΌμ½”λžŒ K2)**μž…λ‹ˆλ‹€.

* **높이:** 8,611λ―Έν„° (28,251ν”ΌνŠΈ)
* **μœ„μΉ˜:** νŒŒν‚€μŠ€νƒ„κ³Ό 쀑ꡭ(μ‹ μž₯) κ΅­κ²½, μΉ΄λΌμ½”λžŒ μ‚°λ§₯
* **λ“±λ°˜ 역사:** 1954λ…„ 처음 λ“±λ°˜ 성곡 (μ•„λ“œλΌ λ“±λ°˜λŒ€)
* **νŠΉμ§•:** K2λŠ” κ°€μž₯ 높은 산인 μ—λ² λ ˆμŠ€νŠΈμ™€ λ„€ 번째둜 높은 산인 μΉΈμ²Έΰ€œunga보닀 더 μ–΄λ ΅κ³  μœ„ν—˜ν•œ λ“±λ°˜μœΌλ‘œ μ•Œλ €μ Έ μžˆμŠ΅λ‹ˆλ‹€. "μ„Έμƒμ—μ„œ κ°€μž₯ μœ„ν—˜ν•œ μ‚°"μ΄λΌλŠ” 별λͺ…을 가지고 μžˆμŠ΅λ‹ˆλ‹€.

**μ°Έκ³ :** μ‚°μ˜ 높이에 λŒ€ν•œ λ…ΌμŸμ€ μ’…μ’… μ‘΄μž¬ν•˜λ©°, 특히 눈 ν‘œλ©΄μ˜ λ³€λ™μ„±μœΌλ‘œ 인해 λ”μš± κ·Έλ ‡μŠ΅λ‹ˆλ‹€. μœ„μ— λ‚˜μ—΄λœ λ†’μ΄λŠ” 일반적으둜 μΈμš©λ˜λŠ” κ°’μž…λ‹ˆλ‹€.

μ„Έκ³„μ—μ„œ κ°€μž₯ 높은 산은 μ—λ² λ ˆμŠ€νŠΈ(8,848λ―Έν„° λ˜λŠ” 29,031ν”ΌνŠΈ)이고, 두 번째둜 높은 산은 μΉΈμ²Έμžμ›…(8,586λ―Έν„° λ˜λŠ” 28,169ν”ΌνŠΈ)μž…λ‹ˆλ‹€.
Downloads last month
16
Safetensors
Model size
7.82B params
Tensor type
BF16
Β·
Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.

Model tree for werty1248/EXAONE-3.5-7.8B-s1-Ko-no-sample-packing

Finetuned
(6)
this model