casperhansen (casperhansen) – Community Activity

New activity in QuixiAI/DeepSeek-R1-AWQ 12 months ago

The inference performance of the DeepSeek-R1-AWQ model is weak compared to the DeepSeek-R1 model

👍 3

8

#3 opened 12 months ago by

qingqingz916

New activity in casperhansen/deepseek-r1-distill-qwen-32b-awq 12 months ago

Generation configs: Unquantised vs AWQ, model weights format

3

#1 opened about 1 year ago by

nfunctor

New activity in casperhansen/gsm8k_synthetic_cot about 1 year ago

Librarian Bot: Add language metadata for dataset

#2 opened about 1 year ago by

librarian-bot

[bot] Conversion to Parquet

#1 opened about 1 year ago by

parquet-converter

New activity in casperhansen/mistral-nemo-instruct-2407-awq over 1 year ago

Create README.md

#3 opened over 1 year ago by

cfahlgren1

New activity in casperhansen/deepseek-coder-v2-instruct-awq over 1 year ago

Original model perplexity

1

#1 opened over 1 year ago by

jacobpfau

New activity in casperhansen/llama-3-8b-instruct-awq over 1 year ago

update eos token

#6 opened over 1 year ago by

Praful932

New activity in casperhansen/llama-3-70b-instruct-awq almost 2 years ago

encountered error when loading model

7

#4 opened almost 2 years ago by

zhouzr

How did you create AWQ-quantized weights?

4

#5 opened almost 2 years ago by

nightdude

Infinity conversation, sould eos be <|eot_id|> ?

👍 5

4

#1 opened almost 2 years ago by

Komposter43

Update generation_config.json

#3 opened almost 2 years ago by

alugowski

New activity in casperhansen/llama-3-8b-instruct-awq almost 2 years ago

Create quant_config.json

1

#1 opened almost 2 years ago by

Suparious

New activity in casperhansen/llama-3-70b-instruct-awq almost 2 years ago

Update generation_config.json

#2 opened almost 2 years ago by

hmellor

New activity in TheBloke/Mixtral-8x7B-Instruct-v0.1-AWQ almost 2 years ago

always getting 0 in output

👍 4

15

#3 opened about 2 years ago by

xubuild

Not supporting with TGI

1

#4 opened about 2 years ago by

abhishek3jangid

OC is not a multiple of cta_N = 64

2

#5 opened about 2 years ago by

lazyDataScientist

TGI - response is an empty string

2

#6 opened almost 2 years ago by

p-christ

RuntimeError: probability tensor contains either `inf`, `nan` or element < 0

2

#7 opened almost 2 years ago by

aaganaie

New activity in TheBloke/zephyr-7B-beta-AWQ about 2 years ago

Is there a way to finetune these AWQ beasts?

4

#1 opened about 2 years ago by

yasmolin

New activity in TheBloke/Mistral-7B-Instruct-v0.1-AWQ over 2 years ago

KeyError: 'mistral'

14

#1 opened over 2 years ago by

tripathiarpan20

casperhansen

AI & ML interests

Organizations

The inference performance of the DeepSeek-R1-AWQ model is weak compared to the DeepSeek-R1 model

Generation configs: Unquantised vs AWQ, model weights format

Librarian Bot: Add language metadata for dataset

[bot] Conversion to Parquet

Create README.md

Original model perplexity

update eos token

encountered error when loading model

How did you create AWQ-quantized weights?

Infinity conversation, sould eos be <|eot_id|> ?

Update generation_config.json

Create quant_config.json

Update generation_config.json

always getting 0 in output

Not supporting with TGI

OC is not a multiple of cta_N = 64

TGI - response is an empty string

RuntimeError: probability tensor contains either `inf`, `nan` or element < 0

Is there a way to finetune these AWQ beasts?

KeyError: 'mistral'

casperhansen

AI & ML interests

Organizations

casperhansen's activity

The inference performance of the DeepSeek-R1-AWQ model is weak compared to the DeepSeek-R1 model

Generation configs: Unquantised vs AWQ, model weights format

Librarian Bot: Add language metadata for dataset

[bot] Conversion to Parquet

Create README.md

Original model perplexity

update eos token

encountered error when loading model

How did you create AWQ-quantized weights?

Infinity conversation, sould eos be <|eot_id|> ?

Update generation_config.json

Create quant_config.json

Update generation_config.json

always getting 0 in output

Not supporting with TGI

OC is not a multiple of cta_N = 64

TGI - response is an empty string

RuntimeError: probability tensor contains either `inf`, `nan` or element < 0

Is there a way to finetune these AWQ beasts?

KeyError: 'mistral'