TypeError: BaseGPTQForCausalLM.generate() takes 1 positional argument but 2 were given
When I try to test the RTLCoder-gptq-4bit
Python tell me that input_ids regarded as two arguments
but isn't following code the solution to this problem?
input_ids = tokenizer(prompt, return_tensors="pt").input_ids.to(0)
Or am I mistaken?
how can i solve this problem?
Hi, thanks for your issue! The following code should work,
'''
from transformers import AutoTokenizer
from auto_gptq import AutoGPTQForCausalLM, BaseQuantizeConfig
prompt = "Please act as a professional verilog designer and provide a half adder. \nmodule half_adder\n(input a, \ninput b, \noutput sum, \noutput carry);\n"
tokenizer = AutoTokenizer.from_pretrained("ishorn5/RTLCoder-v1.1-gptq-4bit")
model = AutoGPTQForCausalLM.from_quantized("ishorn5/RTLCoder-v1.1-gptq-4bit", device=0)
model.eval()
inputs = tokenizer(prompt, return_tensors="pt").to(0)
sample = model.generate(**inputs, max_length=512, temperature=0.5, top_p=0.9, pad_token_id=model.config.eos_token_id,)
print(tokenizer.decode(sample[0], skip_special_tokens=True))
'''
Appreciate the clarification