HuggingFaceTB/smollm-135M-instruct-v0.2-Q8_0-GGUF · Training GGUF model using Transformers library

Hi There,
I have been wanting to make use of smallest model with LoRA based fine-tuning,
I am trying the code below and throws me error due to not having config.json file, is there anything wrong with asuming that it would work with Transformers & pytorch ?
or my complete approach is wrong ?


from transformers import AutoTokenizer, AutoModelForCausalLM, AutoModelForSeq2SeqLM 

from trl import SFTConfig

import bitsandbytes as bnb

from accelerate import init_empty_weights, load_checkpoint_and_dispatch

from peft import get_peft_model, LoraConfig, TaskType

torch.cuda.empty_cache()

checkpoints = "HuggingFaceTB/smollm-135M-instruct-v0.2-Q8_0-GGUF"

peft_config = LoraConfig(task_type=TaskType.CAUSAL_LM, inference_mode=True, r=8, lora_alpha=8, 

                         lora_dropout=0.1)

model = AutoModelForCausalLM.from_pretrained(checkpoints)

model = get_peft_model(model, peft_config)



model.print_trainable_parameters()

----------------------ERROR --------------------
OSError: HuggingFaceTB/smollm-135M-instruct-v0.2-Q8_0-GGUF does not appear to have a file named config.json. Checkout 'https://huggingface.co/HuggingFaceTB/smollm-135M-instruct-v0.2-Q8_0-GGUF/tree/main' for available files.
it