How to configure and run inference via python code?
Noob here. Trying to run inference using the example on the LTX github but on an 8GB card. I followed the readme (https://github.com/Lightricks/LTX-Video?tab=readme-ov-file#run-locally) but got an OutOfMemoryError. So now I'm trying this repo model so I downloaded the ltx-video-2b-v0.9.1-q8_0.gguf file and am trying the following:
python inference.py --ckpt_path 'mymodels/ltx-video-2b-v0.9.1-q8_0.gguf' --prompt "A woman with blonde hair styled up, wearing a black dress with sequins and pearl earrings, looks down with a sad expression on her face. The camera remains stationary, focused on the woman's face." --height 128 --width 128 --num_frames 9 --seed 1
However this gives the following error:
vae = CausalVideoAutoencoder.from_pretrained(ckpt_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
video_vae = cls.from_config(config)
^^^^^^
UnboundLocalError: cannot access local variable 'config' where it is not associated with a value
How do I configure and call the inference.py to use the gguf?
tensor in gguf is different from torch i.e.,ckpt/safetensors; you should use gguf tensor reader to read the tensor instead of torch; you could simply use the node in comfyui to perform that task; or llama.cpp engined cli pack to do it; and your 8gb card is fine since the ltxv model is pretty light/small
Couldn't reply as was rate-limited by being a new user. Thanks I'm new so I don't understand but I'll try to process your answer
i would suggest you just run it from comfyui; since everything inside this pack