RCNA MINI

RCNA MINI is a compact LoRA (Low-Rank Adaptation) model designed for generating high-quality, 4-step text-to-video outputs. It can create video clips ranging from 4 to 16 seconds long, making it ideal for generating short animations with rich details and smooth transitions.

Key Features:

  • 4-step Text-to-Video: Generates videos from a text prompt in just 4 steps.
  • Video Length: Can generate videos from 4 seconds to 16 seconds long.
  • High Quality: Supports high-resolution and detailed outputs (up to 8K).
  • Fast Sampling: Leveraging decoupled consistency learning, the model is optimized for speed while maintaining quality.

Example Outputs:

  • Prompt: "Astronaut in a jungle, cold color palette, muted colors, detailed, 8K"
    • Generates a high-quality video with rich details and smooth motion.

How it Works:

RCNA MINI is based on the LoRA architecture, which fine-tunes diffusion models using low-rank adaptations. This results in faster generation and less computational overhead compared to full model retraining.

Applications:

  • Short-form animations for social media content
  • Video generation for creative projects
  • Artistic video generation based on textual descriptions

Model Details:

  • Architecture: LoRA applied to diffusion models
  • Inference Steps: 4-step generation
  • Output Length: 4 to 16 seconds

Using AnimateLCM with Diffusers

import torch
from diffusers import AnimateDiffPipeline, LCMScheduler, MotionAdapter, DiffusionPipeline
from diffusers.utils import export_to_gif

# Load AnimateLCM for video generation
adapter = MotionAdapter.from_pretrained("Binarybardakshat/RCNA_MINI")
pipe = AnimateDiffPipeline.from_pretrained("emilianJR/epiCRealism", motion_adapter=adapter, torch_dtype=torch.float16)
pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config, beta_schedule="linear")
pipe.load_lora_weights("Binarybardakshat/RCNA_MINI", weight_name="RCNA_LORA_MINI_1.safetensors", adapter_name="lcm-lora")
pipe.set_adapters(["lcm-lora"], [0.8])
pipe.enable_vae_slicing()
pipe.enable_model_cpu_offload()

# Generate video using RCNA MINI
output = pipe(
    prompt="A space rocket with trails of smoke behind it launching into space from the desert, 4k, high resolution",
    negative_prompt="bad quality, worse quality, low resolution",
    num_frames=16,
    guidance_scale=2.0,
    num_inference_steps=6,
    generator=torch.Generator("cpu").manual_seed(0),
)
frames = output.frames[0]
export_to_gif(frames, "animatelcm.gif")
print("Video and image generation complete!")

License:

This model is licensed under the MIT License.

Downloads last month
54
Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API: The HF Inference API does not support text-to-video models for diffusers library.