Lighteval documentation

Pipeline

Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

Pipeline

Pipeline

class lighteval.pipeline.Pipeline

< >

( tasks: strpipeline_parameters: PipelineParametersevaluation_tracker: EvaluationTrackermodel_config = Nonemodel = Nonemetric_options = None )

PipelineParameters

class lighteval.pipeline.PipelineParameters

< >

( launcher_type: ParallelismManagerenv_config: EnvConfig = <factory>job_id: int = 0dataset_loading_processes: int = 1nanotron_checkpoint_path: str | None = Nonecustom_tasks_directory: str | None = Noneoverride_batch_size: int | None = Nonenum_fewshot_seeds: int = 1max_samples: int | None = Noneuse_chat_template: bool = Falsesystem_prompt: str | None = Noneload_responses_from_details_date_id: str | None = None )

ParallelismManager

class lighteval.pipeline.ParallelismManager

< >

( valuenames = Nonemodule = Nonequalname = Nonetype = Nonestart = 1 )

An enumeration.

< > Update on GitHub