. It can be managed through Hugging Face settings.namespace parameter is used for listing and creating the inference endpoints. It can take any of the following values, username or organization name or * depending on where the inference endpoint should be created.

We can now use the model deployer in our stack.

zenml stack update <CUSTOM_STACK_NAME> --model-deployer=<MODEL_DEPLOYER_NAME>

See the huggingface_model_deployer_step for an example of using the Hugging Face Model Deployer to deploy a model inside a ZenML pipeline step.

Configuration

Within the HuggingFaceServiceConfig you can configure:

model_name: the name of the model in ZenML.

endpoint_name: the name of the inference endpoint. We add a prefix zenml- and first 8 characters of the service uuid as a suffix to the endpoint name.

repository: The repository name in the user’s namespace ({username}/{model_id}) or in the organization namespace ({organization}/{model_id}) from the Hugging Face hub.

framework: The machine learning framework used for the model (e.g. "custom", "pytorch" )

accelerator: The hardware accelerator to be used for inference. (e.g. "cpu", "gpu")

instance_size: The size of the instance to be used for hosting the model (e.g. "large", "xxlarge")

instance_type: Inference Endpoints offers a selection of curated CPU and GPU instances. (e.g. "c6i", "g5.12xlarge")

region: The cloud region in which the Inference Endpoint will be created (e.g. "us-east-1", "eu-west-1" for vendor = aws and "eastus" for Microsoft Azure vendor.).

vendor: The cloud provider or vendor where the Inference Endpoint will be hosted (e.g. "aws").

token: The Hugging Face authentication token. It can be managed through huggingface settings. The same token can be passed used while registering the Hugging Face model deployer.

account_id: (Optional) The account ID used to link a VPC to a private Inference Endpoint (if applicable).

min_replica: (Optional) The minimum number of replicas (instances) to keep running for the Inference Endpoint. Defaults to 0.