Configure a code repository

Connect a Git repository to ZenML to track code changes and collaborate on MLOps projects.

Throughout the lifecycle of a MLOps pipeline, it can get quite tiresome to always wait for a Docker build every time after running a pipeline (even if the local Docker cache is used). However, there is a way to just have one pipeline build and keep reusing it until a change to the pipeline environment is made: by connecting a code repository.

With ZenML, connecting to a Git repository optimizes the Docker build processes. It also has the added bonus of being a better way of managing repository changes and enabling better code collaboration. Here is how the flow changes when running a pipeline:

You trigger a pipeline run on your local machine. ZenML parses the @pipeline function to determine the necessary steps.

The local client requests stack information from the ZenML server, which responds with the cloud stack configuration.

The local client detects that we're using a code repository and requests the information from the git repo.

Instead of building a new Docker image, the client checks if an existing image can be reused based on the current Git commit hash and other environment metadata.

The client initiates a run in the orchestrator, which sets up the execution environment in the cloud, such as a VM.

The orchestrator downloads the code directly from the Git repository and uses the existing Docker image to run the pipeline steps.

Pipeline steps execute, storing artifacts in the cloud-based artifact store.

Throughout the execution, the pipeline run status and metadata are reported back to the ZenML server.

By connecting a Git repository, you avoid redundant builds and make your MLOps processes more efficient. Your team can work on the codebase simultaneously, with ZenML handling the version tracking and ensuring that the correct code version is always used for each run.

Creating a GitHub Repository