Understanding stacks

Learning how to switch the infrastructure backend of your code.

Now that we have ZenML deployed, we can take the next steps in making sure that our machine learning workflows are production-ready. As you were running your first pipelines, you might have already noticed the term stack in the logs and on the dashboard.

A stack is the configuration of tools and infrastructure that your pipelines can run on. When you run ZenML code without configuring a stack, the pipeline will run on the so-called default stack.

Separation of code from configuration and infrastructure

As visualized in the diagram above, there are two separate domains that are connected through ZenML. The left side shows the code domain. The user's Python code is translated into a ZenML pipeline. On the right side, you can see the infrastructure domain, in this case, an instance of the default stack. By separating these two domains, it is easy to switch the environment that the pipeline runs on without making any changes in the code. It also allows domain experts to write code/configure infrastructure without worrying about the other domain.

The default stack

zenml stack describe lets you find out details about your active stack:

...

Stack Configuration

┏━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┓

┃ COMPONENT_TYPE │ COMPONENT_NAME ┃

┠────────────────┼────────────────┨

┃ ARTIFACT_STORE │ default        ┃

┠────────────────┼────────────────┨

┃ ORCHESTRATOR   │ default        ┃

┗━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┛

'default' stack (ACTIVE)

Stack 'default' with id '...' is owned by user default and is 'private'.

...

zenml stack list lets you see all stacks that are registered in your zenml deployment.

...

┏━━━━━━━━┯━━━━━━━━━━━━┯━━━━━━━━━━━┯━━━━━━━━┯━━━━━━━━━┯━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━┓

┃ ACTIVE │ STACK NAME │ STACK ID  │ SHARED │ OWNER   │ ARTIFACT_STORE │ ORCHESTRATOR ┃

┠────────┼────────────┼───────────┼────────┼─────────┼────────────────┼──────────────┨