<p>Can I run a computer vision pipeline with Kubeflow? Is it a good idea, would it run efficiently?</p>

<p>Let's say the steps of the pipeline would need to be image segmentation, some filtering and what not (gpu enabled opencv until now) and maybe a tensorflow serving for a CNN at the end.</p>

<p>Any useful resources?</p>

<p>Thanks,</p>

<p>The <a href="https://cloud.google.com/blog/products/ai-machine-learning/getting-started-kubeflow-pipelines" rel="nofollow noreferrer">kubeflow pipelines</a> would make a good fit with your specific use case.
The idea is that you containerize all the individual steps that you want to have decoupled, something like: 1/ preprocessing, 2/ training, 3/ serving. Each container is designed so it can take the relevant arguments that you want to modify over time to run different versions of the pipeline. </p>

<ul>
<li>For the preprocessing image I would suggest to start from a GPU image with opencv installed that drops the output on Google Cloud Storage. </li>
<li>For the training you could leverage the <code>google/cloud-sdk:latest</code> image that comes with gcloud command so you just copy over your code and run the ml engine command. </li>
<li>For serving, you could use ml engine to deploy the model and thus start building the image from again <code>google/cloud-sdk:latest</code>, alternatively you could use the TF serving <a href="https://hub.docker.com/r/tensorflow/serving/tags/" rel="nofollow noreferrer">images</a> that are available off the shelf and you only need to specify the bucket where your saved model is stored and the model name <a href="https://www.tensorflow.org/serving/docker" rel="nofollow noreferrer">see instructions</a>. </li>
</ul>

<p>This <a href="https://towardsdatascience.com/how-to-create-and-deploy-a-kubeflow-machine-learning-pipeline-part-1-efea7a4b650f" rel="nofollow noreferrer">blog posts</a> describes how to build a similar pipeline.</p>
