<p>I want to read file paths from a persistent volume and store these file paths into a persistent queue of sorts. This would probably be done with an application contained within a pod. This persistent volume will be updated constantly with new files. This means that I will need to constantly update the queue with new file paths. What if this application that is adding items to the queue crashes? Kubernetes would be able to reboot the application, but I do not want to add in file paths that are already in the queue. The app would need to know what exists in the queue before adding in files, at least I would think. I was leaning on RabbitMQ, but apparently you cannot search a queue for specific items with this tool. What can I do to account for this issue? I am running this cluster on Google Kubernetes Engine, so this would be on the Google Cloud Platform.</p>

<p>Have you ever heard about <a href="https://kubemq.io/" rel="nofollow noreferrer">KubeMQ</a>? There is <a href="https://github.com/kubemq-io/kubemq-community" rel="nofollow noreferrer">a KubeMQ community</a> where you can refer to with the guides and help.</p>
<p>As an alternative solution you can find useful <a href="https://kubernetes.io/docs/tasks/job/fine-parallel-processing-work-queue/" rel="nofollow noreferrer">guide on official Kubernetes documentation</a> on creating working queue with Redis</p>
