text: EndpointSearxNG Search APISerpAPIShale ProtocolSingleStoreDBscikit-learnSlackspaCySpreedlyStarRocksStochasticAIStripeTairTelegramTigris2MarkdownTrelloTruLensTwitterTypesenseUnstructuredVectaraVespaWeights & BiasesWeatherWeaviateWhatsAppWhyLabsWikipediaWolfram AlphaWriterYeager.aiYouTubeZepZillizIntegrationsGrouped by providerRay ServeOn this pageRay ServeRay Serve is a scalable model serving library for building online inference APIs. Serve is particularly well suited for system composition, enabling you to build a complex inference service consisting of multiple chains and business logic all in Python code. Goal of this notebookâ€‹This notebook shows a simple example of how to deploy an OpenAI chain into production. You can extend it to deploy your own self-hosted models where you can easily define amount of hardware resources (GPUs and CPUs) needed to run your model in production efficiently. Read more about available options including autoscaling in the Ray Serve documentation.Setup Ray Serveâ€‹Install ray with pip install ray[serve]. General Skeletonâ€‹The general skeleton for deploying a service is the following:# 0: Import ray serve and request from starlettefrom ray import servefrom starlette.requests import Request# 1: Define a Ray Serve deployment.@serve.deploymentclass LLMServe:    def __init__(self) -> None:        # All the initialization code goes here        pass    async def __call__(self, request: Request) -> str:        # You can parse the request here        # and return a response        return "Hello World"# 2: Bind the model to deploymentdeployment = LLMServe.bind()# 3: Run