title: RuntimeError: CUDA out of memory with FastAPI
body: <p>I try to make a backend server based on fastapi.</p>
  <p>My backend server works well with almost no errors, but I found an error situation.</p>
  <p>I allocate two services to fastapi apps using <code>api.route</code>.</p>
  <p>If you look at the code below, you can see that the <code>detected_images</code> and <code>filtered_images</code> services are assigned.</p>
  <p>This is my code:</p>
  <pre><code>import uvicorn
  from fastapi import FastAPI
  
  app = FastAPI()
  
  from api.routes import detected_images
  app.include_router(detected_images.router)
  
  from api.routes import filtered_images
  app.include_router(filtered_images.router)
  
  if __name__ == '__main__':
      uvicorn.run(app, port='8000', host=&quot;127.0.0.1&quot;)
  </code></pre>
  <p>But if I run this code as it is, I'll face an error.</p>
  <p>In my opinion, it is estimated that GPU memory is temporarily exceeded while running at the same time.</p>
  <p>This is my error message:</p>
  <p><code>RuntimeError: CUDA out of memory. Tried to allocate 2.00 MiB (GPU 0; 8.00 GiB total capacity; 6.23 GiB already allocated; 0 bytes free; 6.25 GiB reserved in total by PyTorch)</code></p>
  <p>However, if this is not executed in one python code, divided into two, and executed in order, no errors will occur.</p>
  <p>The code below is the way I tried to avoid errors.</p>
  <p>This is my process1:</p>
  <pre><code>import uvicorn
  from fastapi import FastAPI
  
  app = FastAPI()
  
  from api.routes import detected_images
  app.include_router(detected_images.router)
  
  if __name__ == '__main__':
      uvicorn.run(app, port='8000', host=&quot;127.0.0.1&quot;)
  </code></pre>
  <p>This is my process2:</p>
  <pre><code>import uvicorn
  from fastapi import FastAPI
  
  app = FastAPI()
  
  from api.routes import filtered_images
  app.include_router(filtered_images.router)
  
  if __name__ == '__main__':
      uvicorn.run(app, port='8001', host=&quot;127.0.0.1&quot;)
  </code></pre>
  <p>In addition, I tried the following method, but it didn't work.</p>
  <pre><code>import uvicorn
  from fastapi import FastAPI
  
  app = FastAPI()
  
  from api.routes import detected_images
  app.include_router(detected_images.router)
  
  import torch, gc
  gc.collect()
  torch.cuda.empty_cache()
  
  from api.routes import filtered_images
  app.include_router(filtered_images.router)
  
  if __name__ == '__main__':
      uvicorn.run(app, port='8000', host=&quot;127.0.0.1&quot;)
  </code></pre>
  <p>Is there any good way to solve this problem?</p>