Aniket Maurya

aniketmaurya

AI & ML interests

Computer vision, NLP, MLOps

Recent Activity

upvoted an article 6 months ago

Tensor Parallelism

updated a model 6 months ago

aniketmaurya/receipt-model-2025

liked a model 6 months ago

aniketmaurya/receipt-model-2025

View all activity

Organizations

upvoted an article 6 months ago

Article

Tensor Parallelism

•

Aug 20, 2024

• 20

updated a model 6 months ago

aniketmaurya/receipt-model-2025

Visual Question Answering • Updated Jan 26 • 1

liked a model 6 months ago

aniketmaurya/receipt-model-2025

Visual Question Answering • Updated Jan 26 • 1

published a model 6 months ago

aniketmaurya/receipt-model-2025

Visual Question Answering • Updated Jan 26 • 1

replied to singhsidhukuldeep's post 6 months ago

woohoo thanks for checking LitServe @singhsidhukuldeep ! LitServe now has OpenAI API-compatible endpoint and you can also serve a LLM using vLLM engine with LitServe so you get both speed + flexibility.

reacted to singhsidhukuldeep's post with 🚀 6 months ago

Post

926

Just tried LitServe from the good folks at @LightningAI !

Between llama.cpp and vLLM, there is a small gap where a few large models are not deployable!

That's where LitServe comes in!

LitServe is a high-throughput serving engine for AI models built on FastAPI.

Yes, built on FastAPI. That's where the advantage and the issue lie.

It's extremely flexible and supports multi-modality and a variety of models out of the box.

But in my testing, it lags far behind in speed compared to vLLM.

Also, no OpenAI API-compatible endpoint is available as of now.

But as we move to multi-modal models and agents, this serves as a good starting point. However, it’s got to become faster...

GitHub: https://github.com/Lightning-AI/LitServe