Hanshi

preminstrel's activity

• All
• Models
• Datasets
• Spaces
• Papers
• Collections
• Community
• Posts
• Upvotes
• Likes
• Articles

upvoted 2 papers 4 months ago

ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inference

Paper • 2410.21465 • Published Oct 28, 2024 • 11

Fast Best-of-N Decoding via Speculative Rejection

Paper • 2410.20290 • Published Oct 26, 2024 • 10

upvoted a paper 10 months ago

TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding

Paper • 2404.11912 • Published Apr 18, 2024 • 17