Pretam Ray's picture

2 4 7

Pretam Ray

Pretam

·

raypretam

AI & ML interests

NLP

Recent Activity

published a model 15 days ago

Pretam/t5-small-finetuned-xsum

upvoted an article 7 months ago

Making LLMs even more accessible with bitsandbytes, 4-bit quantization and QLoRA

reacted to vladbogo's post with 👍 11 months ago

A recent paper titled "ShortGPT: Layers in Large Language Models are More Redundant Than You Expect" proposes a simple and effective approach to pruning Large Language Models (LLMs) by removing redundant layers. Key points: * Discovers significant redundancy across layers in LLMs, with some layers playing a negligible role for the final performance. * Defines a new metric called Block Influence (BI) to quantify the importance of each layer in an LLM. * Removes layers with low BI scores, achieving up to 25% reduction in parameters and computation while maintaining 92% of the LLM's performance. Congrats to the authors for their work! Paper: https://huggingface.co/papers/2403.03853

View all activity

Organizations

Pretam's activity

upvoted an article 7 months ago

Article

Making LLMs even more accessible with bitsandbytes, 4-bit quantization and QLoRA

May 24, 2023

• 114

upvoted a paper 11 months ago

Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model

Paper • 2402.07827 • Published Feb 12, 2024 • 47

upvoted a paper about 1 year ago

OFA: A Framework of Initializing Unseen Subword Embeddings for Efficient Large-scale Multilingual Continued Pretraining

Paper • 2311.08849 • Published Nov 15, 2023 • 5

upvoted a paper over 1 year ago

Zephyr: Direct Distillation of LM Alignment

Paper • 2310.16944 • Published Oct 25, 2023 • 123