Pretam Ray's picture

2 4 7

Pretam Ray

Pretam

·

raypretam

AI & ML interests

NLP

Recent Activity

published a model 13 days ago

Pretam/t5-small-finetuned-xsum

upvoted an article 7 months ago

Making LLMs even more accessible with bitsandbytes, 4-bit quantization and QLoRA

reacted to vladbogo's post with 👍 11 months ago

A recent paper titled "ShortGPT: Layers in Large Language Models are More Redundant Than You Expect" proposes a simple and effective approach to pruning Large Language Models (LLMs) by removing redundant layers. Key points: * Discovers significant redundancy across layers in LLMs, with some layers playing a negligible role for the final performance. * Defines a new metric called Block Influence (BI) to quantify the importance of each layer in an LLM. * Removes layers with low BI scores, achieving up to 25% reduction in parameters and computation while maintaining 92% of the LLM's performance. Congrats to the authors for their work! Paper: https://huggingface.co/papers/2403.03853

View all activity

Organizations

Pretam's activity

New activity in Pretam/ramayana about 1 year ago

[bot] Conversion to Parquet

#1 opened about 1 year ago by

parquet-converter

New activity in MultiCoNER/multiconer_v2 almost 2 years ago

Dataset Generation Error when loading this dataset using load_dataset('MultiCoNER/multiconer_v2' ,'English (EN)')

#1 opened almost 2 years ago by