Home Community Meet LLM Surgeon: A Latest Machine Learning Framework for Unstructured, Semi-Structured, and Structured Pruning of Large Language Models (LLMs)

Meet LLM Surgeon: A Latest Machine Learning Framework for Unstructured, Semi-Structured, and Structured Pruning of Large Language Models (LLMs)

0
Meet LLM Surgeon: A Latest Machine Learning Framework for Unstructured, Semi-Structured, and Structured Pruning of Large Language Models (LLMs)

The recent advancements in Artificial Intelligence have enabled the event of Large Language Models (LLMs) with a significantly large variety of parameters, with a few of them reaching into billions (for instance, LLaMA-2 that is available in sizes of 7B, 13B, and even 70B parameters). With such specifications, the model is capable of achieve very high performances across diverse tasks, making it a strong tool for various AI applications. The downside to this, nonetheless, is that the deployment of such models comes with an expensive cost, and devices like phones don’t possess enough memory to host them. 

Various pruning techniques have emerged up to now to beat this issue. Nonetheless, many result in a big performance degradation after pruning. Furthermore, these methods don’t readily extend to structured pruning. Subsequently, a team of researchers from Imperial College London, Qualcomm AI Research, QUVA Lab, and the University of Amsterdam have introduced a framework for unstructured, semi-structured, and structured LLM pruning that prunes the model in multiple steps, updating the weights and curvature estimates between each step. In accordance with the experiments conducted by the researchers, their framework allows for the pruning of LLMs by as much as 30% with none significant performance degradation, demonstrating its effectiveness.

The framework uses weight magnitude and activations from forward passes and gradient information from backward passes to relate weight removal costs to the true final objective. The researchers have improved the previous works in weight pruning through the use of more accurate approximations to the loss curvature and more weight correlations to update remaining weights.

The accuracy of pruning is determined by accurately estimating the local curvature and concurrently overcoming the memory cost that’s related to storing the precise curvature. 

LLM Surgeon uses the KFAC approximation for this task, a preferred method for curvature approximation, due to its memory efficiency. This method allows the framework to compute the dynamic allocation of structures that may be removed. Furthermore, it also allows the updation of the remaining weights, accounting for the removal.

The framework prunes multiple weights directly to succeed in the goal model size while inflicting the least possible cost. Moreover, LLM Surgeon prunes in multiple steps to enhance the performance-to-sparsity. The researchers justified their approach by showing that the pruning performance increased with more shots.

The researchers evaluated the performance of LLM Surgeon on language modeling tasks on models like OPT and LLaMA-2, using data from the wikitext-2 dataset. For structured compression, the framework allows the model size to be reduced by as much as 30% with none significant loss. Furthermore, it performs higher than all baselines, achieving the most effective performance for every goal size. For semi-structured and unstructured compression as well, LLM Surgeon outperforms all baselines, demonstrating the most effective performance across goal sizes.

In conclusion, LLM Surgeon addresses the issue posed by LLMs with a significantly large variety of parameters when it comes to deployment. The outcomes show that it will probably prune rows and columns from a spread of LLMs by 20-30% without significant loss in performance. It also achieves state-of-the-art leads to unstructured and semi-structured pruning of LLMs, enabling a better deployment process.


Take a look at the Paper. All credit for this research goes to the researchers of this project. Also, don’t forget to hitch our 35k+ ML SubReddit, 41k+ Facebook Community, Discord Channel, LinkedIn Group, and Email Newsletter, where we share the newest AI research news, cool AI projects, and more.

If you happen to like our work, you’ll love our newsletter..


Asif Razzaq is the CEO of Marktechpost Media Inc.. As a visionary entrepreneur and engineer, Asif is committed to harnessing the potential of Artificial Intelligence for social good. His most up-to-date endeavor is the launch of an Artificial Intelligence Media Platform, Marktechpost, which stands out for its in-depth coverage of machine learning and deep learning news that’s each technically sound and simply comprehensible by a large audience. The platform boasts of over 2 million monthly views, illustrating its popularity amongst audiences.


🚀 Boost your LinkedIn presence with Taplio: AI-driven content creation, easy scheduling, in-depth analytics, and networking with top creators – Try it free now!.

LEAVE A REPLY

Please enter your comment!
Please enter your name here