Home Artificial Intelligence Speculative Sampling — Intuitively and Exhaustively Explained Language Models Are Getting Too Big

Speculative Sampling — Intuitively and Exhaustively Explained Language Models Are Getting Too Big

0
Speculative Sampling — Intuitively and Exhaustively Explained
Language Models Are Getting Too Big

Machine Learning | Natural Language Processing | Data Science

Exploring the drop-in strategy that’s speeding up language models by 3x

Towards Data Science
“Speculators” by Daniel Warfield using MidJourney and Affinity Design 2. All images by the creator unless otherwise specified.

In this text we’ll discuss “Speculative Sampling”, a technique that makes text generation faster and more cost-effective without compromising on performance.

Empirical results of using speculative sampling on a wide range of text generation tasks. Notice how, in all cases, generation time is significantly faster. Source

First we’ll discuss a serious problem that’s slowing down modern language models, then we’ll construct an intuitive understanding of how speculative sampling elegantly speeds them up, then we’ll implement speculative sampling from scratch in Python.

Who’s this convenient for? Anyone inquisitive about natural language processing (NLP), or leading edge AI advancements.

How advanced is that this post? The concepts in this text are accessible to machine learning enthusiasts, and are leading edge enough to interest seasoned data scientists. The code at the top could also be useful to developers.

Pre-requisites: It is perhaps useful to have a cursory understanding of Transformers, OpenAI’s GPT models, or each. In case you end up confused, you possibly can check with either of those articles:

During the last 4 years OpenAI’s GPT models have grown from 117 million parameters in 2018 to an estimated 1.8 Trillion parameters in 2023. This rapid growth can largely be attributed to the incontrovertible fact that, in language modeling, greater is healthier.

LEAVE A REPLY

Please enter your comment!
Please enter your name here