The inherent capabilities of pretrained large language models are notable, yet achieving desired behaviors often requires additional adaptation. When coping with models whose weights are kept private, the challenge intensifies, rendering tuning either excessively costly or outright not possible. Because of this, striking the suitable balance between customization and resource efficiency stays a persistent concern in optimizing the performance of those advanced language models.

Despite the growing versatility of enormous pretrained language models, they predominantly profit from additional fine-tuning to reinforce specific behaviors. Tremendous-tuning has turn out to be more resource-intensive, posing challenges, especially when coping with private model weights, as GPT-4 from OpenAI in 2023. Consequently, efficiently customizing increasingly expansive language models for diverse user and application needs stays a distinguished challenge.

The researchers from the University of Washington and Allen Institute for AI present proxy-tuning, a decoding-time algorithm designed to fine-tune large black-box language models (LMs) without accessing their internal weights. This method leverages a smaller tuned LM and computes the difference between its predictions and the untuned version. Using decoding-time experts, the unique predictions of the larger base model are adjusted based on this difference, effectively achieving the advantages of direct tuning.

Proxy-tuning goals to bridge the disparity between a base language model and its directly tuned version without altering the bottom model’s parameters. This approach includes tuning a smaller LM and using the contrast between its predictions and the untuned version to regulate the unique predictions of the bottom model toward the tuning direction. Importantly, proxy-tuning preserves the benefits of extensive pretraining while effectively achieving the specified behaviors within the language model.

The bottom models need assistance with AlpacaFarm and GSM questions, achieving low win rates and accuracy. Proxy-tuning significantly improves performance, reaching 88.0% on AlpacaFarm and 32.0% on GSM for 70B-BASE. On Toxigen, proxy-tuning reduces toxicity to 0%. TruthfulQA’s open-ended setting sees proxy-tuning surpassing CHAT models in truthfulness. Across different scenarios, proxy-tuning closes 91.1% of the performance gap on the 13B scale and 88.1% on the 70B scale, demonstrating its effectiveness in enhancing model behavior without direct fine-tuning.

To summarise, The researchers from the University of Washington and Allen Institute for AI have proposed Proxy-tuning, which emerges as a promising approach for fine-tuning large language models at decoding time by modifying output logits. It’s an efficient alternative to traditional fine-tuning, making large language models more accessible, especially for those with limited resources. The tactic also addresses the challenge of adapting proprietary models to diverse use cases. The conclusion invites model-producing organizations to share output probabilities for broader utilization. 

Try the Paper. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter. Join our 36k+ ML SubReddit, 41k+ Facebook Community, Discord Channel, and LinkedIn Group.

For those who like our work, you’ll love our newsletter..

Don’t Forget to affix our Telegram Channel