The appearance of open foundation models, equivalent to BERT, CLIP, and Stable Diffusion, has ushered in a brand new era in artificial intelligence, marked by rapid technological development and significant societal impact. These models are characterised by their widely available model weights, allowing for greater customization and broader access, which, in turn, offers a bunch of advantages and introduces recent risks. This evolution has sparked a critical debate on the open versus closed release of foundation models, with significant attention from policymakers globally.

Current state-of-the-art methods in AI development often involve closed foundation models, where model weights are usually not publicly available, limiting the flexibility of researchers and developers to customize or inspect these models. Open foundation models challenge this paradigm by offering an alternate that promotes innovation, competition, and transparency. These models enable local adaptation and inference, making them particularly precious in fields where data sensitivity is paramount. Nevertheless, their open nature also means once released, controlling access or use becomes nearly unimaginable, raising concerns about misuse and the issue of moderating or monitoring their application.

The advantages of open foundation models are significant, spanning from fostering innovation and accelerating scientific research to enhancing transparency and reducing market concentration. By allowing broader access and customization, these models distribute decision-making power regarding acceptable model behavior, enabling a diversity of applications that may be tailored to specific needs. Additionally they play an important role in scientific research by providing essential tools for exploration in AI interpretability, security, and safety. Nevertheless, these benefits include caveats, equivalent to potential comparative disadvantages in model improvement over time attributable to the shortage of user feedback and the fragmented use of heavily customized models.

Despite these advantages, open foundation models present risks, especially by way of societal harm through misuse in areas like cybersecurity, biosecurity, and the generation of non-consensual intimate imagery. To know the character of those risks, this study presents a framework that centers marginal risk: what additional risk is society subject to due to open foundation models relative to pre-existing technologies, closed models, or other relevant reference points? This framework considers the threat identification, existing risks, defenses, evidence of marginal risk, ease of defending against recent risks, and the underlying uncertainties and assumptions. It highlights the importance of a nuanced approach to evaluating the risks and advantages of open foundation models, underscoring the necessity for empirical research to validate theoretical advantages and risks.

In conclusion, open foundation models represent a pivotal shift within the AI landscape, offering substantial advantages while posing recent challenges. Their impact on innovation, transparency, and scientific research is undeniable, yet additionally they introduce significant risks that require careful consideration and governance. Because the AI community and policymakers navigate these waters, a balanced approach, informed by empirical evidence and a deep understanding of the distinctive properties of open foundation models, will likely be essential for harnessing their potential while mitigating their risks.

Try the Paper. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter. Join our Telegram Channel, Discord Channel, and LinkedIn Group.

In case you like our work, you’ll love our newsletter..

Don’t Forget to affix our 38k+ ML SubReddit