Open source artificial intelligence (AI) models have been gaining traction in the tech industry over the past year, fueled by events such as a popular gathering in San Francisco dubbed the “Woodstock of AI.” At this event, hosted by open source AI hub Hugging Face, thousands of participants congregated to explore the potential of publicly available AI models and their underlying codes. The impact of this event has been long-lasting, resulting in the emergence of new unicorn startups like Mistral and Together AI, as well as a continuous stream of open source AI models that are progressively challenging the performance benchmarks set by OpenAI’s flagship GPT-4.
However, despite the growing popularity of open source AI models, a recent survey conducted by venture capital firm a16z indicates that large enterprises still heavily rely on closed, proprietary models offered by OpenAI. This preference is particularly evident when it comes to models that are actually put into production. Nevertheless, the survey reveals signs of change as more organizations are now willing to experiment with different AI model options, including those that are open source.
Frequently Asked Questions:
Q: What are open source AI models?
Open source AI models refer to AI models whose underlying code and sometimes model weights and training methods are publicly available for researchers and developers to access and build upon. They promote collaboration and transparency within the AI community.
Q: What are closed, proprietary models?
Closed, proprietary models are AI models that are developed and owned by specific organizations, with restricted access to their underlying code and model details. These models are typically commercialized and require licensing agreements to use.
Q: How do open source AI models differ from closed, proprietary models?
Open source AI models offer greater accessibility and transparency as their code is openly available for exploration and enhancement. Closed, proprietary models, on the other hand, provide more control and security for proprietary data but limit collaboration and customization.
According to Sarah Wang, a general partner at a16z and co-author of the survey, one of OpenAI’s advantages so far has been its early entrance into the market. Additionally, OpenAI’s GPT-4 model has been widely regarded as one of the best available, easily accessible through an API or Microsoft Azure. As a result, many organizations initially relied on OpenAI’s models for experimentation, particularly in areas like marketing, coding, and customer support.
The survey results estimate that closed source models, with OpenAI leading the way, accounted for 80%-90% of the market share in 2023. However, 46% of respondents in the survey expressed a preference for open source models. Wang further highlighted that more enterprises are now exploring multiple model families, with open source options like Llama and Mistral gaining traction.
OpenAI, recognizing the growing competition, is intensifying its efforts to solidify its position among corporate customers. The company recently announced new features for its “self-serve fine-tuning API,” allowing for customization, and shared success stories of companies, including SK Telecom, that have tailored and fine-tuned OpenAI models. OpenAI’s Chief Operating Officer, Brad Lightcap, also touted the significant growth in ChatGPT Enterprise, which now boasts over 600,000 registered users, emphasizing that 2024 is positioned to be a year of widespread AI adoption in the enterprise.
Despite OpenAI’s dominant market position, Kjell Carlsson, head of AI strategy at Domino Data Lab, argues that the generative AI model market is starting to shift based on enterprise use cases. While OpenAI benefits from its early advantage and collaboration with Microsoft’s sales teams, many companies turn to alternative vendors and open source models when seeking differentiation and data protection for specialized AI applications. Carlsson emphasizes that most companies currently use OpenAI’s models due to their market position, rather than inherent technological advantages.
Alongside cost considerations, respondents in the survey identified control and customization as key reasons for adopting AI models. Open source models allow enterprises to self-host and fine-tune models based on their own data, offering greater control, security, and the ability to align models with specific use cases. Ali Ghodsi, CEO of Databricks, echoes this sentiment, emphasizing the significance of open source AI in 2024 as a trend that is often overlooked. Enterprises aspire to customize AI models to their unique data and tasks as a means of gaining a competitive edge and retaining intellectual property.
It is worth noting that current predictions about corporate adoption of generative AI do not consider the impact of OpenAI’s upcoming GPT-5 model, which is highly anticipated. However, Wang highlights that the cost of switching models is relatively low for enterprises, suggesting that organizations will likely continue experimenting with a mixture of open source and proprietary models based on their evolving needs.
The open source artificial intelligence (AI) model industry has seen significant growth in recent years. The emergence of platforms like Hugging Face, which hosted the “Woodstock of AI” gathering in San Francisco, has fueled the popularity of publicly available AI models and their underlying codes. This has led to the rise of new startups such as Mistral and Together AI, as well as a continuous stream of open source models that challenge the performance benchmarks set by proprietary models like OpenAI’s GPT-4.
Despite the increasing popularity of open source AI models, a survey conducted by venture capital firm a16z reveals that large enterprises still heavily rely on closed, proprietary models offered by organizations like OpenAI. This preference is particularly evident when it comes to models that are put into production. However, the survey also indicates a shift in mindset, as more organizations are now willing to experiment with different AI model options, including those that are open source.
Closed, proprietary models are AI models that are developed and owned by specific organizations, with restricted access to their underlying code and model details. On the other hand, open source AI models offer greater accessibility and transparency as their code is openly available for exploration and enhancement. While closed, proprietary models provide more control and security for proprietary data, they limit collaboration and customization.
OpenAI’s early entrance into the market and the high performance of its GPT-4 model have contributed to its market dominance. According to the survey results, closed source models, with OpenAI leading the way, accounted for 80%-90% of the market share in 2023. However, 46% of respondents expressed a preference for open source models, indicating a growing interest in alternative options.
To solidify its position in the face of increasing competition, OpenAI has introduced new features for its self-serve fine-tuning API, allowing for customization. The company has also shared success stories of companies that have tailored and fine-tuned OpenAI models, emphasizing its appeal to corporate customers. OpenAI’s Chief Operating Officer, Brad Lightcap, points out the significant growth in ChatGPT Enterprise, which now has over 600,000 registered users, and predicts that 2024 will be a year of widespread AI adoption in the enterprise.
Despite OpenAI’s dominant market position, the generative AI model market is starting to shift based on enterprise use cases. Companies are looking for differentiation and data protection in specialized AI applications, leading them to explore alternative vendors and open source models. Control and customization are key factors driving the adoption of AI models, with open source models allowing enterprises to self-host and fine-tune models based on their own data. This offers greater control, security, and the ability to align models with specific use cases.
Although the predictions about corporate adoption of generative AI do not take into account the impact of OpenAI’s upcoming GPT-5 model, which is highly anticipated, the survey suggests that organizations are willing to experiment with a mixture of open source and proprietary models based on their evolving needs. The relatively low cost of switching models for enterprises further supports this notion.
Overall, the open source AI model industry is experiencing growth and competition, with both open source and closed, proprietary models playing key roles. As enterprises seek control, customization, and specialization in their AI applications, the landscape is likely to continue evolving.