In a significant move, xAI Corp., an artificial intelligence startup spearheaded by Elon Musk, has made its Grok-1 large language model (LLM) available as open source. This release comes shortly after Apple Inc. unveiled its own research on multimodal LLMs. While Musk had announced the intention to release Grok as open source on March 11, it is today that the company has shared the core architecture and weights of the model.
The released code encompasses a crucial aspect of Grok’s structural design – the arrangement and interconnection of layers and nodes for data processing. Additionally, the base model weights, which are the adjusted parameters determining the transformation of input data into output, have also been shared.
Grok-1 is an extensive Mixture-of-Experts model developed by xAI from scratch. A Mixture-of-Experts model combines the outputs of specialized sub-models, known as experts, to generate a final prediction. Through the convergence of diverse tasks and data subsets, this approach leverages the expertise of each individual model to optimize performance.
What has been released is a raw base model checkpoint from the pre-training phase, which concluded in October 2023. It is important to note that the model has not been fine-tuned for any specific application, such as dialogue.
Elon Musk’s xAI aims to compete with AI offerings from industry giants like Google and OpenAI. Their inaugural model, Grok, draws inspiration from Douglas Adams’ renowned book “The Hitchhiker’s Guide to the Galaxy.” According to xAI, Grok is designed to provide answers to a wide range of questions and even suggest relevant inquiries.
Meanwhile, Apple has taken a step forward in its research on multimodal LLMs. The company recently published a paper detailing its work on MM1, a set of multimodal models capable of captioning images, answering visual questions, and understanding natural language. Apple’s researchers claim that multimodal LLMs represent the next frontier in foundation models and offer superior capabilities.
Multimodal LLMs, like MM1, possess the ability to comprehend and generate responses across various data types, including text, images, and audio. By integrating diverse forms of information, these models excel at complex tasks. Apple’s breakthrough with MM1 is expected to facilitate the scaling of such models to larger datasets, substantially improving their performance and reliability.
It is worth mentioning that Apple had previously made progress in multimodal LLMs with Ferret, which was quietly open-sourced in October and gained attention in December.
With the availability of Grok-1’s open source code, xAI Corp. has entered the race against established AI firms. The move not only contributes to the collective knowledge in the AI community but also invites collaboration and innovation. As the field of artificial intelligence continues to evolve, the release of such models will undoubtedly shape the future of AI research and applications.
FAQ
What is a large language model (LLM)?
A large language model (LLM) is an artificial intelligence system designed to understand and generate human-like text based on vast amounts of training data.
What is a Mixture-of-Experts model?
A Mixture-of-Experts model is a machine learning approach that combines the outputs of multiple specialized sub-models to make a final prediction. Each sub-model focuses on a particular aspect of the task at hand, contributing to an overall optimized performance.
What are multimodal LLMs?
Multimodal LLMs are AI systems capable of comprehending and generating responses across different data types, such as text, images, and audio. By integrating diverse forms of information, these models exhibit enhanced capabilities for complex tasks.
(Source: Domain of article source)
The release of xAI Corp.’s Grok-1 large language model (LLM) as open source marks a significant move in the artificial intelligence (AI) industry. Spearheaded by Elon Musk, xAI Corp. aims to compete with industry giants like Google and OpenAI. Grok-1 is an extensive Mixture-of-Experts model, which combines the outputs of specialized sub-models to generate predictions.
In comparison, Apple Inc. has also made strides in multimodal LLMs with its research on MM1. Multimodal LLMs, like MM1, have the ability to comprehend and generate responses across various data types, including text, images, and audio. Apple’s researchers claim that these models represent the next frontier in foundation models and offer superior capabilities. Their breakthrough is expected to facilitate the scaling of such models to larger datasets and enhance their performance and reliability.
With the release of Grok-1’s open source code, xAI Corp. has entered the race against established AI firms. This move not only adds to the collective knowledge in the AI community but also encourages collaboration and innovation. As the field of artificial intelligence continues to evolve, the availability of open source models like Grok-1 and the advancements made by companies like Apple will shape the future of AI research and applications.
For more information and related news, please visit the xAI Corp. website and the Apple Inc. website.
The source of the article is from the blog maestropasta.cz