Meta’s Open Source Llama disrupts AI horse race.

Meta's Open Source Llama disrupts AI horse race.

Meta’s Open Source Llama 2: A Game Changer in the AI Landscape

In May, an anonymous memo leaked online, raising concerns about the future of Google. The memo highlighted the threat posed by OpenAI’s text-generation technology and argued that open source software was quietly gaining ground. One example cited was Llama, a large language model created by Meta, which initially had limited availability but quickly became popular after leaking on 4Chan. Variants like Alpaca and Vicuna were developed, offering similar capabilities to OpenAI’s ChatGPT but with the added advantage of customizability. According to the leaked memo, the impact on the community was significant, empowering programmers to experiment freely.

Recently, Meta released Llama 2, the second version of their model, which is open source and available for commercial use right from the start. Meta claims that Llama 2, created using 40 percent more data than its predecessor, can produce results on par with OpenAI’s ChatGPT. What sets Meta apart is that, unlike other generative AI models that cost millions to create, their system is accessible to developers, startups, and anyone interested in customizing the model. By providing a more affordable option, Meta’s Llama 2 has the potential to fuel innovation and accelerate the current AI boom.

Meta’s open source AI initiative has gained significant support from major partners in the field. AI startups like Hugging Face, Databricks, and OctoML have already made the model available to their customers. Even Microsoft, despite investing $10 billion in OpenAI, will offer Llama 2 downloads to developers for use in the cloud or on Windows. At a recent conference, CEO Satya Nadella expressed enthusiasm for developers being able to use Meta’s open source AI alongside OpenAI’s proprietary offerings. Amazon’s cloud division, AWS, also provides access to Llama 2, further expanding its reach.

Ahmad Al-Dahle, Meta’s vice president for generative AI, emphasizes the company’s commitment to open source, citing the example of PyTorch, a popular tool for machine learning developers. According to Al-Dahle, Meta recognized the demand for working on and improving models beyond just researchers. While work is already underway on Llama 3, Al-Dahle refrains from providing specific details about how it will differ from its predecessors.

However, while Llama 2 establishes Meta as a leader in open source AI, some aspects of the release deviate from the traditional open source ethos. The training data used for the model’s creation is only described as “publicly available online sources,” without further clarification. Additionally, Meta’s licensing terms require companies with over 700 million monthly active users to establish a separate license agreement, creating a barrier for other tech giants looking to build upon the system. The model also has an acceptable use policy prohibiting malicious code generation, violence promotion, criminal activity enablement, abuse, or harassment. Meta has not elaborated on the actions it would take if Llama 2 were used in violation of this policy.

Jon Turow, an investor at Madrona Ventures, perceives Meta’s shift from trying to restrict the distribution of the first Llama model to open-sourcing the second as a catalyst for a new wave of creativity using large language models. Turow draws a parallel between Meta’s strategy and Google’s 2007 introduction of the Android mobile operating system to rival Apple’s iOS. By providing a cheap yet powerful alternative, Meta aims to counterbalance proprietary systems like those developed by OpenAI and ignite innovation that can in turn enhance Meta’s products and services.

Nathan Lambert, an AI researcher at Hugging Face, a startup specializing in open source machine learning software, including generative models, considers Llama 2 a significant milestone. While he doesn’t classify the project as fully open source due to Meta’s limited disclosure about its development, Lambert is astonished by the numerous variations of Llama 2 he encounters across social media. For instance, the latest version of WizardLM, an AI system designed to follow complex instructions, is one of several examples influenced by Llama 2. Lambert believes that Llama 2 will eventually catch up to and perform most tasks for which people currently rely on ChatGPT.

Nevertheless, Lambert acknowledges that Meta’s release of Llama 2 leaves several questions unanswered, particularly regarding the lack of documentation on training data. It remains true that only major players like Meta, Google, Microsoft, and OpenAI possess the computing resources and staff necessary to develop leading large language models. However, Lambert remains hopeful that language models are moving towards a more transparent era. A voluntary agreement between the White House and seven major AI companies underscores the need for tests to evaluate potential discrimination, societal impact, and national security implications before deployment.

Still, this transparency trend could face challenges as questions regarding legal liability for AI systems grow, accompanied by increasing regulatory pressure from politicians concerned about potential misuse of open source models.

Like Demis Hassabis, the AI researcher now leading Google’s AI development, Turow disagrees with the leaked Google memo’s assertion that major AI companies are threatened by open source AI. While data, talent, and access to computing power will continue to serve as protective barriers for tech giants, they won’t make them invincible. Turow eagerly anticipates seeing what startups and researchers can accomplish with Llama 2, expecting rapid improvements similar to those observed with Meta’s initial model. This progress could create new possibilities not just for startups but also for the broader AI field, potentially challenging the current leaders in the market.

In conclusion, Meta’s release of the open source Llama 2 model has the potential to revolutionize the AI landscape. By providing an accessible and customizable alternative to proprietary systems, Meta empowers developers, startups, and the wider community to experiment freely and foster innovation. While questions remain regarding transparency and access to resources, Llama 2 represents a significant milestone that could reshape the field of AI and pave the way for future advancements.