Meta releases new version of its code generation AI model, Code Llama 70B, with improved code accuracy and a Python-optimized variant



Meta, the company behind Facebook and Instagram, has announced the release of a 70 billion parameter model for Code Llama, an AI that generates program code based on text input. The model is released under the same

Llama 2 Community License as Llama 2, and can be used commercially free of charge for companies with up to 700 million monthly active users.

Introducing Code Llama, a state-of-the-art large language model for coding
https://ai.meta.com/blog/code-llama-large-language-model-coding/



Code Llama is a version of Llama 2, released by Meta in July 2023, further trained on code-specific datasets. It supports Python, C++, Java, PHP, Typescript & Javascript, C#, and Bash, and in addition to generating code continuations, it can also generate code based on natural language input and generate commentary about the code. Models with 7 billion, 13 billion, and 34 billion parameters were released in August 2023 .

On January 29, 2024, Meta released the 'Code Llama 70B' model, which is based on the 70 billion parameter 'Llama 2 70B' model and has been additionally trained on a code-specific dataset. While other models of this size require 500 billion tokens of training data, the 70 billion parameter model was trained on 1 trillion tokens. Furthermore, in addition to the basic 'Code Llama' model, which is fine-tuned for each purpose, variants such as 'Code Llama - Instruct,' which is specialized for programming from natural language, and 'Code Llama - Python,' which is specialized for handling Python, are available.



The performance of the 70 billion parameter model compared to other models measured using

HumanEval and MBPP is shown in the figure below. The Instruct model outperformed GPT-4, and all three variants showed steady improvements.



Code Llama models are available through submission to Meta and are also hosted on Hugging Face .

◆ Forum is currently open
A forum related to this article has been set up on the official GIGAZINE Discord server . Anyone can post freely, so please feel free to comment! If you don't have a Discord account, please refer to the account creation instructions article to create one!

Discord | 'Have you tried a Llama-style language model? What can you do with it?' | GIGAZINE
https://discord.com/channels/1037961069903216680/1201822215050641428

Related Posts:

in AI,   Software, Posted by log1d_ts