Meta has released 'TRIBE v2,' an AI model that accurately predicts the human brain's response to images and sounds.



' TRIBE v2 ' is a foundational model trained to predict how the human brain will react to almost any visual or auditory stimuli.

TRIBE v2

https://aidemos.atmeta.com/tribev2/

TRIBE v2 is a foundational model that leverages over 500 hours of fMRI recordings collected from more than 700 subjects to create a digital twin of human neural activity, enabling zero-shot learning for new subjects, languages, and tasks.




The foundation for this is TRIBE , which was announced in July 2025. While TRIBE made predictions on 1,000 cortical cells, TRIBE v2 predicts whole-brain activity across 70,000 voxels. Furthermore, while TRIBE was trained on only four subjects, TRIBE v2 achieves zero-shot learning by combining vast amounts of recorded data and a large cohort.

TRIBE v2 predicts brain activity through a three-stage pipeline.

The first stage is called 'trimodal encoding,' in which the model uses pre-trained audio , video , and text embeddings to capture features common to both the AI model and the human brain.

The second stage is a process called 'universal integration.' These embeddings are processed by transformers that can learn universal representations common to all stimuli, tasks, and individuals.

In the third stage, 'brain mapping' is performed. The subject layer maps these universal representations to individual fMRI voxels (3D pixels that track neural activity through gradual changes in blood flow and oxygenation).

The following diagram illustrates the three-stage pipeline of TRIBE v2.



TRIBE v2 can predict an individual's brain responses, even those never observed before, with high accuracy, without retraining. For example, when predicting brain responses during activities such as watching movies or listening to audiobooks, it succeeded in predicting brain responses with approximately 2 to 3 times higher accuracy compared to conventional methods.




According to Meta, TRIBE v2's brain activity prediction accuracy often reflects typical responses more accurately than actual fMRI scans. Raw brain activity data can contain a lot of noise due to factors such as heart rate, movement, and equipment. On the other hand, TRIBE v2 predicts standard brain responses and has been shown to correlate more highly with the average neural activity of a group than with individual fMRI recordings.

Meta makes not only the models themselves, but also the codebase, papers, and demos publicly available so that researchers can further advance neuroscience, build better AI by leveraging insights into the brain, and accelerate breakthroughs in the diagnosis and treatment of neurological diseases using computational simulations.

The paper can be viewed below.

A foundation model of vision, audition, and language for in-silico neuroscience | Research - AI at Meta
https://ai.meta.com/research/publications/a-foundation-model-of-vision-audition-and-language-for-in-silico-neuroscience/

TRIBE v2 can also be downloaded from the following link.

facebook/tribev2 · Hugging Face
https://huggingface.co/facebook/tribev2

The source code is publicly available on GitHub.

GitHub - facebookresearch/tribev2: This repository contains the code to train and evaluate TRIBE v2, a multimodal model for brain response prediction · GitHub
https://github.com/facebookresearch/tribev2

A demo version of TRIBE v2 is available from the link below.

TRIBE v2
https://aidemos.atmeta.com/tribev2/

Related Posts:

in AI, Posted by logu_ii