A vulnerability called 'GitLost' has been reported that allows AI to extract information about private repositories using only public issues on GitHub.
Security firm Noma Labs has reported a GitLost vulnerability, a prompt injection vulnerability that allows attackers to inject malicious instructions into public GitHub repositories, causing an AI agent to post content from a private repository as a public comment. It is believed that attackers could potentially carry out the attack simply by creating an issue in an organization's public repository, even without authentication credentials or access to the repository.
GitLost: How We Tricked GitHub's AI Agent into Leaking Private Repos - Noma Security
GitLost is a vulnerability that occurs through a combination of Issues and GitHub Agentic Workflows. Issues are a GitHub feature that allows users to request new features or report bugs, and it is becoming increasingly common for public repositories to accept Issues from external users. GitHub Agentic Workflows is a mechanism for delegating tasks around a repository to an AI agent. By describing natural language workflows in Markdown, tasks such as classifying Issues can be processed by an AI agent on GitHub Actions.
While AI agents are convenient, they have a problem: they can interpret text they read as instructions. Humans can distinguish between user-written requests and system rules, but if an AI agent processes content such as issue text and comments in the same context, there is a risk that text written by an attacker will be treated as an instruction. This type of attack is called 'indirect prompt injection,' and it is particularly problematic when the AI agent is designed to read external text.
In the vulnerable GitHub Agentic Workflows configuration identified by Noma Labs, the workflow was set to be triggered by the issue assignment event, read the issue title and body, and post a reply comment.
Furthermore, the workflow had permission to read both public and private repositories managed by the same organization. The attacker was able to post a seemingly legitimate issue to the organization's public repository and embed instructions for the AI agent within the issue body, causing the AI agent to retrieve the README.md from the private repository and post it in the comments section of the public issue.
GitHub Agentic Workflows includes defenses such as read-only permissions, sandboxed execution, secure output validation, and threat detection. However, according to Noma Labs, the research team found that adding the keyword 'Additionally' altered the AI agent's behavior, causing the defenses to malfunction.
Noma Labs states that 'indirect prompt injection in agent AI is the equivalent of SQL injection in web applications; it's a problem common to all systems of the same type.'
Noma Labs recommends the following countermeasures: 'Do not treat user-controlled text as trusted commands,' 'Minimize AI agent permissions,' 'Restrict what AI agents can post to public forums,' and 'Separate or sanitize user input before passing it to AI agents.' They emphasize that agents that can read across multiple repositories are particularly valuable targets for attacks, making a review of permission design crucial.
Noma Labs has stated that they have responsibly reported the GitLost incident to GitHub, and that GitHub has made the details public after gaining a full understanding of the situation.
Related Posts:
in AI, Web Service, Security, Posted by log1d_ts