Carding
Professional
- Messages
- 2,871
- Reaction score
- 2,371
- Points
- 113
The neural network hacking training platform itself has become a privacy threat.
The Swiss company Lakera AI, developer of the Gandalf educational platform designed to teach the risks of "prompt injection" attacks in Large Language models (LLM), recently fixed a vulnerability in its system. Security experts discovered a publicly available analytics dashboard that provided information about requests sent by players and related metrics.
Gandalf was launched in May and is a web form that allows users to gamely attempt to cheat basic LLM using an API from OpenAI. The goal is to identify passwords in the game by consistently completing increasingly complex tasks.
The analytics dashboard was discovered by Australian security consulting firm Dvuln. According to Dvuln, the server showed more than 18 million requests from users, 4 million attempts to guess the password, as well as other game metrics.
It is noted that the data does not contain personal information and is not confidential. However, some players entered their email addresses into the game, which became available through the dashboard.
Dvuln stressed that while the Gandalf task is a simulation designed to illustrate the risks associated with LLM, the lack of adequate security measures for data storage deserves attention.
The incident calls into question not only the security of Gandalf, but also privacy and security issues related to the use of LLM models in general. The situation highlights the need for strict security protocols, even in systems designed for educational or demonstration purposes.
The Swiss company Lakera AI, developer of the Gandalf educational platform designed to teach the risks of "prompt injection" attacks in Large Language models (LLM), recently fixed a vulnerability in its system. Security experts discovered a publicly available analytics dashboard that provided information about requests sent by players and related metrics.
Gandalf was launched in May and is a web form that allows users to gamely attempt to cheat basic LLM using an API from OpenAI. The goal is to identify passwords in the game by consistently completing increasingly complex tasks.
The analytics dashboard was discovered by Australian security consulting firm Dvuln. According to Dvuln, the server showed more than 18 million requests from users, 4 million attempts to guess the password, as well as other game metrics.
It is noted that the data does not contain personal information and is not confidential. However, some players entered their email addresses into the game, which became available through the dashboard.
Dvuln stressed that while the Gandalf task is a simulation designed to illustrate the risks associated with LLM, the lack of adequate security measures for data storage deserves attention.
The incident calls into question not only the security of Gandalf, but also privacy and security issues related to the use of LLM models in general. The situation highlights the need for strict security protocols, even in systems designed for educational or demonstration purposes.
