Man
Professional
- Messages
- 3,223
- Reaction score
- 915
- Points
- 113
Programmer Keyu Tian got a job at ByteDance as an intern and sabotaged a neural network development project from the inside for two months, adding bugs to the code. Because of Keyu's actions, a team of three dozen developers searched around the clock and fixed the constantly appearing bugs in the project.
The investigation revealed that the intern:
• methodically and consistently uploaded special Pickle files with hidden code that was full of viruses and malicious components. The code in such scripts is executed randomly and automatically, so no one could understand why all the team's efforts did not bring proper stabilization;
• gained access to and completely changed the version of the PyTorch library on which the projects were based. He made small changes there every day, and they brought down the programs. At the same time, none of the team's developers even thought to look at the source code — tasks continued to crash with errors and a ton of bugs, and all the experiments brought only incorrect results;
• brought real chaos to checkpoints — these are files that help train neurons and preserve their intermediate states. He juggled model parameters, changed training data, or simply deleted many checkpoints, and the team's developments disappeared, were deleted, and there was no way to repeat them;
• did not hide from meetings. He could not be caught quickly, because he went to every rally and collected the team's solutions to fix problems. After the calls, the intern knew how they were going to fix bugs and came up with new ones. The team did not understand what was happening every time.
As a result, the intern was caught from the logs. ByteDance also explained that a team of thirty programmers worked in vain for two whole months due to the destructive actions of one developer. The team's project lost all deadlines, and the customers' money was wasted.
After Tian was expelled from ByteDance, he was not punished or censured by his mentors at the higher education institution.
The investigation revealed that the intern:
• methodically and consistently uploaded special Pickle files with hidden code that was full of viruses and malicious components. The code in such scripts is executed randomly and automatically, so no one could understand why all the team's efforts did not bring proper stabilization;
• gained access to and completely changed the version of the PyTorch library on which the projects were based. He made small changes there every day, and they brought down the programs. At the same time, none of the team's developers even thought to look at the source code — tasks continued to crash with errors and a ton of bugs, and all the experiments brought only incorrect results;
• brought real chaos to checkpoints — these are files that help train neurons and preserve their intermediate states. He juggled model parameters, changed training data, or simply deleted many checkpoints, and the team's developments disappeared, were deleted, and there was no way to repeat them;
• did not hide from meetings. He could not be caught quickly, because he went to every rally and collected the team's solutions to fix problems. After the calls, the intern knew how they were going to fix bugs and came up with new ones. The team did not understand what was happening every time.
As a result, the intern was caught from the logs. ByteDance also explained that a team of thirty programmers worked in vain for two whole months due to the destructive actions of one developer. The team's project lost all deadlines, and the customers' money was wasted.
After Tian was expelled from ByteDance, he was not punished or censured by his mentors at the higher education institution.
