Man
Professional
- Messages
- 3,061
- Reaction score
- 586
- Points
- 113
I continue the series of articles about the complex relationship between the hoster and the client.
Today we will talk about two "vulnerabilities" in quotes, because at least one of them is not a bug, but a feature
0. Who is to blame?
The reason for the occurrence of both "vulnerabilities" is that BMCs absolutely trust the operating system running on the server , and not necessarily installed, it is enough to boot from LiveCD.
1) the first "vulnerability" allows you to add a BMC user with administrator rights, without knowing the current administrator password, and / or change his password to any other, and / or reset the password to the default ("ADMIN", "calvin", and so on in different BMCs)
the vast majority of all servers of all brands in the world are subject to this "vulnerability", regardless of the manufacturer of the BMC chip or its firmware.
An example of commands for creating a new user:
where "10" = user id (check first that this id is not taken)
and "0x4" = administrator rights.
You can simply run "ipmitool user set password id password" to change the password or "ipmitool user priv id privileges" to change the rights of an existing user, if the hoster gave you access to the BMC with user rights.
sometimes you need to add another "channel number", for example in Supermicro it is "1", and in HP - "2" ("ipmitool user priv 10 0x4 1" and "ipmitool user priv 10 0x4 2" respectively).
sometimes you need to separately issue rights through an additional utility, for example, in Dell after creating a user through ipmitool you also need to run "idracadm set iDRAC.Users.10.Privilege 0x1ff" (after downloading iDRACTools from the Dell website)
2) the second "vulnerability" allows you to make a dump of the current firmware and / or upload a new firmware. Subsequently, you can get user hashes.
From this dump, and, having brute-forced these hashes, try to substitute the received passwords on other servers in the organization's network, and by adding useful software such as socat or plink to the firmware dump, you can move along the organization's network to other servers. The vast majority of all servers are also susceptible to this "vulnerability", but I am only sure about servers using BMC from ASPEED: Supermicro, Lenovo, IBM, Tyan, AsRock, Quanta, ASUS, Gigabyte and some other brands. It is definitely possible to replace the firmware on BMC ASPEED models up to and including AST2400, these models were used in servers manufactured before 2018. In AST2500 it is most likely possible too, but I have not checked. But starting with AST2600 (in servers manufactured somewhere since 2020) - there is a chance that it will not be possible to upload modified firmware, because this model has added the functionality of digital signature and firmware verification, "Silicon Root of Trust". However, it is still possible to dump the current firmware on AST2600.
Flittle about firmware signing
As I understand it, I may be wrong.
Old models have nowhere to get data to verify the authenticity of the firmware, so they are forced to trust the data received from the user. firmware manufacturers, in turn, do not bother much and make a simple "signature" of the firmware in the form of a special string (for example, "ATENs_FW" for Supermicro (https://xss.is/threads/94447/post-655853) or "$MODULE$" for Lenovo), hidden in a special place inside the firmware, and also containing CRC32 checksums of the firmware sections.
and new BMC models have an additional, proprietary SPI flash drive or proprietary TPM chip (sometimes in a software implementation on FPGA), in which the keys are stored, with the help of which the authenticity of the firmware is verified. these keys are filled in once at the factory, and without physical access to the server the user will not be able to replace these keys (and with physical access he will be able to
https://github.com/google/security-research/security/advisories/GHSA-v9gx-jrwm-3f78), these keys written in the SPI/TPM chip are the "Silicon Root of Trust".
1. What does hosting have to do with it?
A server rental service is the most obvious target for these "vulnerabilities", because a hosting provider gives third parties access to its hardware.
The danger of these "vulnerabilities" for hosting is that a malicious client can rent a server for a month, backdoor the BMC, and then have access to this server (and through it to other servers in the hosting network) through the backdoor even after the paid rental period has expired.
And from the hosting client's point of view, the danger is that he can rent an already backdoored server, or the client's server can be backdoored by third parties: for example, a cybercriminal hacks the client's site, root the server through the site, gains access to the BMC through root, and gains a foothold on the server at the BMC level, and then no matter how you clean the site from shells and reinstall the OS on the server, the fraudster will still be able to restore his access to the site.
a cybercriminal (or malicious client) does not even have to log into the hosting admin panel to find out the BMC access data for uploading modified firmware - if the hosting does not provide access to the BMC Web interface, then there is a chance to upload a trojanized firmware through some software from firmware developers, such as "Yafuflash" from AMI, "AlUpdate" from ATEN, "socflash" from ASPEED or "SUM" from Supermicro.
All this software comes without clear documentation, may or may not work on a specific server and, depending on the specific version of the software, may not work in a "large" OS like Linux or Windows, but work in DOS or UEFIin short, I have had varying success with all this.
So, unlike my previous articles with an evil hoster, this time the client can harm the hosting, and even third parties can harm the first two at once.
2. What to do? there is no way to protect yourself from the first "vulnerability" - it is not a bug, but a feature, this is how the IPMI standard was intended
¯ \ (ツ) /¯ from the second - use new servers, the BMC of which supports the technologies of "secure boot" and digital signature of firmware "Secure Boot" and "Silicon Root of Trust": - Supermicro 13th generation, and some models of the 12th generation with BMC ASPEED AST2600 - do not use Lenovo at all, it is dog shit, not a server
That is, in order to avoid problems, hosters need to spend money on updating the entire fleet of servers (and also properly configure the network so that BMC does not have access to the Internet and neighboring servers), and clients will have to spend more on renting modern models so as not to be afraid of getting an already trojanized server.
3. Demonstration on a live example
The video demonstrates the actions of a malicious client: potential pwnage of the entire data center by renting a server and uploading trojanized BMC firmware to it (sorry for the English language - I was making a report for the FBI curators, and I'm too lazy to record a separate video with comments in Russian).
Direct link to the video (100MB): https://files.catbox.moe/mtsh6w.mp4
Vimeo:
If you only knew how my ass got torn apart.
It was found out experimentally that a bug in the Peek program does not allow saving videos longer than 5 minutes 10 seconds, so after the beloved first hour of filming, I had to film everything from the very beginning, in parts of a maximum of 5 minutes, and some sections several times, so do not be surprised by sharp transitions, the user "DOOR" and the suddenly appearing rootfs.bin_backup file
4. Outro
Once again I draw your attention to the fact that the described vulnerabilities apply not only to the specific hosting shown in the video, but to ALL hostings using servers of not the latest generations (older than 2020). I notified this company that their BMCs can not only see each other, but also get onto the Internet, so I think that now this is already blocked on the data center firewall. so break other hostings.
Today we will talk about two "vulnerabilities" in quotes, because at least one of them is not a bug, but a feature
0. Who is to blame?
The reason for the occurrence of both "vulnerabilities" is that BMCs absolutely trust the operating system running on the server , and not necessarily installed, it is enough to boot from LiveCD.
1) the first "vulnerability" allows you to add a BMC user with administrator rights, without knowing the current administrator password, and / or change his password to any other, and / or reset the password to the default ("ADMIN", "calvin", and so on in different BMCs)
the vast majority of all servers of all brands in the world are subject to this "vulnerability", regardless of the manufacturer of the BMC chip or its firmware.
An example of commands for creating a new user:
Code:
ipmitool user set name 10 BACKDOOR
ipmitool user set password 10 BACKDOOR
ipmitool user priv 10 0x4
ipmitool user enable 10and "0x4" = administrator rights.
You can simply run "ipmitool user set password id password" to change the password or "ipmitool user priv id privileges" to change the rights of an existing user, if the hoster gave you access to the BMC with user rights.
sometimes you need to add another "channel number", for example in Supermicro it is "1", and in HP - "2" ("ipmitool user priv 10 0x4 1" and "ipmitool user priv 10 0x4 2" respectively).
sometimes you need to separately issue rights through an additional utility, for example, in Dell after creating a user through ipmitool you also need to run "idracadm set iDRAC.Users.10.Privilege 0x1ff" (after downloading iDRACTools from the Dell website)
2) the second "vulnerability" allows you to make a dump of the current firmware and / or upload a new firmware. Subsequently, you can get user hashes.
From this dump, and, having brute-forced these hashes, try to substitute the received passwords on other servers in the organization's network, and by adding useful software such as socat or plink to the firmware dump, you can move along the organization's network to other servers. The vast majority of all servers are also susceptible to this "vulnerability", but I am only sure about servers using BMC from ASPEED: Supermicro, Lenovo, IBM, Tyan, AsRock, Quanta, ASUS, Gigabyte and some other brands. It is definitely possible to replace the firmware on BMC ASPEED models up to and including AST2400, these models were used in servers manufactured before 2018. In AST2500 it is most likely possible too, but I have not checked. But starting with AST2600 (in servers manufactured somewhere since 2020) - there is a chance that it will not be possible to upload modified firmware, because this model has added the functionality of digital signature and firmware verification, "Silicon Root of Trust". However, it is still possible to dump the current firmware on AST2600.
Flittle about firmware signing
As I understand it, I may be wrong.
Old models have nowhere to get data to verify the authenticity of the firmware, so they are forced to trust the data received from the user. firmware manufacturers, in turn, do not bother much and make a simple "signature" of the firmware in the form of a special string (for example, "ATENs_FW" for Supermicro (https://xss.is/threads/94447/post-655853) or "$MODULE$" for Lenovo), hidden in a special place inside the firmware, and also containing CRC32 checksums of the firmware sections.
and new BMC models have an additional, proprietary SPI flash drive or proprietary TPM chip (sometimes in a software implementation on FPGA), in which the keys are stored, with the help of which the authenticity of the firmware is verified. these keys are filled in once at the factory, and without physical access to the server the user will not be able to replace these keys (and with physical access he will be able to
https://github.com/google/security-research/security/advisories/GHSA-v9gx-jrwm-3f78), these keys written in the SPI/TPM chip are the "Silicon Root of Trust".1. What does hosting have to do with it?
A server rental service is the most obvious target for these "vulnerabilities", because a hosting provider gives third parties access to its hardware.
The danger of these "vulnerabilities" for hosting is that a malicious client can rent a server for a month, backdoor the BMC, and then have access to this server (and through it to other servers in the hosting network) through the backdoor even after the paid rental period has expired.
And from the hosting client's point of view, the danger is that he can rent an already backdoored server, or the client's server can be backdoored by third parties: for example, a cybercriminal hacks the client's site, root the server through the site, gains access to the BMC through root, and gains a foothold on the server at the BMC level, and then no matter how you clean the site from shells and reinstall the OS on the server, the fraudster will still be able to restore his access to the site.
a cybercriminal (or malicious client) does not even have to log into the hosting admin panel to find out the BMC access data for uploading modified firmware - if the hosting does not provide access to the BMC Web interface, then there is a chance to upload a trojanized firmware through some software from firmware developers, such as "Yafuflash" from AMI, "AlUpdate" from ATEN, "socflash" from ASPEED or "SUM" from Supermicro.
All this software comes without clear documentation, may or may not work on a specific server and, depending on the specific version of the software, may not work in a "large" OS like Linux or Windows, but work in DOS or UEFI
in short, I have had varying success with all this.So, unlike my previous articles with an evil hoster, this time the client can harm the hosting, and even third parties can harm the first two at once.
2. What to do? there is no way to protect yourself from the first "vulnerability" - it is not a bug, but a feature, this is how the IPMI standard was intended
¯ \ (ツ) /¯ from the second - use new servers, the BMC of which supports the technologies of "secure boot" and digital signature of firmware "Secure Boot" and "Silicon Root of Trust": - Supermicro 13th generation, and some models of the 12th generation with BMC ASPEED AST2600 - do not use Lenovo at all, it is dog shit, not a server
- HP starting from 10th generation, with BMC iLO5
- Dell since 14th generation, with BMC Nuvoton NPCM7xx
That is, in order to avoid problems, hosters need to spend money on updating the entire fleet of servers (and also properly configure the network so that BMC does not have access to the Internet and neighboring servers), and clients will have to spend more on renting modern models so as not to be afraid of getting an already trojanized server.
3. Demonstration on a live example
The video demonstrates the actions of a malicious client: potential pwnage of the entire data center by renting a server and uploading trojanized BMC firmware to it (sorry for the English language - I was making a report for the FBI curators, and I'm too lazy to record a separate video with comments in Russian).
Direct link to the video (100MB): https://files.catbox.moe/mtsh6w.mp4
Vimeo:
If you only knew how my ass got torn apart.
It was found out experimentally that a bug in the Peek program does not allow saving videos longer than 5 minutes 10 seconds, so after the beloved first hour of filming, I had to film everything from the very beginning, in parts of a maximum of 5 minutes, and some sections several times, so do not be surprised by sharp transitions, the user "DOOR" and the suddenly appearing rootfs.bin_backup file
4. Outro
Once again I draw your attention to the fact that the described vulnerabilities apply not only to the specific hosting shown in the video, but to ALL hostings using servers of not the latest generations (older than 2020). I notified this company that their BMCs can not only see each other, but also get onto the Internet, so I think that now this is already blocked on the data center firewall. so break other hostings.
