A long time ago in a network far far away…
Like many in our field, I am a Star Wars nerd. I could quickly fill this post with several thoughts and opinions on Star Wars such as the treacherous waters of comparing the 3 trilogies and little-known Star Wars facts (e.g. Willrow Hood). You can find plenty of that information on the Internet. Instead, I want to relate three Star Wars examples to supply chain risk. You need only be familiar with Episode IV: A New Hope (and to an extent Star Wars: Rogue One), Episode III: Revenge of the Sith, and the latest series called “Andor” (I promise there will be no Andor spoilers in this post).
“Order 66”
It was Episode II: Attack of the Clones that hinted at what was to come later in the series as we learn that a secret clone army has been in development for some time. Little did we know that each of the clone troopers was fitted with a special chip that responds to a command, which in turn instructs the clone trooper to execute all Jedi and treat them as traitors to the republic.
The physical tampering or implantation of a chip that is not disclosed as part of the documented design has surfaced in the past. The Bloomberg/Super Micro article chronicles one such tale, claiming hardware from Super Micro was tampered with. Super Micro, in turn, refutes this claim. Recently, Wired ran a story detailing a proof-of-concept that could implant malicious hardware titled “Planting Tiny Spy Chips in Hardware Can Cost as Little as $200”.
Of course, in all of the above examples, tampering (changing the behavior to cause unintended consequences that are unbeknownst to the user) occurs in the hardware layer. This very same attack could also be executed in the software layer, as we’ve all read about in the Solar Winds attack (a software-based attack was also referenced in the Bloomberg/Super Micro article). But what makes these supply chain attacks really stand out is the difficulty of detection. Whether the tampering and backdoors are in hardware, firmware, or software, detection is difficult. Why? If we look at the examples, including “Order 66”, there was no behavior that would cause concern, until the actual attack. Solar Winds customers did not know something was wrong until they were able to observe behavior indicative of an attack (the same goes for the Jedi, everything was great until their own troops turned against them). To detect this type of attack before something bad happens, we have to be looking for it. We have to inherently distrust our supply chain and verify the integrity of the components if we want to get ahead of the supply chain threats.
The “Thermal Exhaust Port”
While I always thought there was someone pouring over the technical readouts of the battle station we all know as the “Death Star” to find a weakness that was put there accidentally by the Empire, turns out it was an insider. We don’t find out until much later (at least later when Star Wars: Rogue One was released) that the vulnerability leading to the destruction of the death star was implanted by trusted insider Galen Erso.
I like this example because we can relate it to vulnerabilities in the supply chain in two different contexts. The first is how it plays out in the movie, an insider (or threat actor with enough access) has implanted a vulnerability in the supply chain. I don’t recall occurrences of this in software, though it’s an interesting attack vector to consider. Given that many vulnerabilities, especially in older software such as sudo, can exist undetected for years, this type of supply chain attack could prove valuable for attackers. Of course, once attackers start exploiting the “hidden” vulnerability, all bets are off and the investigation begins and likely the access vector is essentially burned by the threat actors. Putting this in the context of firmware is a bit more exciting, I mean scary because fewer people are looking at the increasing number of lower-level vulnerabilities that lie in firmware-based systems.
The second is the more common scenario where a vulnerability is discovered in a component that is widely used. We’ve seen countless examples of this, including Log4j and multiple occurrences of OpenSSL (though the most recent vulnerability was not as impactful as we once thought). In traditional (non-firmware-based) software, such as a library, organizations have some control over the remediation. Provided the organization can identify the vulnerable component, the fixes are implemented through the software development process. Some options include updating the library, patching the library (backporting the security fix but not updating the rest of the software to the latest version), switching to a new third-party library that has equivalent functionality, or writing the library yourself and maintaining it moving forward. Firmware, and many lower-level components such as bootloaders and kernels, are far more complex and tied to physical hardware. Typically organizations must rely on an upstream supplier to release a fix and then apply it to their existing systems. In certain circumstances with firmware (such as SPI flash configurations), you simply cannot update it yourself. You have to wait, and sometimes you could be waiting for a very long time (or forever for that matter).
Stolen Navigation Equipment (Andor)
Released just this year, Andor is a new Star Wars TV series that follows Cassian Andor, one of the lead characters from Star Wars: Rogue One. Given the show is so new, I don’t want to introduce any spoilers (If you are highly sensitive to spoilers, you can stop reading now and skip this section). One of the early events introduced in Andor is how the rebels managed to steal a piece of imperial equipment. Let’s just say they manage to do exactly that, and the Empire is not happy about losing a piece of proprietary technology that is critical to some of their “systems”.
Supply chain attacks encompass events beyond the control of suppliers, manufacturers, and users. Any party in the supply chain could be the source of a problem that affects all links in the supply chain, and impacts can flow both downstream and upstream. Recently, Intel was in the news because a partner leaked source code and other proprietary materials. While this particular incident has not had an immediate security impact, researchers may use the information disclosed to find vulnerabilities in the future (and hopefully submit them to Intel’s bug bounty program). This type of attack is also difficult to defend against, especially in the enterprise that is the end-consumer of technology that may fall victim to vulnerabilities well outside the control of the IT department.
The recent Lenovo vulnerabilities share a similar story to the stolen imperial equipment as they represent tools that allow users to modify the system. The vulnerabilities were not in firmware per se, but in the drivers that allow a user to make changes to variables used by the firmware. The variables (stored in NVRAM) in this case could allow attackers to circumvent the Secure Boot process. The Lenovo drivers represent a supply chain issue that incorporates vulnerabilities along with tools that can be used against you falling into the wrong hands.
Conclusion
In both Star Wars and the real world, supply chain attacks have devastating consequences. Order 66 wiped out most of the Jedi Order. The Solar Winds attack is estimated to have cost companies an average of $12 million in damages. Firmware-based attacks that take advantage of supply chain weaknesses are becoming more and more common (See Firmware Attacks: An Endpoint Timeline for examples). The question is what do we do about it? While my recommendation stands to always update your firmware, what happens if you can’t? Taking Lenovo as an example, the Ideapad Y700-14ISK is end-of-life and no longer supported, which means it will not receive an update. I don’t fault manufacturers for putting limits on the amount of time a product will be supported, as it’s not economically feasible to support every product from the release date until the end of time. However, it’s important to identify the vulnerable systems, which ones can be patched, and which ones cannot be patched to formulate an effective remediation strategy. Perhaps your only feasible solution is to replace the older equipment. For the supported equipment, rolling out new firmware is the best option, but also comes with operational risk. Within your organization, the supply chain, and the component within, should not be blindly trusted. Take steps to identify your assets (especially the ones below the surface), verify their integrity (is this really the firmware/software I should be running?), and fortify them with timely firmware/software updates.
