This month’s report is going to offer up some insights in the device/IT supply chain space, relevant to our industry, and based on existing trends that all point to The Supplyocalpyse: Appetite for Destruction. We almost called it the Firmocalypse, but it just didn’t hit the same way. We’ve solicited input from trust groups as well as from several of our own researchers and field engineers. Some of these insights will be attributed to their authors, while others will stand on their own. After collecting and analyzing them all, we quickly realized the themes they all centered around: Destructive attacks and attacks both targeting and stemming from the IT Supply Chain, and the convergence of these two themes.
Grab a cup of hot chocolate and take a seat. You’ll want to be sitting down.
Vigilante hacktivism may become a significant threat actor profile, advancing timelines for destructive attacks and collateral damage
Historically hacktivism hasn’t been a major focus for many enterprises or federal missions, but things have changed in 2022, and we are just getting started. In the Russo-Ukraine war alone there are over 81 Groups overtly attacking one another, 36 Pro-Ukraine and 40 Pro-Russia. The number of countries hosting hacktivist actors is expanding every day. These actors are loud and proud, racing each other to claim victories against some of the most important victim organizations on the planet, from Ministries of Defense across Europe and Russia to major brands like Disney, Nasdaq, Coca-Cola, and McDonalds. Major news networks like the BBC and CNN are being attacked. From the White House all the way down to US Citizens and their medical data, everyone has become a target.
NATO (currently preparing for an all-out cyber war) and the West aren’t the only targets though. Russia and Iran are being targeted more than any time in history. Even Russia’s centralized EISA (The national Unified System of Identification and Authentication) used by government personnel is being targeted. And this is just one day’s worth of news and claims. In speaking with industry peers, the actual cyber impact on Russia’s government and military has been their “OPM” moment, and then some. Looking forward to 2023, we will see these attacks escalate in both scope and severity, transitioning from wipers and DDOS attacks to compromised data centers, cloud environments, and enterprise networks. The same criminal marketplaces used to acquire such services also provide access to VPN exploits and credentials. Or worse, malware that used to be nation-state level tradecraft, such as Black Lotus, a well-documented UEFI bootkit that can at once persist indefinitely, or brick a device indefinitely too.
The net effect of this new vigilante (and often state-backed) hacktivism is the advancement of destructive timelines and collateral impact. The political will component of such attacks is no longer a mitigating factor when hacktivist clout, intent, and public encouragement of such activity is now so well formed.
The Conti Group apparatus will likely continue to push forward with new tactics for evasion, impact, indefinite persistence, resilient infrastructure, flexible c2 tunneling, and device-centric persistence
Conti are the most prolific, powerful, and well-resourced cyber crime apparatus in the world. They constantly push the envelope of malware TTPs and are able to canvas entire verticals or regions. They heavily influence the course of ransomware and extortion dynamics for the entire criminal underground, and they pour tremendous resources into recruiting the foremost developer talent in the space. Much of their tradecraft is on par with the very best nation-state capabilities and is often modeled off of such (see our TrickBoot and Conti-leaks blogs exposing their focus on tradecraft associated with low-level firmware attacks leveraging systemic supply chain vulnerabilities). The group is now ‘split’ (only in names and their respective focus within the overall apparatus) into a dozen or so ransomware groups, and continues to cause grave impact to western victims. While there are rumors of pending arrests underway, the group as a whole will carry on as they have always done.
Conti operators focus on a wide swath of verticals, and many of their targets are ‘hard’ targets; organizations that have the resources and tooling to detect modern-day threats. This forces them to continuously re-invent their tactics and develop evasive methods to avoid a robust security stack.
This, combined with their overall allegiance to Russian interests in the context of the ever-expanding cyber war being fought, provides the pretext for our insight, that over the next 12-18 months, they will be able to scale APT level persistence, evasion, and ultimately, destructive capability beyond the typical ransomware encryption payloads and double-extorion tactics seen today. The ability to destroy devices at the hardware level by attacking BMCs or the BIOS/UEFI, gives them leverage in the context of extortion while synergistically aligning such tactics with the Russian government’s heightened goal of destructive cyber capacity. Quite simply, both critical infrastructure and the Fortune 500 value public, patient, and worker safety (and uptime) more than they value stolen data. This dynamic is the primary premise for this sobering theory. The greater the leverage against a victim, the better the odds they will pay.
The terms ‘malware’ and ‘ransomware’ are increasingly becoming less useful
Why? Because malware can no longer be confined to malicious software, but rather, must be expanded to include any and all software that is used maliciously. This isn’t the first time this distinction has been made. The reason it is so important to address looking forward, is because legitimate, signed, and trusted software being used maliciously has become the rule, not the exception, for many an attack chain. The legitimate firmware running on a VPN device that an attacker commandeers, offers them unfettered access to the devices inside the network, as well as tunneling, C2, data exfiltration, credential harvesting, and denial of service capabilities. The signed drivers shipped as part of the OEM technology supply chain present attackers with the ability to escalate privileges and perform system-level commands. Collectively, this activity represents more than an incremental step beyond legacy “living off the land” tactics: It paves the way for suppliers to incorporate additional functionality in firmware updates that can be used maliciously. Is a feature or vulnerability that has been placed in such firmware on purpose considered malware? If you answered yes, then we know you are paying attention and can see the writing on the wall against today’s backdrop of chip wars, the re-Balkanization of the IT supply chain, and state-sponsored offensive cyber campaigns.
Similarly, the term “ransomware” needs to either be replaced or summarily expanded to include more than just encryption payloads and wipers operating at the user OS level. Ransomware, going forward, needs to encompass any software, operating at any level (Firmware, hypervisor, or OS level) that provides a criminal, hacktivist, or nation-state leverage over the humans in a given attack scenario. That leverage, itself, needs to expand in scope well beyond stereotypical profit motives, to include political, hacktivist, anti-competitive commercial, and destructive leverage. As evidenced in the NotPetya attack, the many wiper attacks observed in Ukraine, and the recent CryWiper campaign targeting Russian courts, even ransomware payloads aren’t always there to serve the profit motive. In fact, several ‘ransomware’ operations of late have been carried out by none other than the Russian GRU’s Sandworm organization. Are the encryption and wiper payloads meant to feed the coffers of the Russian military? Or are the disruptive payloads meant to both disrupt and distract from other operations being carried out? This same actor is extremely adept at targeting devices at the firmware level. Why stop at ‘ransomware’?
In the end, extortion itself reduces to nothing more than an exchange of power between two adversarial entities. Our industry will need to speak and operate independently from the vendor marketese and myopic solution providers that have so narrowly defined the ransomware problem space in recent years.
Offensive security tactics from victim organizations, victim nations, and trust groups could increase dramatically, out of necessity alone
It was only a year ago we began hearing chatter within trust groups and in closed-door meetings at the executive level, about “hacking back”, aka “returning fire to the enemy”. While that dynamic continues to play out in the context of incidents, it does not encompass the current dynamic at play: Nation states on all sides of the current cyber war have shifted to overt, proactive cyber attacks against opposing actors, critical infrastructure, and industry. Put differently, the political will of countries has shifted. Nay, the political appetite for destruction (to borrow from a Guns and Roses album we all know and love) is now more hungry than ever.
What’s more, this shift is born out of necessity alone. Even law enforcement (including the FBI) around the world is participating in a mix of both covert and overt (publicly-acknowledged) offensive operations, whether to bring down cyber-criminal organizations, disrupt IT supply chains or recruit academic and industry experts to perform narrow-scope offensive research and operations.
A right-wing deputy in Russia is proposing to assign military ranks to hackers that Russia is proactively recruiting for the ongoing cyber war, drawing from its many cadres of hackers known for their creative and sophisticated prowess during hacking competitions throughout the years. Since March of this year, Russia has experienced a tremendous amount of cyber attacks, more than any other time in its history. Russia has effectively had their ‘OPM moment” and have quickly realized (in the face of DDOS and Ransomware attacks now targeting them within their own border) that like every country, they have a soft underbelly in the form of weak infrastructure. Going forward this challenge will be exacerbated by the severe and effective IT sanctions that have been placed against them.
Ukraine, well known for supporting and funding offensive hacking efforts via the IT Army, has recently set up an EU-funded Cyber Lab in order to better model and train their experts on how to prepare for the threats stemming from Russia’s cyber aggression against their critical infrastructure.
Not to be outdone, the United States is proposing to adopt a similar strategy in the bipartisan endorsed National Defense Authorization Act (NDAA) for fiscal year 2023. Inside there are nearly 1000 mentions of the word “cyber”. There’s even a US Senate amendment with a provision (sec. 6101) that would require CISA to conduct a pilot program that would hire civilian cybersecurity reserves to aid in response to significant cybersecurity incidents. While defensive in nature, once established, it would be a small jump to use the same program to recruit cyber offensive reserves down the road. The US is also strengthening ties with Japan on this front, with the US National Cyber Director making a recent visit to Japan.
China, no stranger to recruiting hackers for nation-state activities, has been formalizing a nationwide program to streamline and systematically recruit entire armies of cyber warriors, each with specialized skills to be matrixed out based on mission objectives. It plans to bring the overall deficit of cyber skilled workers down by one million workers by 2027, from an estimated 1.4 million work deficit in 2017, as described in the in-depth study this article is centered around.
Just as evidenced throughout this World Cup, often the best defense against a strong opponent is a strong offense that keeps the other team in check and on their heels. Going forward this same dynamic will become the norm in the global cyber conflict, even though absolutely no one wants to see it play out.
Extortion, hacktivism, and disruption attacks may move more and more toward cyber-physical and cyber-human tactics
This has already been evidenced in recent years in the context of ransomware, with actors threatening individuals in the victim organization should they choose not to pay the ransom. With so much of one’s life easily discoverable on social media and via the mounds of already leaked data, threat actors are spending trivial amounts of time and extra effort to ‘know’ their victims prior to the negotiation phase.
Attackers targeting public figures and executives have long since leveraged a hybrid-approach to instill the most amount of fear and uncertainty in order to get a victim to pay, or reveal closely guarded secrets; combing cyber activities with mailing packages or postcards to victim homes, or describing the environment around a victim in order to let them know they are being watched. Military attacks against power grids have also combined cyber, telephone, and physical tactics in a single campaign.
Going forward, expect to see this hybrid approach proliferate into every attack campaign of significance. Why? Because it is an effective way to increase leverage and FUD, and it helps advance the attacker’s objective timelines to better position the victim’s capitulation. Why threaten to merely encrypt data or steal it when you can further threaten to brick a critical asset indefinitely via firmware attack, destroy a victim’s career and reputation, or fire a long-arm rifle at a substation instead of only hacking its SCADA network? Whether such actions manifest or not isn’t the point: attackers of all ilk have realized the power of combining such threats to achieve the desired outcome.
The risk of destructive attacks against data center and cloud infrastructure is growing
This and the subsequent two insights are all closely related and tie into the destructive motive theme overall. To date, most attacks against the hypervisor and cloud infrastructure have been espionage and data-theft focused. However, the same dynamics that make cloud environments attractive for those motives make them equally attractive for disruptive and destructive motive attacks: multiple tenants hosted in one environment whose infrastructure is centrally managed by a single provider.
There’s been a tremendous amount of research into vulnerabilities and tactics related to attacking such environments over the last few years. The overall understanding of where and how such environments are vulnerable is well-known now in the hacking community. It’s also well known by adversaries, who have begun to create malware and campaign strategies specifically focused on attacking virtual and cloud environments. Those same tactics end up being a single OS or firmware payload away from causing destructive havoc. If and once a motive turns from profit or espionage into destruction, tremendous impact can be felt across wide swaths of critical infrastructure and commerce. That threshold only gets closer and closer with every passing day. Only now, attackers understand how and where to attack the infrastructure itself at the device level, which leads us to our next related insight…
Homogeneous environments should take absolute control over their assets from the hardware all the way up to the application layer
The cost and scalability advantages of cloud environments, and indeed the entire concept of immutable virtualized images that can be spun up anew over and over again, are also one of their greatest weaknesses: the homogeneity of the platforms they sit upon that allows for scalable management, tech-refresh cycles, and streamlined operations. Having homogeneous platforms that leverage single vendor and model lines, creates an opportunity for threat actors to research what platform vulnerabilities are present on any given day, and craft automation and tactics into their campaigns. It’s not hard to imagine that this weakness will be exploited more and more in the years to come.
Take the recent BMC&C vulnerabilities Eclpysium discovered and disclosed this month as an example: Baseboard management controls are omnipresent in cloud and virtualized data center environments. A single high-impact remote code execution vulnerability or hard-coded default admin password vulnerability across many devices presents an ideal attack scenario, whether the motive is espionage, profit, or destruction. BMC’s provide an attacker with low-level persistence below the hypervisor or host operating system and remain accessible even when the device is powered down, regardless of what security controls exist above them in the stack. They provide a bi-directional means for attackers to deploy malware payloads to the OS, or move from the OS down to the BMC. They also allow attackers to move laterally across management networks, or even hop from production to management networks or vice versa. Attackers can use them to go from one guest tenant to another 3rd party tenant’s guest images, or to survive the entire re-provisioning process, as evidenced in our CloudBourne research tied to this class of vulnerabilities. They can even be used to jump from VDI environments to production/OT networks. Or, attackers can brick the BMC itself, creating indefinite downtime, and ironically, preventing operators from being able to restore the system via the BMC (one of its primary use cases).
All of this to say, homogeneous environments will need to cease operating under any form of implicit trust in the vendors and devices their platforms rely on, and take back absolute security control over those devices; whether it’s their provenance, their firmware updates, or vulnerability-discovery and the subsequent patching and mitigations at the device platform level. Detection of firmware backdoors and implants will become paramount, and the ability to attest to platform security itself will be demanded by both federal and private customers. Data centers are already under attack more than ever before, and they will have even more challenges going forward.
On the upside, with the right tooling and visibility, this is now possible to manage at scale as formalized IT supply chain SBOM and FBOM are taking shape and platform vendor participation with the security industry at the hardware and firmware level continues to manifest.
Threats both to and from the Technology Supply Chain may increase exponentially due to exposed weaknesses in the firmware SDLC, vulnerability research advancements, and economic computing challenges
The current global threat landscape is nothing, if not directly centered on the global challenges within the technology supply chain. Incident response teams have been adapting and are working diligently to help organizations better prepare for such attacks. CISA’s newly formed Cyber Safety Review Board is allocating significant resources to address the supply chain threat posed by the Lapsus$ group’s infiltration of numerous technology suppliers ranging from LG, Microsoft, NVIDIA, Okta, Samsung, Ubisoft, to Vodafone, and many others (potentially including this recent Uber supply chain attack). The list of actors targeting the supply chain is quite frankly, innumerable.
While threat actors of all stripes have been routinely targeting the supply chain more than ever before, the future state of this battleground will be unlike anything we’ve seen to date. Whether it’s the design phase or the production and delivery phases of the supply chain, nation-states will be looking to leverage every advantage in the most competitive technology race we’ve seen. Naturally cyber will play a huge role, and offer offensive opportunities to any willing participant. The majority of known attacks to date have been leveraging the inherent lack of secure development practices throughout the supply chain, and the resulting vulnerabilities stemming from them. Hive ransomware, for example, has hit over 1300 companies and collected over $100m in ransom, in part by targeting supply chain vulnerabilities like the recent FortiOS vulnerability. It’s hard to even name a single threat actor of any significance who hasn’t leveraged vulnerabilities found in VPN’s, load balancers, and other IOT devices. This is why the GAO recently published an 80-page report titled: Critical Infrastructure: Actions Needed to Better Secure Internet-Connected Devices.
Meanwhile, advancements in the research community have produced a bow-wave of vulnerabilities that are being discovered at a cyclical rate that defenders and vendors alike are unable to address before attack campaigns that manifest just hours or days after major vulnerabilities are disclosed. It’s already an untenable situation.
Yet, looking forward, the exploitation of vulnerabilities might end up being the least of our challenges. There’s nothing in place from a global governance perspective, let alone a cyber industry perspective, to prevent or deter vendors from implanting backdoors, malicious logic, spyware, or ‘kill-switch’ code into device firmware, and we believe this will become so common as to become one of the foremost challenges in cyber security the industry has ever faced. There’s a long history of this happening in edge cases, and indeed the ICT supply chain is already under a microscope as it is.
In the near future, we anticipate a new wave of these types of threats above and beyond current activity, and at a much broader level and at a much faster pace. The dynamic nature of firmware alone is a primary consideration, with vendors now pushing routine firmware updates and those updates evolving into a process that need not involve the end user’s action or awareness. The industry’s over-reliance on cryptographic trust via code signing has also been fully leveraged by every adversary. Stolen and leaked private keys are used just days after their theft, to sign malware, and this same implicit trust problem extends all the way down to firmware and the secure boot process itself.
It won’t be enough to scan and assess devices at the time of procurement. Malicious device firmware won’t be confined to only enterprise-class devices, either: with the workforce now largely remote, the firmware in SOHO and home electronics will become just as effective in targeting the enterprise or mission.
As my colleague Paul Asadoorian remarks in this blog drawing a parallel between the Star Wars trilogy and the firmware supply chain, every single player can pose a threat, including trusted insiders:
“While I always thought there was someone pouring over the technical readouts of the battle station we all know as the “Death Star” to find a weakness that was put there accidentally by the Empire, turns out it was an insider. We don’t find out until much later (at least later when Star Wars: Rogue One was released) that the vulnerability leading to the destruction of the death star was implanted by trusted insider Galen Erso.” -Paul Asadoorian, Eclpysium Evangelist
And we all know too well the fate of the Death Star…
The collection of software, firmware, and hardware bill of materials is not a sufficient mitigation for supply chain threats
This insight comes from our VP of Strategy, John Loucaides, who states that enterprise leadership will demand options, start making supplier comparisons, and require interoperability and dependencies between software, firmware, and hardware components as critical capabilities for any SCRM (Supply Chain Risk Management) program. SCRM is already a significant challenge for organizations, but as the threat landscape evolves, it simply won’t be enough for leadership to consult an SBOM to address provenance and track vulnerable code libraries throughout the enterprise. SBOM, HBOM, and FBOM tools will serve to address a part of the challenge but will be unable to address all the factors that tie into decision-making related to risk management. No device exists on its own; it is a part of a larger system of devices working together, and therein lies the rub. Should an FBOM reveal a malicious or vulnerable component in one device make and model, one cannot simply decide to remove it from production or not procure it. Instead, dependencies and device interoperability factor in.
The moment enterprises are forced to take security control over their environments instead of implicitly trusting vendors in their supply chain, they become further responsible for making sure related decisions don’t impact uptime and productivity.
This may extend down to the component level as well. Put differently, the same complexity and implicit trust challenges that attackers have begun to leverage to their advantage, are the same ones that will make addressing these challenges all the more difficult. Thanks for that, John…we knew we were in for trouble when we asked you for your input!
The barrage of critical vulnerabilities found in infrastructure systems is unlikely to subside, and attackers will continue to run rampant as defenders struggle under the weight of technical debt accrued by the vendors they’re expected to rely on
We’ve mentioned how prevalent and challenging vulnerabilities in the technology supply chain are, but this insight from Eclypsium Director of Threat Research Nate Warfield calls out a specific dynamic that will continue to play out well into the future. Today’s technology supply chain vendors are carrying a technical debt in the form of lax secure development practices and cyber hygiene. It is difficult for even the most well-resourced organizations to address the technical, process, and cultural change management required to ultimately ship secure devices and firmware updates over time.
What is more, the industry is one of the most low-margin, competitive, and fast-paced in the world. As mentioned earlier in this report, it’s only going to get more competitive as chip manufacturers race to design and produce the next generation of computing hardware. Just in the last few weeks the biggest vendors in the industry (Intel and AMI) routinely exhibit flaws (Lenovo) in the most critical firmware (Acer) on the devices they ship, or even have their UEFI source code leaked (Intel). The technical debt Nate mentions takes many forms; everything from securing production environments, static and dynamic code analysis, QA, building security requirements directly into the firmware SDLC itself, bug bounty and vulnerability management programs. You know, the very same challenges we are familiar with already in the context of application development; only more complex, more interdependent, and with far less expertise across industry.
One of the best corollaries to Murphy’s Law is; “Everything that can go wrong, will…and it probably has, you just don’t know it yet”. This is something the USMC instills in every rifleman. Once that becomes the mindset, it allows one to anticipate encountering the things that have gone wrong when they are discovered, and to know how each wrong thing should be handled in advance.
Vlad Babkin, a threat researcher on the Eclypsium team, regularly encounters an entire universe of unknown unknowns taking the form of vulnerabilities, backdoors, implants, stolen and exposed source code, signing certificates, and much more beyond. This applies to vulnerabilities, but also to tactics adversaries might be using already in the wild we simply aren’t aware of, as Eclypsium’s Nate Warfield endeavored to explore in recent research demonstrating how attackers can commandeer and persist indefinitely (even surviving reboots and firmware updates!) on F5 load balancers and Citrix devices:
“The techniques used are within reach of an average attacker, utilize readily available open-source tooling, and are only detectable from the advanced administrative shell; they are invisible to the web management interface and restricted shell.” – Nate Warfield, Eclypsium Dir of Research.
Case in point: If a single researcher can discover techniques an average attacker can exploit using freely available tools, how do we know if (and for how long) these techniques have already been used in the wild? We don’t. And that’s the whole problem. We only get to learn of campaigns well after victims have been compromised, and organizations like the NSA put out threat-hunting guidance specific to one actor’s TTPs, such as this recent guide on hunting for APT5 activity actively targeting the very same Citrix ADC’s that Nate researched. APT5 is a China-nexus actor that routinely targets routers and gateways, hitting telecommunications and technology companies in both the US and SouthEast Asia.
As the industry (and even the DoD) begins to finally provide true zero trust level visibility into just how exposed and compromised our devices are, we will all be stunned at what we find. Already we know that threat campaigns targeting device firmware often run for many years prior to their discovery in the wild. In those cases, it has taken an almost serendipitous confluence of an incident with specialized forensics and even luck, to find such threats. Indeed this is what such attackers have been relying on entirely, and is the reason they have gravitated to low-level tactics that evade the current-day cyber security stack.
All that is about to change going forward. Solutions exist today that allow enterprises to prevent and detect device firmware-level attacks. These solutions will only get better over time, and we will all soon come to realize just how prevalent and nefarious device-level threats have become. The more organizations leverage them, the more the collective telemetry and insight will illuminate the true nature of these attacks.
Welcome to the Supplyocalypse. We did warn you, and we hope you were sitting down as you read these. They aren’t the typical generic cyber predictions cast about this time of the year. These are a true window into the future of where we are headed. We know, we’re from the future, and we’ve been building solutions for it for five strong years now.
The good news is there is a tremendous amount of energy, resources, new requirements, and board-level interest in this area. If we can anticipate such future threats, we can begin today preparing ourselves as both professionals and organizations.
