The Precipice

Eclypsium Threat Report September 2022 Firmware

Check our discussion of this between Paul Asadoorian and Scott Scheferman.

The world is on the precipice of significant economic and societal change. This winter in Europe may end up seeing the worst energy crisis in 50 years, an all encompassing crisis that expands beyond just the fuel shortages of the 70’s, to include natural gas, nuclear, and solar energy challenges never before seen concurrently. Needless to say, the war between Russia and Ukraine/ NATO allies has also escalated to a level none of us want to even contemplate, with open threats of nuclear strikes on the table from Russia, along with promises to attack the energy infrastructure of Ukraine and its allies. The energy crisis is quickly becoming a global one, even tapping the US’s energy reserves by more than 25% already.

What does this mean for the broader cyber landscape as we head into this uncertain future?

Quite simply, all major OT environments in affected geographies will be tested given the extremely thin margins, just-in-time supply chain, and heavy reliance upon energy. Margins will be squeezed and businesses will be faced with the decision to close shop or remain operational. Against this pretext, cyber security will suffer, as it has historically during economic downturns. Opex costs in the form of subscription cyber services will be reduced or cut completely. Human resources, including high-cost specialized cyber roles, will be trimmed.

In Europe there will be an uncertain mix of strength through unity, and state-to-state competition to protect energy resources and advantages. These strains may hamper cyber threat intelligence sharing between states and same-vertical industries that would otherwise engage in “coopetition” and sharing of cyber threat intelligence.

To frame the level of intensity best, we might turn to these words from Ukraine’s Directorate of Intelligence, Ministry of Defense:

“The Kremlin plans to carry out massive cyberattacks on critical infrastructure facilities of Ukrainian enterprises and critical infrastructure institutions of Ukraine’s allies. First of all, the blow will be aimed at enterprises of the energy sector. The experience of cyberattacks on Ukraine’s energy systems in 2015 and 2016 will be used when conducting operations.” (Earily similar time frame to those of when Russia last attacked Montenegro too, and they’re at it again there, too)

A day after this warning, explosions ruptured the Nord 1 and 2 pipelines between Russia and Europe. Even though both lines were not sending fuel to Europe, the lines were both pressurized, sending gas into the baltic sea and kicking off a whirlwind of ‘who dun it?’. Fingers have been pointed at Russia, Ukraine, and even the United States, even though the US had warned Germany about the potential for such an attack months ago. Yet, here we are: a (highly likely to be) state actor has used destructive force against a strategic element of the ensuing energy crisis. That thin red line has been crossed, and cracks the door wide open for other types of attacks, including cyber, to play out against allied energy targets. While there has been much written about the need for Russia’s espionage capabilities under SVR needing to turn even more towards cyber tradecraft, what we must always remember is that today’s espionage campaign can quickly turn destructive, even targeting the same victim footprint. The most likely espionage targets will be those that support the Kremlin’s re-vamped focus to rebuild its own ICT supply chain from the ground up in order to counter the risk of their own reliance on Western suppliers. In turn, those same ICT suppliers end up being potential destructive targets when and if the time is right.

When it comes to the Energy Sector, CISA has been warning of attacks against for months, right down to the TTPs (Tools Tactics and Procedures) certain Russian adversaries (in this case the FSB) are known to have already used for over a decade. Importantly, these actors are those behind the Trisis campaign, a specifically destructive attempt to disable industrial safety systems to make them appear to be functioning properly even after disabling them. They are also the actors behind the Havex malware campaigns as far back as 2013. Recall that this campaign poisoned software updates via ICS supply chain vendors. Recall, too, that these campaigns often began with DNS redirects to malicious websites. Finally, recall that Havex was really good at pulling VPN configuration files. Regular readers of this BtS threat report will understand the significance of that tactic. And this was a decade ago. Further, anyone reading this in the energy sector may want to review one of the single best sources for uncovering the TTPs of any threat actor: unsealed indictments on the DOJ website.

Meanwhile, Sandworm, the very same group behind NotPetya and the 2015-2016 power grid attacks referenced in the Ukraine MOD quote above, is displaying heightened activity, setting up for ensuing campaigns by masquerading as telecoms in order to facilitate subsequent spear phishing and redirection campaigns. You guessed it, just like the FSB, the GRU also relies on tried and true tactics like redirected websites. They are also masters of targeting third-party infrastructure to conduct spear phishing, watering-hole, and supply chain attacks against vendors and service providers in order to target their downstream Energy victims.

And, what of criminal organizations pledging allegiance to Putin that might provide the Russian government with access to key energy targets? Conti (now disbanded only in name but not in the relentless activity of its past and new members acting under different names and groups), is notorious for hacking the energy sector. Recall when they attacked SEA-Invest, causing all 24 sea ports handling oil and gas liquid bulk to cease operations this past February. Or when they hit CS Energy 10 months ago, one of Australia’s largest electricity providers, who narrowly escaped operational impact. In the same month they also hit a large Italian Natural Gas distribution company. Although a murky relationship, some researchers have also tied HIVE ransomware (known to primarily target the energy sector, including 186 attempts to compromise such machines) to Evil Corp and Conti affiliates. The head figure of Evil Corp is of course none other than Maksim Yakubetssaid to have worked for the FSB sometime after 2018. 

Perhaps it is no surprise that when we focus on Critical Infrastructure threats in the Rewards for Justice bounty page, the only named actors are six members of the prolific Conti organization, who have amassed over one thousand ransomware victims:

While the latest activities out of Russia highlighted above are compelling reasons for every reader to take imminent destructive threats seriously, they only broach the surface of what the collective Russian cyber threat is capable of. This well-written piece by the Center for European Policy Analysis (CEPA) lays out Russia’s long history of cyber capabilities, and the past and present structures and organizations that support them. One particular observation resonates with what many cyber professionals closer to the region have often said; it isn’t just the formal SVR, FSB, and GRU components, nor their dotted-line ties to cyber criminal apparatuses like ransomware and fraud groups that is of concern. It is also the cadres of engineers and cyber experts whose pedigree extends back to Soviet era polytechnic training. While many of these engineers and their families have fled and taken their talents elsewhere or have even started their own cyber security companies, many still remain in-country and active. Some of them may be considered some of the finest talent Russia has in the area of offensive cyber capability. They represent talent that simply cannot be underestimated. These cadres and the body of knowledge and tradecraft extending from their lineage, help Russia maintain some of the most talented cyber security hackers in the world. Put differently, in the most competitive and sophisticated CTF (Capture the Flag) events held in this broader region, the cyber talent exhibited is at a level rarely observed elsewhere. We cannot afford to under-estimate what may come should this talent pool be tapped for attacks against NATO and the West, to include what might be possible should they turn their sights on the ICT supply chain and firmware attack surface.

So what is the West going to do about all this? In a rare twist, the US government seems to be well ahead of industry, perhaps for the first time in cyber history. That should tell us something. Whether it’s the incredibly detailed and actionable alerts and guidance from CISA, or whether it is the US Executive Office putting out a much needed and forward-leaning executive order designed to better protect our ICT supply chain from software and firmware threats. The United States government is keenly aware of these threats, and is asking all of industry to step up their game. In fact, they aren’t just asking, they are commanding us to do so.

Recall in February the US Government finished an extensive 1-year study into our ICT supply chain risk. The results were nothing short of eye-opening for many and appear in this 96-page report highlighting how:

“Firmware can also be a lucrative target with a relatively low cost of attack. Over the past few years, hackers have increasingly targeted firmware to launch devastating attacks.”

They found that firmware presented “a large and ever-expanding attack surface” that threatened the core of modern computing itself. “Securing the firmware layer is often overlooked, but it is a single point of failure in devices and is one of the stealthiest methods in which an attacker can compromise devices at scale. Attackers can subvert OS and hypervisor visibility and bypass most security systems, hide, and persist in networks and devices for extended periods of time while conducting attack operations, and inflict irrevocable damage”.

It was not a surprise that NIST soon after released formal guidance requiring attestation from ICT and device vendors (and even formal “show your work” type proof for critical systems), that the software and firmware they are shipping (to include bundled Open-Source libraries and software) has been developed securely and is without backdoors and vulnerabilities. As of September 2022, however, this guidance has now become policy per a new Memorandum M-22-18, “Enhancing the Security of the Software Supply Chain through Secure Software Development Practices.” The word ‘shall’ appears 19 times, and directs NIST, OMB, CISA and Agencies to comply within specified periods of time. To that end, our readers will be happy to learn that Eclypsium is now the first and only vendor on CISA’s Continuous Diagnostic and Monitoring Approved Product List able to address these low-level supply chain firmware risks. Agencies will finally be able register risk for the firmware attack surface and couch it against mission risk. This bodes well for commercial enterprise and critical infrastructure too, who will benefit from vendors needing to comply. The stage has finally been set, and the clock is now ticking formally for all parties. 

Stepping back to understand how we will collectively solve the ICT supply chain challenge, we must realize that there is no silver bullet. In the end this all boils down to a software complexity problem. The complexity is multi-faceted beyond compare. It isn’t just about the devices themselves. It’s a problem centered on the broader definition of what we mean by the word ‘trust’, as fully explored in last months’ BtS report. A large portion of that trust has to do with the collective bundle of software that comes from any given OEM. Bundles include firmware as well as the drivers and OEM flashing tools often packaged to support the development and maintenance of that hardware over time. Just like firmware, these drivers and tools present exceptional risk to mission and enterprise, and are commonly leveraged by adversaries to escalate privileges by leveraging the inherent trust that user operating systems place in them. This month, an arbitrary code execution in select ACER DXE drivers was discovered, allowing an attacker to escalate from userland to below-the-kernel privileges. In other news, hackers have been bringing along a driver used by a popular Chinese video game to perform functions they couldn’t do otherwise. The driver is a low-level anti-cheat driver, but is easily abused by an attacker. The attacker only needs to bring the trusted driver module to a victim host. For an entire expose on how and why the intersection of video games and low-level attacks are connected, this is a must-read blog by one of our researchers. And all-too-ironically, attackers leverage vulnerable anti-virus drivers to disable other anti-virus products, as Allan Liska explains in this video.

As a quintessential example of just how pervasive these drivers are, new research on DPRK’s Lazarus group highlights how effective leveraging these signed drivers can be. In this case it was a hardware-related module made by “ENE Technology” that used an old OSS library called “WinIO,” developed by Yariv Kaplan in 1999. In closed-circle DFIR trust groups, this module has been known to be targeted at least since 2015 by Chinese and other actors as well during high-profile public and non-public incidents. From first hand experience, know that it can be found in just about any environment of any size or operation. It was kept under wraps in the DFIR community for years for one reason: we considered it a ‘forever-day’ that would be ever-present in critical infrastructure environments, and we didn’t want more actors to become aware of it by exposing its use publicly. Clearly, that ship has now sailed. It’s used by many dozens of hardware manufacturers. It’s everywhere, including sensitive ICS systems found in production environments in many sectors. Even Russian robots and bitcoin miner applications use it. It’s been a ‘cheat code’ for OEMs and OSS project devs for decades, and still is. When it comes to abusing trusted 3rd party drivers, there isn’t a single class of actor that doesn’t leverage this tactic, whether the actor finds the driver already in the environment, or brings it with them (now known as BYOVD (Bring Your Own Vulnerable Driver)).

At an even higher level, drivers are a massive part of the supply chain cyber challenge. A few years ago Eclypsium researchers debuted their “Screwed Drivers” research, and the rest is history, as they say. It’s become a core part of what Microsoft is trying to solve in Windows 11 via WHCP (Windows Hardware Compatibility Program), where they check 3rd party drivers for both compatibility and malware. Yet, note that neither compatibility nor malware are the real challenge here: it’s the vulnerabilities and raw functionality of the drivers themselves that present the threat attackers are leveraging. It isn’t just a Windows problem either. There are plenty of OSS tools that allow attackers to modify BIOS settings from the Linux OS, like this one.

The title of this month’s BtS is “The Precipice”. Ironically, as we bring this month’s report to a close, word of active in-the-wild campaigns targeting new MS Exchange 0-days are emerging. It turns out they’ve been used for over a month already. Worse, unlike earlier Exchange-based campaigns, the vulnerability in question is pre 2FA auth, removing a step entirely for the attacker. Finally, there are many hundreds of exposed Exchange servers, including many that are a portion of hybrid customers that would otherwise be protected by subscribing to Exchange Online services. CISA has now added both vulnerabilities to their Known Exploited Vulnerabilities (KEV) database. 

This, just as word of a new Lazarus campaign emerges leveraging a low level CVE (once again in an OEM supply chain driver!) that hadn’t yet been observed being exploited in the wild. The DPRK-backed hackers are using a rootkit named FudModule.dll, which modifies kernel variables and removes kernel callbacks in order to disable monitoring of all security solutions on the system. The vulnerability being exploited to do this is CVE-2021-21551, found in Dell DBUtil drivers. These drivers are found in hundreds of millions of devices since 2009. 

Speculatively speaking, let’s all hope they aren’t combined with these new MS Exchange 0-day campaigns currently ramping up. MS Exchange blue teams have gotten much better at detection and incident management from the last recent scourge of Exchange attacks. Attackers will need to be even better at evasion. Where might they turn to buy themselves more time and  evade revamped blue team capabilities to detect them? Disabling kernel callbacks would be a good place to start. Or, they could pivot to targeting BMCs (Baseboard Management Controllers) to persist underneath the operating system. Or they could look for BiosWE vulnerabilities like the Russian GRUTrickbot, and other groups do. (An absolute must-read this month is Paul Assadorian’s blog on SPI flash write protections!) Or, if they really want to ratchet up, why not go from a compromised Exchange server, grab AD creds, and pivot to vSphere to embed into a VIB via one of these latest techniques to evade the rest of the entire security stack if Secure Boot isn’t enabled? By targeting ESXi infrastructure and using malware that infects and transfers files either direction to the underlying VM’s, attackers would gain significant advantage. In practice many ESXi and vSphere servers don’t run EDR or even much logging, buying attackers even more time. Imagine an attacker being able to execute arbitrary commands from one guest VM to another guest VM, both running on the same hypervisor. Or, since we are speculating, imagine an attacker being able to use OEM tools (akin to the ones mentioned earlier in this report) from within these signed malicious VIBs that would allow them to interact with or even flash firmware, or ones that allow interaction with IPMI/BMC’s. Either scenario would allow an attacker to move even further down the stack and embed below the hypervisor. 

Indeed, we are upon the precipice: Of even more evasive techniques that circumvent the entire security stack. Of indefinite persistence, below the hypervisor even. Of the destructive firmware element DHS describes as having devastatingirrevocable damage potential. And of an escalation in cyber warfare, just as Ukraine plans to join NATO, placing the entire war itself upon this new precipice.

We as defenders are also on the precipice of mandatory change. Change that is further born out of necessity alone when it comes to visibility, 3rd party attestation, and detection of vulnerabilities and threats at the ICT supply chain firmware level. For a quick and dirty summary of how the September 14th White House memo M-22-18, (which directs federal agencies to comply with the NIST ICT supply chain guidance that has firmware scoped in), check out this blog. After understanding that context and its mandate, head here to learn how Eclypsium helps address this challenge for enterprises and missions alike.

We are on the precipice, and it’s now go-time for us all.