Part 1: On Being a Sausage Expert
The last 12 months have seen a tremendous focus on securing digital supply chains. Not because we’ve grown bored with our day jobs but because we’ve had to. For example:
- The supply chain for Pulse Secure network devices was exploited through known firmware-level vulnerabilities in VPNs, routers and switches, and the attackers were effectively able to create their own “supply chain effect” by identifying end users who had employed this gear.
- The supply chains for ASUS computers were compromised by ShadowHammer, where the manufacturer’s Live Update Utility was exploited to deliver properly signed, yet entirely malicious, updates to more than 57,000 downstream users.
- The supply chain for Fortinet network devices were targeted heavily in 2022 impacting cloud-based, on-prem and virtual networks around the world.
- The SolarWinds supply chain was compromised through SUNBURST, a sophisticated attack that implanted malicious code within the source code of SolarWinds “Orion” IT Monitoring and Management platform. This compromised code was subsequently published and delivered to more than 18,000 federal and private sector customers including a variety of leading technology companies.
- A common supply chain component called Log4j was attacked via LOG4SHELL, a security flaw so widespread that most organizations are still unaware they’re impacted. Security researchers have identified approximately 10 million Log4Shell exploitation attempts every hour led by attack on retail, technology, financial services and manufacturing.
We’ve had to focus our security efforts on digital supply chains because our adversaries – whether nation states or criminal hackers or next-gen script-kiddies – discovered the easiest path to money, influence and power was through a weak spot in our digital culture. As a report by the U.S. Departments of Homeland Security and Commerce said in a joint report, that weak spot was in our technology supply chains.
In Part 1 of this series, On Being a Sausage Expert, we explained how even though we can’t all be experts in the making of sausages and fine meats, we can start thinking like experts. We can use SBOM technology to commonize our language about ingredients, dependencies and reputation, and at least deliver a report that attests to what we know to be true.
Span of Control
Technology supply chains – especially those that support design and delivery of digital devices – are vast to the point of being indescribable. They cross international boundaries, ideologies, and legal jurisdictions. While cybersecurity teams enjoy privilege and responsibility in most digitally-oriented organizations, there are limits.
“Span of Control is the area of activity and number of functions, people, or things for which an individual or organization is responsible.”
“Span of control” is a business management term that refers to the degree of influence individuals have to change the course of a business or its strategies. Each individual has a certain span of control, as does each team, each department, and each internal organization. Some of these are absolute and cast in stone, like internal audit or governance processes and the teams that define them.
But for most of us “span of control” is a constantly-shifting lens that interprets for us “What we can do” as well as “What we can’t do.” When tectonic changes occur in our area of activity (and for a primer on “tectonic changes” you can review the five bullets at the beginning of this post) we have to make conscious decisions about what we’ll attempt to pull under our span of control.
When it comes to the devices we use in our networks, whether endpoints, servers or networking equipment, most of us cannot in all likelihood:
- Fire our vendors
- Force a change to more expensive but more traceable gear
- Force a change to more expensive but more reliable gear
- Advocate for an out-of-band replacement of millions of dollars of computer equipment
- Propose a true “Zero Trust” device approach that includes default-deny postures for any gear that has questionable manufacturing provenance
In the best of times, cybersecurity teams don’t have that sort of Span of Control.
We also cannot enforce oversight and cybersecurity signoff on the supply chain in all its entirety, from obscure sub-suppliers who are writing firmware in Indonesia to equally obscure Chinese component manufacturers who may have questionable motives or allegiances.
What You Can Do
With large, convoluted and sprawling problem spaces like digital supply chains, it’s best to remember the advice of Bishop Desmond Tutu, the first black African Bishop of Johannesburg, the first black African Archbishop of Cape Town, and the winner of the Nobel Peace Prize in 1984. Among many brilliant things he said about human nature, society and theology was this gem:
“There is only one way to eat an elephant: one bite at a time.”
As big and seemingly uncontainable as the problem of securing digital supply chains might be, it’s not as big or as rooted as the problems created by the practices of apartheid. Cybersecurity practitioners responsible for securing devices, specifically, can take a similar approach by securing the smallest element within their span of control: the firmware that enables, instructs and empowers their devices.
For a more detailed discussion on how firmware is the linchpin to securing digital supply chains, readers can review our white paper, Need Secure Supply Chains? Start With Their DNA. But if we agree firmware represents the lowest, most atomic unit of “control” we can exert over our supply chains, a focus on controlling that firmware can give us the confidence we desire, despite the seeming enormity and complexity of the overarching problem.
Like Bishop Tutu’s four standards, we can do this by naming the problem – securing device supply chains by securing and assuring the hardware they depend on – if we break the Herculean task down to just three new standards we must adopt:
- Identify our device firmware
- Verify our firmware’s authenticity and integrity
- Fortify our firmware as needed
Unfortunately, many cybersecurity strategists and practitioners have mental blocks on the subject of “firmware.” The first thought that comes to mind regarding firmware is not usually, “That’s something I can manage.”
It’s more often, “That’s the manufacturer’s problem” or “that’s beyond the scope of my tools” or “our operations teams don’t want us to access that.”
How We Can Help
But firmware is becoming a part of the mainstream conversation. The joint report report cited earlier, authored by the Departments of Homeland Security and Commerce, said this about the firmware in our modern devices:
“Firmware presents a large and ever-expanding attack surface, as the population of electronic devices grows.
“Securing the firmware layer is often overlooked, but it is a single point of failure in devices and is one of the stealthiest methods in which an attacker can compromise devices at scale.
“Over the past few years, hackers have increasingly targeted firmware to launch devastating attacks.”
Eclypsium has, of course, built the Eclypsium Platform, the world’s first security solution purpose-built to identify, verify and fortify the low-level firmware and software embedded in the supply chain’s chips, processors and technology components.
Identify – Discover, inventory, and classify your organization’s devices and device-level firmware whether in endpoints, servers, network devices, or embedded and hard-to-track IT supply chains
- Gain visibility: create a firmware inventory across the enterprise
- Include firmware-level inspection of servers, VMs, endpoints, networked devices, connected devices
- Assess across vendors of all kinds
- Gain deep insight into hardware and software supply chains
- Assure IT has visibility down to the sub-OS hardware component level
Verify – Validate current firmware and device configurations by comparing them against a database of known-good and up-to-date hardware and firmware profiles
- Verify all devices for current firmware versions
- Verify for appropriate and recommended firmware configurations
- Maintain an always up-to-date database across the entire firmware ecosystem
- Enable automated device scanning, analysis, and reporting of embedded code
implants and back doors
Fortify – Patch, configure, update and repair firmware as needed.
- Automated remediation of vulnerable or misconfigured firmware components
- Hardens systems in conjunction with existing patch, remediation, and threat analysis tools
- Intelligent automation helps answer the questions “should I update” and “how do I update”?
- Coordinate investigation and response via REST API
- An update is available, but should you apply it?
- Work with existing deployment and asset management tools, like Microsoft SCCM, Tanium, and BigFix
Heading Upstream
Bishop Tutu said and wrote a lot of inspirational and thought-provoking things, on a myriad of different subjects. Without taking too many liberties, we think one of those aphorisms is uniquely applicable to the problem of securing modern technology supply chains:
I’ve no doubt Bishop Tutu was talking about societal failings when he said this: why people resort to crime; why rates of recidivism are so stubborn; how social power is more often abused than shared.
But it applies to our problem of securing digital supply chains just as easily. How many of the supply chain exploits bulleted at the beginning of this post could have been avoided – or at least managed and controlled – if we had made a practice of Identifying, Verifying and Fortifying the firmware embedded in our devices? It’s not necessarily a small set of tasks, but they’re manageable and measurable.
If we go upstream and focus on firmware, we might find fewer organizations have fallen into the River of Supply Chain Breach. If we do a good job at these smaller tasks we might find, eventually, that we’ve eaten the whole elephant.
The first article in this series, On Being a Sausage Expert, can be found here.
