The SolarWinds attack and best practices for code-signing

Since the announcement of the SolarWinds supply chain attack, intensive analysis has been done by Crowdstrike, FireEye (with additional details), Microsoft, Symantec, SolarWinds, and many others, to understand the attack’s workings both within SolarWinds and in the targeted networks. Here we focus on the code signing procedures, which seemingly failed at SolarWinds but likely could have mitigated the risk of the attack if they had been implemented and enforced to a higher standard.

To recapitulate what is known so far: Intruders placed the so-called Sunspot malware into the SolarWinds system. The Sunspot malware was used to monitor and hijack the build process of the SolarWindsOrion app. This way, at compilation time, source code file content was replaced with a version containing the Sunburst malware.

The resulting executable (called SolarWindsOrion.Core.BusinessLayer.dll) was digitally signed by SolarWinds on March 24, 2020. Along with Orion updates between March and June 2020, the Sunburst malware then infiltrated over 18,000 government and private networks. The malware gathered info on the infected networks and sent data to a remote server. At selected targets, Sunburst downloaded and installed the Teardrop or Raindrop malwares, which subsequently allowed the attackers to start a human-operated attack.

As part of the hands-on keyboard attacks in the infiltrated networks, the attackers leveraged X.509 certificates to create legitimate OAuth access to the networks, or they forged SAML tokens and signed them with legitimate, compromised certificates to impersonate trusted users and accounts. 

Code-Signing

Apparently SolarWinds had code-signing in place, and possibly followed some of the best practices, such as: 

• holding code-signing keys within a FIPS 140-2 level 3 hardware security module (HSM) to ensure the strictest security of all key material,  
• secure workflows for the signing process, involving several users for necessary approvals before signing (“endorsed code-signing”), 
• using policies to rigorously control from a central service (Policy Engine) who can use which keys incl. parameter choices, and
• keeping tamper-evident logs to record who used a key and when and for what.

It is being reported that SolarWinds’ code-signing certificate was not compromised at the time of signing the tampered code. (Note, though, that SolarWinds is digitally re-signing products and the existing certificate is scheduled to be revoked on March 8, 2021.)

What seems to have failed at SolarWinds was the integrity of the in-enterprise supply chain: there was no coupling between the source code repository and the code signing system. The bad code was injected outside of source control before the source code was being read by the compiler. If the code signing system has no knowledge what it signs, it does not help to have the signing key secured in an HSM. 

For robust code signing, the code signing system must validate that what it is signing matches what is in the source code repository:

• Any code, or code revision, that a developer checks in must be digitally signed by that developer using their own signing key. 
• During the build process, all developer signatures are to be verified, thereby executing integrity checks on every single module used to build the final product. This makes sure that only legitimate code is being used. 
• Only with all these checks being successfully completed, the final build is to be signed following the customary code signing schemes. 

This would ensure the integrity of the source code that the developer checks in, up until the final build and linking of the delivery, and prevent any adversary in the network from inserting their own malicious code.

All of the above can be achieved, for example, by deploying Cryptomathic’s Crypto Service Gateway (CSG), and integrating Cryptomathic’s Code Signing Platform (powered by CSG) with the build environment.