Large-scale ransomware attack via managed service providers likely to have significant impact

In or around April 2021, researchers working with the Dutch Institute for Vulnerability Disclosure (DIVD) identified multiple high severity vulnerabilities in Kaseya Virtual Server Administration (VSA), a tool commonly used by MSPs to manage virtual servers used by their downstream clients. These vulnerabilities, collectively tracked under CVE-2021-30116, were reported to Kaseya, which in early July 2021 was reportedly in the process of developing a patch.

On 02 July 2021, an unknown group began bulk exploitation of these vulnerabilities, reportedly targeting on-premises VSA installations. The majority of these servers were reportedly operated by MSPs and used to manage hosts on behalf of their customers. Sophos reported that following successful exploitation the VSA agent update service, used to push software updates to VSA agents deployed on MSP client endpoints, was used to push a malicious executable (agentmon.exe) and certificate (agent.crt) file to Windows hosts running the VSA agent. As the VSA servers were operated by MSPs, each compromised host was able to push the malicious files to all client hosts managed via that server. These files were deployed to the VSA working directory, which Kaseya advises allow-listing in host security tools to prevent disruption of important processes by false positive detections.

Following deployment, the VSA agent ran a one-line command via a Windows command shell. This concatenated multiple individual commands to achieve the following: 

1. Wait for c. 94 minutes before any further action.
2. Disable Windows Defender security features using PowerShell. PowerShell v1 was used, likely to prevent security features in more recent versions from preventing successful execution.
3. Copy certutil.exe from its default location (C:\Windows\System32) to the VSA working directory.
4. Modify certutil.exe with a randomly generated five digit string, potentially to prevent signature-based detection of malicious certutil usage.
5. Use certutil to decode agent.crt, a base64 encoded malicious executable, and save this file (agent.exe) to the VSA working directory.
6. Delete agent.crt and the modified copy of certutil.
7. Run agent.exe. Because this executable was run by the VSA agent, it inherited this process’s SYSTEM level privileges.

After being run, agent.exe downloaded and ran an outdated version of Microsoft Malware Protection Engine (msmpeng.exe) which is vulnerable to DLL side loading, a technique which can be used to force legitimate binaries to run malicious DLLs. Agent.exe wrote a malicious DLL (mpsvc.dll) to the same folder as msmpeng.exe and then ran the legitimate executable, resulting in it running the malicious DLL. This DLL encrypted files on the host and dropped a ransom note, then modified host firewall rules to allow detection of other networked machines. Sophos have published IoCS for this attack chain, which are available here. At the time of writing, it is not known how ransomware propagated from hosts running the VSA agent to other networked machines.

At an unknown time on 4 July 2021, the Tor hidden service used by REvil affiliates to announce successful attacks and host data stolen from victims was updated with a statement claiming responsibility for the Kaseya operation. The statement claimed that the operators responsible will release a “universal decryptor” which can purportedly be used by all victims to recover encrypted files in exchange for a payment of $70 million in Bitcoin.

While the full impacts remain unknown, due to the number of victims this operation has almost certainly had substantial disruptive impact. The fact that the attack was launched on 2 July 2021, immediately prior to a federal holiday weekend in the United States, is likely to have exacerbated this, as many IT and security staff would have been unavailable to respond. The technical elements of this operation were executed (and were very likely informed by an understanding of how Kaseya VSA works) at both the server and client level.

The operators responsible developed a well-designed payload deployment and execution flow which took advantage of the high-level privileges with which VSA agents run and the common practice of excluding the VSA working directory from host security tool detections to run ransomware on a large number of hosts in a manner which minimised the likelihood of detection. This was highly likely designed, developed, and tested ahead of time. Due to the bulk deployment method used it is unlikely that the operators established engaged in manual lateral movement on most compromised networks, but it cannot be ruled out that priority targets were chosen.

This contrasts with the monetisation element of the operation, which was poorly designed and likely created significant monetisation challenges for the group responsible. Concurrently handling communications and payment processing for 1,500 victims is likely to be beyond the capabilities of the group responsible, particularly given that the payments site has remained unavailable following the attack. This means that even victims who do attempt to pay their individual ransom are unlikely to promptly receive decryption keys. Demanding a single ransom payment for a ”universal decryptor” is also unlikely to result in actual payments, particularly given the high fee; the victims of this operation are very unlikely to start pooling funds to make a collective payment.

There are several realistically possible explanations for the decision to simultaneously deploy ransomware to the networks of the compromised MSPs’ customers, as opposed to a payload which would enable persistent covert access which could be exploited over a longer time horizon. These include the possibility that the operation was executed at short notice, potentially due to awareness that a patch release for CVE-2021-30116 was imminent, or that the group perceived that it did not have the resources to concurrently manage such a large volume of network intrusions. At the time of writing there is insufficient information to enable an assessment of which of these possibilities is more likely.

Organisations which operate on-premises VSA deployments have been advised to shut down these servers until patches for CVE-2021-30116 are available. Kaseya has released a set of PowerShell scripts which can be used to search across VSP servers and managed endpoints for evidence of compromise associated with this activity. MSPs which use Kaseya VPS and their customers are advised to use these tools to identify potentially compromised hosts even if managed hosts are not showing obvious evidence of ransomware infection; it is possible that payloads were deployed but did not properly execute.

US CISA has released a more general set of security guidance around manging MSP network access, which is available here.