Twilio Suffers Information Leakage from Tampered Codecov’s Bash Uploader Tool: Supply Chain Attack

Twilio, a provider of cloud communications, has revealed that it was affected by the recent Codecov software tampering supply chain attack.

It turns out that Codecov, a popular code coverage tool, was actually the victim of a supply chain attack for two months.

What is a supply chain attack?

Over the past two months, attackers have been tampering with the legitimate Codecov Bash Uploader tool to leak environment variables (including sensitive information such as keys, tokens, and credentials) from Codecov customers’ CI/CD environments.

The attackers who attacked Codecov reportedly used credentials obtained from a tampered Bash Uploader to infiltrate the networks of hundreds of customers.

Twilio: Email addresses of a small number of customers compromised

Twilio, a cloud communications and VoIP platform, announced that it has been affected by the Codecov supply chain attack.

In April 2021, shortly after we published a security incident regarding Codecov’s Bash Uploader, we were notified that Twilio was also affected.

Twilio, however, says that the malformed Bash Uploader component is actively used in a small number of Twilio projects and CI pipelines, and is not related to any critical systems.

“Projects and CI pipelines that use this tool are not on the critical path to provide updates or functionality to our communication APIs. Our subsequent investigation has revealed that this resulting small number of email addresses was likely exposed by an unknown attacker. We have avoided further potential damage by personally notifying the affected individuals and allowing them to thoroughly review and correct their potentially exposed credentials.”

“Although we were not able to prevent the security breach caused by this supply chain attack, to prevent inadvertent security leaks, we run an internal service called Deadshot that scans GitHub pull requests. The service scans pull requests in real time to identify secrets and other common insecure coding practices in code being merged into GitHub. If insecure code is found, Deadshot notifies the user who made the pull request and notifies the Product Security team if a specific type of secret is found. This allows developers to remove or change the code before merging it into GitHub.”

“In addition, the product security team maintains a tool in our environment to perform static application security testing. This tool scans for secrets in the code and looks for security issues such as insecure coding practices and vulnerabilities such as OWASP Top 10. In this way, if there is unauthorized access to the GitHub repository containing our code, we limit the further damage that could be caused by such unauthorized actors.”

“Again, at this time, with the exception of a few email addresses, there is no indication that any customer data has been accessed and is not at risk. We also do not foresee or anticipate any problems with the availability or functionality of Twilio’s products. If there are any changes, Twilio’s security incident response team will keep you updated on this page. If you have any further questions, please contact your customer support partner.”

and Twilio commented.

Twilio’s Response to the Recent Codecov Vulnerability
On April 15, 2021, Codecov publicly disclosed a security event with the Bash Uploader component. Here's how it affected Twilio customer data.

Email address found in GitHub repository

On April 22, GitHub detected suspicious activity related to Codecov and notified Twilio that Twilio’s user token had been exposed.

“ had identified a series of GitHub repositories cloned by the attackers before we were notified by Codecov,” Twilio reports. Twilio reports.

Twilio’s security team discovered “a small number of email addresses belonging to Twilio customers” in one of these GitHub repositories, but the exact nature of this “small number” has not been disclosed.

Twilio has stated that at this time there is no indication or evidence that other customer data has been compromised, nor has Twilio’s repository been altered in any way by the attackers.

Twilio is also taking steps to detect such incidents in the future, such as scanning GitHub pull requests in real time to discover publicly available secrets and general insecure coding.

In fact, Twilio was not the only company affected

Twilio isn’t the only company affected by Codecov’s supply chain attack.

Last month, HashiCorp announced that its GPG private key had been exposed in an attack.

This key was used to sign and validate software releases, so it was necessary to update the key.

Before Codecov discovered the attack, Bash Uploader was being used in thousands of open source projects, and we know that members of the Mozilla Firefox community posted that they were updating their confidential information in response to the Codecov attack. It is also known that members of the Mozilla Firefox community have posted that they have been attacked by Codecov to update their confidential information.

Mozilla said, “In response to the Codecov tampering attack announced on April 15, 2021, Mozilla’s security team followed Codecov’s guidance to regenerate credentials and tokens. However, no evidence of a breach was detected and we do not expect any impact to Mozilla products or services.

In early May 2021, Codecov began additional notifications to affected customers and released a detailed list of Indicators of Compromise (IOC), the IP addresses of the attackers associated with this supply chain attack.

Bash Uploader Security Update:

Codecov users should check their CI/CD environments and networks for signs of compromise, and erase all sensitive information that may have been compromised as a safety measure.