CQ: Log4j Exposes Blind Spot in Digital Supply Chain

CONTRIBUTED ARTICLE

 The CQ Prime Threat Research team found additional unpatched servers with the Log4j vulnerability hidden within their digital supply chain, dubbed LoNg4j.

The Risks

  • The Log4j vulnerability is more widespread than thought, spread across the digital software supply chain.
  • Testing reveals LoNg4j discovery can take as long as 15 hours to discover in contrast to traditional Log4j vulnerability results, which can be immediate.
  • Unpatched Log4j instances are more common than we think across the digital supply chain.
  • Initial testing may be too focused on immediate blast radius.
  • CISA identified 15 vulnerabilities that malicious actors exploited in 2021 and Log4j was number one.

What the CQ Prime Threat Research team found:

While helping customers validate their Log4j patching efforts, we found that not patching one’s own estate is only part of the problem. We found unpatched servers within customers’ digital supply chain that appeared some 15 hours after the initial test results were received. Dubbed “LoNg4j,” this new exploit demonstrates the widespread implications of the Log4j vulnerability and organizations are potentially at risk of it for years to come.

We also found similar attack vectors that exploit Log4j but are performed through the DNS redirection of popular enterprise solutions such as web conferencing. Detecting this type of LoNg4j exploit requires extensive test infrastructure that most organizations have not allocated. These two pervasive LoNg4j attack vectors mean a patient attacker eventually can gain access to a victim’s application through sheer persistence.

Risks from Log4j Here for Foreseeable Future

Much like lingering COVID-19 symptoms experienced by those who have recovered from the virus, security professionals agree Log4j risks will exist for the foreseeable future. As a proof point, as of February 14, 36 percent of the 50 organizations analyzed remain vulnerable to the Log4j vulnerability with specific verticals like retail and health care fare well, while universities and financial services fare poorly.

The statistics show that even after a globally known vulnerability has been disclosed and a patch made available, organizations still suffer from a lack of applied patches.

Number of vulnerable systems tripled in one organization

While testing our customer’s applications to validate their Log4j patches, we discovered on numerous occasions that they were still responding back as vulnerable to the Log4j vulnerability. With one customer we had tested across their patched applications, we saw the number of Log4j vulnerabilities go from 10 vulnerable systems, then to 8, then 6, and then suddenly 14 vulnerable systems. In total, 38 vulnerable systems were found that contained the Log4j vulnerability.

When we looked deeper, each of the applications that responded back with a positive confirmation that they still had unpatched Log4j components were in fact using a third-party log storage and analysis cloud service. The service still used an unpatched Log4j logging component somewhere within its service but did not realize it had the vulnerability.

How does the Log4j exploit testing work?  

The primary method to figure out if an application has the Log4j vulnerability is to embed a malicious domain lookup request such as sending the following command within a user-agent within a browser:

{jndi:ldap://uniqueIDperTEST.example.com}

The vulnerable Log4j logging component will generate a DNS query to resolve the malicious domain contained in the above command request. The DNS query to the malicious domain confirms the logging component is vulnerable and there is a high likelihood of it containing the Log4j vulnerability.

Going Long on Long Log4j

As we researched the Log4j vulnerability across our customers, we recognized a pattern that we labeled LoNg4j. LoNg4j describes how interconnected modern enterprise IT infrastructure is and how this digital supply chain extends far beyond our known applications. Modern cloud-based application software is often written where code can make calls to other libraries or services.

In our customer example, the application was making calls to a popular third-party log analysis cloud service that had an unpatched log4j component residing somewhere within the application. This log analysis cloud service is used by customers to extract meaningful insights from their applications such as who is logging in, how often they log in and what is their general behavior.

As the log4j component read the JNDI command stored in the transferred logs, the vulnerable Log4j component repeated the unique DNS query of the malicious domain sent above by our test team to the customer application. We know this happened because we received the DNS query to the malicious domain about 12 to 15 hours after we had originally sent it.

Depth & Breadth of Digital Software Supply Chain

We suspect we will continue to see other LoNg4j for years to come, because of the depth and breadth of the deployment of Log4j across our organizations around the world. Today’s modern organizations are layered in software that has been written using open-source software, third-party software and API-driven cloud software services, helping to ensure that software can be written and deployed quickly. Unfortunately, these pieces of software often pull along with them, the vulnerabilities that exist within those third-party components.

What has resulted is a far-reaching digital supply chain with potentially vulnerable applications running across thousands of organizations. Most security and IT teams probably do not know the full scope and severity of the security blind spots that exist within their organization with LoNg4j.

Recommendations:

Organizations need to be aware of the full extent of how deeply embedded the Log4j component is in today’s digital supply chain. We expect even with extensive patching programs that Log4J will continue to remain a hidden risk for years to come. What that means is that unless there is an expanded remediation program to include software that has Log4j components hidden in their software, organizations will remain exposed to cyber-attacks that target the Log4J vulnerability.

We recommend each organization take a two-stage process to evaluate software applications if they have a Log4j software component. Working with internal development teams, you can enable extensive patching programs to ensure each Log4J patch is applied.

The second stage should look to identify application partners in your digital supply chain that have hidden Log4j components in their software. Working with the respective software vendor, you can determine if they are susceptible to vulnerability and if a patch can be applied. An alternative organization can use its WAF to develop virtual patching rules that ensure any incoming request is blocked to ensure you have time to remediate your internal systems.

Not sure if your servers or servers in your digital supply chain are vulnerable to Long Log4J?

Click here to see your organization from an outside-in hacker’s view and determine the impact of this vulnerability in your organization.

CONTRIBUTED ARTICLE