Overview

Log4Shell is a remote code execution vulnerability discovered in December 2021 in log4j Java library. The vulnerability allows attacker to abuse logging library found in most Java apps to execute any code. The exploit is quite easy to use and does not require any complicated infrastructure. Yet it can lead to serious data breach and was therefore classified as critical vulnerability (with a score of 10 out of 10 which should tell you how serious it is).

At CloverDX we’ve detected the vulnerability on December 10th and immediately started to work on mitigation and fixes. We’ve published a security advisory where you can find more details.

In this article, I will show you how to verify your CloverDX instances are safe after you’ve applied mitigation steps or installed our security patches. I will also show how you can determine if someone has tried to attack you in the past.

Are you vulnerable to Log4Shell?

Verification of the vulnerability is simple and can be done if you have access to the environment you wish to test as well as any machine that is connected to the internet and which allows you to run command-line utilities on it.

Note that the steps below do not cause any harm to your CloverDX environment – you cannot break anything, and it will not affect your CloverDX instances in any way.

Preparation

For full verification we’ll need to verify three different parts of CloverDX platform:

• CloverDX Server: each Server runs “Server Core” which takes care of Server Console (the UI), orchestration and many other services.
• Server Worker: each Server runs a single instance of Worker. This is a separate process in which your jobs (graphs etc.) run when you execute them. Worker protects Server from variety of failures and runs in its own JVM with its own settings.
• CloverDX Designer: this is the development environment which runs on developer’s machines. It runs a small copy of Server called Runtime.

Each part has its own JVM with its own settings. To be able to test all of them, we’ll first create an endpoint to which we’ll try to connect from each tool. If the connection is successful, we know that we are vulnerable and need to install security patch or apply mitigation steps.

Note that this approach can also be used to test other applications, not just CloverDX. You can use the same listener and just try reusing the payload to see whether your other apps are safe or not.

We will use nmap toolkit to create a listener that will monitor incoming network connections. If you are on Linux, you can use your favourite package manager to install nmap. On Windows, download the command-line versions of nmap here (direct link to nmap 7.92 zip).

To start monitoring the network, we will use ncat tool. We can use the following command-line to start it:

Windows:ncat -k -v -l 8888 -c "echo PWNED! Remote host %NCAT_REMOTE_ADDR%:%NCAT_REMOTE_PORT%, protocol %NCAT_PROTO% 1>&2"

Linux:
ncat -k -v -l 8888 -c "echo PWNED! Remote host ${NCAT_REMOTE_ADDR}:${NCAT_REMOTE_PORT}, protocol ${NCAT_PROTO} 1>&2"

The command-line options we are using:

• -k: keep listening and do not exit after the first connection.
• -v: verbose to show more output, to see even more details use -vv instead.
• -l: listen on specific port. We are using port 8888 in the example above, but you are free to use different port if needed (e.g., to get around firewalls).
• -c: execute a command. In this case we are just printing some information about the source of the call. The NCAT_* are ncat's "variables" that provide more information about the remote side. We print remote address, port and protocol.

Once ncat starts, you will see an output similar to this:

With the above, we are now ready to test whether different parts of CloverDX platform have been properly secured.

Once you are done with your testing, you can safely stop the ncat process (with Ctrl+C).

Verify CloverDX Server

On Server, we can test by trying to log-in with a specially crafted user-name that will force Log4j to perform JNDI lookup:

When trying to log-in, replace the IP address and port with address and port of the listener you started during preparation. Once you try to log-in, you will see the log-in failed like this:

If your CloverDX Server Core is vulnerable, you will see the following in the ncat output:In my case, all above messages were generated by a single log-in attempt.

If your CloverDX Server Core is not vulnerable, you will not see any connection attempts in ncat. CloverDX Server will never perform any JNDI lookups during user log-in and will never try to connect you your ncat.

Verify CloverDX Server Worker

Worker runs in its own JVM and therefore has its own instance of log4j with its own settings. We can verify it very easily simply by trying to log our payload in a graph.

To do this, create a simple graph with a Success component and paste the following code into it:

//#CTL2
const string URL = "jndi:ldap://127.0.0.1:8888/pwned";

function integer transform() {
    printLog(info, "${" + URL + "}");

    return ALL;
}


Run the graph and observe its behaviour.

If your Server Worker is vulnerable, you will see output in ncat just like you did when trying the Server log-in above.

If your Server Worker is not vulnerable, you will not see any output in ncat.

Verify CloverDX Designer

CloverDX Designer runs its own limited version of Server as its Runtime. We can use the same job we used for Server Worker verification and need to run it in a local project.

If your Designer is vulnerable, you will see output in ncat just like before and can also see a warning in Runtime log:2021-12-14 00:45:38,309 SUCCESS_1 WARN Error looking up JNDI resource [ldap://127.0.0.1:8888/pwned]. javax.naming.NamingException: LDAP response read timed out, timeout used:-1ms.

If your Designer is not vulnerable, you will not see any warnings and no new connections attempts will be visible in ncat output.

Verify if someone attacked your CloverDX Server in the past

Once you’ve verified that your Server is not vulnerable, you can try to see if anyone has made attempts to attack you using this vulnerability before. To see that, you can simply search through CloverDX Server logs like this:

Windows:
findstr "${jndi" %CLOVER_HOME%\temp\cloverlogs\*.log

Linux:
grep '${jndi' $CLOVER_HOME/temp/cloverlogs/*.log

Where CLOVER_HOME is path to you CloverDX installation (the above assumes default installation in Tomcat, your log location might differ depending on your application server and your configuration).

If you were attacked in the past, you will see messages like these in the log:2021-12-13 23:09:15,686[io-23080-exec-2] userAction WARN Login unsuccessful; username=$ {jndi:ldap://127.0.0.1:8888/pwned},address=127.0.0.1

If you were not attacked, you will not see any output of the above command since CloverDX never performs JNDI lookups like this.