Stack Exchange
NoClassDefFoundError when running JAR file with Apache Kafka dependencies
I'm trying to run a Java application that integrates with Apache Kafka. The code compiles and runs successfully from within IntelliJ IDEA, but when I try to run the compiled JAR file, I get the following error:
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/kafka/clients/consumer/KafkaConsumer
    at com.blockonic.balances.ChronicleMapWebSocketServer.main(ChronicleMapWebSocketServer.java:112)
Caused by: java.lang.ClassNotFoundException: org.apache.kafka.clients.consumer.KafkaConsumer
    at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
    at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527)

Here is my java code:
// ChronicleMapWebSocketServer.java (simplified for brevity)
import org.apache.kafka.clients.consumer.*;
import org.apache.kafka.common.serialization.*;

public class ChronicleMapWebSocketServer {
    public static void main(String[] args) throws Exception {
        // Configure consumer properties
        Properties config = new Properties();
        config.put(ConsumerConfig.GROUP_ID_CONFIG, "java-group-1");
        // ... other config properties

        // Create a new Kafka consumer instance
        String topic = "user-balances";
        KafkaConsumer<String, String> consumer = new KafkaConsumer<>(config);
        consumer.subscribe(Arrays.asList(topic));

        // ... rest of the code
    }
}

and here is my build.gradle
plugins {
    id 'application'
    id 'java'
}

repositories {
    mavenCentral()
}

dependencies {
    implementation 'net.openhft:chronicle-map:3.24ea1'
    implementation files('build/libs/kafka-clients-3.3.1.jar')
}

jar {
    manifest {
        attributes(
            'Main-Class': 'com.blockonic.balances.ChronicleMapWebSocketServer',
            'Class-Path': 'libs/kafka-clients-3.3.1.jar'
        )
    }
}

I've tried adding the Apache Kafka client JAR (kafka-clients-3.3.1.jar) to the build/libs directory and specifying it in the dependencies block of my build.gradle file. I've also added it to the Class-Path attribute in the JAR manifest.
However, when I run the compiled JAR file, I still get the NoClassDefFoundError related to the KafkaConsumer class.
Can someone please help me understand what I'm doing wrong and how to resolve this issue?

be sure the JAR file is in the same directory as the 'libs' folder

( the Class-Path  entries are relative to the location of the JAR file )

I have doubly ensured, the Jar file is in the same directory!

that comment does not help much... be more specific (and include in question) e.g. show the relevant directories and files -- I am assuming posted error is the full message since Class-Path in posted build.gradle is apparently missing dependencies

After some research what worked for me was the shadowJar plugin from the com.github.johnrengelman.shadow. this creates a fat JAR file that includes all the required dependencies.
Here's what I did:
I applied the shadowJar plugin in my build.gradle file:
plugins {
    id 'com.github.johnrengelman.shadow' version '7.1.2'
    // ... other plugins
}

I included the Apache Kafka client libraries and the required SLF4J dependencies in the dependencies block: (because it was giving another error, that it could not find SLF4J Logger's class definitions )
dependencies {
    implementation 'net.openhft:chronicle-map:3.24ea1'
    implementation 'org.apache.kafka:kafka-clients:3.3.1'
    implementation 'org.slf4j:slf4j-simple:1.7.36'
    implementation 'org.slf4j:slf4j-api:1.7.36'
}

Then I configured the shadowJar task to include the required dependencies:
shadowJar {
    dependencies {
        include(dependency('org.apache.kafka:kafka-clients:3.3.1'))
        include(dependency('org.slf4j:slf4j-simple:1.7.36'))
        include(dependency('org.slf4j:slf4j-api:1.7.36'))
    }
}

and build the JAR file using the shadowJar task:
./gradlew shadowJar
Voila! it works!

As checked on JavaDoc there is no such class named "Properties" available in Package org.apache.kafka.clients.consumer. Then on what basis you have created the object, Could you please clarify?
Thanks.

Properties belongs to the java.utils package.

Where did you import it ? @RameshPareek

from the error message "NoClassDefFoundError: org/apache/kafka/clients/consumer/KafkaConsumer" we clearly see that the problem is not Properties, but KafkaConsumer

I did not include the import for brevity, here in this code. In my actual code I have imported it. But the problem lies somewhere else! I would appreciate if someone can answer the question without using shadowJar which has currently solved my problem, but I do not consider it to be a great solution!

Yes,  @user85421 , upvote to your comment :)

I cannot upvote because of my reputation right now. that's why commented!

Understood, thank you for clarification, learned things
