[SOLVED] Cannot use custom-formatter

Issue

Running on java-11-openjdk-11.0.9.11-2.el8_3.x86_64, WildFly Core 10.1.12.SP1-redhat . I'm trying to use a custom formatter like this:

module add --name=com.my.keycloak.log-formatter-json --resources=/extensions/log-formatter-json.jar --dependencies=org.jboss.logmanager

/subsystem=logging/custom-formatter=custom:add(class=com.my.keycloak.logging.formatter.json.JsonFormatter, module=com.my.keycloak.log-formatter-json, properties={prettyPrint=true,printDetails=true,bufferSize=1024})

but always get the following errors:

16:41:29,921 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) WFLYCTL0013: Operation ("add") failed - address: ([
    ("subsystem" => "logging"),
    ("custom-formatter" => "CUSTOM")
]): java.lang.IllegalArgumentException: Failed to load module "com.my.keycloak.log-formatter-json" for formatter "CUSTOM"

...

Caused by: java.lang.NoClassDefFoundError: org/jboss/modules/ModuleLoader
    at org.jboss.logmanager.config.AbstractPropertyConfiguration$ModuleFinder.getClassLoader(AbstractPropertyConfiguration.java:556) [jboss-logmanager-2.1.17.Final-redhat-00001.jar:2.1.17.Final-redhat-00001]
    at org.jboss.logmanager.config.AbstractPropertyConfiguration.<init>(AbstractPropertyConfiguration.java:63) [jboss-logmanager-2.1.17.Final-redhat-00001.jar:2.1.17.Final-redhat-00001]
    ... 18 more

if I tried to add

JAVA_OPTS="$JAVA_OPTS -Xbootclasspath/a:$JBOSS_HOME/jboss-modules.jar"

then I get:

java.lang.IllegalStateException: The LogManager was not properly installed (you must set the "java.util.logging.manager" system property to "org.jboss.logmanager.LogManager")
    at org.jboss.logmanager.Logger.getLogger(Logger.java:57)
    at org.jboss.as.server.Main.main(Main.java:89)
    at org.jboss.modules.Module.run(Module.java:352)
    at org.jboss.modules.Module.run(Module.java:320)
    at org.jboss.modules.Main.main(Main.java:593)

then, I added

JAVA_OPTS="$JAVA_OPTS -Djava.util.logging.manager=org.jboss.logmanager.LogManager"

and the issue still persists. Please, help.

P.S. If you think that JAVA_OPTS were not updated, then please see the entire JAVA_OPTS logged during the running.

  JBoss Bootstrap Environment

  JBOSS_HOME: /opt/eap

  JAVA: /usr/lib/jvm/java-11/bin/java

  JAVA_OPTS:  -server -Xlog:gc*:file="/opt/eap/standalone/log/gc.log":time,uptimemillis:filecount=5,filesize=3M -Xbootclasspath/a:/opt/eap/jboss-modules.jar -Djava.util.logging.manager=org.jboss.logmanager.LogManager -XX:+UseParallelOldGC -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -XX:MaxMetaspaceSize=256m -XX:+ExitOnOutOfMemoryError -Djava.security.egd=file:/dev/./urandom  -javaagent:/opt/jboss/container/jolokia/jolokia.jar=config=/opt/jboss/container/jolokia/etc/jolokia.properties -Xbootclasspath/a:/opt/eap/modules/system/layers/base/.overlays/layer-base-jboss-eap-7.3.3.CP/org/jboss/logmanager/main/jboss-logmanager-2.1.17.Final-redhat-00001.jar:/opt/eap/modules/system/layers/base/org/glassfish/jakarta/json/main/jakarta.json-1.1.6.redhat-00001.jar:/opt/eap/modules/system/layers/base/javax/json/api/main/jakarta.json-api-1.1.6.redhat-00001.jar:/opt/eap/modules/system/layers/base/.overlays/layer-base-jboss-eap-7.3.3.CP/org/wildfly/common/main/wildfly-common-1.5.2.Final-redhat-00002.jar -Djava.util.logging.manager=org.jboss.logmanager.LogManager -Dsun.util.logging.disableCallerCheck=true  --add-exports=java.base/sun.nio.ch=ALL-UNNAMED --add-exports=jdk.unsupported/sun.misc=ALL-UNNAMED --add-exports=jdk.unsupported/sun.reflect=ALL-UNNAMED

Solution

You're error is caused by the issue described in WFCORE-4748. This should is fixed in WildFly 21 and should be fixed in JBoss EAP 7.4.

Another thing to note is there is no need to use the KeyCloak formatter like that. There is one built into JBoss EAP 7.3 which it seems you're using.

/subsystem=logging/json-formatter=custom:add(pretty-print=true, print-details=true})

Answered By - James R. Perkins