Grails 2 fails on WebSphere

I have had very similar issues trying to run an empty grails 2.0 app on Websphere 7.0.0.21.

I was able to put a break point on where the root cause NullPointerException was happening (websphere logs truncated the important parts). It was at FiltersGrailsPlugin.groovy:183. On that line, the return value of applicationContext.getAutowireCapableBeanFactory() was null causing the exception.

I have tried this with grails 2.0.0, 2.0.1 and 2.0.1 together with websphere 7.0.0.21. Running a blank grails 1.3.7 app had no such issues so this is a regression in grails 2.0.x.

Like everyone else, switching to another container is not an option. This issue is a blocker for those forced to use Websphere for deployment.

Tried with Grails 2.0.2 and exactly same error and stacktrace.

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'pluginManager' defined in ServletContext resource [/WEB-INF/applicationContext.xml]: Invocation of init method failed; nested exception is java.lang.NullPointerException

The full stacktrace.log is in WebSphere/AppServer/profiles/AppSrv01/stacktrace.log file by default (GRAILS-2730).

This is the missing information about where the NPE really happens:

Caused by: java.lang.NullPointerException
        at org.springframework.core.io.AbstractFileResolvingResource.exists(AbstractFileResolvingResource.java:122)
        at org.codehaus.groovy.grails.plugins.BinaryGrailsPlugin.initializeViewMap(BinaryGrailsPlugin.java:82)
        at org.codehaus.groovy.grails.plugins.BinaryGrailsPlugin.<init>(BinaryGrailsPlugin.java:67)
        at org.codehaus.groovy.grails.plugins.DefaultGrailsPluginManager.createBinaryGrailsPlugin(DefaultGrailsPluginManager.java:346)
        at org.codehaus.groovy.grails.plugins.DefaultGrailsPluginManager.findCorePlugins(DefaultGrailsPluginManager.java:333)
        at org.codehaus.groovy.grails.plugins.DefaultGrailsPluginManager.attemptLoadPlugins(DefaultGrailsPluginManager.java:285)
        at org.codehaus.groovy.grails.plugins.DefaultGrailsPluginManager.loadPlugins(DefaultGrailsPluginManager.java:240)
        at org.codehaus.groovy.grails.plugins.GrailsPluginManagerFactoryBean.afterPropertiesSet(GrailsPluginManagerFactoryBean.java:105)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1514)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1452)
        ... 107 more

later on in stacktrace.log there is also the full stack trace

2012-04-12 11:23:01,159 [WebContainer : 6] ERROR StackTrace  - Full Stack Trace:
java.lang.NullPointerException
        at org.springframework.core.io.AbstractFileResolvingResource.exists(AbstractFileResolvingResource.java:122)
        at org.codehaus.groovy.grails.plugins.BinaryGrailsPlugin.initializeViewMap(BinaryGrailsPlugin.java:82)
        at org.codehaus.groovy.grails.plugins.BinaryGrailsPlugin.<init>(BinaryGrailsPlugin.java:67)
        at org.codehaus.groovy.grails.plugins.DefaultGrailsPluginManager.createBinaryGrailsPlugin(DefaultGrailsPluginManager.java:346)
        at org.codehaus.groovy.grails.plugins.DefaultGrailsPluginManager.findCorePlugins(DefaultGrailsPluginManager.java:333)
        at org.codehaus.groovy.grails.plugins.DefaultGrailsPluginManager.attemptLoadPlugins(DefaultGrailsPluginManager.java:285)
        at org.codehaus.groovy.grails.plugins.DefaultGrailsPluginManager.loadPlugins(DefaultGrailsPluginManager.java:240)
        at org.codehaus.groovy.grails.plugins.GrailsPluginManagerFactoryBean.afterPropertiesSet(GrailsPluginManagerFactoryBean.java:105)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:15
14)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1452)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:519)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
        at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294)
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:225)
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291)
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:567)
        at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:913)
        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:464)
        at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:384)
        at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:283)
        at org.codehaus.groovy.grails.web.context.GrailsContextLoader.initWebApplicationContext(GrailsContextLoader.java:66)
        at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:111)
        at com.ibm.ws.wswebcontainer.webapp.WebApp.notifyServletContextCreated(WebApp.java:653)
        at com.ibm.ws.webcontainer.webapp.WebApp.commonInitializationFinish(WebApp.java:370)
        at com.ibm.ws.wswebcontainer.webapp.WebApp.initialize(WebApp.java:293)
(continues)

I think this should be reported to Spring Jira since there isn't proper null checks in org.springframework.core.io.AbstractFileResolvingResource.exists(AbstractFileResolvingResource.java:122)

btw. In the WAS documentation, the invokeFiltersCompatibility property is "com.ibm.ws.webcontainer.invokeFiltersCompatibility" (not com.ibm.ws.webcontainer.invokefilterscompatibility).
Maybe the property name isn't case-sensitive?

Hints for testing Grails on WAS:

• get VirtualBox from https://www.virtualbox.org/ .
• I used CentOS 5.7 (32-bit Linux) from virtualboxes.org to install trial releases of WAS on it (use the VM image at your own risk).
  • Update the VirtualBox guest additions to the CentOS image
  • Setup a shared folder in VirtualBox to share files between the host and VM (downloading files from IBM site in the host vm is easier)
• I had to first install IBM Java 6 to get the WAS 6.1 installer running (after that "java -jar setup.jar" worked, launchpad.sh didn't do anything).

Links:

• WAS 6.1 Trial download: https://www14.software.ibm.com/webapp/iwm/web/preLogin.do?lang=en_US&source=was60exp&S_CMP=web_dw_rt_swd
• UpdateInstaller download (for installing the fixpack): http://www-01.ibm.com/support/docview.wss?rs=180&uid=swg24020446
• Fixpack 29 http://www-01.ibm.com/support/docview.wss?uid=swg24025188 (get AppServer & JDK fixpacks for 32-bit Linux)

The url that causes the NPE in my test is:
"wsjar:file:/opt/was60/WebSphere/AppServer/profiles/AppSrv01/installedApps/centos57Node02Cell/petclinic_war.ear/petclinic.war/WEB-INF/lib/grails-plugin-codecs-2.0.3.jar!/META-INF/views.properties"

The com.ibm.ws.classloader.Handler$ClassLoaderURLConnection instance returns null when the resource isn't found. This must be a bug in Spring that this isn't handled properly in AbstractFileResolvingResource.

I opened an issue in Spring Jira https://jira.springsource.org/browse/SPR-9323

Pull request to spring-framework 3.1.x branch https://github.com/SpringSource/spring-framework/pull/65 .

This is actually a WebSphere bug. It should be fixed in a more recent fixpack:
PK68016: URLConnection.getInputStream for the wsjar returns null rather than throwing a FileNotException.

The bug in WebSphere isn't fixed by the PK68016 fix . I installed fixpack 43 (6.1.0.43) and the problem still exists. SPR-9323 seems to be the best option to fix this problem.

Now I read the PK68016 release notes again. It requires setting a separate property in WebSphere.

The wsjar protocol handler is changed to throw
FileNotFoundException from URLConnection.getInputStream when
the JVM custom property
com.ibm.ws.classloader.getInputStream.enableIOException is set
to true. If the property is unset or set to false, then the
current behavior of returning null is retained.

Adding "com.ibm.ws.classloader.getInputStream.enableIOException=true" JVM custom property fixed the NPE problem.

This is the next problem with Grails 2.0.3:

Caused by: org.codehaus.groovy.grails.exceptions.NewInstanceCreationException: Could not create a new instance of class [HibernateGrailsPlugin]!
        ... 17 more
Caused by: java.lang.UnsupportedClassVersionError: (org/grails/datastore/mapping/core/AbstractDatastore) bad major version at offset=6
        at java.lang.ClassLoader.defineClass(ClassLoader.java:267)
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:151)
        at com.ibm.ws.classloader.CompoundClassLoader._defineClass(CompoundClassLoader.java:583)
        at com.ibm.ws.classloader.CompoundClassLoader.findClass(CompoundClassLoader.java:531)
        at com.ibm.ws.classloader.CompoundClassLoader.loadClass(CompoundClassLoader.java:405)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:616)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:267)
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:151)

grails-datastore-core-1.0.4.RELEASE.jar is compiled for Java 1.6

Lari, many many thanks for your investigations on this!
I can confirm that after I set this JVM custom property I have now the same UnsupportedClassVersionError as you.
(I'm still on WebSphere 6.1.0.29. According to IBM, PK68016 is included since Fix Pack 6.1.0.21)

I got a Grails 2.0.3 app running after removing WEB-INF/lib/grails-datastore-core-1.0.4.RELEASE.jar , WEB-INF/lib/grails-datastore-gorm-1.0.4.RELEASE.jar and WEB-INF/lib/grails-datastore-simple-1.0.4.RELEASE.jar files from the war file manually and replacing them with these files:
http://hudson.grails.org/view/All/job/grails_data_mapping/lastSuccessfulBuild/artifact/grails-datastore-core/build/libs/grails-datastore-core-1.0.6.BUILD-SNAPSHOT.jar
http://hudson.grails.org/view/All/job/grails_data_mapping/lastSuccessfulBuild/artifact/grails-datastore-gorm/build/libs/grails-datastore-gorm-1.0.6.BUILD-SNAPSHOT.jar
http://hudson.grails.org/view/All/job/grails_data_mapping/lastSuccessfulBuild/artifact/grails-datastore-simple/build/libs/grails-datastore-simple-1.0.6.BUILD-SNAPSHOT.jar

grails-datastore release 1.0.6 will be compiled for Java 1.5 (tracked by GRAILS-8565 , fix committed by Graeme yesterday: https://github.com/SpringSource/grails-data-mapping/commit/56d2700caac2dd1ca3ade39aa945afd14cc67201).

The above workarounds do not work when dealing with websphere 7. Websphere 7 comes with jdk6 so the class version error will not happen there. Also the workaround for null pointer by setting com.ibm.ws.classloader.getInputStream.enableIOException to true does not affect the behaviour of websphere 7. We are still encountering a NullPointerException at FiltersGrailsPlugin.groovy:183 due to applicationContext.getAutowireCapableBeanFactory() returning null on that line.

Craig, your problem must be different.

I tested deploying to WAS 7.0.0.11 version (downloaded and installed the trial version).
I only had to set "-Xverify:none" to the JVM arguments (Application servers > server1 > Process definition > Java Virtual Machine -> Generic JVM arguments). Without this setting, the JVM crashed.

There was no need for the com.ibm.ws.webcontainer.invokefilterscompatibility setting on WAS 7.

Craig, can you find a file name stacktrace.log (WebSphere/AppServer/profiles/AppSrv01/stacktrace.log seems to be the default location because of GRAILS-2730) ? You'll find a full stack trace in that file that's causing the startup to fail. Usually the problem is related to database connections / drivers. Classloader problems are also typical (f.e. XML or logging library conflicts).

Hi Lari,

Here is the extract from stacktrace.log when starting a blank grails project. As I mentioned earlier the null ptr exception is happening at FiltersGrailsPlugin.groovy line 183. In a remote debugger, I can see that getAutowireCapableBeanFactory() is returning null. We are using a slightly different version of WAS 7 than you (express 7.0.0.21).

2012-04-23 16:04:18,699 [server.startup : 0] ERROR StackTrace - Full Stack Trace:
java.lang.NullPointerException: Cannot invoke method autowireBeanProperties() on null object
at org.codehaus.groovy.runtime.NullObject.invokeMethod(NullObject.java:77)
at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:45)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
at org.codehaus.groovy.runtime.callsite.NullCallSite.call(NullCallSite.java:32)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:54)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:124)
at org.codehaus.groovy.grails.plugins.web.filters.FiltersGrailsPlugin.reloadFilters(FiltersGrailsPlugin.groovy:183)
at org.codehaus.groovy.grails.plugins.web.filters.FiltersGrailsPlugin$reloadFilters.callStatic(Unknown Source)
at org.codehaus.groovy.grails.plugins.web.filters.FiltersGrailsPlugin$_closure2.doCall(FiltersGrailsPlugin.groovy:70)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:45)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
at java.lang.reflect.Method.invoke(Method.java:599)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1047)
at groovy.lang.ExpandoMetaClass.invokeMethod(ExpandoMetaClass.java:1110)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:877)
at groovy.lang.Closure.call(Closure.java:412)
at org.codehaus.groovy.grails.plugins.DefaultGrailsPlugin.doWithApplicationContext(DefaultGrailsPlugin.java:489)
at org.codehaus.groovy.grails.plugins.AbstractGrailsPluginManager.doPostProcessing(AbstractGrailsPluginManager.java:174)
at org.codehaus.groovy.grails.commons.spring.GrailsRuntimeConfigurator.performPostProcessing(GrailsRuntimeConfigurator.java:238)
at org.codehaus.groovy.grails.commons.spring.GrailsRuntimeConfigurator.configure(GrailsRuntimeConfigurator.java:178)
at org.codehaus.groovy.grails.commons.spring.GrailsRuntimeConfigurator.configure(GrailsRuntimeConfigurator.java:127)
at org.codehaus.groovy.grails.web.context.GrailsConfigUtils.configureWebApplicationContext(GrailsConfigUtils.java:121)
at org.codehaus.groovy.grails.web.servlet.GrailsDispatcherServlet.createWebApplicationContext(GrailsDispatcherServlet.java:174)
at org.springframework.web.servlet.FrameworkServlet.initWebApplicationContext(FrameworkServlet.java:508)
at org.springframework.web.servlet.FrameworkServlet.initServletBean(FrameworkServlet.java:449)
at org.springframework.web.servlet.HttpServletBean.init(HttpServletBean.java:133)
at javax.servlet.GenericServlet.init(GenericServlet.java:241)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.init(ServletWrapper.java:358)
at com.ibm.ws.webcontainer.servlet.ServletWrapperImpl.init(ServletWrapperImpl.java:169)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.initialize(ServletWrapper.java:1809)
at com.ibm.wsspi.webcontainer.extension.WebExtensionProcessor.createServletWrapper(WebExtensionProcessor.java:98)
at com.ibm.ws.webcontainer.webapp.WebApp.getServletWrapper(WebApp.java:1038)
at com.ibm.ws.webcontainer.webapp.WebApp.getServletWrapper(WebApp.java:959)
at com.ibm.ws.webcontainer.webapp.WebApp.initializeTargetMappings(WebApp.java:638)
at com.ibm.ws.webcontainer.webapp.WebApp.commonInitializationFinally(WebApp.java:436)
at com.ibm.ws.webcontainer.webapp.WebAppImpl.initialize(WebAppImpl.java:304)
at com.ibm.ws.webcontainer.webapp.WebGroupImpl.addWebApplication(WebGroupImpl.java:100)
at com.ibm.ws.webcontainer.VirtualHostImpl.addWebApplication(VirtualHostImpl.java:166)
at com.ibm.ws.webcontainer.WSWebContainer.addWebApp(WSWebContainer.java:732)
at com.ibm.ws.webcontainer.WSWebContainer.addWebApplication(WSWebContainer.java:617)
at com.ibm.ws.webcontainer.component.WebContainerImpl.install(WebContainerImpl.java:376)
at com.ibm.ws.webcontainer.component.WebContainerImpl.start(WebContainerImpl.java:668)
at com.ibm.ws.runtime.component.ApplicationMgrImpl.start(ApplicationMgrImpl.java:1127)
at com.ibm.ws.runtime.component.DeployedApplicationImpl.fireDeployedObjectStart(DeployedApplicationImpl.java:1319)
at com.ibm.ws.runtime.component.DeployedModuleImpl.start(DeployedModuleImpl.java:610)
at com.ibm.ws.runtime.component.DeployedApplicationImpl.start(DeployedApplicationImpl.java:944)
at com.ibm.ws.runtime.component.ApplicationMgrImpl.startApplication(ApplicationMgrImpl.java:740)
at com.ibm.ws.runtime.component.ApplicationMgrImpl.start(ApplicationMgrImpl.java:2051)
at com.ibm.ws.runtime.component.CompositionUnitMgrImpl.start(CompositionUnitMgrImpl.java:385)
at com.ibm.ws.runtime.component.CompositionUnitImpl.start(CompositionUnitImpl.java:123)
at com.ibm.ws.runtime.component.CompositionUnitMgrImpl.start(CompositionUnitMgrImpl.java:328)
at com.ibm.ws.runtime.component.CompositionUnitMgrImpl.access$300(CompositionUnitMgrImpl.java:113)
at com.ibm.ws.runtime.component.CompositionUnitMgrImpl$CUInitializer.run(CompositionUnitMgrImpl.java:895)
at com.ibm.wsspi.runtime.component.WsComponentImpl$_AsynchInitializer.run(WsComponentImpl.java:349)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1604)

Craig, it's possible that some earlier problem is causing this NPE / stacktrace.

Please try to locate the stacktrace.log file and check if it contains any errors that happen before this one.

The stacktrace.log file can usually be found in WebSphere/AppServer/profiles/AppSrv01/stacktrace.log since that is the the default location because of GRAILS-2730.

I'll see if I can test with the same version as you have.

I downloaded WAS 7.0 Express and installed Fixpack 21 to it (JDK & AppServer FPs).
I added "-Xverify:none" to server1 JVM arguments (Application servers > server1 > Process definition > Java Virtual Machine -> Generic JVM arguments) and restarted it.
The deployment of the Grails app was successfull and there weren't any problems running it.

I couldn't reproduce the problem Craig has.

Upgraded to datastore 1.0.7 which is compiled with JDK 1.5 so all should be good now

https://github.com/grails/grails-core/commit/419db8d6e79977dbf202e5c51ffd9f1896e0b223

Thanks a lot for your effort.

With following dependency in BuildConfig.groovy it's now possible to run a Grails 2.0.3 application on WAS 6.1 (which runs on Java 1.5)

compile("org.grails:grails-datastore-gorm:1.0.7.RELEASE", "org.grails:grails-datastore-simple:1.0.7.RELEASE", "org.grails:grails-datastore-core:1.0.7.RELEASE") {
	transitive=false
}