Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 1.2 final
    • Fix Version/s: 2.0.4, 2.1-RC2
    • Component/s: Project infrastructure
    • Labels:
      None
    • Environment:
      Win32, Tomcat 6.0.20 / 6.0.24

      Description

      When deploying a minimal Grails application to Tomcat 6.0.20 on Windows, everything works fine. When the same application is undeployed, the following libraries are locked and cannot be removed by Tomcat.

      • ehcache-core-1.7.1.jar
      • groovy-all-1.6.7.jar
      • grails-core-1.2.0.jar

      The same problem also happens if the application is configured to use the MSSQL JDBC 2.0 driver. The first time the application is deployed the database connections work fine, when the application is undeployed it does not remove the driver's JAR file and also subsequent deployments fail with JDBC connection issues.

      When deploying the exact same application on Tomcat 6.0.24 on Windows, none of the problems mentioned above occur. However the Tomcat log contains a bunch of errors like the following...

      SEVERE: A web application created a ThreadLocal with key of type [org.codehaus.groovy.runtime.GroovyCategorySupport.MyThreadLocal] (value [org.codehaus.groovy.runtime.GroovyCategorySupport$MyThreadLocal@d32ea0]) and a value of type [java.lang.ref.SoftReference] (value [java.lang.ref.SoftReference@9a288b]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

      The full log file is attached to this issue as well as the sample grails application which triggers these errors.

      The purpose of the issue would be to fix the locking issues under Tomcat 6.0.20 and to clean up the SEVERE messages in the log file under Tomcat 6.0.24.

      Please let me know if you require any additional information. Thanks!

      1. abstract-save-persistent-leak.patch
        2 kB
        R
      2. catalina.2010-01-29.log
        13 kB
        Daniel Mikusa
      3. grails121.log
        16 kB
        Daniel Mikusa
      4. hibernate-persistence-context-leak.patch
        2 kB
        R

        Activity

        Hide
        Daniel Mikusa added a comment -

        I've upgraded the "help" demo app and checked again with Grails 1.2.1. Unfortunately I am still seeing the same issue.

        The list of locked JAR files is as follows...

        • ehcache-core-1.7.1.jar
        • grails-core-1.2.1.jar
        • groovy-all-1.6.7.jar

        I've also attached the log produced by Tomcat 6.0.24 as it forces the app to undeploy.

        Show
        Daniel Mikusa added a comment - I've upgraded the "help" demo app and checked again with Grails 1.2.1. Unfortunately I am still seeing the same issue. The list of locked JAR files is as follows... ehcache-core-1.7.1.jar grails-core-1.2.1.jar groovy-all-1.6.7.jar I've also attached the log produced by Tomcat 6.0.24 as it forces the app to undeploy.
        Hide
        Daniel Mikusa added a comment -

        Tomcat 6.0.24 log with Grails 1.2.1 app.

        Show
        Daniel Mikusa added a comment - Tomcat 6.0.24 log with Grails 1.2.1 app.
        Hide
        Graeme Rocher added a comment -

        As far as I understand it there were specific changes made to 6.0.24 that improved Tomcat's ability to reload deployed applications. These changes aren't present in 6.0.20 so nothing can be done other than the user upgrading to the latest Tomcat version.

        We can however looked at why these SEVERE errors are appearing

        Show
        Graeme Rocher added a comment - As far as I understand it there were specific changes made to 6.0.24 that improved Tomcat's ability to reload deployed applications. These changes aren't present in 6.0.20 so nothing can be done other than the user upgrading to the latest Tomcat version. We can however looked at why these SEVERE errors are appearing
        Hide
        Burt Beckwith added a comment -

        Most of the ThreadLocal warnings are Spring-related; see this thread: http://forum.springsource.org/showthread.php?t=84202

        Show
        Burt Beckwith added a comment - Most of the ThreadLocal warnings are Spring-related; see this thread: http://forum.springsource.org/showthread.php?t=84202
        Hide
        Graeme Rocher added a comment -

        Tomcat 6.0.26 has some new resource clean up and permgen leak detection logic which will hopefully help fix some of the issues. Note that many of the issues are not just in Grails but in libraries like Spring and Apache commons

        Show
        Graeme Rocher added a comment - Tomcat 6.0.26 has some new resource clean up and permgen leak detection logic which will hopefully help fix some of the issues. Note that many of the issues are not just in Grails but in libraries like Spring and Apache commons
        Hide
        Daniel Mikusa added a comment -

        I tried again on Tomcat 6.0.26 with Grails 1.2.1. The number of SEVERE items has significantly dropped, but there are still a few. Including the list below.

        I'm not certain about the first error message in the list since it is related to the MSSQL Driver, but the other six appear to be referencing classes from the org.codehaus.groovy and org.codehaus.groovy.grails packages. Are these issues with Groovy / Grails or are these as you mentioned before errors with the underlying Spring and Apache Commons libraries?

        Thanks

        SEVERE: A web application registered the JBDC driver [com.microsoft.sqlserver.jdbc.SQLServerDriver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.
        Mar 24, 2010 4:03:11 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
        SEVERE: A web application created a ThreadLocal with key of type [org.codehaus.groovy.runtime.GroovyCategorySupport.MyThreadLocal] (value [org.codehaus.groovy.runtime.GroovyCategorySupport$MyThreadLocal@4a17d8]) and a value of type [java.lang.ref.SoftReference] (value [java.lang.ref.SoftReference@63725d]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.
        Mar 24, 2010 4:03:11 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
        SEVERE: A web application created a ThreadLocal with key of type [org.codehaus.groovy.runtime.GroovyCategorySupport.MyThreadLocal] (value [org.codehaus.groovy.runtime.GroovyCategorySupport$MyThreadLocal@4a17d8]) and a value of type [java.lang.ref.SoftReference] (value [java.lang.ref.SoftReference@e94c62]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.
        Mar 24, 2010 4:03:11 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
        SEVERE: A web application created a ThreadLocal with key of type [null] (value [org.codehaus.groovy.grails.commons.spring.ReloadAwareAutowireCapableBeanFactory$2@95c2b4]) and a value of type [java.lang.Boolean] (value [false]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.
        Mar 24, 2010 4:03:11 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
        SEVERE: A web application created a ThreadLocal with key of type [org.codehaus.groovy.runtime.GroovyCategorySupport.MyThreadLocal] (value [org.codehaus.groovy.runtime.GroovyCategorySupport$MyThreadLocal@4a17d8]) and a value of type [java.lang.ref.SoftReference] (value [java.lang.ref.SoftReference@9212f4]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.
        Mar 24, 2010 4:03:11 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
        SEVERE: A web application created a ThreadLocal with key of type [null] (value [org.codehaus.groovy.grails.commons.spring.ReloadAwareAutowireCapableBeanFactory$2@95c2b4]) and a value of type [java.lang.Boolean] (value [false]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.
        Mar 24, 2010 4:03:11 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
        SEVERE: A web application created a ThreadLocal with key of type [org.codehaus.groovy.runtime.GroovyCategorySupport.MyThreadLocal] (value [org.codehaus.groovy.runtime.GroovyCategorySupport$MyThreadLocal@4a17d8]) and a value of type [java.lang.ref.SoftReference] (value [java.lang.ref.SoftReference@1a755e]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

        Show
        Daniel Mikusa added a comment - I tried again on Tomcat 6.0.26 with Grails 1.2.1. The number of SEVERE items has significantly dropped, but there are still a few. Including the list below. I'm not certain about the first error message in the list since it is related to the MSSQL Driver, but the other six appear to be referencing classes from the org.codehaus.groovy and org.codehaus.groovy.grails packages. Are these issues with Groovy / Grails or are these as you mentioned before errors with the underlying Spring and Apache Commons libraries? Thanks SEVERE: A web application registered the JBDC driver [com.microsoft.sqlserver.jdbc.SQLServerDriver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered. Mar 24, 2010 4:03:11 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap SEVERE: A web application created a ThreadLocal with key of type [org.codehaus.groovy.runtime.GroovyCategorySupport.MyThreadLocal] (value [org.codehaus.groovy.runtime.GroovyCategorySupport$MyThreadLocal@4a17d8] ) and a value of type [java.lang.ref.SoftReference] (value [java.lang.ref.SoftReference@63725d] ) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed. Mar 24, 2010 4:03:11 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap SEVERE: A web application created a ThreadLocal with key of type [org.codehaus.groovy.runtime.GroovyCategorySupport.MyThreadLocal] (value [org.codehaus.groovy.runtime.GroovyCategorySupport$MyThreadLocal@4a17d8] ) and a value of type [java.lang.ref.SoftReference] (value [java.lang.ref.SoftReference@e94c62] ) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed. Mar 24, 2010 4:03:11 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap SEVERE: A web application created a ThreadLocal with key of type [null] (value [org.codehaus.groovy.grails.commons.spring.ReloadAwareAutowireCapableBeanFactory$2@95c2b4] ) and a value of type [java.lang.Boolean] (value [false] ) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed. Mar 24, 2010 4:03:11 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap SEVERE: A web application created a ThreadLocal with key of type [org.codehaus.groovy.runtime.GroovyCategorySupport.MyThreadLocal] (value [org.codehaus.groovy.runtime.GroovyCategorySupport$MyThreadLocal@4a17d8] ) and a value of type [java.lang.ref.SoftReference] (value [java.lang.ref.SoftReference@9212f4] ) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed. Mar 24, 2010 4:03:11 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap SEVERE: A web application created a ThreadLocal with key of type [null] (value [org.codehaus.groovy.grails.commons.spring.ReloadAwareAutowireCapableBeanFactory$2@95c2b4] ) and a value of type [java.lang.Boolean] (value [false] ) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed. Mar 24, 2010 4:03:11 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap SEVERE: A web application created a ThreadLocal with key of type [org.codehaus.groovy.runtime.GroovyCategorySupport.MyThreadLocal] (value [org.codehaus.groovy.runtime.GroovyCategorySupport$MyThreadLocal@4a17d8] ) and a value of type [java.lang.ref.SoftReference] (value [java.lang.ref.SoftReference@1a755e] ) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.
        Hide
        Graeme Rocher added a comment -

        yes. The GroovyCategory ones are problems in Groovy (not Grails) that we'll need a new release of Groovy for. I'm not sure whether the issue with org.codehaus.groovy.grails.commons.spring.ReloadAwareAutowireCapableBeanFactory is a problem with Grails or Spring at this stage. Will need to investigate further...

        Show
        Graeme Rocher added a comment - yes. The GroovyCategory ones are problems in Groovy (not Grails) that we'll need a new release of Groovy for. I'm not sure whether the issue with org.codehaus.groovy.grails.commons.spring.ReloadAwareAutowireCapableBeanFactory is a problem with Grails or Spring at this stage. Will need to investigate further...
        Hide
        Graeme Rocher added a comment -

        Related Groovy issue GROOVY-4130

        Show
        Graeme Rocher added a comment - Related Groovy issue GROOVY-4130
        Hide
        Felipe Cypriano added a comment - - edited

        I'm having this problem and couldn't find a way to solve it. I'm trying to deploy the application on Tomcat 6.0.26 and these are the errors:

        SEVERE: A web application registered the JBDC driver [org.firebirdsql.jdbc.FBDriver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.
        19/04/2010 10:40:17 org.apache.catalina.loader.WebappClassLoader clearReferencesStopTimerThread
        SEVERE: A web application appears to have started a TimerThread named [Timer-2] via the java.util.Timer API but has failed to stop it. To prevent a memory leak, the timer (and hence the associated thread) has been forcibly cancelled.
        19/04/2010 10:40:17 org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
        SEVERE: A web application created a ThreadLocal with key of type [org.codehaus.groovy.runtime.GroovyCategorySupport.MyThreadLocal] (value [org.codehaus.groovy.runtime.GroovyCategorySupport$MyThreadLocal@a0416a]) and a value of type [java.lang.ref.SoftReference] (value [java.lang.ref.SoftReference@126d2fe]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

        Grails version is 1.1.2

        Is there any workaround to make it work?

        Show
        Felipe Cypriano added a comment - - edited I'm having this problem and couldn't find a way to solve it. I'm trying to deploy the application on Tomcat 6.0.26 and these are the errors: SEVERE: A web application registered the JBDC driver [org.firebirdsql.jdbc.FBDriver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered. 19/04/2010 10:40:17 org.apache.catalina.loader.WebappClassLoader clearReferencesStopTimerThread SEVERE: A web application appears to have started a TimerThread named [Timer-2] via the java.util.Timer API but has failed to stop it. To prevent a memory leak, the timer (and hence the associated thread) has been forcibly cancelled. 19/04/2010 10:40:17 org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap SEVERE: A web application created a ThreadLocal with key of type [org.codehaus.groovy.runtime.GroovyCategorySupport.MyThreadLocal] (value [org.codehaus.groovy.runtime.GroovyCategorySupport$MyThreadLocal@a0416a] ) and a value of type [java.lang.ref.SoftReference] (value [java.lang.ref.SoftReference@126d2fe] ) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed. Grails version is 1.1.2 Is there any workaround to make it work?
        Hide
        Dan Lynn added a comment -

        At this point it looks like you have to restart tomcat after enough redeploys. Not really a workaround, I know (Tomcat 6.0.26, Grails 1.2.2 here)

        Show
        Dan Lynn added a comment - At this point it looks like you have to restart tomcat after enough redeploys. Not really a workaround, I know (Tomcat 6.0.26, Grails 1.2.2 here)
        Hide
        Alex Tkachman added a comment -

        I just commited fix for GROOVY-4130 both to Groovy 1.7.x & 1.8.x
        It solves issue with ThreadLocals but the issue has nothing to do with locked jars
        Graeme, please let me know if you have other problems between Groovy runtime & Grails

        Show
        Alex Tkachman added a comment - I just commited fix for GROOVY-4130 both to Groovy 1.7.x & 1.8.x It solves issue with ThreadLocals but the issue has nothing to do with locked jars Graeme, please let me know if you have other problems between Groovy runtime & Grails
        Hide
        Graeme Rocher added a comment -

        Thanks Alex!

        Show
        Graeme Rocher added a comment - Thanks Alex!
        Hide
        Raphael Miranda added a comment -

        The fix Alex commited entered groovy 1.7.3 and 1.8-beta-1. Since Grails 1.3.5 ships with groovy 1.7.5, does it contain this fix?

        Show
        Raphael Miranda added a comment - The fix Alex commited entered groovy 1.7.3 and 1.8-beta-1. Since Grails 1.3.5 ships with groovy 1.7.5, does it contain this fix?
        Hide
        Marcus Better added a comment -

        It is only partially fixed. Alex' fix took care of the category support stuff, but my Grails 1.3.5 app still gives messages on undeploy because of the following ThreadLocal objects:

        org.codehaus.groovy.grails.orm.hibernate.validation.HibernateDomainClassValidator
        org.codehaus.groovy.grails.web.sitemesh.GrailsContentBufferingResponse
        org.springframework.security.core.context.SecurityContextImpl

        Show
        Marcus Better added a comment - It is only partially fixed. Alex' fix took care of the category support stuff, but my Grails 1.3.5 app still gives messages on undeploy because of the following ThreadLocal objects: org.codehaus.groovy.grails.orm.hibernate.validation.HibernateDomainClassValidator org.codehaus.groovy.grails.web.sitemesh.GrailsContentBufferingResponse org.springframework.security.core.context.SecurityContextImpl
        Hide
        Graeme Rocher added a comment -

        @Raphael Yes the fix has been applied in Groovy

        @Marcus Is it possible to post a reproducible example, there are clearly cases where this still happens but a vanilla Grails application doesn't exhibit any problems as far as I can tell

        Show
        Graeme Rocher added a comment - @Raphael Yes the fix has been applied in Groovy @Marcus Is it possible to post a reproducible example, there are clearly cases where this still happens but a vanilla Grails application doesn't exhibit any problems as far as I can tell
        Hide
        Chaitanya Kulkarni added a comment -

        @Graeme
        I am using tomcat 6.0.29, grails 1.3.7, and java 1.5 version. But when I undeploy or stop the tomcat server then i am getting below error in the catalina.log file of tomcat...

        SEVERE: The web application [/application] created a ThreadLocal with key of type [null] (value [org.codehaus.groovy.grails.commons.spring.ReloadAwareAutowireCapableBeanFactory$2@1115340]) and a value of type [java.lang.Boolean] (value [false]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.

        How to resolve this issue? Do you think there is a memory leak in ReloadAwareAutowireCapableBeanFactory class?
        Or am i doing something wrong?

        Show
        Chaitanya Kulkarni added a comment - @Graeme I am using tomcat 6.0.29, grails 1.3.7, and java 1.5 version. But when I undeploy or stop the tomcat server then i am getting below error in the catalina.log file of tomcat... SEVERE: The web application [/application] created a ThreadLocal with key of type [null] (value [org.codehaus.groovy.grails.commons.spring.ReloadAwareAutowireCapableBeanFactory$2@1115340] ) and a value of type [java.lang.Boolean] (value [false] ) but failed to remove it when the web application was stopped. This is very likely to create a memory leak. How to resolve this issue? Do you think there is a memory leak in ReloadAwareAutowireCapableBeanFactory class? Or am i doing something wrong?
        Hide
        Felipe Nascimento added a comment -

        I am using apache-tomcat-6.0.32 , grails 1.3.7, Java jdk1.6.0_24 on a CentoOS 5.5.
        My application just stops responding at some point and I can see the following logs:
        Any workaround?

        SEVERE: The web application [/nu] created a ThreadLocal with key of type [org.codehaus.groovy.grails.orm.hibernate.support.HibernatePersistenceContextInterceptor$2] (value [org.codehaus.groovy.grails.orm.hibernate.support.HibernatePersistenceContextInterceptor$2@7649a07e]) and a value of type [java.lang.Integer] (value [0]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.
        Jun 3, 2011 4:23:13 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
        SEVERE: The web application [/nu] created a ThreadLocal with key of type [org.codehaus.groovy.grails.orm.hibernate.validation.HibernateDomainClassValidator$1] (value [org.codehaus.groovy.grails.orm.hibernate.validation.HibernateDomainClassValidator$1@fb0a82c]) and a value of type [java.util.ArrayList] (value [[]]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.

        Show
        Felipe Nascimento added a comment - I am using apache-tomcat-6.0.32 , grails 1.3.7, Java jdk1.6.0_24 on a CentoOS 5.5. My application just stops responding at some point and I can see the following logs: Any workaround? SEVERE: The web application [/nu] created a ThreadLocal with key of type [org.codehaus.groovy.grails.orm.hibernate.support.HibernatePersistenceContextInterceptor$2] (value [org.codehaus.groovy.grails.orm.hibernate.support.HibernatePersistenceContextInterceptor$2@7649a07e] ) and a value of type [java.lang.Integer] (value [0] ) but failed to remove it when the web application was stopped. This is very likely to create a memory leak. Jun 3, 2011 4:23:13 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap SEVERE: The web application [/nu] created a ThreadLocal with key of type [org.codehaus.groovy.grails.orm.hibernate.validation.HibernateDomainClassValidator$1] (value [org.codehaus.groovy.grails.orm.hibernate.validation.HibernateDomainClassValidator$1@fb0a82c] ) and a value of type [java.util.ArrayList] (value [[]]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.
        Hide
        Graeme Rocher added a comment -

        That message is Tomcat stopping the application and automatically clearing out thread locals, but is not related to your application becoming unresponsive.

        @Chaitanya Tomcat is auto-preventing a memory leak in this case. There are a few thread locals here and there that are not cleared and I'll look and getting them cleared for 1.4 M2

        Show
        Graeme Rocher added a comment - That message is Tomcat stopping the application and automatically clearing out thread locals, but is not related to your application becoming unresponsive. @Chaitanya Tomcat is auto-preventing a memory leak in this case. There are a few thread locals here and there that are not cleared and I'll look and getting them cleared for 1.4 M2
        Hide
        Graeme Rocher added a comment -

        The errors seen here should be resolved by this commit:

        https://github.com/grails/grails-core/commit/0f2c0631a8ed47e1d0d34bc240a894eed4b86261

        If further errors are seen in 1.4 M2 please re-open

        Show
        Graeme Rocher added a comment - The errors seen here should be resolved by this commit: https://github.com/grails/grails-core/commit/0f2c0631a8ed47e1d0d34bc240a894eed4b86261 If further errors are seen in 1.4 M2 please re-open
        Hide
        Kwang Lee added a comment -

        Hi,
        I'm sorry but I'm still seeing this leak with tomcat 7 even with minimum application build by grails 2.
        First I tried to undeploy and see the following error log by checking tomcat's new leak detection

        SEVERE: The web application [/hellog2-0.1] appears to have started a thread named [Timer-1] but has failed to stop it. This is very likely to create a memory leak.
        16-Dec-2011 9:29:26 AM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
        SEVERE: The web application [/hellog2-0.1] created a ThreadLocal with key of type [org.codehaus.groovy.grails.orm.hibernate.metaclass.AbstractSavePersistentMethod$1] (value [org.codehaus.groovy.grails
        .orm.hibernate.metaclass.AbstractSavePersistentMethod$1@4552a650]) and a value of type [java.util.HashSet] (value [[]]) but failed to remove it when the web application was stopped. Threads are going
        to be renewed over time to try and avoid a probable memory leak.
        16-Dec-2011 9:29:57 AM org.apache.catalina.util.LifecycleBase stop
        INFO: The stop() method was called on component [StandardEngine[Catalina].StandardHost[localhost].StandardContext[/hellog2-0.1]] after stop() had already been called. The second call will be ignored.

        To be sure, I've monitored the PermGen graph from visual-gc, then found the PermGen stay same after undeployment done, and when I re-deploy the same application then the PermGen just increased.

        So, whenever I'm repeating undeploy/deploy the same app, the PermGen keeps increasing.

        The exact versions I've tested:

        Java 1.6.0_17 (build 1.6.0_17-b04)
        Grails 2.0.0
        apache-tomcat-7.0.22

        Show
        Kwang Lee added a comment - Hi, I'm sorry but I'm still seeing this leak with tomcat 7 even with minimum application build by grails 2. First I tried to undeploy and see the following error log by checking tomcat's new leak detection SEVERE: The web application [/hellog2-0.1] appears to have started a thread named [Timer-1] but has failed to stop it. This is very likely to create a memory leak. 16-Dec-2011 9:29:26 AM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks SEVERE: The web application [/hellog2-0.1] created a ThreadLocal with key of type [org.codehaus.groovy.grails.orm.hibernate.metaclass.AbstractSavePersistentMethod$1] (value [org.codehaus.groovy.grails .orm.hibernate.metaclass.AbstractSavePersistentMethod$1@4552a650]) and a value of type [java.util.HashSet] (value [[]]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak. 16-Dec-2011 9:29:57 AM org.apache.catalina.util.LifecycleBase stop INFO: The stop() method was called on component [StandardEngine [Catalina] .StandardHost [localhost] .StandardContext [/hellog2-0.1] ] after stop() had already been called. The second call will be ignored. To be sure, I've monitored the PermGen graph from visual-gc, then found the PermGen stay same after undeployment done, and when I re-deploy the same application then the PermGen just increased. So, whenever I'm repeating undeploy/deploy the same app, the PermGen keeps increasing. The exact versions I've tested: Java 1.6.0_17 (build 1.6.0_17-b04) Grails 2.0.0 apache-tomcat-7.0.22
        Hide
        Lari Hotari added a comment -

        This isn't related to the threadlocal problems, it's about the "jar locking" problem:
        http://docs.oracle.com/javase/7/docs/technotes/guides/net/ClassLoader.html

        Since Java 1.7.0 there is a close() method in URLClassLoader.

        Show
        Lari Hotari added a comment - This isn't related to the threadlocal problems, it's about the "jar locking" problem: http://docs.oracle.com/javase/7/docs/technotes/guides/net/ClassLoader.html Since Java 1.7.0 there is a close() method in URLClassLoader .
        Hide
        R added a comment - - edited

        Here is a simple (untested) patch for the issue with the AbstractSavePersistentMethod noted in the next-to-previous comment. The problem is that an (anonymous) subclass of ThreadLocal is being created, and that therefore keeps a reference to the classloader. By replacing the initialValue() method with a "manual" setting of the initial value we avoid that subclass and hence the issue. As I said, though, this is untested.

        Btw., the shutdown handler here is mostly useless: it only removes the thread-local entry for the thread that happens to run the shutdown handler, but does nothing for all others. I think it could just as well be removed.

        Show
        R added a comment - - edited Here is a simple (untested) patch for the issue with the AbstractSavePersistentMethod noted in the next-to-previous comment. The problem is that an (anonymous) subclass of ThreadLocal is being created, and that therefore keeps a reference to the classloader. By replacing the initialValue() method with a "manual" setting of the initial value we avoid that subclass and hence the issue. As I said, though, this is untested. Btw., the shutdown handler here is mostly useless: it only removes the thread-local entry for the thread that happens to run the shutdown handler, but does nothing for all others. I think it could just as well be removed.
        Hide
        R added a comment -

        I have actually tested the patch now in tomcat 7.0.27 and it works. In the course of testing I also noticed that tomcat was complaining about another place where thread-locals were causing a leak:

        org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
        SEVERE: The web application [/site-0.1] created a ThreadLocal with key of type [org.codehaus.groovy.grails.orm.hibernate.support.HibernatePersistenceContextInterceptor$2] (value [org.codehaus.groovy.grails.orm.hibernate.support.HibernatePersistenceContextInterceptor$2@57f16388]) and a value of type [java.lang.Boolean] (value [false]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
        org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
        SEVERE: The web application [/site-0.1] created a ThreadLocal with key of type [org.codehaus.groovy.grails.orm.hibernate.support.HibernatePersistenceContextInterceptor$3] (value [org.codehaus.groovy.grails.orm.hibernate.support.HibernatePersistenceContextInterceptor$3@28bda2d3]) and a value of type [java.lang.Integer] (value [0]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
        

        I have attached a second patch for this one (same problem, same solution - and previous comment about shutdown hook basically being useless applies here too).

        Show
        R added a comment - I have actually tested the patch now in tomcat 7.0.27 and it works. In the course of testing I also noticed that tomcat was complaining about another place where thread-locals were causing a leak: org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks SEVERE: The web application [/site-0.1] created a ThreadLocal with key of type [org.codehaus.groovy.grails.orm.hibernate.support.HibernatePersistenceContextInterceptor$2] (value [org.codehaus.groovy.grails.orm.hibernate.support.HibernatePersistenceContextInterceptor$2@57f16388]) and a value of type [java.lang.Boolean] (value [false]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak. org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks SEVERE: The web application [/site-0.1] created a ThreadLocal with key of type [org.codehaus.groovy.grails.orm.hibernate.support.HibernatePersistenceContextInterceptor$3] (value [org.codehaus.groovy.grails.orm.hibernate.support.HibernatePersistenceContextInterceptor$3@28bda2d3]) and a value of type [java.lang.Integer] (value [0]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak. I have attached a second patch for this one (same problem, same solution - and previous comment about shutdown hook basically being useless applies here too).
        Hide
        Graeme Rocher added a comment -

        I've applied to the patches, if problems persist please re-open

        Show
        Graeme Rocher added a comment - I've applied to the patches, if problems persist please re-open
        Hide
        Raphael Miranda added a comment -

        I've tested with master latest and I got:

        SEVERE: The web application [/leakapp] appears to have started a thread named [Timer-1] but has failed to stop it. This is very likely to create a memory leak.

        I created a virtual testing environment and sample app: https://github.com/xymor/grails-5823-test

        Show
        Raphael Miranda added a comment - I've tested with master latest and I got: SEVERE: The web application [/leakapp] appears to have started a thread named [Timer-1] but has failed to stop it. This is very likely to create a memory leak. I created a virtual testing environment and sample app: https://github.com/xymor/grails-5823-test
        Hide
        R added a comment -

        I noticed that too, but only when using H2 as the database, not when using PostgreSQL (i.e. only in dev/testing, not production). Don't know who the actual culprit is, though.

        Show
        R added a comment - I noticed that too, but only when using H2 as the database, not when using PostgreSQL (i.e. only in dev/testing, not production). Don't know who the actual culprit is, though.

          People

          • Assignee:
            Graeme Rocher
            Reporter:
            Daniel Mikusa
          • Votes:
            28 Vote for this issue
            Watchers:
            24 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:
              Last Reviewed:

              Development