Issue Details (XML | Word | Printable)

Key: HUDSON-5119
Type: Bug Bug
Status: Resolved Resolved
Resolution: Fixed
Priority: Blocker Blocker
Assignee: Unassigned
Reporter: tjuerge
Votes: 5
Watchers: 7
Operations

If you were logged in you would be able to see more operations.
Hudson

Winstone: Memory leak due to default session timeout of -1

Created: 17/Dec/09 09:24 AM   Updated: 02/Aug/10 01:18 PM   Resolved: 06/May/10 06:48 PM
Return to search
Component/s: core
Affects Version/s: None
Fix Version/s: None

File Attachments: None
Image Attachments:

1. VisualVM 1.2.2 4192010 12616 PM.jpg
(136 kB)

2. Winstone-Session-Leak.png
(64 kB)
Environment: Hudson 1.337 (Winstone), Solaris 10 SPARC, Sun JDK 1.6.0_16


 Description  « Hide

We are running Hudson with its embedded Winstone servlet engine. User authentication within Hudson is done via Hudsons LDAP security realm using our corporate ActiveDirectory via LDAP.

After running Hudson a few weeks we're running into OutOfMemory exceptions. Here the culprit is a single winstone.WebAppConfiguration instance holding a lot of winstone.WinStoneSession instances each holding an Acegi RememberMeAuthenticationToken instance with 36k data from the authenticated users AD record.

It seems that Hudson isn't setting Winstones session timeout. Without specifying a session timeout winstone.WebAppConfiguration.makeNewSession() is using a session timeout of -1.



Sort Order: Ascending order - Click to sort in descending order
tjuerge added a comment - 17/Dec/09 04:16 PM

This can be fixed by specifying a session timeout in hudson/main/war/resources/WEB-INF/web.xml:

Index: resources/WEB-INF/web.xml
===================================================================
--- resources/WEB-INF/web.xml	(revision 24689)
+++ resources/WEB-INF/web.xml	(working copy)
@@ -30,6 +30,10 @@
   <display-name>Hudson</display-name>
   <description>Build management system</description>
 
+  <session-config>
+    <session-timeout>60</session-timeout>
+  </session-config>
+
   <servlet>
     <servlet-name>Stapler</servlet-name>
     <servlet-class>org.kohsuke.stapler.Stapler</servlet-class>

tjuerge added a comment - 17/Dec/09 04:20 PM

Answered my question: winstone.HostConfiguration.run() is responsible for calling WebAppCOnfiguration.invalidateExpiredSessions()


slurdge added a comment - 15/Mar/10 02:36 AM

We have also the same issue.
I tried tjuerge patch to web.xml but it did not change anything (by the way I had to remove a sha1 - computed file to do this).
We have a ActiveDirectory that is configured via LDAP. Every connection leaks a little bit. The problem is worsened by the fact that we use a tray applet that checks the build status. This tray applet authenticates itself with Hudson. I believe that is creating a WinStoneSession each time, leaking memory slowly.
I've also looked at the memory dump and my findings are the same as tjuerge.
In one week, Hudson takes the maximum of 512Mb allocated and starts to fail builds.

Any solution is welcomed, I would be happy to try a build with a fix for this.
Also, if there is a workaround in forcing the flush of expired sessions, this would allow us to periodically do it and not interrupt operation by restarting Hudson (the only available option so far, but not very good for us).


StevenTheAmusing added a comment - 19/Apr/10 12:39 PM

We're also seeing a large number of objects 6,143 sessions with 17.9Mb. Can this timeout be added to the standard distribution?

We're going OOM as often as 3 times a day. When monit restarts Hudson we blow away the expanded WAR directory which messing with the web.xml a bit much pain.

We are also using the Hudson tray applet. Our primary OOM problem seems to be related to the Cobertura reports, by the way, but this is not a small leak either.

Hudson ver. 1.353 on Linux, Sun JDK 1.6.0_18


StevenTheAmusing added a comment - 19/Apr/10 12:41 PM

VisualVM showing a large number of sessions.


kohsuke added a comment - 23/Apr/10 07:52 PM

Fixed in Winstone 0.9.10-hudson-17 to have the finite default session timeout value. This change still needs to be integrated into Hudson.


amrangan added a comment - 26/Apr/10 09:08 AM

Will this change be integrated into the next release of Hudson (i.e. 1.356) ?


scm_issue_link added a comment - 06/May/10 06:48 PM

Code changed in hudson
User: : kohsuke
Path:
trunk/hudson/main/war/pom.xml
trunk/www/changelog.html
http://hudson-ci.org/commit/30797
Log:
[FIXED HUDSON-5119] integrated a newer version of Winstone with memory leak fix and SSL certificate handling improvement.


scm_issue_link added a comment - 02/Aug/10 11:36 AM

Code changed in hudson
User: : kohsuke
Path:
trunk/hudson/main/war/pom.xml
trunk/www/changelog.html
http://hudson-ci.org/commit/33533
Log:
[FIXED HUDSON-5119] integrated another Winstone that fixes session object leak.


dogfood added a comment - 02/Aug/10 01:18 PM

Integrated in hudson_main_trunk #154
[FIXED HUDSON-5119] integrated another Winstone that fixes session object leak.

kohsuke :
Files :

  • /trunk/hudson/main/war/pom.xml
  • /trunk/www/changelog.html