Out of memory and issues with running with 64-bit Java

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Out of memory and issues with running with 64-bit Java

PetrikL
Hi,
I have 2 issues and was wondering, if anybody already solved them somehow.

1. I have a lot of tests and each produces quite some output. The Hudson seems to aggregate the tests in junitResults.xml file. For some jobs this file has anywhere from 100-500MBs and this is causing a serious problems with memory. To successfully parse this 500 junit XML file Hudson needs around 7GB of memory. Any ways to reduce the memory footprint?
2. When running Hudson with 64-bit Java on Solaris the --daemon option doesn't seem to work. It reports that -Xmx8192 is invalid value (too high). I guess is somehow still starts in 32-bit. How do you start Hudson with 64-bit as a daemon?

Many thanks,
   Lubos.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Out of memory and issues with running with 64-bit Java

Bruno Bonacci
Hi,
one of my build process more then 10.000 tests (2500 xml files) and I use Java-32bit.
My advice is:

1) check the content and the size of your xml files, if they are too big it might be due to excessive console output during the tests. In this case probably the easiest way is to reduce the log level or configure the logger to dump output in a file (then accessible via workspace)

2) -Xmx8192 is invalid value
the correct settings would be
-Xmx8192M, as otherwise you try to set as max memory only 8Kb.

bye
Bruno
Reply | Threaded
Open this post in threaded view
|

Re: Out of memory and issues with running with 64-bit Java

PetrikL
Thanks Bruno for answering, but I still have issues.

> My advice is:
>
> 1) check the content and the size of your xml files, if they are too big it
> might be due to excessive console output during the tests. In this case
> probably the easiest way is to reduce the log level or configure the logger
> to dump output in a file (then accessible via workspace)

That is exactly true. A lot of console output. However I need to keep it for future reference. I was thinking maybe storing the complete individual XML files somewhere in the workspace and before Hudson parses the results, create a trimmed version of console output for successfully completed tests (most of them). Any better ideas?

>
> 2) -Xmx8192 is invalid value
> the correct settings would be
> -Xmx8192M, as otherwise you try to set as max memory only 8Kb.

My bad, of course I used -Xmx8192M, but if I do that I get:
Forking into background to run as a daemon.
Invalid maximum heap size: -Xmx8192M
The specified size exceeds the maximum representable size.
Could not create the Java virtual machine.

Is anyone able to run Hudson with 64-bit Java (with lots of memory) as a daemon?

Thanks,
   Lubos.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Out of memory and issues with running with 64-bit Java

Bruno Bonacci
HI,

for the first point:
if your project is a java project and you are using a log system like log4j it's very easy to dump the full debug log into a file and only the errors into the console.
If you need the log for future reference you can add them as "archive artifacts" and they will be stored on every build. (This is the way I solved this problem in my builds)

for the second point:
I haven't tried to run hudson with 64-bit. But I guess is only matter to make sure that the env is set properly and JAVA_HOME as well as the PATH are pointing to the 64-bit.
Regarding the 8Gb memmory, unless your server has not enough memory, if you are using java64 bit you can allocate even more (One of my projects uses 20Gb as Xmx).

to make sure that hudson uses the right java version
add an explict;
export JAVA_HOME=/path/to/java64
export PATH=$JAVA_HOME/bin:$PATH

in the hudson startup script.

bye
bruno