Blog

13
February 2015

Gavin Pickin

How to get your Tomcat to pounce on startup, not crawl

Apache, CFML Server, Server Admin

As promised in my last few posts, I am finally going to give you a couple of Tomcat tips and tricks, that can seriously speed up your Tomcat startup time. In all fairness, they are simple configurations, and I am not the first to find them, not even the first CFMLer to find them. This is how I got my Tomcat startup from 164 seconds, down to 8 seconds, a gain of 156 seconds, in 1 line of code.

The first thing you have to remember, whether we are running Lucee, or Railo, we are running on the JVM. Tomcat is a Java Servlet and it is much more powerful than our lil old cfml engines. Knowing this, Tomcat has a few settings in given files that don’t apply to MOST of our CFML servers… the big one, is the JAR scanning.
Note: There could be an odd chance this might affect some weird setup, I’m just not sure what at this point.

"The Jar Scanner element represents the component that is used to scan the web application for JAR files and directories of class files. It is typically used during web application start to identify configuration files such as TLDs or web-fragment.xml files that must be processed as part of the web application initialization."

Taken from the Apache Tomcat 8 Configuration Referencehttp://tomcat.apache.org/tomcat-8.0-doc/config/jar-scanner.html

For every host / context, tomcat is scanning every folder it knows about (slight exaggeration), and you can see in the logs that it fails to find TLDs and configuration information, that it warns you that you could save time.

org.apache.jasper.servlet.TldScanner.scanJars At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.

So we are wasting time here, great, how do we fix it. There are two solutions.

// File: conf/context.xml

 

Inside the Context element, you have watched resources, but you can also set JarScanner with some params. By default scanAllDirectories, scanAllFiles, and scanBootstrapClassPath are set to false, but scanClassPath is true. Setting this, will set that to false too.

<JarScanner scanClassPath="false" />

 

Restart Tomcat, and you should see the startup time disappear.
My dev machine right now, has only a few Host setup in my Server.xml, since i have been experimenting with lots of alternatives to using manually coded Hosts, but even with that, my startup times looked like this.

12-Feb-2015 21:43:39.592 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 23576 ms
// Versus 
12-Feb-2015 21:48:40.840 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 3287 ms

 

Now, if you are concerned, and you want some of your JARs to be scanned, the 2nd option will be better for you.
Note, I am referring to Tomcat 8, Tomcat 7 had 3 settings in the catalina.properties, not just 1.

// File: conf/catalina.properties approx line 96 for standard Lucee Express

tomcat.util.scan.StandardJarScanFilter.jarsToSkip= \
bootstrap.jar,commons-daemon.jar,tomcat-juli.jar,\
annotations-api.jar,el-api.jar,jsp-api.jar,servlet-api.jar,websocket-api.jar,\
catalina.jar,catalina-ant.jar,catalina-ha.jar,catalina-storeconfig.jar,\
catalina-tribes.jar,\
jasper.jar,jasper-el.jar,ecj-*.jar,\
tomcat-api.jar,tomcat-util.jar,tomcat-util-scan.jar,tomcat-coyote.jar,\
tomcat-dbcp.jar,tomcat-jni.jar,tomcat-spdy.jar,tomcat-websocket.jar,\
tomcat-i18n-en.jar,tomcat-i18n-es.jar,tomcat-i18n-fr.jar,tomcat-i18n-ja.jar,\
tomcat-juli-adapters.jar,catalina-jmx-remote.jar,catalina-ws.jar,\
tomcat-jdbc.jar,\
tools.jar,\
commons-beanutils*.jar,commons-codec*.jar,commons-collections*.jar,\
commons-dbcp*.jar,commons-digester*.jar,commons-fileupload*.jar,\
commons-httpclient*.jar,commons-io*.jar,commons-lang*.jar,commons-logging*.jar,\
commons-math*.jar,commons-pool*.jar,\
jstl.jar,\
geronimo-spec-jaxrpc*.jar,wsdl4j*.jar,\
ant.jar,ant-junit*.jar,aspectj*.jar,jmx.jar,h2*.jar,hibernate*.jar,httpclient*.jar,\
jmx-tools.jar,jta*.jar,log4j*.jar,mail*.jar,slf4j*.jar,\
xercesImpl.jar,xmlParserAPIs.jar,xml-apis.jar,\
junit.jar,junit-*.jar,ant-launcher.jar,\
cobertura-*.jar,asm-*.jar,dom4j-*.jar,icu4j-*.jar,jaxen-*.jar,jdom-*.jar,\
jetty-*.jar,oro-*.jar,servlet-api-*.jar,tagsoup-*.jar,xmlParserAPIs-*.jar,\
xom-*.jar

 

This property lists all of the jars to skip… so if you want, you can name all of the jars to skip, but it extensive, and daunting.
Or you can do the equivalent of the last setting, and set this line to

tomcat.util.scan.StandardJarScanFilter.jarsToSkip=*

 

This will skip all JARS, and you will see almost identical startup.
That is all of the magic, its 1 setting, and it saves about 5-6 seconds per Host on each and every startup. I previous had 20-30 sites and was waiting for up to 5 minutes, which is ridiculous. Especially considering we do not need scanning.

As I mentioned earlier, Tony Junkes was looking into this when he first moved to Tomcat 8, and posted his article on Fixing Slow Railo Startup on Tomcat 8
Tony goes into more details on this findings, plus a few other bugs he found, so I will not repeat that, please go read his article.

At the same time we posted that, I was still running Tomcat 7 and round that there were two other settings, shown below, but the best performance increase was the same setting still available in Tomcat 8

org.apache.catalina.startup.ContextConfig.jarsToSkip=* = 150 second gain
org.apache.catalina.startup.TldConfig.jarsToSkip=*     = No real gain
tomcat.util.scan.DefaultJarScanner.jarsToSkip=*  156 second gain

Although this post is aimed at Tomcat 8, if you are still on Tomcat 7, it still might be fruitful to test this out.

If anyone knows why you might need to leave this on, in our normal CFML running servers, please share that information.

I hope it was worth the wait, go try it.
Please give some feedback in the comments to the BEFORE and AFTER times, and the number of Host records if you are up to counting them, please.

My dev server before the Host cutting was 95% savings on 25 hosts
Current dev server, it was 86.1% savings with 3 hosts.

by Jamie Jackson
02/13/2015 01:24:06 PM

Thanks! In my local development environment, it knocked the time down from 20-40s to 1.5s. Now I'll just wait to see what blows up.

by Jean
02/16/2015 01:48:24 AM

Hello,

Thanks Gavin for posting this.

I have a question about what you call startup times: is it the time it takes for the Tomcat service to restart, or is it the time it takes to get a response when first requesting a Lucee page?

In my case (Tomcat7, Lucee 4.5), I was not able to improve the time it takes Lucee to first answer after the Tomcat service restarts (service restarts in 3 to 5 seconds, first hit takes a solid 20 seconds).

by Gavin
02/16/2015 10:36:10 AM

Are you using mod_cfml? If you are using mod_cfml to create the Host Context, it will take 12-15 seconds for the first response.

The startup times I refer to are in the Catalina log file itself. You should see a line like I included in my post, like the following:

org.apache.catalina.startup.Catalina.start Server startup in 3287 ms

This focuses on Catalina / Tomcat starting up. At that point, there will be other initialization... I think I mentioned earlier with the Pros and Cons with mod_cfml, a normal page is about 3 seconds for first request with lucee vs 15 ish if you have mod_cfml creating the Host Context.

Of course, these times are based on my Mac Book Pro. Look in yout catalina log, and see what it shown there.

This will help you if you have a lot of Hosts in your Server.xml. If you use Mod_CFML it saves you startup time, but you will pay for it with first request on those hosts.

Read my Pros vs Cons on Mod_CFML for more info on my findings there.

by Julian Halliwell
02/17/2015 01:29:20 AM

Excellent info, Gavin, thanks. Poor startup times have been really troubling me as I add contexts. Currently have 23 of them on Lucee/Tomcat 7 and startups have been exceeding 150 seconds. Now down to 13 seconds in dev.

Would this mean, though, that newly installed extensions wouldn't be picked up? These may involve adding new jars/TLDs to contexts.

by Jean
02/17/2015 08:01:35 AM

No, not using mod_cfml, using ajp directly. I will check the catalina logs for startup times. Thanks!

by Gavin
02/17/2015 08:48:08 AM

Julian, that is the big question. I haven't run across any issues with it yet, but I haven't tried extensions.

My understanding would be if those JARs had configuration information needed for the context at loading, it might be an issue... but I'm not sure if some/all/none of the extensions do this... my experience with extensions is limited.

Most Java Access I deal with is through javaLoaders.

If you use extensions, experiment when you have time, and if you find anything, please share.

by Julian Halliwell
02/18/2015 01:24:39 AM

Hi Gavin. I just installed the spreadsheet extension to one of my local Lucee apps/contexts and am pleased to report that it worked as expected after restarting, despite the Tomcat jar scanning being disabled.

I suspect that Lucee does its own jar scanning independently of Tomcat.

Blog Search