[Bug 63685] New: WebappClassLoaderBase and StandardRoot reload classes from jar files on each call of getResource, getResourceAsStream and etc. that hit application startup performance

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

[Bug 63685] New: WebappClassLoaderBase and StandardRoot reload classes from jar files on each call of getResource, getResourceAsStream and etc. that hit application startup performance

Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=63685

            Bug ID: 63685
           Summary: WebappClassLoaderBase and StandardRoot reload classes
                    from jar files on each call of getResource,
                    getResourceAsStream and etc. that hit application
                    startup performance
           Product: Tomcat 8
           Version: 8.5.x-trunk
          Hardware: All
                OS: All
            Status: NEW
          Severity: normal
          Priority: P2
         Component: Catalina
          Assignee: [hidden email]
          Reporter: [hidden email]
  Target Milestone: ----

During profiling of the application I found interesting behavior related to
class loading and caching in org.apache.catalina.loader.WebappClassLoaderBase
and org.apache.catalina.webresources.StandardRoot.

Situation.
I have very big Spring application which consists of a bit more than 1500 jar
files.
Startup of this application takes quite a lot of time. After investigation with
logs analysis, profiling and debugging including Tomcat source code it was
found out that one of the reasons is laid in Tomcat classloading behavior.
A lot of methods of Tomcat's org.apache.catalina.loader.WebappClassLoaderBase
calls for org.apache.catalina.WebResourceRoot#getClassLoaderResource(String
path), for example:
- public URL getResource(String name);
- public URL findResource(final String name);
- public InputStream getResourceAsStream(String name).

Every time as one of these methods is invoked for .class files it leads to
invocation of
org.apache.catalina.webresources.StandardRoot#getResourceInternal(String path,
boolean useClassLoaderResources) method, which searches for the class file
across all the jars in application WEB-INF\lib directory. And in my case it's
about 1500 jar files.


In org.apache.catalina.webresources.StandardRoot already there is cache for
WebResources. But currently it's used for static content only.
There is special check with comment in
org.apache.catalina.webresources.Cache.class:

private boolean noCache(String path) {
        // Don't cache classes. The class loader handles this.
        // Don't cache JARs. The ResourceSet handles this.
        if ((path.endsWith(".class") &&
                (path.startsWith("/WEB-INF/classes/") ||
path.startsWith("/WEB-INF/lib/")))
                ||
                (path.startsWith("/WEB-INF/lib/") && path.endsWith(".jar"))) {
            return true;
        }
        return false;
    }
Comment tells that classes cache is handled by class loader, but in fact
classloader caches only instances of the created classes, but not the content
of the .class files or their URLs.


After modification of the noCache method as below:
private boolean noCache(String path) {
        // Don't cache JARs. The ResourceSet handles this.
        return (path.startsWith("/WEB-INF/lib/") && path.endsWith(".jar"));
    }
and setting cacheMaxSize and cacheTtl parameters, time of application startup
was reduced in average about 5 minutes.
I understand that it's not an issue for a small application, but as we see from
the result in case of a huge application it can give noticeable speedup of
application launch.


In my case I faced with performance problem mainly in case of application
startup but theoretically this behavior can bring negative impact on work of
application in runtime as well.

Would you be so kind to let me know why classes cache was intentionally skipped
in org.apache.catalina.webresources.Cache.class and if it's possible to enable
it and promote such fix?
Or if it's a bad idea what are the potential drawbacks I can face if I decide
to enable cache of classes in Tomcat for my application?

Thank you!

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 63685] WebappClassLoaderBase and StandardRoot reload classes from jar files on each call of getResource, getResourceAsStream and etc. that hit application startup performance

Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=63685

Alexandr <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Severity|normal                      |major

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 63685] WebappClassLoaderBase and StandardRoot reload classes from jar files on each call of getResource, getResourceAsStream and etc. that hit application startup performance

Bugzilla from bugzilla@apache.org
In reply to this post by Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=63685

Alexandr <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Hardware|All                         |PC

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 63685] WebappClassLoaderBase and StandardRoot reload classes from jar files on each call of getResource, getResourceAsStream and etc. that hit application startup performance

Bugzilla from bugzilla@apache.org
In reply to this post by Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=63685

--- Comment #1 from Mark Thomas <[hidden email]> ---
It is expected that a class file is read once and once only when the class is
loaded into memory. Therefore there is no point caching the contents of a class
file.

What use case requires that classes are loaded as resource files rather than
classes? If there is sufficient justification we can always add an option to
cache class files.

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 63685] WebappClassLoaderBase and StandardRoot reload classes from jar files on each call of getResource, getResourceAsStream and etc. that hit application startup performance

Bugzilla from bugzilla@apache.org
In reply to this post by Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=63685

Mark Thomas <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |NEEDINFO

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 63685] WebappClassLoaderBase and StandardRoot reload classes from jar files on each call of getResource, getResourceAsStream and etc. that hit application startup performance

Bugzilla from bugzilla@apache.org
In reply to this post by Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=63685

--- Comment #2 from Mark Thomas <[hidden email]> ---
Leaving this open for now but it will be closed as WONTFIX if no justification
for caching classes is provided.

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 63685] WebappClassLoaderBase and StandardRoot reload classes from jar files on each call of getResource, getResourceAsStream and etc. that hit application startup performance

Bugzilla from bugzilla@apache.org
In reply to this post by Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=63685

--- Comment #3 from Andrei Ivanov <[hidden email]> ---
I remember having a problem with AOP and load time weaving where Tomcat was
scanning the WAR for servlet annotations and caching the classes so by the time
the app was starting and LTW enabled, the classloader was returning the cached
classes instead of passing them through the instrumentation agent,

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]