[Bug 64858] New: Allow to deploy a WAR whose dependencies are on a Maven repository

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

[Bug 64858] New: Allow to deploy a WAR whose dependencies are on a Maven repository

Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=64858

            Bug ID: 64858
           Summary: Allow to deploy a WAR whose dependencies are on a
                    Maven repository
           Product: Tomcat 10
           Version: unspecified
          Hardware: PC
                OS: All
            Status: NEW
          Severity: normal
          Priority: P2
         Component: Catalina
          Assignee: [hidden email]
          Reporter: [hidden email]
  Target Milestone: ------

The war in the Maven repository will not have any lib in WEB-INF/lib, Tomcat
should use Maven dependency resolving to get the dependencies.

Thanks to the missing WEB-INF/lib of the war its size will allow it to be
publish in a Maven repository without a disk usage explosion.

Tomcat could use directly the Maven resolver project
(https://github.com/apache/maven-resolver), however there is a lot of work on
the classloader construction.

I created Vestige (https://gaellalire.fr/vestige/) which is able to manage
classloader very precisely: you can group interdependent jar in same
classloader, you can tell which jar should not be shared because of its use of
static variable ...

I decided to create an API
(https://github.com/vestige-java/vestige.spi.resolver.maven) that dynamic
application can use to create and share classloader.

Then I modified some Tomcat to be sure my API is enough and the POC
(https://github.com/vestige-java-app/tomcat_vestige) is able to load a this WAR
(https://github.com/vestige-java-app/mywar) using it.

This POC was done on Tomcat 8.0.32, you could copy and correct the code in
Tomcat master or maybe better create an extension point API which will allow to
enable this functionality without overloading any Tomcat class.

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 64858] Allow to deploy a WAR whose dependencies are on a Maven repository

Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=64858

--- Comment #1 from Michael Osipov <[hidden email]> ---
Why don't you do this yourself with Resolver Ant Tasks? The likelihood to have
this in zTomcsg is almost zero.

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 64858] Allow to deploy a WAR whose dependencies are on a Maven repository

Bugzilla from bugzilla@apache.org
In reply to this post by Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=64858

--- Comment #2 from Gael Lalire <[hidden email]> ---
Yes, it is also possible to rebuild the WAR locally with maven resolver.
However you will have multiple copies of same lib in your disk (more disk
usage) and you lose the possibility to share lib's classes between WAR (more
memory usage).
In addition classloader created by Vestige are faster to load classes, because
it does not try each lib jar, it evaluates a minimized DFA to know in which jar
the class should be loaded.

Also I made not many changes to only 6 classes
(https://github.com/vestige-java-app/tomcat_vestige/commit/67dea6054c9da30047ebba3e9a376fa44b544f13#diff-e9240c9af00e44b55b8036434d325bea52d194dc43615ef31478c531d24751cc)
in Tomcat and one (WebappClassLoaderBase.java) does not count because it is
only removing illegal operations on JVM > 9 which newer version of Tomcat will
not have.

With an extension point all the code to create classloader in another way will
not be in Tomcat.

But I need someone, who find reduced disk and memory usage useful, to work on
how to create this extension point.

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 64858] Allow to deploy a WAR whose dependencies are on a Maven repository

Bugzilla from bugzilla@apache.org
In reply to this post by Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=64858

Remy Maucherat <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Severity|normal                      |enhancement

--- Comment #3 from Remy Maucherat <[hidden email]> ---
It's great and all, but such a proprietary extension with extra dependencies
has almost zero chances to be integrated with Tomcat. Anyway, this is an
enhancement.

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 64858] Allow to deploy a WAR whose dependencies are on a Maven repository

Bugzilla from bugzilla@apache.org
In reply to this post by Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=64858

--- Comment #4 from Gael Lalire <[hidden email]> ---
There is nothing proprietary, all my code is open source.

And by extension point I mean a Tomcat extension point.

So for example in Tomcat :

interface MainResourceSetLoader {
  String getExtension();
  DirResourceSet load(File file);
}

in Tomcat-vestige :

class VestigeMainResourceSetLoader implements MainResourceSetLoader {
  String getExtension() { return ".vwar" }
  DirResourceSet load(File file) { /* read the vwar and create the resourceset
*/ }
}


So in StandardRoot.java you can do

private MainResourceSetLoader externalMainResourceSetLoader;
...
  if (externalMainResourceSetLoader != null &&
    file.endsWith(externalMainResourceSetLoader.getExtension())) {
    mainResourceSet = externalMainResourceSetLoader.load(file);
  }

That is what I mean by extension point.

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 64858] Allow to deploy a WAR whose dependencies are on a Maven repository

Bugzilla from bugzilla@apache.org
In reply to this post by Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=64858

--- Comment #5 from Mark Thomas <[hidden email]> ---
I have always viewed WebResource and friends as the extension point for things
like this. Easy extensibility rather took a back seat as those interfaces
evolved but I'd be happy to consider changes to them to make it easier to
provide custom extensions (or complete replacement) of Tomcat's resources
implementation.

I think it highly unlikely the necessary extension points to add additional
deployment types identified by file extension will be added.

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 64858] Allow to deploy a WAR whose dependencies are on a Maven repository

Bugzilla from bugzilla@apache.org
In reply to this post by Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=64858

--- Comment #6 from Gael Lalire <[hidden email]> ---
Yes, WebResource are a terrific extension point. However it's a pity that
scanners don't use it and rely on constructed jar: URL.

Anyway I already use WebResource but I need an entry point.

As long as JPMS named module are not activated I can overload Tomcat classes by
a classpath hack.

But with JPMS named module activated I cannot use an already defined package
that why I propose this PR https://github.com/apache/tomcat/pull/375.

Of course I created the extension point for my need, and it may be changed to a
more generic way or split into multiple extension points.

Anyway I'm sorry to read that such integration has (almost) no change to get
merged and I would like to know why do you think that.

------------

I also saw you now provide generated named bundle through BND.
I tested it and got 3 issues :
- missing uses, you can either configure BND (el-api.jar.tmp.bnd) to add it or
add "ExpressionFactory.class.getModule().addUses(ExpressionFactory.class);" in
the ExpressionFactory class.
- missing Module package on org.apache.tomcat.jasper.el, the includepackage
option of jasper-el.jar.tmp.bnd seems ignored.
- JSP compilation failed, but I did not look onto it yet

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 64858] Allow to deploy a WAR whose dependencies are on a Maven repository

Bugzilla from bugzilla@apache.org
In reply to this post by Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=64858

--- Comment #7 from Mark Thomas <[hidden email]> ---
(In reply to Gael Lalire from comment #6)
> Yes, WebResource are a terrific extension point. However it's a pity that
> scanners don't use it and rely on constructed jar: URL.

Which code are you referring to here? The StandardJarScanner obtains URLs for
the JARs via the WebResources implementation.

> Anyway I already use WebResource but I need an entry point.

That is expected to be the definition of a custom implementation for one or
more components of the web resources implementation.

> As long as JPMS named module are not activated I can overload Tomcat classes
> by a classpath hack.
>
> But with JPMS named module activated I cannot use an already defined package
> that why I propose this PR https://github.com/apache/tomcat/pull/375.

My preference continues to be doing this via WebResources and making changes
there as necessary to permit extensions like this.

> Anyway I'm sorry to read that such integration has (almost) no change to get
> merged and I would like to know why do you think that.

Too invasive.


> I also saw you now provide generated named bundle through BND.
> I tested it and got 3 issues :

We are aware of those and they are being fixed via other bug reports.

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 64858] Allow to deploy a WAR whose dependencies are on a Maven repository

Bugzilla from bugzilla@apache.org
In reply to this post by Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=64858

--- Comment #8 from Gael Lalire <[hidden email]> ---
(In reply to Mark Thomas from comment #7)
> Which code are you referring to here? The StandardJarScanner obtains URLs
> for the JARs via the WebResources implementation.

StandardJarScanner.process is expecting either "jar:" or "file:" URL, so even
if WebResource is allowing any URL we are forced to use one of these.
The good think is that JarScannerCallback is now using a org.apache.tomcat.Jar
so if I can overload StandardJarScanner.process it will be good but :
- UrlJar is opening a connection and cast in JarURLConnection
- FragmentJarScannerCallback.extractJarFileName is expecting a jar URL.
- TagLibraryInfoImpl.<init> is opening a connection instead of using an
inexistent Jar.getLastModified(<no-arg>) which could call
WebResource.getLastModified
- StandardRoot.BaseLocation.<init> is expecting either jar or file. With jar it
is expected a file inside. Should also use JarFactory.newInstance instead.
- maybe other ...

However it is not an issue for the moment I can use file and jar URL and still
load from Maven repository.

(In reply to Mark Thomas from comment #7)
> My preference continues to be doing this via WebResources and making changes > there as necessary to permit extensions like this.

OK, I updated my PR putting only what I need. I changed only 5 tomcat files
with minor changes (https://github.com/apache/tomcat/pull/375/files). I
understand that the TomcatController I proposed is too invasive. Can you guide
me on how to transform it so it becomes acceptable ?

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]