Java 9 TC shutdown warnings (reflection in WebappClassLoaderBase)

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

Java 9 TC shutdown warnings (reflection in WebappClassLoaderBase)

Rainer Jung-3
Hi all,

when running TC 8.5.21 (9.0 should be the same) with Java 9 I get the
following warnings on STDOUT during shutdown:

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by
org.apache.catalina.loader.WebappClassLoaderBase
(file:/.../lib/catalina.jar) to field java.lang.Thread.threadLocals
WARNING: Please consider reporting this to the maintainers of
org.apache.catalina.loader.WebappClassLoaderBase
WARNING: Use --illegal-access=warn to enable warnings of further illegal
reflective access operations
WARNING: All illegal access operations will be denied in a future release


If I add the mentioned flag, I get more such lines, all referring to
org.apache.catalina.loader.WebappClassLoaderBase and then:

field java.lang.Thread.threadLocals
field java.lang.Thread.inheritableThreadLocals
field java.lang.ThreadLocal$ThreadLocalMap.table
method java.lang.ThreadLocal$ThreadLocalMap.expungeStaleEntries()
field java.lang.ThreadLocal$ThreadLocalMap$Entry.value
field sun.rmi.transport.Target.ccl
field sun.rmi.transport.Target.stub
field sun.rmi.transport.ObjectTable.objTable
field sun.rmi.transport.ObjectTable.implTable

Details may vary depending on the cleanup flags set in the loader, but
this is 8.5 with default settings.

Regards,

Rainer

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Java 9 TC shutdown warnings (reflection in WebappClassLoaderBase)

markt
On 23/09/17 15:55, Rainer Jung wrote:
> Hi all,
>
> when running TC 8.5.21 (9.0 should be the same) with Java 9 I get the
> following warnings on STDOUT during shutdown:

Tomcat 9 has the Java 9 fixes. 8.5.x doesn't. I've put together a list
of the Java 9 commits that need to be back-ported. Now that Java 9 is
final those back-ports need to happen.

This is on my TODO list for after TomcatCon London on Tuesday.

Mark


>
> WARNING: An illegal reflective access operation has occurred
> WARNING: Illegal reflective access by
> org.apache.catalina.loader.WebappClassLoaderBase
> (file:/.../lib/catalina.jar) to field java.lang.Thread.threadLocals
> WARNING: Please consider reporting this to the maintainers of
> org.apache.catalina.loader.WebappClassLoaderBase
> WARNING: Use --illegal-access=warn to enable warnings of further illegal
> reflective access operations
> WARNING: All illegal access operations will be denied in a future release
>
>
> If I add the mentioned flag, I get more such lines, all referring to
> org.apache.catalina.loader.WebappClassLoaderBase and then:
>
> field java.lang.Thread.threadLocals
> field java.lang.Thread.inheritableThreadLocals
> field java.lang.ThreadLocal$ThreadLocalMap.table
> method java.lang.ThreadLocal$ThreadLocalMap.expungeStaleEntries()
> field java.lang.ThreadLocal$ThreadLocalMap$Entry.value
> field sun.rmi.transport.Target.ccl
> field sun.rmi.transport.Target.stub
> field sun.rmi.transport.ObjectTable.objTable
> field sun.rmi.transport.ObjectTable.implTable
>
> Details may vary depending on the cleanup flags set in the loader, but
> this is 8.5 with default settings.
>
> Regards,
>
> Rainer
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Java 9 TC shutdown warnings (reflection in WebappClassLoaderBase)

Rainer Jung-3
Thanks Mark!

You might also want to look at r1809434 as a candidate and check whether
that workaround looks OK for you.

Wishing you a nice and successful TomcatCon!

Regards,

Rainer

Am 23.09.2017 um 17:01 schrieb Mark Thomas:

> On 23/09/17 15:55, Rainer Jung wrote:
>> Hi all,
>>
>> when running TC 8.5.21 (9.0 should be the same) with Java 9 I get the
>> following warnings on STDOUT during shutdown:
>
> Tomcat 9 has the Java 9 fixes. 8.5.x doesn't. I've put together a list
> of the Java 9 commits that need to be back-ported. Now that Java 9 is
> final those back-ports need to happen.
>
> This is on my TODO list for after TomcatCon London on Tuesday.
>
> Mark
>
>
>>
>> WARNING: An illegal reflective access operation has occurred
>> WARNING: Illegal reflective access by
>> org.apache.catalina.loader.WebappClassLoaderBase
>> (file:/.../lib/catalina.jar) to field java.lang.Thread.threadLocals
>> WARNING: Please consider reporting this to the maintainers of
>> org.apache.catalina.loader.WebappClassLoaderBase
>> WARNING: Use --illegal-access=warn to enable warnings of further illegal
>> reflective access operations
>> WARNING: All illegal access operations will be denied in a future release
>>
>>
>> If I add the mentioned flag, I get more such lines, all referring to
>> org.apache.catalina.loader.WebappClassLoaderBase and then:
>>
>> field java.lang.Thread.threadLocals
>> field java.lang.Thread.inheritableThreadLocals
>> field java.lang.ThreadLocal$ThreadLocalMap.table
>> method java.lang.ThreadLocal$ThreadLocalMap.expungeStaleEntries()
>> field java.lang.ThreadLocal$ThreadLocalMap$Entry.value
>> field sun.rmi.transport.Target.ccl
>> field sun.rmi.transport.Target.stub
>> field sun.rmi.transport.ObjectTable.objTable
>> field sun.rmi.transport.ObjectTable.implTable
>>
>> Details may vary depending on the cleanup flags set in the loader, but
>> this is 8.5 with default settings.
>>
>> Regards,
>>
>> Rainer

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Java 9 TC shutdown warnings (reflection in WebappClassLoaderBase)

markt
On 23/09/17 16:10, Rainer Jung wrote:
> Thanks Mark!
>
> You might also want to look at r1809434 as a candidate and check whether
> that workaround looks OK for you.

Looks good to me. I've added to the list of J9 back-port candidates. It
does beg the question whether or not we want to look at switching to
multi-release JARs for all of the compatibility code. WDYT?

> Wishing you a nice and successful TomcatCon!

Thanks!

Mark


>
> Regards,
>
> Rainer
>
> Am 23.09.2017 um 17:01 schrieb Mark Thomas:
>> On 23/09/17 15:55, Rainer Jung wrote:
>>> Hi all,
>>>
>>> when running TC 8.5.21 (9.0 should be the same) with Java 9 I get the
>>> following warnings on STDOUT during shutdown:
>>
>> Tomcat 9 has the Java 9 fixes. 8.5.x doesn't. I've put together a list
>> of the Java 9 commits that need to be back-ported. Now that Java 9 is
>> final those back-ports need to happen.
>>
>> This is on my TODO list for after TomcatCon London on Tuesday.
>>
>> Mark
>>
>>
>>>
>>> WARNING: An illegal reflective access operation has occurred
>>> WARNING: Illegal reflective access by
>>> org.apache.catalina.loader.WebappClassLoaderBase
>>> (file:/.../lib/catalina.jar) to field java.lang.Thread.threadLocals
>>> WARNING: Please consider reporting this to the maintainers of
>>> org.apache.catalina.loader.WebappClassLoaderBase
>>> WARNING: Use --illegal-access=warn to enable warnings of further illegal
>>> reflective access operations
>>> WARNING: All illegal access operations will be denied in a future
>>> release
>>>
>>>
>>> If I add the mentioned flag, I get more such lines, all referring to
>>> org.apache.catalina.loader.WebappClassLoaderBase and then:
>>>
>>> field java.lang.Thread.threadLocals
>>> field java.lang.Thread.inheritableThreadLocals
>>> field java.lang.ThreadLocal$ThreadLocalMap.table
>>> method java.lang.ThreadLocal$ThreadLocalMap.expungeStaleEntries()
>>> field java.lang.ThreadLocal$ThreadLocalMap$Entry.value
>>> field sun.rmi.transport.Target.ccl
>>> field sun.rmi.transport.Target.stub
>>> field sun.rmi.transport.ObjectTable.objTable
>>> field sun.rmi.transport.ObjectTable.implTable
>>>
>>> Details may vary depending on the cleanup flags set in the loader, but
>>> this is 8.5 with default settings.
>>>
>>> Regards,
>>>
>>> Rainer
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Multi-release Jars [Was: Java 9 TC shutdown warnings (reflection in WebappClassLoaderBase)]

Rainer Jung-3
Am 23.09.2017 um 17:22 schrieb Mark Thomas:
> On 23/09/17 16:10, Rainer Jung wrote:
>> Thanks Mark!
>>
>> You might also want to look at r1809434 as a candidate and check whether
>> that workaround looks OK for you.
>
> Looks good to me. I've added to the list of J9 back-port candidates. It
> does beg the question whether or not we want to look at switching to
> multi-release JARs for all of the compatibility code. WDYT?

Indeed, I ran my first multi-release Jar experiment (successfully)
today. For those who haven't yet seen it: starting with Java 9 one can
include instances of a class in a jar file that will only get used when
a specific JVM (major) version is detected at runtime. You can e.g. have
a default one as usual plus additional ones for specific Java runtime
versions. For Java 9 one would add a class at
META-INF/versions/9/org/apache/juli/... which would then be picked when
the runtime Java version is 9.

The enable the feature, one simply has to add the attribute

Multi-Release: true

to the Manifest file of the jar.

Thinks that come to my mind:

- you can have multiple versions of the same class. If we would use it
like that, we would have to slightly reorganize our svn layout to
reflect that, e.g. java/org/... plus new java9/org/....

- accordingly we would need e.g. a directory output/classes9 or similar.

- If we need Java 9 for compilation of the Java 9 classes, we would
mimic the Java 7 lines from the TC 7 build.xml.

- To add the Java 9 classes to one of our jars it would suffice in
build.xml to do something like

<jar jarfile="${somefile.jar}" update="true">
   <manifest>
     <attribute name="Multi-Release" value="true"/>
   </manifest>
   <zipfileset prefix="META-INF/versions/9/" dir="${tomcat.classes}9"/>
</jar>

- to reduce redundant maintenance it would be good to factor out JVM
dependent code, so that the classes we have to maintain multiply mostly
contain the code that's really version-dependent.

For instance when using the feature for my Java 9 change in Juli
http://svn.apache.org/viewvc?rev=1809434&view=rev one could use e.g. a
small but version dependent Constants.java which would provide the
correct directory name ("lib" versus "conf"). Or more easily a one line
properties file available in two versions providing the directory name.

I have not made any check on performance implications for the slightly
more complex class loading. What I did see, is that -verbose:class does
not indicate, which of the versions was chosen.

I currently do not have a real insight on how our most important JVM
version dependencies look. I guess how useful multi-release is, might
depend on the type of code dependencies we have.

Regards,

Rainer


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Multi-release Jars [Was: Java 9 TC shutdown warnings (reflection in WebappClassLoaderBase)]

Rainer Jung-3
Forgot the reference: http://openjdk.java.net/jeps/238

Am 23.09.2017 um 20:01 schrieb Rainer Jung:

> Am 23.09.2017 um 17:22 schrieb Mark Thomas:
>> On 23/09/17 16:10, Rainer Jung wrote:
>>> Thanks Mark!
>>>
>>> You might also want to look at r1809434 as a candidate and check whether
>>> that workaround looks OK for you.
>>
>> Looks good to me. I've added to the list of J9 back-port candidates. It
>> does beg the question whether or not we want to look at switching to
>> multi-release JARs for all of the compatibility code. WDYT?
>
> Indeed, I ran my first multi-release Jar experiment (successfully)
> today. For those who haven't yet seen it: starting with Java 9 one can
> include instances of a class in a jar file that will only get used when
> a specific JVM (major) version is detected at runtime. You can e.g. have
> a default one as usual plus additional ones for specific Java runtime
> versions. For Java 9 one would add a class at
> META-INF/versions/9/org/apache/juli/... which would then be picked when
> the runtime Java version is 9.
>
> The enable the feature, one simply has to add the attribute
>
> Multi-Release: true
>
> to the Manifest file of the jar.
>
> Thinks that come to my mind:
>
> - you can have multiple versions of the same class. If we would use it
> like that, we would have to slightly reorganize our svn layout to
> reflect that, e.g. java/org/... plus new java9/org/....
>
> - accordingly we would need e.g. a directory output/classes9 or similar.
>
> - If we need Java 9 for compilation of the Java 9 classes, we would
> mimic the Java 7 lines from the TC 7 build.xml.
>
> - To add the Java 9 classes to one of our jars it would suffice in
> build.xml to do something like
>
> <jar jarfile="${somefile.jar}" update="true">
>    <manifest>
>      <attribute name="Multi-Release" value="true"/>
>    </manifest>
>    <zipfileset prefix="META-INF/versions/9/" dir="${tomcat.classes}9"/>
> </jar>
>
> - to reduce redundant maintenance it would be good to factor out JVM
> dependent code, so that the classes we have to maintain multiply mostly
> contain the code that's really version-dependent.
>
> For instance when using the feature for my Java 9 change in Juli
> http://svn.apache.org/viewvc?rev=1809434&view=rev one could use e.g. a
> small but version dependent Constants.java which would provide the
> correct directory name ("lib" versus "conf"). Or more easily a one line
> properties file available in two versions providing the directory name.
>
> I have not made any check on performance implications for the slightly
> more complex class loading. What I did see, is that -verbose:class does
> not indicate, which of the versions was chosen.
>
> I currently do not have a real insight on how our most important JVM
> version dependencies look. I guess how useful multi-release is, might
> depend on the type of code dependencies we have.
>
> Regards,
>
> Rainer

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Java 9 TC shutdown warnings (reflection in WebappClassLoaderBase)

Coty Sutherland
In reply to this post by markt
On Sat, Sep 23, 2017 at 11:01 AM, Mark Thomas <[hidden email]> wrote:
> On 23/09/17 15:55, Rainer Jung wrote:
>> Hi all,
>>
>> when running TC 8.5.21 (9.0 should be the same) with Java 9 I get the
>> following warnings on STDOUT during shutdown:
>
> Tomcat 9 has the Java 9 fixes. 8.5.x doesn't. I've put together a list
> of the Java 9 commits that need to be back-ported. Now that Java 9 is
> final those back-ports need to happen.

Have you shared/can you share the list of commits? I'd like to take a
look and see if I can help get 7.0.x and 8.0.x ready to go too.

> This is on my TODO list for after TomcatCon London on Tuesday.
>
> Mark
>
>
>>
>> WARNING: An illegal reflective access operation has occurred
>> WARNING: Illegal reflective access by
>> org.apache.catalina.loader.WebappClassLoaderBase
>> (file:/.../lib/catalina.jar) to field java.lang.Thread.threadLocals
>> WARNING: Please consider reporting this to the maintainers of
>> org.apache.catalina.loader.WebappClassLoaderBase
>> WARNING: Use --illegal-access=warn to enable warnings of further illegal
>> reflective access operations
>> WARNING: All illegal access operations will be denied in a future release
>>
>>
>> If I add the mentioned flag, I get more such lines, all referring to
>> org.apache.catalina.loader.WebappClassLoaderBase and then:
>>
>> field java.lang.Thread.threadLocals
>> field java.lang.Thread.inheritableThreadLocals
>> field java.lang.ThreadLocal$ThreadLocalMap.table
>> method java.lang.ThreadLocal$ThreadLocalMap.expungeStaleEntries()
>> field java.lang.ThreadLocal$ThreadLocalMap$Entry.value
>> field sun.rmi.transport.Target.ccl
>> field sun.rmi.transport.Target.stub
>> field sun.rmi.transport.ObjectTable.objTable
>> field sun.rmi.transport.ObjectTable.implTable
>>
>> Details may vary depending on the cleanup flags set in the loader, but
>> this is 8.5 with default settings.
>>
>> Regards,
>>
>> Rainer
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Java 9 TC shutdown warnings (reflection in WebappClassLoaderBase)

markt
On 25/09/17 13:33, Coty Sutherland wrote:

> On Sat, Sep 23, 2017 at 11:01 AM, Mark Thomas <[hidden email]> wrote:
>> On 23/09/17 15:55, Rainer Jung wrote:
>>> Hi all,
>>>
>>> when running TC 8.5.21 (9.0 should be the same) with Java 9 I get the
>>> following warnings on STDOUT during shutdown:
>>
>> Tomcat 9 has the Java 9 fixes. 8.5.x doesn't. I've put together a list
>> of the Java 9 commits that need to be back-ported. Now that Java 9 is
>> final those back-ports need to happen.
>
> Have you shared/can you share the list of commits? I'd like to take a
> look and see if I can help get 7.0.x and 8.0.x ready to go too.

I haven't shared them and now I go back to re-read them my notes are
somewhat disorganized.

I'll try and post something more coherent on Wednesday.


Mark


>
>> This is on my TODO list for after TomcatCon London on Tuesday.
>>
>> Mark
>>
>>
>>>
>>> WARNING: An illegal reflective access operation has occurred
>>> WARNING: Illegal reflective access by
>>> org.apache.catalina.loader.WebappClassLoaderBase
>>> (file:/.../lib/catalina.jar) to field java.lang.Thread.threadLocals
>>> WARNING: Please consider reporting this to the maintainers of
>>> org.apache.catalina.loader.WebappClassLoaderBase
>>> WARNING: Use --illegal-access=warn to enable warnings of further illegal
>>> reflective access operations
>>> WARNING: All illegal access operations will be denied in a future release
>>>
>>>
>>> If I add the mentioned flag, I get more such lines, all referring to
>>> org.apache.catalina.loader.WebappClassLoaderBase and then:
>>>
>>> field java.lang.Thread.threadLocals
>>> field java.lang.Thread.inheritableThreadLocals
>>> field java.lang.ThreadLocal$ThreadLocalMap.table
>>> method java.lang.ThreadLocal$ThreadLocalMap.expungeStaleEntries()
>>> field java.lang.ThreadLocal$ThreadLocalMap$Entry.value
>>> field sun.rmi.transport.Target.ccl
>>> field sun.rmi.transport.Target.stub
>>> field sun.rmi.transport.ObjectTable.objTable
>>> field sun.rmi.transport.ObjectTable.implTable
>>>
>>> Details may vary depending on the cleanup flags set in the loader, but
>>> this is 8.5 with default settings.
>>>
>>> Regards,
>>>
>>> Rainer
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [hidden email]
>>> For additional commands, e-mail: [hidden email]
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Java 9 backports [Was: Java 9 TC shutdown warnings (reflection in WebappClassLoaderBase)]

Rainer Jung-3
I checked the TC 9 svn log for messages containing the regexp
/(java|jvm|jdk) ?9/i. All of them actually contained "Java 9" and the
following is the list. I marked the ones where I found the revision in
the 8.5 mergeinfo with a leading "*", but I have not checked how
complete the 8.5 backport was. The oldest ones would also not be in
mergeinfo, because they happened before the separat 8.5 branch.

r1809434 | rjung | 2017-09-23 16:45:48 +0200 (Sat, 23 Sep 2017) | 3 lines
Use the correct path when loading the JVMlogging.properties file for Java 9.
------------------------------------------------------------------------
r1806973 | markt | 2017-09-01 17:04:45 +0200 (Fri, 01 Sep 2017) | 1 line
Java 9 allows us to be more selective with the JRE memory leak protection.
------------------------------------------------------------------------
r1806932 | markt | 2017-09-01 13:10:37 +0200 (Fri, 01 Sep 2017) | 1 line
Update comment. We now have features that depend on Java 9.
------------------------------------------------------------------------
r1800617 | markt | 2017-07-03 12:19:46 +0200 (Mon, 03 Jul 2017) | 1 line
Add necessary Java 9 configuration options to the startup scripts to prevent
warnings being generated on web application stop.
------------------------------------------------------------------------
r1800614 | markt | 2017-07-03 11:48:01 +0200 (Mon, 03 Jul 2017) | 1 line
Restore the local definition of the web service annotations since the JRE
provided versions are deprecated and Java 9 does not provide them by
default.
------------------------------------------------------------------------
*r1791050 | markt | 2017-04-12 00:36:01 +0200 (Wed, 12 Apr 2017) | 1 line
Refactoring in preparation for Java 9. Refactor to avoid using some methods
that will be deprecated in Java 9 onwards.
------------------------------------------------------------------------
r1791036 | markt | 2017-04-11 23:40:13 +0200 (Tue, 11 Apr 2017) | 1 line
Java 8 and Java 9 friendly alternative
------------------------------------------------------------------------
r1791032 | markt | 2017-04-11 23:34:50 +0200 (Tue, 11 Apr 2017) | 1 line
Revert the Java 9 change that breaks in Java 8
------------------------------------------------------------------------
r1791028 | markt | 2017-04-11 23:16:04 +0200 (Tue, 11 Apr 2017) | 1 line
Refactoring in preparation for Java 9. Refactor to avoid using some methods
that will be deprecated in Java 9 onwards.
------------------------------------------------------------------------
r1791027 | markt | 2017-04-11 22:40:36 +0200 (Tue, 11 Apr 2017) | 1 line
Refactoring in preparation for Java 9. Refactor to avoid using some methods
that will be deprecated in Java 9 onwards.
------------------------------------------------------------------------
*r1782857 | markt | 2017-02-13 21:39:14 +0100 (Mon, 13 Feb 2017) | 3 lines
Java 9 support for annotation scanningBased
on:http://cr.openjdk.java.net/~mr/jigsaw/spec/lang-vm.html#jigsaw-2.6
------------------------------------------------------------------------
r1779932 | markt | 2017-01-23 15:16:32 +0100 (Mon, 23 Jan 2017) | 2 lines
Remove unused code, effectively reverting 1779370 and 1779612.
Java 9 is now handled in other branch of if/else.
------------------------------------------------------------------------
r1779622 | remm | 2017-01-20 14:09:56 +0100 (Fri, 20 Jan 2017) | 1 line
Restore Java 9 direct byte buffer cleanup code, for now. The last way to
access it is through the private Unsafe singleton, which will likely stop
working at some point :(
------------------------------------------------------------------------
r1779612 | markt | 2017-01-20 12:58:51 +0100 (Fri, 20 Jan 2017) | 1 line
Log message that includes command line option required when running on
Java 9
------------------------------------------------------------------------
r1779545 | markt | 2017-01-20 01:12:16 +0100 (Fri, 20 Jan 2017) | 2 lines
Adding ALPN support for JSSE with Java 9Enable ALPN and also, therefore,
HTTP/2 for the NIO and NIO2 HTTP connectors when using the JSSE
implementation
for TLS when running on Java 9.
------------------------------------------------------------------------
r1779370 | markt | 2017-01-18 19:46:27 +0100 (Wed, 18 Jan 2017) | 1 line
Java 9 can throw a Java 9 specific exception here
(InaccessibleObjectException)
so tweak the handling
------------------------------------------------------------------------
r1779313 | markt | 2017-01-18 12:23:17 +0100 (Wed, 18 Jan 2017) | 1 line
ws police (I need to configure my Java 9 dev environment correctly)
------------------------------------------------------------------------
r1778603 | markt | 2017-01-13 15:42:01 +0100 (Fri, 13 Jan 2017) | 2 lines
Adding ALPN support for JSSE with Java 9Add some plumbing to exposed the
client
requested application protocols to the method that configures the SSLEngine
------------------------------------------------------------------------
r1778575 | markt | 2017-01-13 13:50:01 +0100 (Fri, 13 Jan 2017) | 2 lines
Adding ALPN support for JSSE with Java 9Expand the data extracted from
the TLS
client hello to include the client requested ALPN names.
------------------------------------------------------------------------
*r1766822 | markt | 2016-10-27 15:59:41 +0200 (Thu, 27 Oct 2016) | 2 lines
ThreadLocal leak detection is now hitting Java 9 module issues.
Catch the error and provide a useful error message if this happens.
------------------------------------------------------------------------
*r1762753 | remm | 2016-09-29 12:20:27 +0200 (Thu, 29 Sep 2016) | 1 line
Java 9 compatibility for direct ByteBuffer cleaner.
------------------------------------------------------------------------
*r1758556 | markt | 2016-08-31 11:09:47 +0200 (Wed, 31 Aug 2016) | 1 line
The latest Java 9 early access builds have fixed some more memory leaks.
------------------------------------------------------------------------
*r1744323 | markt | 2016-05-17 22:36:54 +0200 (Tue, 17 May 2016) | 1 line
Make checking for RMI Target memory leaks optional and log a warning if
running
on Java 9 without the necessary command line options
------------------------------------------------------------------------
r1744149 | markt | 2016-05-16 23:36:39 +0200 (Mon, 16 May 2016) | 1 line
Tomcat needs to know if it is running on Java 9 since the reflection used by
the RMI memory leak detection will break unless the right command line
option
is specified.
------------------------------------------------------------------------
r1739492 | markt | 2016-04-16 21:08:19 +0200 (Sat, 16 Apr 2016) | 2 lines
Java 9 The JRE class loaders no longer extend URLClassLoader
------------------------------------------------------------------------
r1720196 | markt | 2015-12-15 18:05:45 +0100 (Tue, 15 Dec 2015) | 1 line
Minor hack to get the unit tests passing on Java 9.
------------------------------------------------------------------------
r1705771 | markt | 2015-09-28 22:45:08 +0200 (Mon, 28 Sep 2015) | 1 line
No need for LogManager when using stop. Can't use the Tomcat config
anyway since
that would resut in two processes trying to write to the same file. This
was also
causing an ugly stack trace on stop with Java 9 as it tried to find a
logging
configuration to use.
------------------------------------------------------------------------
r1692896 | markt | 2015-07-27 17:18:49 +0200 (Mon, 27 Jul 2015) | 1 line
Add Java 9 support for JSPs
------------------------------------------------------------------------
r1653475 | markt | 2015-01-21 11:39:59 +0100 (Wed, 21 Jan 2015) | 1 line
Remove use of java.endorsed.dirs since causes errors when starting with
Java 9.
Users that need to can still use this via setenv.sh
------------------------------------------------------------------------
r1633595 | markt | 2014-10-22 13:04:31 +0200 (Wed, 22 Oct 2014) | 1 line
Java 9 Javadoc issues
------------------------------------------------------------------------

Regards,

Rainer

Am 25.09.2017 um 21:40 schrieb Mark Thomas:

> On 25/09/17 13:33, Coty Sutherland wrote:
>> On Sat, Sep 23, 2017 at 11:01 AM, Mark Thomas <[hidden email]> wrote:
>>> On 23/09/17 15:55, Rainer Jung wrote:
>>>> Hi all,
>>>>
>>>> when running TC 8.5.21 (9.0 should be the same) with Java 9 I get the
>>>> following warnings on STDOUT during shutdown:
>>>
>>> Tomcat 9 has the Java 9 fixes. 8.5.x doesn't. I've put together a list
>>> of the Java 9 commits that need to be back-ported. Now that Java 9 is
>>> final those back-ports need to happen.
>>
>> Have you shared/can you share the list of commits? I'd like to take a
>> look and see if I can help get 7.0.x and 8.0.x ready to go too.
>
> I haven't shared them and now I go back to re-read them my notes are
> somewhat disorganized.
>
> I'll try and post something more coherent on Wednesday.
>
>
> Mark
>
>
>>
>>> This is on my TODO list for after TomcatCon London on Tuesday.
>>>
>>> Mark
>>>
>>>
>>>>
>>>> WARNING: An illegal reflective access operation has occurred
>>>> WARNING: Illegal reflective access by
>>>> org.apache.catalina.loader.WebappClassLoaderBase
>>>> (file:/.../lib/catalina.jar) to field java.lang.Thread.threadLocals
>>>> WARNING: Please consider reporting this to the maintainers of
>>>> org.apache.catalina.loader.WebappClassLoaderBase
>>>> WARNING: Use --illegal-access=warn to enable warnings of further illegal
>>>> reflective access operations
>>>> WARNING: All illegal access operations will be denied in a future release
>>>>
>>>>
>>>> If I add the mentioned flag, I get more such lines, all referring to
>>>> org.apache.catalina.loader.WebappClassLoaderBase and then:
>>>>
>>>> field java.lang.Thread.threadLocals
>>>> field java.lang.Thread.inheritableThreadLocals
>>>> field java.lang.ThreadLocal$ThreadLocalMap.table
>>>> method java.lang.ThreadLocal$ThreadLocalMap.expungeStaleEntries()
>>>> field java.lang.ThreadLocal$ThreadLocalMap$Entry.value
>>>> field sun.rmi.transport.Target.ccl
>>>> field sun.rmi.transport.Target.stub
>>>> field sun.rmi.transport.ObjectTable.objTable
>>>> field sun.rmi.transport.ObjectTable.implTable
>>>>
>>>> Details may vary depending on the cleanup flags set in the loader, but
>>>> this is 8.5 with default settings.
>>>>
>>>> Regards,
>>>>
>>>> Rainer

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Java 9 backports [Was: Java 9 TC shutdown warnings (reflection in WebappClassLoaderBase)]

markt
On 26/09/17 10:47, Rainer Jung wrote:
> I checked the TC 9 svn log for messages containing the regexp
> /(java|jvm|jdk) ?9/i. All of them actually contained "Java 9" and the
> following is the list. I marked the ones where I found the revision in
> the 8.5 mergeinfo with a leading "*", but I have not checked how
> complete the 8.5 backport was. The oldest ones would also not be in
> mergeinfo, because they happened before the separat 8.5 branch.

Thanks for this. I've copied the list to the wiki so it can be updated
as progress is made.

https://cwiki.apache.org/confluence/display/TOMCAT/Java+9+Status+tracking

Mark


>
> r1809434 | rjung | 2017-09-23 16:45:48 +0200 (Sat, 23 Sep 2017) | 3 lines
> Use the correct path when loading the JVMlogging.properties file for
> Java 9.
> ------------------------------------------------------------------------
> r1806973 | markt | 2017-09-01 17:04:45 +0200 (Fri, 01 Sep 2017) | 1 line
> Java 9 allows us to be more selective with the JRE memory leak protection.
> ------------------------------------------------------------------------
> r1806932 | markt | 2017-09-01 13:10:37 +0200 (Fri, 01 Sep 2017) | 1 line
> Update comment. We now have features that depend on Java 9.
> ------------------------------------------------------------------------
> r1800617 | markt | 2017-07-03 12:19:46 +0200 (Mon, 03 Jul 2017) | 1 line
> Add necessary Java 9 configuration options to the startup scripts to
> prevent
> warnings being generated on web application stop.
> ------------------------------------------------------------------------
> r1800614 | markt | 2017-07-03 11:48:01 +0200 (Mon, 03 Jul 2017) | 1 line
> Restore the local definition of the web service annotations since the JRE
> provided versions are deprecated and Java 9 does not provide them by
> default.
> ------------------------------------------------------------------------
> *r1791050 | markt | 2017-04-12 00:36:01 +0200 (Wed, 12 Apr 2017) | 1 line
> Refactoring in preparation for Java 9. Refactor to avoid using some methods
> that will be deprecated in Java 9 onwards.
> ------------------------------------------------------------------------
> r1791036 | markt | 2017-04-11 23:40:13 +0200 (Tue, 11 Apr 2017) | 1 line
> Java 8 and Java 9 friendly alternative
> ------------------------------------------------------------------------
> r1791032 | markt | 2017-04-11 23:34:50 +0200 (Tue, 11 Apr 2017) | 1 line
> Revert the Java 9 change that breaks in Java 8
> ------------------------------------------------------------------------
> r1791028 | markt | 2017-04-11 23:16:04 +0200 (Tue, 11 Apr 2017) | 1 line
> Refactoring in preparation for Java 9. Refactor to avoid using some methods
> that will be deprecated in Java 9 onwards.
> ------------------------------------------------------------------------
> r1791027 | markt | 2017-04-11 22:40:36 +0200 (Tue, 11 Apr 2017) | 1 line
> Refactoring in preparation for Java 9. Refactor to avoid using some methods
> that will be deprecated in Java 9 onwards.
> ------------------------------------------------------------------------
> *r1782857 | markt | 2017-02-13 21:39:14 +0100 (Mon, 13 Feb 2017) | 3 lines
> Java 9 support for annotation scanningBased
> on:http://cr.openjdk.java.net/~mr/jigsaw/spec/lang-vm.html#jigsaw-2.6
> ------------------------------------------------------------------------
> r1779932 | markt | 2017-01-23 15:16:32 +0100 (Mon, 23 Jan 2017) | 2 lines
> Remove unused code, effectively reverting 1779370 and 1779612.
> Java 9 is now handled in other branch of if/else.
> ------------------------------------------------------------------------
> r1779622 | remm | 2017-01-20 14:09:56 +0100 (Fri, 20 Jan 2017) | 1 line
> Restore Java 9 direct byte buffer cleanup code, for now. The last way to
> access it is through the private Unsafe singleton, which will likely stop
> working at some point :(
> ------------------------------------------------------------------------
> r1779612 | markt | 2017-01-20 12:58:51 +0100 (Fri, 20 Jan 2017) | 1 line
> Log message that includes command line option required when running on
> Java 9
> ------------------------------------------------------------------------
> r1779545 | markt | 2017-01-20 01:12:16 +0100 (Fri, 20 Jan 2017) | 2 lines
> Adding ALPN support for JSSE with Java 9Enable ALPN and also, therefore,
> HTTP/2 for the NIO and NIO2 HTTP connectors when using the JSSE
> implementation
> for TLS when running on Java 9.
> ------------------------------------------------------------------------
> r1779370 | markt | 2017-01-18 19:46:27 +0100 (Wed, 18 Jan 2017) | 1 line
> Java 9 can throw a Java 9 specific exception here
> (InaccessibleObjectException)
> so tweak the handling
> ------------------------------------------------------------------------
> r1779313 | markt | 2017-01-18 12:23:17 +0100 (Wed, 18 Jan 2017) | 1 line
> ws police (I need to configure my Java 9 dev environment correctly)
> ------------------------------------------------------------------------
> r1778603 | markt | 2017-01-13 15:42:01 +0100 (Fri, 13 Jan 2017) | 2 lines
> Adding ALPN support for JSSE with Java 9Add some plumbing to exposed the
> client
> requested application protocols to the method that configures the SSLEngine
> ------------------------------------------------------------------------
> r1778575 | markt | 2017-01-13 13:50:01 +0100 (Fri, 13 Jan 2017) | 2 lines
> Adding ALPN support for JSSE with Java 9Expand the data extracted from
> the TLS
> client hello to include the client requested ALPN names.
> ------------------------------------------------------------------------
> *r1766822 | markt | 2016-10-27 15:59:41 +0200 (Thu, 27 Oct 2016) | 2 lines
> ThreadLocal leak detection is now hitting Java 9 module issues.
> Catch the error and provide a useful error message if this happens.
> ------------------------------------------------------------------------
> *r1762753 | remm | 2016-09-29 12:20:27 +0200 (Thu, 29 Sep 2016) | 1 line
> Java 9 compatibility for direct ByteBuffer cleaner.
> ------------------------------------------------------------------------
> *r1758556 | markt | 2016-08-31 11:09:47 +0200 (Wed, 31 Aug 2016) | 1 line
> The latest Java 9 early access builds have fixed some more memory leaks.
> ------------------------------------------------------------------------
> *r1744323 | markt | 2016-05-17 22:36:54 +0200 (Tue, 17 May 2016) | 1 line
> Make checking for RMI Target memory leaks optional and log a warning if
> running
> on Java 9 without the necessary command line options
> ------------------------------------------------------------------------
> r1744149 | markt | 2016-05-16 23:36:39 +0200 (Mon, 16 May 2016) | 1 line
> Tomcat needs to know if it is running on Java 9 since the reflection
> used by
> the RMI memory leak detection will break unless the right command line
> option
> is specified.
> ------------------------------------------------------------------------
> r1739492 | markt | 2016-04-16 21:08:19 +0200 (Sat, 16 Apr 2016) | 2 lines
> Java 9 The JRE class loaders no longer extend URLClassLoader
> ------------------------------------------------------------------------
> r1720196 | markt | 2015-12-15 18:05:45 +0100 (Tue, 15 Dec 2015) | 1 line
> Minor hack to get the unit tests passing on Java 9.
> ------------------------------------------------------------------------
> r1705771 | markt | 2015-09-28 22:45:08 +0200 (Mon, 28 Sep 2015) | 1 line
> No need for LogManager when using stop. Can't use the Tomcat config
> anyway since
> that would resut in two processes trying to write to the same file. This
> was also
> causing an ugly stack trace on stop with Java 9 as it tried to find a
> logging
> configuration to use.
> ------------------------------------------------------------------------
> r1692896 | markt | 2015-07-27 17:18:49 +0200 (Mon, 27 Jul 2015) | 1 line
> Add Java 9 support for JSPs
> ------------------------------------------------------------------------
> r1653475 | markt | 2015-01-21 11:39:59 +0100 (Wed, 21 Jan 2015) | 1 line
> Remove use of java.endorsed.dirs since causes errors when starting with
> Java 9.
> Users that need to can still use this via setenv.sh
> ------------------------------------------------------------------------
> r1633595 | markt | 2014-10-22 13:04:31 +0200 (Wed, 22 Oct 2014) | 1 line
> Java 9 Javadoc issues
> ------------------------------------------------------------------------
>
> Regards,
>
> Rainer
>
> Am 25.09.2017 um 21:40 schrieb Mark Thomas:
>> On 25/09/17 13:33, Coty Sutherland wrote:
>>> On Sat, Sep 23, 2017 at 11:01 AM, Mark Thomas <[hidden email]> wrote:
>>>> On 23/09/17 15:55, Rainer Jung wrote:
>>>>> Hi all,
>>>>>
>>>>> when running TC 8.5.21 (9.0 should be the same) with Java 9 I get the
>>>>> following warnings on STDOUT during shutdown:
>>>>
>>>> Tomcat 9 has the Java 9 fixes. 8.5.x doesn't. I've put together a list
>>>> of the Java 9 commits that need to be back-ported. Now that Java 9 is
>>>> final those back-ports need to happen.
>>>
>>> Have you shared/can you share the list of commits? I'd like to take a
>>> look and see if I can help get 7.0.x and 8.0.x ready to go too.
>>
>> I haven't shared them and now I go back to re-read them my notes are
>> somewhat disorganized.
>>
>> I'll try and post something more coherent on Wednesday.
>>
>>
>> Mark
>>
>>
>>>
>>>> This is on my TODO list for after TomcatCon London on Tuesday.
>>>>
>>>> Mark
>>>>
>>>>
>>>>>
>>>>> WARNING: An illegal reflective access operation has occurred
>>>>> WARNING: Illegal reflective access by
>>>>> org.apache.catalina.loader.WebappClassLoaderBase
>>>>> (file:/.../lib/catalina.jar) to field java.lang.Thread.threadLocals
>>>>> WARNING: Please consider reporting this to the maintainers of
>>>>> org.apache.catalina.loader.WebappClassLoaderBase
>>>>> WARNING: Use --illegal-access=warn to enable warnings of further
>>>>> illegal
>>>>> reflective access operations
>>>>> WARNING: All illegal access operations will be denied in a future
>>>>> release
>>>>>
>>>>>
>>>>> If I add the mentioned flag, I get more such lines, all referring to
>>>>> org.apache.catalina.loader.WebappClassLoaderBase and then:
>>>>>
>>>>> field java.lang.Thread.threadLocals
>>>>> field java.lang.Thread.inheritableThreadLocals
>>>>> field java.lang.ThreadLocal$ThreadLocalMap.table
>>>>> method java.lang.ThreadLocal$ThreadLocalMap.expungeStaleEntries()
>>>>> field java.lang.ThreadLocal$ThreadLocalMap$Entry.value
>>>>> field sun.rmi.transport.Target.ccl
>>>>> field sun.rmi.transport.Target.stub
>>>>> field sun.rmi.transport.ObjectTable.objTable
>>>>> field sun.rmi.transport.ObjectTable.implTable
>>>>>
>>>>> Details may vary depending on the cleanup flags set in the loader, but
>>>>> this is 8.5 with default settings.
>>>>>
>>>>> Regards,
>>>>>
>>>>> Rainer
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Multi-release Jars

markt
In reply to this post by Rainer Jung-3
Hi,

Following up on this because I have been looking at a multi-release JAR
approach for bug 61597 [1] and the current JreCompat implementation.

On 23/09/17 19:02, Rainer Jung wrote:

> Forgot the reference: http://openjdk.java.net/jeps/238
>
> Am 23.09.2017 um 20:01 schrieb Rainer Jung:
>> Am 23.09.2017 um 17:22 schrieb Mark Thomas:
>>> On 23/09/17 16:10, Rainer Jung wrote:
>>>> Thanks Mark!
>>>>
>>>> You might also want to look at r1809434 as a candidate and check
>>>> whether
>>>> that workaround looks OK for you.
>>>
>>> Looks good to me. I've added to the list of J9 back-port candidates. It
>>> does beg the question whether or not we want to look at switching to
>>> multi-release JARs for all of the compatibility code. WDYT?
>>
>> Indeed, I ran my first multi-release Jar experiment (successfully)
>> today. For those who haven't yet seen it: starting with Java 9 one can
>> include instances of a class in a jar file that will only get used
>> when a specific JVM (major) version is detected at runtime. You can
>> e.g. have a default one as usual plus additional ones for specific
>> Java runtime versions. For Java 9 one would add a class at
>> META-INF/versions/9/org/apache/juli/... which would then be picked
>> when the runtime Java version is 9.
>>
>> The enable the feature, one simply has to add the attribute
>>
>> Multi-Release: true
>>
>> to the Manifest file of the jar.
>>
>> Thinks that come to my mind:
>>
>> - you can have multiple versions of the same class. If we would use it
>> like that, we would have to slightly reorganize our svn layout to
>> reflect that, e.g. java/org/... plus new java9/org/....

Having looked at a couple of options I settled on that one too.

I haven't yet found an IDE with a GA release that handles this. Support
in the tools is fairly embryonic as well.

>> - accordingly we would need e.g. a directory output/classes9 or similar.

I went for classes/META-INF/versions/9 so it mirrored the JAR structure
but that probably isn't essential. One advantage was that - with my
other build.xml changes - supporting additional versions was minimal
effort.

>> - If we need Java 9 for compilation of the Java 9 classes, we would
>> mimic the Java 7 lines from the TC 7 build.xml.

For the JreCompat code, we would need this. This is where most of the
complexity was added for this approach.

>> - To add the Java 9 classes to one of our jars it would suffice in
>> build.xml to do something like
>>
>> <jar jarfile="${somefile.jar}" update="true">
>>    <manifest>
>>      <attribute name="Multi-Release" value="true"/>
>>    </manifest>
>>    <zipfileset prefix="META-INF/versions/9/" dir="${tomcat.classes}9"/>
>> </jar>

I took a slightly different approach. I extended the existing jarIt
macro and provided a default.mr.manifest. We'd also need to decide how
to handle the source JARs in this case. I didn't implement it but my
conclusion was we should mirror the structure of the binary JARs.

>> - to reduce redundant maintenance it would be good to factor out JVM
>> dependent code, so that the classes we have to maintain multiply
>> mostly contain the code that's really version-dependent.

Big +1. I looked at removing JreCompat entirely but that would have
required a lot of duplication. We have some large classes that only need
one or two lines of Java 9 code.

>> For instance when using the feature for my Java 9 change in Juli
>> http://svn.apache.org/viewvc?rev=1809434&view=rev one could use e.g. a
>> small but version dependent Constants.java which would provide the
>> correct directory name ("lib" versus "conf"). Or more easily a one
>> line properties file available in two versions providing the directory
>> name.
>>
>> I have not made any check on performance implications for the slightly
>> more complex class loading. What I did see, is that -verbose:class
>> does not indicate, which of the versions was chosen.

I haven't checked performance either. My working assumption is that if
it is an issue, the JVM vendors will have to address it somehow.

>> I currently do not have a real insight on how our most important JVM
>> version dependencies look. I guess how useful multi-release is, might
>> depend on the type of code dependencies we have.

With the fix for bug 61597 applied the MR JAR approach looks to use
slightly less code (maybe ~20 lines overall including comments).

So, comparing the MR approach vs continuing with the reflection approach:

MR uses less code but requires more complexity in the build script.
Overall, MR requires slightly less code.

MR means release builds will required Java 8 and Java 9 (or just Java 9
with a risk we add a hard Java 9 dependency without realising).

MR is not handled well by IDEs requiring manual inclusion / exclusion of
source files (or multiple projects) to switch between the different
versions.

I do like the reduction in code and the build script complexity doesn't
concern me overly - it is mostly adding volume rather than an complex
logic. The IDE issues are more annoying.

Overall, I'm torn. I guess I'm +0 on switching at this point. What does
everyone else think?

Mark


[1] https://bz.apache.org/bugzilla/show_bug.cgi?id=61597


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Multi-release Jars

Konstantin Kolinko
2017-10-11 14:26 GMT+03:00 Mark Thomas <[hidden email]>:
>
> Having looked at a couple of options I settled on that one too.
>
> I haven't yet found an IDE with a GA release that handles this. Support
> in the tools is fairly embryonic as well.

Several thoughts:

1. I think Eclipse does not support having different version of Java
for different subsets of files in a project.
(I am on 4.6 Neon now, haven't tried 4.7 Oxygen yet)

As such, we will have to use two separate projects,
similar to the two projects in Tomcat 7.0 ("tomcat-7.0.x" and
"tomcat-7.0.x-java7").

There are two separate goals in build.xml to generate the two
projects. Documented in
http://tomcat.apache.org/tomcat-7.0-doc/building.html#Building_with_Eclipse

It was easier for Tomcat 7 with clear separation of WebSocket APIs.
I wonder how it will go with Tomcat 9, and with further backport of
Java 9 support to Tomcat 8.5 and 7.0


2. I think of changing JDK configuration in build files to use
explicit paths to JDK versions.

In Tomcat 7 we have "java.7.home" property to specify location of JDK 7.
My idea is to have explicit
java.6.home=
java.7.home=
java.8.home=

If either of them is missing, ${env.JAVA_HOME} provides the value -
like it is done in Tomcat 7 build.xml.

The goal is to be able to run build.xml with a later version of Java.

Advantages of this approach:

1) It resolves problem with download of some of dependent libraries.
(Those libraries require https connection and cannot be downloaded
with an old version of Java (Java 6), but can be downloaded with Java
8).

2) It allows us to upgrade to a never version of Checkstyle.

As 1) and 2) are not much of a problem, I have not pursued this idea yet.


>>> - accordingly we would need e.g. a directory output/classes9 or similar.
>
> I went for classes/META-INF/versions/9 so it mirrored the JAR structure
> but that probably isn't essential. One advantage was that - with my
> other build.xml changes - supporting additional versions was minimal
> effort.
>
>>> - If we need Java 9 for compilation of the Java 9 classes, we would
>>> mimic the Java 7 lines from the TC 7 build.xml.
>
> For the JreCompat code, we would need this. This is where most of the
> complexity was added for this approach.
>
>>> - To add the Java 9 classes to one of our jars it would suffice in
>>> build.xml to do something like
>>>
>>> <jar jarfile="${somefile.jar}" update="true">
>>>    <manifest>
>>>      <attribute name="Multi-Release" value="true"/>
>>>    </manifest>
>>>    <zipfileset prefix="META-INF/versions/9/" dir="${tomcat.classes}9"/>
>>> </jar>
>
> I took a slightly different approach. I extended the existing jarIt
> macro and provided a default.mr.manifest. We'd also need to decide how
> to handle the source JARs in this case. I didn't implement it but my
> conclusion was we should mirror the structure of the binary JARs.
>
>>> - to reduce redundant maintenance it would be good to factor out JVM
>>> dependent code, so that the classes we have to maintain multiply
>>> mostly contain the code that's really version-dependent.
>
> Big +1. I looked at removing JreCompat entirely but that would have
> required a lot of duplication. We have some large classes that only need
> one or two lines of Java 9 code.
>
> [...]
>
> MR means release builds will required Java 8 and Java 9 (or just Java 9
> with a risk we add a hard Java 9 dependency without realising).
>
> MR is not handled well by IDEs requiring manual inclusion / exclusion of
> source files (or multiple projects) to switch between the different
> versions.
>
> I do like the reduction in code and the build script complexity doesn't
> concern me overly - it is mostly adding volume rather than an complex
> logic. The IDE issues are more annoying.

I agree with the above. And the IDE issues are annoying.

> Overall, I'm torn. I guess I'm +0 on switching at this point. What does
> everyone else think?
>
> Mark
> [1] https://bz.apache.org/bugzilla/show_bug.cgi?id=61597

Best regards,
Konstantin Kolinko

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Multi-release Jars

markt
On 11/10/17 17:12, Konstantin Kolinko wrote:

> 2017-10-11 14:26 GMT+03:00 Mark Thomas <[hidden email]>:
>>
>> Having looked at a couple of options I settled on that one too.
>>
>> I haven't yet found an IDE with a GA release that handles this. Support
>> in the tools is fairly embryonic as well.
>
> Several thoughts:
>
> 1. I think Eclipse does not support having different version of Java
> for different subsets of files in a project.
> (I am on 4.6 Neon now, haven't tried 4.7 Oxygen yet)

Correct. I'm working with the latest (4.7.1a) and this is still the case.

(Aside: My initial impressions are 4.7.1a is a lot faster then 4.6.x.
You might want to consider an upgrade)

> As such, we will have to use two separate projects,
> similar to the two projects in Tomcat 7.0 ("tomcat-7.0.x" and
> "tomcat-7.0.x-java7").

That is one way and the only one I'm aware of that allows working in
both versions at the same time. There are other options if you don't
mind changing config to switch between versions. Whatever the approach,
none of them are ideal.

> There are two separate goals in build.xml to generate the two
> projects. Documented in
> http://tomcat.apache.org/tomcat-7.0-doc/building.html#Building_with_Eclipse
>
> It was easier for Tomcat 7 with clear separation of WebSocket APIs.
> I wonder how it will go with Tomcat 9, and with further backport of
> Java 9 support to Tomcat 8.5 and 7.0

It would be worth testing. There might be a conflict between the
multiple JreCompat versions.

> 2. I think of changing JDK configuration in build files to use
> explicit paths to JDK versions.
>
> In Tomcat 7 we have "java.7.home" property to specify location of JDK 7.
> My idea is to have explicit
> java.6.home=
> java.7.home=
> java.8.home=
>
> If either of them is missing, ${env.JAVA_HOME} provides the value -
> like it is done in Tomcat 7 build.xml.
>
> The goal is to be able to run build.xml with a later version of Java.

I agree that should always be possible. I thought it was at the moment.
However, my preference would always be to use the minimum for a release
build as a safety check that features from a later version hadn't
accidentally crept in.

> Advantages of this approach:
>
> 1) It resolves problem with download of some of dependent libraries.
> (Those libraries require https connection and cannot be downloaded
> with an old version of Java (Java 6), but can be downloaded with Java
> 8).
>
> 2) It allows us to upgrade to a never version of Checkstyle.
>
> As 1) and 2) are not much of a problem, I have not pursued this idea yet.

Makes sense.

>>>> - accordingly we would need e.g. a directory output/classes9 or similar.
>>
>> I went for classes/META-INF/versions/9 so it mirrored the JAR structure
>> but that probably isn't essential. One advantage was that - with my
>> other build.xml changes - supporting additional versions was minimal
>> effort.
>>
>>>> - If we need Java 9 for compilation of the Java 9 classes, we would
>>>> mimic the Java 7 lines from the TC 7 build.xml.
>>
>> For the JreCompat code, we would need this. This is where most of the
>> complexity was added for this approach.
>>
>>>> - To add the Java 9 classes to one of our jars it would suffice in
>>>> build.xml to do something like
>>>>
>>>> <jar jarfile="${somefile.jar}" update="true">
>>>>    <manifest>
>>>>      <attribute name="Multi-Release" value="true"/>
>>>>    </manifest>
>>>>    <zipfileset prefix="META-INF/versions/9/" dir="${tomcat.classes}9"/>
>>>> </jar>
>>
>> I took a slightly different approach. I extended the existing jarIt
>> macro and provided a default.mr.manifest. We'd also need to decide how
>> to handle the source JARs in this case. I didn't implement it but my
>> conclusion was we should mirror the structure of the binary JARs.
>>
>>>> - to reduce redundant maintenance it would be good to factor out JVM
>>>> dependent code, so that the classes we have to maintain multiply
>>>> mostly contain the code that's really version-dependent.
>>
>> Big +1. I looked at removing JreCompat entirely but that would have
>> required a lot of duplication. We have some large classes that only need
>> one or two lines of Java 9 code.
>>
>> [...]
>>
>> MR means release builds will required Java 8 and Java 9 (or just Java 9
>> with a risk we add a hard Java 9 dependency without realising).
>>
>> MR is not handled well by IDEs requiring manual inclusion / exclusion of
>> source files (or multiple projects) to switch between the different
>> versions.
>>
>> I do like the reduction in code and the build script complexity doesn't
>> concern me overly - it is mostly adding volume rather than an complex
>> logic. The IDE issues are more annoying.
>
> I agree with the above. And the IDE issues are annoying.
>
>> Overall, I'm torn. I guess I'm +0 on switching at this point. What does
>> everyone else think?
>>
>> Mark
>> [1] https://bz.apache.org/bugzilla/show_bug.cgi?id=61597
>
> Best regards,
> Konstantin Kolinko

Thanks,

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]