[Bug 63815] New: Expansion of JAVA_OPTS in catalina.sh containing '*' stops startup on linux

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

[Bug 63815] New: Expansion of JAVA_OPTS in catalina.sh containing '*' stops startup on linux

Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=63815

            Bug ID: 63815
           Summary: Expansion of JAVA_OPTS in catalina.sh containing '*'
                    stops startup on linux
           Product: Tomcat 8
           Version: 8.5.x-trunk
          Hardware: Other
                OS: Linux
            Status: NEW
          Severity: normal
          Priority: P2
         Component: Catalina
          Assignee: [hidden email]
          Reporter: [hidden email]
  Target Milestone: ----

On Linux, the catalina.sh does not correctly quote use of $JAVA_OPTS so if you
specify an option containing a bash shell special character, in particular '*'
then tomcat fails to start.

For example set

JAVA_OPTS="-Dfoo.cronExpression=\"0 0 0/12 * * ?\""

And then try to start tomcat, it will fail with a non zero exit code.



Part of the fix is to stop the expansion by simply put the use of JAVA_OPTS in
double quotes.   i.e. "$JAVA_OPTS"

An additional complication is that the unquoted expression strips newline
characters so for backwards compatibility its also necessary to strip newline
characters.

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 63815] Expansion of JAVA_OPTS in catalina.sh containing '*' stops startup on linux

Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=63815

--- Comment #1 from Mark Thomas <[hidden email]> ---
I'm wondering how important it is to remove newlines backwards compatibility.
might it be sufficient to document the change in behaviour?

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 63815] Expansion of JAVA_OPTS in catalina.sh containing '*' stops startup on linux

Bugzilla from bugzilla@apache.org
In reply to this post by Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=63815

--- Comment #2 from Konstantin Kolinko <[hidden email]> ---
What exact change are you asking for?

Please note that on the last step (when invoking a java executable) the
JAVA_OPTS must produce several separate command line arguments: java expects
that each -D is passed as a separate command-line argument. As such, JAVA_OPTS
cannot be quoted there.


I have not tested, but I think that you should have used the quotes around the
whole -D argument:

JAVA_OPTS="\"-Dfoo.cronExpression=0 0 0/12 * * ?\""

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 63815] Expansion of JAVA_OPTS in catalina.sh containing '*' stops startup on linux

Bugzilla from bugzilla@apache.org
In reply to this post by Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=63815

Eugène Adell <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |NEEDINFO

--- Comment #3 from Eugène Adell <[hidden email]> ---
First, please use CATALINA_OPTS when you want to give information to the
application, instead of JAVA_OPTS, and edit setenv.sh instead of catalina.sh
(it will make your maintenance and upgrades easier).

I can't reproduce your problem on Tomcat 7.0 or 8.5. The only side effect that
I get with '*' is of course the variable expanding but I can neutralize this by
adding a set -o noglob to the setenv.sh
Please send the output of 'env|sort', 'bash --version', 'echo $SHELLOPTS' then
maybe we can compare
My tests below.


$ cat bin/setenv.sh
set -o noglob
CATALINA_OPTS="-Dfoo.cronExpression=\"0 0 0/12 * * ?\""

$ ps -eaf  | grep 15394
eadell   15394     1  2 00:22 pts/0    00:00:25 /usr/local/java/1.9/bin/java
-Djava.util.logging.config.file=/home/eadell/tests/85/conf/logging.properties
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
-Djdk.tls.ephemeralDHKeySize=2048
-Djava.protocol.handler.pkgs=org.apache.catalina.webresources
-Dorg.apache.catalina.security.SecurityListener.UMASK=0027
-Dfoo.cronExpression=0 0 0/12 * * ? -Dignore.endorsed.dirs= -classpath
/home/eadell/tests/85/bin/bootstrap.jar:/home/eadell/tests/85/bin/tomcat-juli.jar
-Dcatalina.base=/home/eadell/tests/85 -Dcatalina.home=/home/eadell/tests/85
-Djava.io.tmpdir=/home/eadell/tests/85/temp
org.apache.catalina.startup.Bootstrap start
eadell   15909 14747  0 00:37 pts/0    00:00:00 grep --color=auto 15394


reading system properties from a JSP :
foo.cronExpression: 0 0 0/12 * * ?

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 63815] Expansion of JAVA_OPTS in catalina.sh containing '*' stops startup on linux

Bugzilla from bugzilla@apache.org
In reply to this post by Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=63815

--- Comment #4 from [hidden email] <[hidden email]> ---
Thank you for the comments.

I'm actually using Tomcat within docker so would like to avoid touching any
tomcat code or configuration, whether catalina.sh or setenv.sh.   The pattern
I'm seeing used more and more is to specify a long list of -D options in the
docker-compose.yml file.

Answers to questions
GNU bash, version 4.2.46(2)-release (x86_64-redhat-linux-gnu)
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

braceexpand:emacs:hashall:histexpand:history:interactive-comments:monitor

Agreed I should probably be using CATALINA_OPTS however that seems to have
exactly the same issue with expansion of '*' as JAVA_OPTS

I think the quoting I have used is O.K. since with a quick hack to the tomcat
source files then it works.

What exact change are you asking for?
It's taken me some time to diagnose and work-around this problem.  If I can
share knowledge then that helps the community.

I've fixed this by replacing $JAVA_OPTS with "${JAVA_OPTS//$'\n'/ }" in
catalina.sh.  As pointed out below the same effect can be obtained by set -o
noglob .

Also I couldn't see any discussion about globing in the Tomcat documentation or
comments so perhaps a sentence could be added?

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 63815] Expansion of JAVA_OPTS in catalina.sh containing '*' stops startup on linux

Bugzilla from bugzilla@apache.org
In reply to this post by Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=63815

[hidden email] <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEEDINFO                    |NEW

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 63815] Expansion of JAVA_OPTS in catalina.sh containing '*' stops startup on linux

Bugzilla from bugzilla@apache.org
In reply to this post by Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=63815

--- Comment #5 from Eugène Adell <[hidden email]> ---
Your quoting is OK (confirmed in
https://docs.oracle.com/javase/7/docs/technotes/tools/windows/java.html
although the unix equivalent webpage doesn't mention this).

I'm not convinced Tomcat is guilty of anything here. Probably it fails starting
because the application is unable to run with an incomplete parameter, and
there are several workarounds for no-globbing. It's the normal behaviour of
shells to expand variables. Another workaround is reading a property file,
maybe that's what people are doing when facing this issue, I don't know.

--Also I couldn't see any discussion about globing in the Tomcat documentation
or comments so perhaps a sentence could be added?
Thanks, it's true :) The different variables are explained in the RUNNING.txt
file, we could add a note there but I'm scared very few people read that.

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 63815] Expansion of JAVA_OPTS in catalina.sh containing '*' stops startup on linux

Bugzilla from bugzilla@apache.org
In reply to this post by Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=63815

--- Comment #6 from [hidden email] <[hidden email]> ---
There are various style guides on the web suggesting that it is good practice
to always quote the use of variables in bash scripts

e.g. Google   https://google.github.io/styleguide/shell.xml
     Linux    https://www.tldp.org/LDP/abs/html/quotingvar.html

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 63815] Expansion of JAVA_OPTS in catalina.sh containing '*' stops startup on linux

Bugzilla from bugzilla@apache.org
In reply to this post by Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=63815

--- Comment #7 from Mark Thomas <[hidden email]> ---
(In reply to Konstantin Kolinko from comment #2)

> Please note that on the last step (when invoking a java executable) the
> JAVA_OPTS must produce several separate command line arguments: java expects
> that each -D is passed as a separate command-line argument. As such,
> JAVA_OPTS cannot be quoted there.

I've confirmed that the above is not a concern. The arguments are passed
correct when that use of JAVA_OPTS is quoted. The quoting simply prevents the
expansion of * and strips new lines.

I have a patch locally that applies the quoting fix to JAVA_OPTS and
CATALINA_OPTS in all affected shell scripts.

I'm still leaning towards applying this fix without the removal of newlines and
noting the change of behaviour in the change log.

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 63815] Expansion of JAVA_OPTS in catalina.sh containing '*' stops startup on linux

Bugzilla from bugzilla@apache.org
In reply to this post by Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=63815

Mark Thomas <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |FIXED
             Status|NEW                         |RESOLVED

--- Comment #8 from Mark Thomas <[hidden email]> ---
I didn;t see any objections so I went ahead and applied the patch.

Fixed in:
- master for 9.0.28 onwards
- 8.5.x for 8.5.48 onwards
- 7.0.x for 7.0.98 onwards

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]