Using eval vs. exec in shell scripts

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Using eval vs. exec in shell scripts

Rainer Jung-3
Since Mladens change r918873 in March 2010 we use eval instead of exec
in the shell scripts. The svn log says:

"Use eval instead direct call or exec command so that arguments with
spaces are properly handled"

Eval leaves a copy of the shell process hanging around until Tomcat
shutdown. I want to experiment with different solutions, but have no
idea, what kind of whitespace use the switch from exec to eval was
supposed to change.

Mladen or whoever else understood this: can you give an example which
didn't work with the exec based scripts?

Regards,

Rainer


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Using eval vs. exec in shell scripts

Henri Gomez
Which script ?

Could you send us the svn link ?

2011/6/20 Rainer Jung <[hidden email]>:

> Since Mladens change r918873 in March 2010 we use eval instead of exec
> in the shell scripts. The svn log says:
>
> "Use eval instead direct call or exec command so that arguments with
> spaces are properly handled"
>
> Eval leaves a copy of the shell process hanging around until Tomcat
> shutdown. I want to experiment with different solutions, but have no
> idea, what kind of whitespace use the switch from exec to eval was
> supposed to change.
>
> Mladen or whoever else understood this: can you give an example which
> didn't work with the exec based scripts?
>
> Regards,
>
> Rainer
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Using eval vs. exec in shell scripts

mturk
In reply to this post by Rainer Jung-3
On 06/20/2011 01:39 PM, Rainer Jung wrote:
> Since Mladens change r918873 in March 2010 we use eval instead of exec
> in the shell scripts. The svn log says:
>
> "Use eval instead direct call or exec command so that arguments with
> spaces are properly handled"
>
> Mladen or whoever else understood this: can you give an example which
> didn't work with the exec based scripts?
>

eg.
export JAVA_OPTS="$JAVA_OPTS \"-XX:OnError=gdb - %p\""

... and
We eval for start/stop so we can get the pid.
since eval and exec behave differently when handling quoted args
this was giving different results on catalina.sh run/start


Regards
--
^TM

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Using eval vs. exec in shell scripts

Rainer Jung-3
On 20.06.2011 14:52, Mladen Turk wrote:

> On 06/20/2011 01:39 PM, Rainer Jung wrote:
>> Since Mladens change r918873 in March 2010 we use eval instead of exec
>> in the shell scripts. The svn log says:
>>
>> "Use eval instead direct call or exec command so that arguments with
>> spaces are properly handled"
>>
>> Mladen or whoever else understood this: can you give an example which
>> didn't work with the exec based scripts?
>>
>
> eg.
> export JAVA_OPTS="$JAVA_OPTS \"-XX:OnError=gdb - %p\""

Thanks for the example.

> ... and
> We eval for start/stop so we can get the pid.

And this is broken now. We used $! with exec and we still use it with
eval. But with eval $! returns the pid of a child shell and the java
process id a child of that pid.

> since eval and exec behave differently when handling quoted args
> this was giving different results on catalina.sh run/start

Yes, I got it.

I experimented with "eval exec" (sic!) and it looks promising. The
following tiny script is useful for experimenting:

============================================
#!/bin/sh
dir='/var/tmp/with spaces'
arg1='-Dprop1=val1 with space'
arg2="-DpropX=y -DpropY=Z"
args="\"$arg1\" $arg2"

mkdir -p "$dir"
cp -p /usr/bin/sleep "$dir"

echo "Starting using eval ..."
eval \"$dir\"/sleep 60 $args &
pid=$!
echo "PID is: $pid"
ps -p $pid
kill -9 $pid

echo "Starting using eval exec ..."
eval exec \"$dir\"/sleep 60 $args &
pid=$!
echo "PID is: $pid"
ps -p $pid
kill -9 $pid

rm -rf "$dir"
============================================

I think "eval exec" is the right solution. Seems to behave nice for
whitespace and doesn't add an intermediate process between the startup
script and the Java process.

Regards,

Rainer


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Using eval vs. exec in shell scripts

mturk
On 06/20/2011 03:12 PM, Rainer Jung wrote:
>
>> ... and
>> We eval for start/stop so we can get the pid.
>
> And this is broken now. We used $! with exec and we still use it with
> eval. But with eval $! returns the pid of a child shell and the java
> process id a child of that pid.
>

Are you sure?

cat <<EOF >foo.sh
#!/bin/sh
eval sleep "\$@" "&"
echo "Sleep pid: \$!"
exit 0
EOF

[mturk@fc14x86v0 Temp]$ sh foo.sh 30
Sleep pid: 3998
[mturk@fc14x86v0 Temp]$ ps -ef | grep sleep
...
mturk     3998     1  0 15:58 pts/3    00:00:00 sleep 30
...


Regards
--
^TM

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Using eval vs. exec in shell scripts

Rainer Jung-3
On 20.06.2011 15:59, Mladen Turk wrote:

> On 06/20/2011 03:12 PM, Rainer Jung wrote:
>>
>>> ... and
>>> We eval for start/stop so we can get the pid.
>>
>> And this is broken now. We used $! with exec and we still use it with
>> eval. But with eval $! returns the pid of a child shell and the java
>> process id a child of that pid.
>>
>
> Are you sure?
>
> cat <<EOF >foo.sh
> #!/bin/sh
> eval sleep "\$@" "&"
> echo "Sleep pid: \$!"
> exit 0
> EOF
>
> [mturk@fc14x86v0 Temp]$ sh foo.sh 30
> Sleep pid: 3998
> [mturk@fc14x86v0 Temp]$ ps -ef | grep sleep
> ...
> mturk     3998     1  0 15:58 pts/3    00:00:00 sleep 30
> ...

Aaaah, the fine difference between & and "&" ...

OK, point about PID and about eval exec taken back.

Thanks,

Rainer

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Using eval vs. exec in shell scripts

bradleymccrorey
In reply to this post by Rainer Jung-3
I'll be watching this quite closely. We run thousands of TC6 instances onsite here, and are a daemontools shop. This change breaks daemontools, as the "svc" command will attempt to stop the tomcat service by sending a TERM to the catalina.sh process, which leaves the child java process still running.

For one customer who insists on TC7, I've had to take the measure of changing the catalina.sh script to use the old method for now. This is not maintainable, however. Should there be a bug filed somewhere for this?

Cheers,
Bradley McCrorey

Rainer Jung-3 wrote
Since Mladens change r918873 in March 2010 we use eval instead of exec
in the shell scripts. The svn log says:

"Use eval instead direct call or exec command so that arguments with
spaces are properly handled"

Eval leaves a copy of the shell process hanging around until Tomcat
shutdown. I want to experiment with different solutions, but have no
idea, what kind of whitespace use the switch from exec to eval was
supposed to change.

Mladen or whoever else understood this: can you give an example which
didn't work with the exec based scripts?

Regards,

Rainer


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Reply | Threaded
Open this post in threaded view
|

Re: Using eval vs. exec in shell scripts

Rainer Jung-3
On 26.06.2011 01:45, bradleymccrorey wrote:

>
> I'll be watching this quite closely. We run thousands of TC6 instances onsite
> here, and are a daemontools shop. This change breaks daemontools, as the
> "svc" command will attempt to stop the tomcat service by sending a TERM to
> the catalina.sh process, which leaves the child java process still running.
>
> For one customer who insists on TC7, I've had to take the measure of
> changing the catalina.sh script to use the old method for now. This is not
> maintainable, however. Should there be a bug filed somewhere for this?
>
> Cheers,
> Bradley McCrorey

Did you follow the later messages in this discusison thread?

I made an error in not including all quotes use din catalina.sh in my
simpl test script. So when using the correct scripts, the eval did *not*
leave a copy of the shel process hanging around.

If you think there is a problem, you can discuss the technical details
here, and if the list doesn't find a solution for you, you can open a bug.

Regards,

Rainer

> Rainer Jung-3 wrote:
>>
>> Since Mladens change r918873 in March 2010 we use eval instead of exec
>> in the shell scripts. The svn log says:
>>
>> "Use eval instead direct call or exec command so that arguments with
>> spaces are properly handled"
>>
>> Eval leaves a copy of the shell process hanging around until Tomcat
>> shutdown. I want to experiment with different solutions, but have no
>> idea, what kind of whitespace use the switch from exec to eval was
>> supposed to change.
>>
>> Mladen or whoever else understood this: can you give an example which
>> didn't work with the exec based scripts?
>>
>> Regards,
>>
>> Rainer

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Using eval vs. exec in shell scripts

bradleymccrorey
Rainer Jung-3 wrote
Did you follow the later messages in this discusison thread?

I made an error in not including all quotes use din catalina.sh in my
simpl test script. So when using the correct scripts, the eval did *not*
leave a copy of the shel process hanging around.
I certainly did, but obviously didn't understand that things were working as expected. Please let me know if I'm misunderstanding.

However, if you're saying that the default scripts are working, then I'll definitely have to dispute this.

Observe:

[root@c37f9170a86c583d8a16fc7e60e759cf test]# curl -L http://www.fightrice.com/mirrors/apache/tomcat/tomcat-7/v7.0.16/bin/apache-tomcat-7.0.16.tar.gz |tar xzf -
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 7067k  100 7067k    0     0  10.0M      0 --:--:-- --:--:-- --:--:-- 10.2M
[root@c37f9170a86c583d8a16fc7e60e759cf test]# cd apache-tomcat-7.0.16/
[root@c37f9170a86c583d8a16fc7e60e759cf apache-tomcat-7.0.16]# bin/catalina.sh run
Using CATALINA_BASE:   /root/test/apache-tomcat-7.0.16
Using CATALINA_HOME:   /root/test/apache-tomcat-7.0.16
Using CATALINA_TMPDIR: /root/test/apache-tomcat-7.0.16/temp
Using JRE_HOME:        /usr
Using CLASSPATH:       /root/test/apache-tomcat-7.0.16/bin/bootstrap.jar:/root/test/apache-tomcat-7.0.16/bin/tomcat-juli.jar
Jun 26, 2011 10:50:22 AM org.apache.catalina.core.AprLifecycleListener init
...
INFO: Starting ProtocolHandler ["http-bio-8080"]
Jun 26, 2011 10:50:22 AM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["ajp-bio-8009"]
Jun 26, 2011 10:50:22 AM org.apache.catalina.startup.Catalina start
INFO: Server startup in 648 ms

[1]+  Stopped                 bin/catalina.sh run
[root@c37f9170a86c583d8a16fc7e60e759cf apache-tomcat-7.0.16]# bg
[1]+ bin/catalina.sh run &
[root@c37f9170a86c583d8a16fc7e60e759cf apache-tomcat-7.0.16]# ps auxww |grep catalina
root      1611  0.0  0.4   4576  1116 pts/0    S    10:50   0:00 /bin/sh bin/catalina.sh run
root      1622 14.5 11.0 650752 28236 pts/0    Sl   10:50   0:01 /usr/bin/java -Djava.util.logging.config.file=/root/test/apache-tomcat-7.0.16/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.endorsed.dirs=/root/test/apache-tomcat-7.0.16/endorsed -classpath /root/test/apache-tomcat-7.0.16/bin/bootstrap.jar:/root/test/apache-tomcat-7.0.16/bin/tomcat-juli.jar -Dcatalina.base=/root/test/apache-tomcat-7.0.16 -Dcatalina.home=/root/test/apache-tomcat-7.0.16 -Djava.io.tmpdir=/root/test/apache-tomcat-7.0.16/temp org.apache.catalina.startup.Bootstrap start
root      1643  0.0  0.2   4000   660 pts/0    S+   10:50   0:00 grep catalina
You can clearly see here that there are two processes: one for the shell script, and one for the actual java process. Is this not what I should be seeing?

Cheers,
Bradley McCrorey
Reply | Threaded
Open this post in threaded view
|

Re: Using eval vs. exec in shell scripts

Rainer Jung-3
On 26.06.2011 17:35, bradleymccrorey wrote:

>
>
> Rainer Jung-3 wrote:
>>
>>
>> Did you follow the later messages in this discusison thread?
>>
>> I made an error in not including all quotes use din catalina.sh in my
>> simpl test script. So when using the correct scripts, the eval did *not*
>> leave a copy of the shel process hanging around.
>>
>
> I certainly did, but obviously didn't understand that things were working as
> expected. Please let me know if I'm misunderstanding.
>
> However, if you're saying that the default scripts are working, then I'll
> definitely have to dispute this.

I am :)

> Observe:
>
>
>
>> [root@c37f9170a86c583d8a16fc7e60e759cf test]# curl -L
>> http://www.fightrice.com/mirrors/apache/tomcat/tomcat-7/v7.0.16/bin/apache-tomcat-7.0.16.tar.gz
>> |tar xzf -
>>   % Total    % Received % Xferd  Average Speed   Time    Time     Time
>> Current
>>                                  Dload  Upload   Total   Spent    Left
>> Speed
>> 100 7067k  100 7067k    0     0  10.0M      0 --:--:-- --:--:-- --:--:--
>> 10.2M
>> [root@c37f9170a86c583d8a16fc7e60e759cf test]# cd apache-tomcat-7.0.16/
>> [root@c37f9170a86c583d8a16fc7e60e759cf apache-tomcat-7.0.16]#
>> bin/catalina.sh run
>> Using CATALINA_BASE:   /root/test/apache-tomcat-7.0.16
>> Using CATALINA_HOME:   /root/test/apache-tomcat-7.0.16
>> Using CATALINA_TMPDIR: /root/test/apache-tomcat-7.0.16/temp
>> Using JRE_HOME:        /usr
>> Using CLASSPATH:      
>> /root/test/apache-tomcat-7.0.16/bin/bootstrap.jar:/root/test/apache-tomcat-7.0.16/bin/tomcat-juli.jar
>> Jun 26, 2011 10:50:22 AM org.apache.catalina.core.AprLifecycleListener
>> init
>> ...
>> INFO: Starting ProtocolHandler ["http-bio-8080"]
>> Jun 26, 2011 10:50:22 AM org.apache.coyote.AbstractProtocol start
>> INFO: Starting ProtocolHandler ["ajp-bio-8009"]
>> Jun 26, 2011 10:50:22 AM org.apache.catalina.startup.Catalina start
>> INFO: Server startup in 648 ms
>>
>> [1]+  Stopped                 bin/catalina.sh run
>> [root@c37f9170a86c583d8a16fc7e60e759cf apache-tomcat-7.0.16]# bg
>> [1]+ bin/catalina.sh run &
>> [root@c37f9170a86c583d8a16fc7e60e759cf apache-tomcat-7.0.16]# ps auxww
>> |grep catalina
>> root      1611  0.0  0.4   4576  1116 pts/0    S    10:50   0:00 /bin/sh
>> bin/catalina.sh run
>> root      1622 14.5 11.0 650752 28236 pts/0    Sl   10:50   0:01
>> /usr/bin/java
>> -Djava.util.logging.config.file=/root/test/apache-tomcat-7.0.16/conf/logging.properties
>> -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
>> -Djava.endorsed.dirs=/root/test/apache-tomcat-7.0.16/endorsed -classpath
>> /root/test/apache-tomcat-7.0.16/bin/bootstrap.jar:/root/test/apache-tomcat-7.0.16/bin/tomcat-juli.jar
>> -Dcatalina.base=/root/test/apache-tomcat-7.0.16
>> -Dcatalina.home=/root/test/apache-tomcat-7.0.16
>> -Djava.io.tmpdir=/root/test/apache-tomcat-7.0.16/temp
>> org.apache.catalina.startup.Bootstrap start
>> root      1643  0.0  0.2   4000   660 pts/0    S+   10:50   0:00 grep
>> catalina
>>
>
> You can clearly see here that there are two processes: one for the shell
> script, and one for the actual java process. Is this not what I should be
> seeing?

You should, bot only when using "run", which is precisely meant to not
decouple the console from the process.

If you use "start", which is the normal sart method, your observations
should be different.

Regards,

Rainer

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Using eval vs. exec in shell scripts

bradleymccrorey
Rainer Jung-3 wrote
>
> You can clearly see here that there are two processes: one for the shell
> script, and one for the actual java process. Is this not what I should be
> seeing?

You should, bot only when using "run", which is precisely meant to not
decouple the console from the process.
Hm. Perhaps we're trying to achieve different things here. With the old scripts, the behavior of the "run" switch was that the process would not fork, and would remain in the foreground, and only have one process (the java process). The new version seems to imitate this by leaving the running java process in the foreground of the current shell. However, there are still *2* processes: one for the catalina.sh, and one for java. This is a fundamental change in the way the script behaves.

Does this clarify what I'm trying to achieve? Sorry if I'm being thick-headed, and thanks very much for your consideration.
Reply | Threaded
Open this post in threaded view
|

Re: Using eval vs. exec in shell scripts

Rainer Jung-3
On 26.06.2011 17:59, bradleymccrorey wrote:

>
>
> Rainer Jung-3 wrote:
>>
>>>
>>> You can clearly see here that there are two processes: one for the shell
>>> script, and one for the actual java process. Is this not what I should be
>>> seeing?
>>
>> You should, bot only when using "run", which is precisely meant to not
>> decouple the console from the process.
>>
>>
>
> Hm. Perhaps we're trying to achieve different things here. With the old
> scripts, the behavior of the "run" switch was that the process would not
> fork, and would remain in the foreground, and only have one process (the
> java process). The new version seems to imitate this by leaving the running
> java process in the foreground of the current shell. However, there are
> still *2* processes: one for the catalina.sh, and one for java. This is a
> fundamental change in the way the script behaves.
>
> Does this clarify what I'm trying to achieve? Sorry if I'm being
> thick-headed, and thanks very much for your consideration.

Fixed for "run" in r1139904.

Please try the following change (eval -> eval exec):

Index: catalina.sh
===================================================================
--- catalina.sh (revision 1139151)
+++ catalina.sh (working copy)
@@ -312,7 +312,7 @@
       echo "Using Security Manager"
     fi
     shift
-    eval \"$_RUNJAVA\" \"$LOGGING_CONFIG\" $JAVA_OPTS $CATALINA_OPTS \
+    eval exec \"$_RUNJAVA\" \"$LOGGING_CONFIG\" $JAVA_OPTS $CATALINA_OPTS \
       -Djava.endorsed.dirs=\"$JAVA_ENDORSED_DIRS\" -classpath
\"$CLASSPATH\" \
       -Djava.security.manager \
       -Djava.security.policy==\"$CATALINA_BASE/conf/catalina.policy\" \
@@ -321,7 +321,7 @@
       -Djava.io.tmpdir=\"$CATALINA_TMPDIR\" \
       org.apache.catalina.startup.Bootstrap "$@" start
   else
-    eval \"$_RUNJAVA\" \"$LOGGING_CONFIG\" $JAVA_OPTS $CATALINA_OPTS \
+    eval exec \"$_RUNJAVA\" \"$LOGGING_CONFIG\" $JAVA_OPTS $CATALINA_OPTS \
       -Djava.endorsed.dirs=\"$JAVA_ENDORSED_DIRS\" -classpath
\"$CLASSPATH\" \
       -Dcatalina.base=\"$CATALINA_BASE\" \
       -Dcatalina.home=\"$CATALINA_HOME\" \


Regards,

Rainer

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]