TLS handshake performance

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

TLS handshake performance

Mark Thomas-2
Hi all,

I got asked in the corridor at TomcatCon earlier today what the relative
performance of the TLS handshake was with 8.5.x, the NIO connector and
JSSE vs OpenSSL TLS implementation.

This might be something that is of interest to a wider audience so here
goes...

The following results are very rough and ready and generated with my
(slightly aging now) laptop (4 cores).

I tested trunk but the code is close enough to 8.5.x for this purpose.

I used exacty the same config for each test. The only change was to
add/remove the tc-native library to enable/disable OpenSSL.

Tested with:
ab -n 1000 -c 2 -f TLS1.2 -Z ECDHE-RSA-AES128-GCM-SHA256
https://localhost:8443/test.txt

test.txt is a 3 byte text file.

The results were:
JSSE:    17 reqs/sec
OpenSSL: 23 reqs/sec

So around a 35% increase.

JRE: Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)
OpenSSL: 1.0.2k

YMMV with different versions of TLS and associated ciphers, JREs,
OpenSSl versions etc.

HTH,

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: TLS handshake performance

Christopher Schultz-2
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Mark,

On 5/17/17 5:31 PM, Mark Thomas wrote:
> I got asked in the corridor at TomcatCon earlier today what the
> relative performance of the TLS handshake was with 8.5.x, the NIO
> connector and JSSE vs OpenSSL TLS implementation.
I'm curious about what exactly "TLS handshake" was intended to mean
(by the person who asked the question) in this context.

The handshake itself does not perform any bulk transfer of encrypted
data, so the negotiated cipher suite does not matter. However...

> Tested with: ab -n 1000 -c 2 -f TLS1.2 -Z
> ECDHE-RSA-AES128-GCM-SHA256 https://localhost:8443/test.txt

Here the cipher suite matters very much, since the client is not only
performing the TLS handshake but also transferring the client's
request to the server and the server's response back to the client.

Support for a particular algorithm may dominate the benchmark, here.

What happens if you negotiate a NULL cipher for instance? Or, perform
the TLS handshake but never make an HTTP request after connecting? I
don't know of a tool that can do that out of the box (e.g. ab makes
HTTP requests, not just TLS connections) but one could be written in
Java fairly easily.

> test.txt is a 3 byte text file.
>
> The results were: JSSE:    17 reqs/sec OpenSSL: 23 reqs/sec
>
> So around a 35% increase.

I'd like to see a NULL or very low-overhead cipher under the same
circumstances.

> YMMV with different versions of TLS and associated ciphers, JREs,
> OpenSSl versions etc.

Noted. ;)

- -chris
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlkdK1IACgkQHPApP6U8
pFhfQw/+NGm1CNQcFZ2qVzlCZ36W+TXhaKaBcWeiSCKw60jf/utEFycONRldm5Q3
cRM7Nbrfx1GcPwAs8ufedOtHgsAfzp6JkpzqwVFqZjUX1GODbJhz1vaNgQgB3mL8
YlGBoLqQIRKvQNOcTYJx5bP+tbnqARu96uINH16rMT+GQUF9nIzk+ua7ec0Goe+e
6yO6euDrkV75uOMPArBWDDToSrQVZ9QKiliqlcYpnG2IPDMu1CGWDHZtwO1pxaLG
aMbtqea9gAj42rw3NpFjUNxqYdN4EJHhCFjIIdVCAbiqs5BZQQAjcWjaRPniq45M
ySsuBLNFqPj2sltlhZrdg7CEklvDbVvVgVIWZA21pw0wyfIofZnsiy+KsLo8q/wD
gHcOF/TkQ4pAYGVoP+wh5AnQHwze2SFTJq0RE7kE0s6cohtfXeNSH/Ga6lzbJW5d
B+vHpU8+U6X1Lpha8Hg0A1KxbP7hcANfdLTiRqZNIVMQES8p6Zh+fbIX+DlVYIFR
WLFNmFADdlZ5msxHwRjfdQ8dtL6McwyvM3kmDQeADU/YzN80bhXmr8ZHJJUevTUJ
cya5zcw5MmPrzdlavXhH0VKspbprPoJxrd9llRU0ra5aNfUmJ4xA79jD5VxQmNL/
Cglw5DT8QoxG3knjZEQ8YLRj0gq0NrQXQmzowxqekfMcyNc2EGg=
=+yjT
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: TLS handshake performance

Rémy Maucherat
2017-05-18 7:04 GMT+02:00 Christopher Schultz <[hidden email]>
:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Mark,
>
> On 5/17/17 5:31 PM, Mark Thomas wrote:
> > I got asked in the corridor at TomcatCon earlier today what the
> > relative performance of the TLS handshake was with 8.5.x, the NIO
> > connector and JSSE vs OpenSSL TLS implementation.
> I'm curious about what exactly "TLS handshake" was intended to mean
> (by the person who asked the question) in this context.
>
> The handshake itself does not perform any bulk transfer of encrypted
> data, so the negotiated cipher suite does not matter. However...
>
> > Tested with: ab -n 1000 -c 2 -f TLS1.2 -Z
> > ECDHE-RSA-AES128-GCM-SHA256 https://localhost:8443/test.txt
>
> Here the cipher suite matters very much, since the client is not only
> performing the TLS handshake but also transferring the client's
> request to the server and the server's response back to the client.
>
> Support for a particular algorithm may dominate the benchmark, here.
>

I only tested JSSE/OpenSSL with -k, and the actual encryption is
ridiculously fast compared to the handshake. So Mark's test gives new data
and, IMO, is a good "handshake performance" test where you are supposed to
negotiate a usable cipher.

Rémy

>
> What happens if you negotiate a NULL cipher for instance? Or, perform
> the TLS handshake but never make an HTTP request after connecting? I
> don't know of a tool that can do that out of the box (e.g. ab makes
> HTTP requests, not just TLS connections) but one could be written in
> Java fairly easily.
>
> > test.txt is a 3 byte text file.
> >
> > The results were: JSSE:    17 reqs/sec OpenSSL: 23 reqs/sec
> >
> > So around a 35% increase.
>
> I'd like to see a NULL or very low-overhead cipher under the same
> circumstances.
>
> > YMMV with different versions of TLS and associated ciphers, JREs,
> > OpenSSl versions etc.
>
> Noted. ;)
>
> - -chris
> -----BEGIN PGP SIGNATURE-----
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlkdK1IACgkQHPApP6U8
> pFhfQw/+NGm1CNQcFZ2qVzlCZ36W+TXhaKaBcWeiSCKw60jf/utEFycONRldm5Q3
> cRM7Nbrfx1GcPwAs8ufedOtHgsAfzp6JkpzqwVFqZjUX1GODbJhz1vaNgQgB3mL8
> YlGBoLqQIRKvQNOcTYJx5bP+tbnqARu96uINH16rMT+GQUF9nIzk+ua7ec0Goe+e
> 6yO6euDrkV75uOMPArBWDDToSrQVZ9QKiliqlcYpnG2IPDMu1CGWDHZtwO1pxaLG
> aMbtqea9gAj42rw3NpFjUNxqYdN4EJHhCFjIIdVCAbiqs5BZQQAjcWjaRPniq45M
> ySsuBLNFqPj2sltlhZrdg7CEklvDbVvVgVIWZA21pw0wyfIofZnsiy+KsLo8q/wD
> gHcOF/TkQ4pAYGVoP+wh5AnQHwze2SFTJq0RE7kE0s6cohtfXeNSH/Ga6lzbJW5d
> B+vHpU8+U6X1Lpha8Hg0A1KxbP7hcANfdLTiRqZNIVMQES8p6Zh+fbIX+DlVYIFR
> WLFNmFADdlZ5msxHwRjfdQ8dtL6McwyvM3kmDQeADU/YzN80bhXmr8ZHJJUevTUJ
> cya5zcw5MmPrzdlavXhH0VKspbprPoJxrd9llRU0ra5aNfUmJ4xA79jD5VxQmNL/
> Cglw5DT8QoxG3knjZEQ8YLRj0gq0NrQXQmzowxqekfMcyNc2EGg=
> =+yjT
> -----END PGP SIGNATURE-----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: TLS handshake performance

Rémy Maucherat
In reply to this post by Mark Thomas-2
2017-05-17 23:31 GMT+02:00 Mark Thomas <[hidden email]>:

> Hi all,
>
> I got asked in the corridor at TomcatCon earlier today what the relative
> performance of the TLS handshake was with 8.5.x, the NIO connector and JSSE
> vs OpenSSL TLS implementation.
>
> This might be something that is of interest to a wider audience so here
> goes...
>
> The following results are very rough and ready and generated with my
> (slightly aging now) laptop (4 cores).
>
> I tested trunk but the code is close enough to 8.5.x for this purpose.
>
> I used exacty the same config for each test. The only change was to
> add/remove the tc-native library to enable/disable OpenSSL.
>
> Tested with:
> ab -n 1000 -c 2 -f TLS1.2 -Z ECDHE-RSA-AES128-GCM-SHA256
> https://localhost:8443/test.txt
>
> test.txt is a 3 byte text file.
>
> The results were:
> JSSE:    17 reqs/sec
> OpenSSL: 23 reqs/sec
>
> So around a 35% increase.
>
> JRE: Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)
> OpenSSL: 1.0.2k
>
> YMMV with different versions of TLS and associated ciphers, JREs, OpenSSl
> versions etc.
>
> I did another test run on my own since I had never tested handshake.
Platform: Fedora 25 with its default JVM
(openjdk-1.8.0.131-1.b12.fc25.x86_64), OpenSSL 1.0.2k, a Skylake 6700k CPU.

The results are in req/s:

ab -n 10000 -c 20 -f TLS1.2 -Z ECDHE-RSA-AES128-GCM-SHA256
https://localhost:8443/tomcat.gif
NIO OpenSSL: 1900
NIO JSSE: 650
APR: 1900

+ -k (and also a bigger -n to make the test meaningful):
NIO OpenSSL: 53800
NIO JSSE: 24600
APR: 56900

My conclusion is also that the handshake is much faster with JSSE/OpenSSL
compared to vanilla JSSE, despite the code being complex and having a ton
of native calls. Testing localhost probably introduces a heavy bias as the
ab client also uses a lot of CPU, the real performance of vanilla JSSE on
my Fedora platform is actually worse than what it looks like. So on Fedora
25, it could be around 4 times slower for handshaking. The handshake seems
to be marginally slower with APR compared to JSSE/OpenSSL for some reason,
but then the encryption itself is faster. Undoubtedly the SSL engine adds a
bit of overhead to encryption.

Rémy
Reply | Threaded
Open this post in threaded view
|

Re: TLS handshake performance

Mark Thomas-2
In reply to this post by Christopher Schultz-2
This time to the right list...

On 18/05/2017 06:04, Christopher Schultz wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Mark,
>
> On 5/17/17 5:31 PM, Mark Thomas wrote:
>> I got asked in the corridor at TomcatCon earlier today what the
>> relative performance of the TLS handshake was with 8.5.x, the NIO
>> connector and JSSE vs OpenSSL TLS implementation.
> I'm curious about what exactly "TLS handshake" was intended to mean
> (by the person who asked the question) in this context.

They are using Tomcat in a scenario where clients are making single
requests (so keep alve doesn't help). Given that the handshake uses
asymmetric encryption which is more expensive that symmetric encryption
(which is why the handshake is used to establish a shared secret so
symmetric encryption can used for the actual data) they wanted a sense
of the performance benefit - if any - of NIO and 8.5.x with OpenSSL vs
NIO and 8.5.x with JSSE.

> The handshake itself does not perform any bulk transfer of encrypted
> data, so the negotiated cipher suite does not matter. However...
>
>> Tested with: ab -n 1000 -c 2 -f TLS1.2 -Z
>> ECDHE-RSA-AES128-GCM-SHA256 https://localhost:8443/test.txt
>
> Here the cipher suite matters very much, since the client is not only
> performing the TLS handshake but also transferring the client's
> request to the server and the server's response back to the client. >
> Support for a particular algorithm may dominate the benchmark, here.

Agreed. But it is the handshake that dominates the timings (if you add
-k to use keep-alive the req/sec are an order of magnitude higher).

The cipher suite was the default one chosen by by one of the configs (I
forget which).

The cipher suite will affect the results since it also impacts the
enctrpyion used during the handshake but for any 'reaosnably' secure
cipher suite, I'd expect similar results in terms of the relative
performance.

> What happens if you negotiate a NULL cipher for instance? Or, perform
> the TLS handshake but never make an HTTP request after connecting? I
> don't know of a tool that can do that out of the box (e.g. ab makes
> HTTP requests, not just TLS connections) but one could be written in
> Java fairly easily.
>
>> test.txt is a 3 byte text file.
>>
>> The results were: JSSE:    17 reqs/sec OpenSSL: 23 reqs/sec
>>
>> So around a 35% increase.
>
> I'd like to see a NULL or very low-overhead cipher under the same
> circumstances.

The test was with Tomcat trunk and the test keystore from the Tomcat PMC
private repo so you should be able to set up those tests pretty quickly.
(I did it in around 30 mins which incuded building tc-native).

Mark

>
>> YMMV with different versions of TLS and associated ciphers, JREs,
>> OpenSSl versions etc. >
> Noted. ;)
>
> - -chris
> -----BEGIN PGP SIGNATURE-----
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlkdK1IACgkQHPApP6U8
> pFhfQw/+NGm1CNQcFZ2qVzlCZ36W+TXhaKaBcWeiSCKw60jf/utEFycONRldm5Q3
> cRM7Nbrfx1GcPwAs8ufedOtHgsAfzp6JkpzqwVFqZjUX1GODbJhz1vaNgQgB3mL8
> YlGBoLqQIRKvQNOcTYJx5bP+tbnqARu96uINH16rMT+GQUF9nIzk+ua7ec0Goe+e
> 6yO6euDrkV75uOMPArBWDDToSrQVZ9QKiliqlcYpnG2IPDMu1CGWDHZtwO1pxaLG
> aMbtqea9gAj42rw3NpFjUNxqYdN4EJHhCFjIIdVCAbiqs5BZQQAjcWjaRPniq45M
> ySsuBLNFqPj2sltlhZrdg7CEklvDbVvVgVIWZA21pw0wyfIofZnsiy+KsLo8q/wD
> gHcOF/TkQ4pAYGVoP+wh5AnQHwze2SFTJq0RE7kE0s6cohtfXeNSH/Ga6lzbJW5d
> B+vHpU8+U6X1Lpha8Hg0A1KxbP7hcANfdLTiRqZNIVMQES8p6Zh+fbIX+DlVYIFR
> WLFNmFADdlZ5msxHwRjfdQ8dtL6McwyvM3kmDQeADU/YzN80bhXmr8ZHJJUevTUJ
> cya5zcw5MmPrzdlavXhH0VKspbprPoJxrd9llRU0ra5aNfUmJ4xA79jD5VxQmNL/
> Cglw5DT8QoxG3knjZEQ8YLRj0gq0NrQXQmzowxqekfMcyNc2EGg=
> =+yjT
> -----END PGP SIGNATURE-----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: TLS handshake performance

John.E.Gregg-2
> -----Original Message-----
> From: Mark Thomas [mailto:[hidden email]]
> Sent: Thursday, May 18, 2017 8:47 AM
> To: [hidden email]
> Subject: Re: TLS handshake performance
>
> This time to the right list...
>
> On 18/05/2017 06:04, Christopher Schultz wrote:
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA256
> >
> > Mark,
> >
> > On 5/17/17 5:31 PM, Mark Thomas wrote:
> >> I got asked in the corridor at TomcatCon earlier today what the
> >> relative performance of the TLS handshake was with 8.5.x, the NIO
> >> connector and JSSE vs OpenSSL TLS implementation.
> > I'm curious about what exactly "TLS handshake" was intended to mean
> > (by the person who asked the question) in this context.
>
> They are using Tomcat in a scenario where clients are making single requests
> (so keep alve doesn't help). Given that the handshake uses asymmetric
> encryption which is more expensive that symmetric encryption (which is why
> the handshake is used to establish a shared secret so symmetric encryption
> can used for the actual data) they wanted a sense of the performance
> benefit - if any - of NIO and 8.5.x with OpenSSL vs NIO and 8.5.x with JSSE.
>
> > The handshake itself does not perform any bulk transfer of encrypted
> > data, so the negotiated cipher suite does not matter. However...
> >
> >> Tested with: ab -n 1000 -c 2 -f TLS1.2 -Z
> >> ECDHE-RSA-AES128-GCM-SHA256 https://localhost:8443/test.txt
> >
> > Here the cipher suite matters very much, since the client is not only
> > performing the TLS handshake but also transferring the client's
> > request to the server and the server's response back to the client. >
> > Support for a particular algorithm may dominate the benchmark, here.
>
> Agreed. But it is the handshake that dominates the timings (if you add -k to
> use keep-alive the req/sec are an order of magnitude higher).
>
> The cipher suite was the default one chosen by by one of the configs (I
> forget which).
>
> The cipher suite will affect the results since it also impacts the enctrpyion
> used during the handshake but for any 'reaosnably' secure cipher suite, I'd
> expect similar results in terms of the relative performance.
>


What about SSL session reuse?  Did you have any way of disabling that?  If you're testing against a single server, you'll get 100% reuse, which makes a big difference in the CPU load (lower) on the server.  If your real world case involves dozens or hundreds of servers behind a load balancer, you get almost no reuse.

Thanks


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: TLS handshake performance

Rainer Jung-3
Am 18.05.2017 um 16:03 schrieb [hidden email]:

>> -----Original Message-----
>> From: Mark Thomas [mailto:[hidden email]]
>> Sent: Thursday, May 18, 2017 8:47 AM
>> To: [hidden email]
>> Subject: Re: TLS handshake performance
>>
>> This time to the right list...
>>
>> On 18/05/2017 06:04, Christopher Schultz wrote:
>>> -----BEGIN PGP SIGNED MESSAGE-----
>>> Hash: SHA256
>>>
>>> Mark,
>>>
>>> On 5/17/17 5:31 PM, Mark Thomas wrote:
>>>> I got asked in the corridor at TomcatCon earlier today what the
>>>> relative performance of the TLS handshake was with 8.5.x, the NIO
>>>> connector and JSSE vs OpenSSL TLS implementation.
>>> I'm curious about what exactly "TLS handshake" was intended to mean
>>> (by the person who asked the question) in this context.
>>
>> They are using Tomcat in a scenario where clients are making single requests
>> (so keep alve doesn't help). Given that the handshake uses asymmetric
>> encryption which is more expensive that symmetric encryption (which is why
>> the handshake is used to establish a shared secret so symmetric encryption
>> can used for the actual data) they wanted a sense of the performance
>> benefit - if any - of NIO and 8.5.x with OpenSSL vs NIO and 8.5.x with JSSE.
>>
>>> The handshake itself does not perform any bulk transfer of encrypted
>>> data, so the negotiated cipher suite does not matter. However...
>>>
>>>> Tested with: ab -n 1000 -c 2 -f TLS1.2 -Z
>>>> ECDHE-RSA-AES128-GCM-SHA256 https://localhost:8443/test.txt
>>>
>>> Here the cipher suite matters very much, since the client is not only
>>> performing the TLS handshake but also transferring the client's
>>> request to the server and the server's response back to the client. >
>>> Support for a particular algorithm may dominate the benchmark, here.
>>
>> Agreed. But it is the handshake that dominates the timings (if you add -k to
>> use keep-alive the req/sec are an order of magnitude higher).
>>
>> The cipher suite was the default one chosen by by one of the configs (I
>> forget which).
>>
>> The cipher suite will affect the results since it also impacts the enctrpyion
>> used during the handshake but for any 'reaosnably' secure cipher suite, I'd
>> expect similar results in terms of the relative performance.
>>
>
>
> What about SSL session reuse?  Did you have any way of disabling that?  If you're testing against a single server, you'll get 100% reuse, which makes a big difference in the CPU load (lower) on the server.  If your real world case involves dozens or hundreds of servers behind a load balancer, you get almost no reuse.

Session reuse does not work magically. The client and the server have to
support it, being it using a session cache and IDs or using TLS session
tickets.

I am pretty sure, that "ab", the client used during these tests, does
not support TLS session reuse.

Regards,

Rainer

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: TLS handshake performance

Christopher Schultz-2
In reply to this post by Mark Thomas-2
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Mark,

On 5/18/17 9:47 AM, Mark Thomas wrote:

> On 18/05/2017 06:04, Christopher Schultz wrote: Mark,
>
> On 5/17/17 5:31 PM, Mark Thomas wrote:
>>>> I got asked in the corridor at TomcatCon earlier today what
>>>> the relative performance of the TLS handshake was with 8.5.x,
>>>> the NIO connector and JSSE vs OpenSSL TLS implementation.
> I'm curious about what exactly "TLS handshake" was intended to
> mean (by the person who asked the question) in this context.
>
>> They are using Tomcat in a scenario where clients are making
>> single requests (so keep alve doesn't help). Given that the
>> handshake uses asymmetric encryption which is more expensive that
>> symmetric encryption (which is why the handshake is used to
>> establish a shared secret so symmetric encryption can used for
>> the actual data) they wanted a sense of the performance benefit -
>> if any - of NIO and 8.5.x with OpenSSL vs NIO and 8.5.x with
>> JSSE.
>
> The handshake itself does not perform any bulk transfer of
> encrypted data, so the negotiated cipher suite does not matter.
> However...
>
>>>> Tested with: ab -n 1000 -c 2 -f TLS1.2 -Z
>>>> ECDHE-RSA-AES128-GCM-SHA256 https://localhost:8443/test.txt
>
> Here the cipher suite matters very much, since the client is not
> only performing the TLS handshake but also transferring the
> client's request to the server and the server's response back to
> the client. > Support for a particular algorithm may dominate the
> benchmark, here.
>
>> Agreed. But it is the handshake that dominates the timings (if
>> you add -k to use keep-alive the req/sec are an order of
>> magnitude higher).

Right, but tat only tests one single handshake for many requests, when
they wanted to know the handshake improvement.

>> The cipher suite was the default one chosen by by one of the
>> configs (I forget which).
>
>> The cipher suite will affect the results since it also impacts
>> the enctrpyion used during the handshake but for any 'reaosnably'
>> secure cipher suite, I'd expect similar results in terms of the
>> relative performance.

The cipher suite chosen does not affect the performance of the
handshake at all, since the handshake is 100% asymmetric. That's why I
suggested using a NULL cipher if you want to test just the handshake.
Honestly, I would have made a TLS connection and then town it down had
I been asked the same questions.

But here it's clear that the client wants to know "do I get a
performance benefit swapping-out JSSE for OpenSSL. I think we all knew
what the answer was. Jean-Frederick's slides from yesterday I believe
include such benchmarks as well (NIO/OpenSSL vs NIO/JSSE vs APR/OpenSSL)
.

- -chris
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlke6BUACgkQHPApP6U8
pFgx4BAAw1VDMee6M9B2BqcNnXgFTvq4ReVJNBJaXQiNWSPk+GAQD1bR9gB2/gxO
6/kKaOLEqeFrl54qCo1hHAgEtZXdVV8/Bhx1KwxVSmN5qV1ox4uuul4P5KMMX/xY
Eay1o1mwbee5ayZP3DVcwy8kVwpGAAnrmlSzxTz4LEJVRJAnAjoxx6RV3hGSWFZs
S5HAyCswz8mBQq8DobUr7BG3GmT2KbXL0QKNR28cwrIRXlMnINzXnkcwT0HjpFiE
Zb3fIm6KmDdedtgs8PsCUo6BR13e3y2td2ont4YBz7HC1g9OeYt/3sAPC4rdeBlY
pTFqwvnFA81+9UXIOs+5DSAtn/z7zNYolqshR815+s5Kp1B4UE+oSWeERc4cTZQU
yKd2nFx4dE+6aZKlCjFeb5cs4OmJTShX7aRXXSNFTmixd3HSvj0Xa0NZ9kVwzkji
SpMExUJ5dEyRrVaYbfEhORs6BGeXgnF/ueKvW5VPfs4ZCswD/MWfrxAxMkT8BdHn
7+hkX3J0zWxJoWf6aC3HrjyPXz3ia06+7xpIelJ8vfmHf5X06i1KEv1yo1rG05/4
UnPsxCV1G1xD9Y5qlU+Md6vmLCJ0OBH1+hSlQT4Ib7WHdy6oVQbgSP8cfBLGJR0J
+FlzeuXlu54GnYRDd0CSIPc/CEOpcSxesW+jGk+ru6G+Dull5hw=
=7/Wz
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: TLS handshake performance

Rémy Maucherat
2017-05-19 14:42 GMT+02:00 Christopher Schultz <[hidden email]
>:

> But here it's clear that the client wants to know "do I get a
> performance benefit swapping-out JSSE for OpenSSL. I think we all knew
> what the answer was. Jean-Frederick's slides from yesterday I believe
> include such benchmarks as well (NIO/OpenSSL vs NIO/JSSE vs APR/OpenSSL)
> .
>

I guess the answer is "obvious", but having numbers to back it is better.
JF and myself have never tested without keepalive as we focused more on the
cipher performance, so it was a good idea to do it.

Rémy
Reply | Threaded
Open this post in threaded view
|

Re: TLS handshake performance

Christopher Schultz-2
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Rémy,

On 5/19/17 8:47 AM, Rémy Maucherat wrote:

> 2017-05-19 14:42 GMT+02:00 Christopher Schultz
> <[hidden email]
>> :
>
>> But here it's clear that the client wants to know "do I get a
>> performance benefit swapping-out JSSE for OpenSSL. I think we all
>> knew what the answer was. Jean-Frederick's slides from yesterday
>> I believe include such benchmarks as well (NIO/OpenSSL vs
>> NIO/JSSE vs APR/OpenSSL) .
>>
>
> I guess the answer is "obvious", but having numbers to back it is
> better. JF and myself have never tested without keepalive as we
> focused more on the cipher performance, so it was a good idea to do
> it.

Oh, yes, of course. I had forgotten that your tests did not include
KeepAlives.

A better real-world test for those benchmarks should be to have a very
limited number of KeepAlives allowed, since most clients don't make
10k requests over a single KeepAlive connection... it's more like 2 - 20
.

- -chris
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlkfAPMACgkQHPApP6U8
pFiHEhAAo5cjZOi8g7Qw4FGB8WH8UPyTVdpFezl87rBcMkKw4n/SIM+LPHKN8rJF
KYSzR00jYiDYp07S1GiS9GJB2XSA4MQ3TiVob05eqOVNnBsqKx26D3gsM8ofgy1p
mmsfJGh2jBp2gQrfP7hmx4uNwb3k1DFQopVP78bvmxftONEbGHyMW+Q9ich6SE7w
OqS8I6o8RTDdlPVtgCUaJYLUa0oR6iLha50QqwaJ8yHxzj8O+cTEv2Q4e7EaSYHH
Jw1HX7AvCxUsqk2KBW6CpnAQIXxcuPNMOtnkUo2gQdHaaQniAUh8RVqMkOgLwsYm
9mKu0bV2spf1Xg3qBCbicOeOHpSAMszwsHWmiaNQUBjoBpYU12Xg/gfZYrgCS4bL
f/qwNhbNC9LY9A6gJTTML4iJj5sKmENGg/RzyD2VQhopD4RGdUPOKscHiJs18hYy
FsQ2JuFhIDxPy3an5I/nkjKhHb8Lcqbu0LbAPT9Vw3I8WtN77mYwDvDtEXx9Cmkp
tqaYymRrLDtqK1X8+OEgOoQinBPqv0fjlbfS60IuHohKz8YGKEXrCcjtnMDzysKm
qQdxKq0DrcpR7qOHacf9sLTQF0iWtCdDleb/KT/e5pyI3WWq+SKrK/W6GUknw0hc
8neWjSOnpJ1IhvJdMFZ94t8Dsyklqkrhd8dYK1FhL3wSTPkb+Rk=
=uneW
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: TLS handshake performance

Rémy Maucherat
2017-05-19 16:28 GMT+02:00 Christopher Schultz <[hidden email]
>:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Rémy,
>
> On 5/19/17 8:47 AM, Rémy Maucherat wrote:
> > 2017-05-19 14:42 GMT+02:00 Christopher Schultz
> > <[hidden email]
> >> :
> >
> >> But here it's clear that the client wants to know "do I get a
> >> performance benefit swapping-out JSSE for OpenSSL. I think we all
> >> knew what the answer was. Jean-Frederick's slides from yesterday
> >> I believe include such benchmarks as well (NIO/OpenSSL vs
> >> NIO/JSSE vs APR/OpenSSL) .
> >>
> >
> > I guess the answer is "obvious", but having numbers to back it is
> > better. JF and myself have never tested without keepalive as we
> > focused more on the cipher performance, so it was a good idea to do
> > it.
>
> Oh, yes, of course. I had forgotten that your tests did not include
> KeepAlives.
>
> A better real-world test for those benchmarks should be to have a very
> limited number of KeepAlives allowed, since most clients don't make
> 10k requests over a single KeepAlive connection... it's more like 2 - 20
>
> Yes, with a more typical keepalive count, you get a better mix with the
high performance cost of handshaking.

IMO ab is never real world though, you only request one single resource.
It's a good test when you're trying to optimize and/or have a measure of
raw speed of a certain simple task, but that's it. Benchmarking is serious
business [and it's also all lies obviously :) ].

Rémy