Tomcat on Amazon's Graviton2

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Tomcat on Amazon's Graviton2

Chen Levy
Hello Everyone

We've been running our Staging environment on ARM64 based servers (Amazon EC2 T4g) for a while now, and plan to use similar machines to replace our Production environment's x86-64 based servers in the near future.
All of our servers are based on Amazon Linux 2 with AdoptOpenJDK 15.0.2 and Tomcat 9.
Measured with our application, performance between x86-64 and ARM64 with two similarly sized machines (e.g. T3.large vs T4g.large) is up by an average of 22% on the T4g's.
These numbers are largely academic, and will stay that way until we actually deploy a few machines in Production.
The ARM64 machines are also cheaper by about 20%, so if all goes well, there aren't real downsides to the move.

I read this correspondence about a year ago:
https://marc.info/?l=tomcat-user&w=2&r=1&s=arm64&q=b

I'd like to ask the community if anyone has production experience with Tomcat and ARM64/Graviton2, and whether there are specific issues I should beware of.

Thanks
Chen


Reply | Threaded
Open this post in threaded view
|

Re: Tomcat on Amazon's Graviton2

Martin Grigorov
Hi Chen,

On Wed, Feb 10, 2021 at 6:00 PM Chen Levy <[hidden email]> wrote:

> Hello Everyone
>
> We've been running our Staging environment on ARM64 based servers (Amazon
> EC2 T4g) for a while now, and plan to use similar machines to replace our
> Production environment's x86-64 based servers in the near future.
> All of our servers are based on Amazon Linux 2 with AdoptOpenJDK 15.0.2
> and Tomcat 9.
> Measured with our application, performance between x86-64 and ARM64 with
> two similarly sized machines (e.g. T3.large vs T4g.large) is up by an
> average of 22% on the T4g's.
> These numbers are largely academic, and will stay that way until we
> actually deploy a few machines in Production.
> The ARM64 machines are also cheaper by about 20%, so if all goes well,
> there aren't real downsides to the move.
>
> I read this correspondence about a year ago:
> https://marc.info/?l=tomcat-user&w=2&r=1&s=arm64&q=b
>
> I'd like to ask the community if anyone has production experience with
> Tomcat and ARM64/Graviton2, and whether there are specific issues I should
> beware of.
>

I have experience on ARM64 architecture but on Huawei HiSilicon chips
(Kunpeng), not Graviton2.
TLDR - everything runs just fine! :-)

In the beginning we had some bad performance results but they were caused
by the load testing tools -
https://martin-grigorov.medium.com/compare-apache-tomcat-performance-on-x86-64-and-arm64-cpu-architectures-aacfbb0b5bb6
Initially I used Apache JMeter <https://jmeter.apache.org/> 5.2.1 and it
had the issue of closing the connection after each request. This caused
very bad results for HTTPS. JMeter team helped me by suggesting two
settings which fixed the problem. Those settings are now enabled by default
in 5.3.0+
Then I replaced JMeter with wrk <https://github.com/wg/wrk> - it gives much
better throughput results than JMeter! But at some point I realized (with
the help of Poul Henning Kamp from Varnish Cache team) that it has a nasty
bug in calculating the statistics (
https://martin-grigorov.medium.com/compare-varnish-cache-performance-on-x86-64-and-aarch64-cpu-architectures-cef5ad5fee5f
)
So I moved to Vegeta <https://github.com/tsenart/vegeta> ! It is almost
perfect for my needs. It has just one issue
<https://github.com/tsenart/vegeta/issues/540> - Golang http library sends
RST_STREAM frame after the END_STREAM for HTTP/2. But Mark Thomas improved
Tomcat to keep lightweight versions of the HTTP2 streams for some more time
and this worked around this issue.

For my tests I used two *similar* VMs - one x86_64 and one aarch64. The
have almost identical hardware but due to the different architectures not
exactly the same.
https://martin-grigorov.medium.com/comparing-the-performance-of-several-http2-web-servers-fb5f3787532a
is
my latest blog article related to Tomcat and there x86_64 still won. In the
same tests Netty performed better on aarch64 (same hardware, JDK and
OpenSSL versions!). I guess the reason is in the custom EPoll impl that
Netty uses, but I am not sure.

Apart from Tomcat most of the problems related to moving to ARM64 we found
were related to file format (Protobuf, Parquet) and file compression
(Brotli) libraries because they use native libraries.
The only big problem with pure Java (well, Scala in this case) I am aware
of was in Scala Cats library:
- https://github.com/typelevel/cats-effect/releases/tag/v3.0.0-M4
- code with explanation from OpenJDK expert:
https://gist.github.com/RaasAhsan/8e3554a41e07068536425ca0de46c9e8

Good luck!

Martin


>
> Thanks
> Chen
>
>
>