Hostnames with underscores

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Hostnames with underscores

Amit Pande
Hello all,

Recent Tomcat versions (8.5.32 I think) has made a stricter validation for hostnames with underscores in it. (https://bz.apache.org/bugzilla/show_bug.cgi?id=62371)

This is understandably for addressing security issues (https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-6816) and enforcing RFC compliance, in some way.

Our recent upgrade to Tomcat (8.5.34), we observed:

Note: further occurrences of request parsing errors will be logged at DEBUG level.
java.lang.IllegalArgumentException: The character [_] is never valid in a domain name.
        at org.apache.tomcat.util.http.parser.HttpParser$DomainParseState.next(HttpParser.java:946)
        at org.apache.tomcat.util.http.parser.HttpParser.readHostDomainName(HttpParser.java:842)
        at org.apache.tomcat.util.http.parser.Host.parse(Host.java:66)
        at org.apache.tomcat.util.http.parser.Host.parse(Host.java:40)
        at org.apache.coyote.AbstractProcessor.parseHost(AbstractProcessor.java:286)
        at org.apache.coyote.http11.Http11Processor.prepareRequest(Http11Processor.java:1203)
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:776)
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:806)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1498)
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)


The main issue for us now is that, since we ship Tomcat with our product (IOW Tomcat runs in customer environments), this will break our product functionality if customers have hostnames with underscore. Ideally, they should correct the host names (to be RFC compliant) but customers would be really averse to change hostname as it might a widespread change in their environments.

With Spring also, we ran into same issue but in later releases of Spring this was fixed. I

Would it be a good idea to make this strict check configurable so that we can continue to cater to our customers without breaking the functionality?

Are there any other alternates to help solve this issue?

Thanks,
Amit





Reply | Threaded
Open this post in threaded view
|

Re: Hostnames with underscores

M. Manna
Have you checked the connector config doc for relaxedPathChars and
relaxedQueryChars?



On Fri, 26 Oct 2018 at 18:00, Amit Pande <[hidden email]> wrote:

> Hello all,
>
> Recent Tomcat versions (8.5.32 I think) has made a stricter validation for
> hostnames with underscores in it. (
> https://bz.apache.org/bugzilla/show_bug.cgi?id=62371)
>
> This is understandably for addressing security issues (
> https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-6816) and
> enforcing RFC compliance, in some way.
>
> Our recent upgrade to Tomcat (8.5.34), we observed:
>
> Note: further occurrences of request parsing errors will be logged at
> DEBUG level.
> java.lang.IllegalArgumentException: The character [_] is never valid in a
> domain name.
>         at
> org.apache.tomcat.util.http.parser.HttpParser$DomainParseState.next(HttpParser.java:946)
>         at
> org.apache.tomcat.util.http.parser.HttpParser.readHostDomainName(HttpParser.java:842)
>         at org.apache.tomcat.util.http.parser.Host.parse(Host.java:66)
>         at org.apache.tomcat.util.http.parser.Host.parse(Host.java:40)
>         at
> org.apache.coyote.AbstractProcessor.parseHost(AbstractProcessor.java:286)
>         at
> org.apache.coyote.http11.Http11Processor.prepareRequest(Http11Processor.java:1203)
>         at
> org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:776)
>         at
> org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
>         at
> org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:806)
>         at org.apache.tomcat.util.net
> .NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1498)
>         at org.apache.tomcat.util.net
> .SocketProcessorBase.run(SocketProcessorBase.java:49)
>         at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>         at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
>         at java.lang.Thread.run(Thread.java:748)
>
>
> The main issue for us now is that, since we ship Tomcat with our product
> (IOW Tomcat runs in customer environments), this will break our product
> functionality if customers have hostnames with underscore. Ideally, they
> should correct the host names (to be RFC compliant) but customers would be
> really averse to change hostname as it might a widespread change in their
> environments.
>
> With Spring also, we ran into same issue but in later releases of Spring
> this was fixed. I
>
> Would it be a good idea to make this strict check configurable so that we
> can continue to cater to our customers without breaking the functionality?
>
> Are there any other alternates to help solve this issue?
>
> Thanks,
> Amit
>
>
>
>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [EXTERNAL] Re: Hostnames with underscores

Amit Pande
Yes, I did check the description, but did not find reference to underscore.

From the documentation:
“The value may be any combination of the following characters: " < > [ \ ] ^ ` { | } . “

But I admit that I did not actually verify it and will see if these attributes work for underscore too.

Thanks,
Amit

On Oct 26, 2018, at 12:02 PM, M. Manna <[hidden email]<mailto:[hidden email]>> wrote:

Have you checked the connector config doc for relaxedPathChars and
relaxedQueryChars?



On Fri, 26 Oct 2018 at 18:00, Amit Pande <[hidden email]<mailto:[hidden email]>> wrote:

Hello all,

Recent Tomcat versions (8.5.32 I think) has made a stricter validation for
hostnames with underscores in it. (
https://bz.apache.org/bugzilla/show_bug.cgi?id=62371)

This is understandably for addressing security issues (
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-6816) and
enforcing RFC compliance, in some way.

Our recent upgrade to Tomcat (8.5.34), we observed:

Note: further occurrences of request parsing errors will be logged at
DEBUG level.
java.lang.IllegalArgumentException: The character [_] is never valid in a
domain name.
       at
org.apache.tomcat.util.http.parser.HttpParser$DomainParseState.next(HttpParser.java:946)
       at
org.apache.tomcat.util.http.parser.HttpParser.readHostDomainName(HttpParser.java:842)
       at org.apache.tomcat.util.http.parser.Host.parse(Host.java:66)
       at org.apache.tomcat.util.http.parser.Host.parse(Host.java:40)
       at
org.apache.coyote.AbstractProcessor.parseHost(AbstractProcessor.java:286)
       at
org.apache.coyote.http11.Http11Processor.prepareRequest(Http11Processor.java:1203)
       at
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:776)
       at
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
       at
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:806)
       at org.apache.tomcat.util.net<http://org.apache.tomcat.util.net>
.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1498)
       at org.apache.tomcat.util.net<http://org.apache.tomcat.util.net>
.SocketProcessorBase.run(SocketProcessorBase.java:49)
       at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
       at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
       at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
       at java.lang.Thread.run(Thread.java:748)


The main issue for us now is that, since we ship Tomcat with our product
(IOW Tomcat runs in customer environments), this will break our product
functionality if customers have hostnames with underscore. Ideally, they
should correct the host names (to be RFC compliant) but customers would be
really averse to change hostname as it might a widespread change in their
environments.

With Spring also, we ran into same issue but in later releases of Spring
this was fixed. I

Would it be a good idea to make this strict check configurable so that we
can continue to cater to our customers without breaking the functionality?

Are there any other alternates to help solve this issue?

Thanks,
Amit






Reply | Threaded
Open this post in threaded view
|

Re: [EXTERNAL] Re: Hostnames with underscores

M. Manna
Read the description on RFC 7230 and 3986 - just to be sure. You might be
right after all.



On Fri, 26 Oct 2018 at 18:25, Amit Pande <[hidden email]> wrote:

> Yes, I did check the description, but did not find reference to underscore.
>
> From the documentation:
> “The value may be any combination of the following characters: " < > [ \ ]
> ^ ` { | } . “
>
> But I admit that I did not actually verify it and will see if these
> attributes work for underscore too.
>
> Thanks,
> Amit
>
> On Oct 26, 2018, at 12:02 PM, M. Manna <[hidden email]<mailto:
> [hidden email]>> wrote:
>
> Have you checked the connector config doc for relaxedPathChars and
> relaxedQueryChars?
>
>
>
> On Fri, 26 Oct 2018 at 18:00, Amit Pande <[hidden email]<mailto:
> [hidden email]>> wrote:
>
> Hello all,
>
> Recent Tomcat versions (8.5.32 I think) has made a stricter validation for
> hostnames with underscores in it. (
> https://bz.apache.org/bugzilla/show_bug.cgi?id=62371)
>
> This is understandably for addressing security issues (
> https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-6816) and
> enforcing RFC compliance, in some way.
>
> Our recent upgrade to Tomcat (8.5.34), we observed:
>
> Note: further occurrences of request parsing errors will be logged at
> DEBUG level.
> java.lang.IllegalArgumentException: The character [_] is never valid in a
> domain name.
>        at
>
> org.apache.tomcat.util.http.parser.HttpParser$DomainParseState.next(HttpParser.java:946)
>        at
>
> org.apache.tomcat.util.http.parser.HttpParser.readHostDomainName(HttpParser.java:842)
>        at org.apache.tomcat.util.http.parser.Host.parse(Host.java:66)
>        at org.apache.tomcat.util.http.parser.Host.parse(Host.java:40)
>        at
> org.apache.coyote.AbstractProcessor.parseHost(AbstractProcessor.java:286)
>        at
>
> org.apache.coyote.http11.Http11Processor.prepareRequest(Http11Processor.java:1203)
>        at
> org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:776)
>        at
>
> org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
>        at
>
> org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:806)
>        at org.apache.tomcat.util.net<http://org.apache.tomcat.util.net>
> .NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1498)
>        at org.apache.tomcat.util.net<http://org.apache.tomcat.util.net>
> .SocketProcessorBase.run(SocketProcessorBase.java:49)
>        at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>        at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>        at
>
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
>        at java.lang.Thread.run(Thread.java:748)
>
>
> The main issue for us now is that, since we ship Tomcat with our product
> (IOW Tomcat runs in customer environments), this will break our product
> functionality if customers have hostnames with underscore. Ideally, they
> should correct the host names (to be RFC compliant) but customers would be
> really averse to change hostname as it might a widespread change in their
> environments.
>
> With Spring also, we ran into same issue but in later releases of Spring
> this was fixed. I
>
> Would it be a good idea to make this strict check configurable so that we
> can continue to cater to our customers without breaking the functionality?
>
> Are there any other alternates to help solve this issue?
>
> Thanks,
> Amit
>
>
>
>
>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Hostnames with underscores

markt
In reply to this post by M. Manna
On October 26, 2018 5:02:25 PM UTC, "M. Manna" <[hidden email]> wrote:
>Have you checked the connector config doc for relaxedPathChars and
>relaxedQueryChars?

Those options have no impact on the parsing of the hist name.

>On Fri, 26 Oct 2018 at 18:00, Amit Pande <[hidden email]>
>wrote:
>
>> Hello all,
>>
>> Recent Tomcat versions (8.5.32 I think) has made a stricter
>validation for
>> hostnames with underscores in it. (
>> https://bz.apache.org/bugzilla/show_bug.cgi?id=62371)

<snip/>

>> The main issue for us now is that, since we ship Tomcat with our
>product
>> (IOW Tomcat runs in customer environments), this will break our
>product
>> functionality if customers have hostnames with underscore. Ideally,
>they
>> should correct the host names (to be RFC compliant) but customers
>would be
>> really averse to change hostname as it might a widespread change in
>their
>> environments.
>>
>> With Spring also, we ran into same issue but in later releases of
>Spring
>> this was fixed. I
>>
>> Would it be a good idea to make this strict check configurable so
>that we
>> can continue to cater to our customers without breaking the
>functionality?

The host names are invalid and should be fixed. Generally, Tomcat does not provide options to break spec compliance unless there is a compelling case to do so. I haven't seen such a case made for relaxing the host name parsing.

>> Are there any other alternates to help solve this issue?

Short of patching Tomcat yourself (not a course of action I recommend), no.

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]