[squid-users] deny_info and CONNECT for https request gives SSL error

[Amish]

Hello

I have this simplified ACL in squid.conf

acl denyit src all
deny_info http://192.168.1.1/blocked.html denyit
http_access deny denyit

The purpose is to block everything and redirect to
http://192.168.1.1/blocked.html

It works fine with http (non-secure) CONNECT request ...

curl -ix 192.168.1.1:8080 http://google.com

HTTP/1.1 302 Found
Server: squid/4.3
Mime-Version: 1.0
Date: Tue, 16 Oct 2018 11:02:05 GMT
Content-Type: text/html;charset=utf-8
Content-Length: 0
Location: http://192.168.1.1/blocked.html
X-Squid-Error: 403 Access Denied
X-Cache: MISS from somehost
X-Cache-Lookup: NONE from somehost:8080
Connection: keep-alive

 ... but gives certificate error with https CONNECT request.

curl -ix 192.168.1.1:8080 https://google.com

HTTP/1.1 200 Connection established

curl: (60) SSL certificate problem: self signed certificate in
certificate chain
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.


I believe this is because squid first gives 200 status and hence browser
thinks connection is established expects Google's certificate.

Can squid not return with 302/307 on CONNECT HTTPs requests for deny_info?

Like instead of HTTP/1.1 200 Connection established - it returns:

HTTP/1.1 307 Redirect
Location: http://192.168.1.1/blocked.html
(...)

RFC 7231 https://tools.ietf.org/html/rfc7231#section-4.3.6 states that:

Any response other than a successful response
indicates that the tunnel has not yet been formed and that the
connection remains governed by HTTP.


which means HTTP/1.1 307 Redirect should make browser treat connection
as HTTP and hopefully also follow Location.

Any idea? Or any other workaround (except importing squid certificate)

Thanks and regards,

Amish

[Amish]

Further to this:

I have ssl-bump setup on port 8080.

If I remove ssl-bump squid works just like I mentioned in my earlier e-mail.

curl -ix 192.168.1.1:8080 https://google.com

HTTP/1.1 307 Temporary Redirect
Server: squid/4.3
Mime-Version: 1.0
Date: Tue, 16 Oct 2018 12:01:41 GMT
Content-Type: text/html;charset=utf-8
Content-Length: 0
Location: http://192.168.1.1/blocked.html
X-Squid-Error: 403 Access Denied
X-Cache: MISS from somehost
X-Cache-Lookup: NONE from somehost:8080
Connection: keep-alive

curl: (56) Received HTTP code 307 from proxy after CONNECT


So then I reactivated ssl-bump and added this at top of squid.conf:

http_port 8080 ssl-bump ...
ssl_bump splice all

i.e. http_port has ssl-bump but squid is not really supposed to bump at all.

But squid still goes ahead and replies with 200 Connection established.

So it means that this is a bug in ssl-bump implementation.

It seems that current algorithm for ssl-bump is:

1) Browser sends CONNECT request
2) Squid sends status 200 Connection Established
3) Check ACL
4) If denied, bump the connection with squid certificate
5) If allowed, go ahead with tunneling / bumping as applicable


In my opinion correct flow should be like this:

1) Browser sends CONNECT request
2) Check ACL
3) If denied, return with 307 (or 302)
4) If allowed, go ahead with tunneling / bumping as applicable


Please test / check,

Thank you,

Amish.

Hello
I have this simplified ACL in squid.conf
acl denyit src all
deny_info http://192.168.1.1/blocked.html denyit
http_access deny denyit
The purpose is to block everything and redirect to
http://192.168.1.1/blocked.html
It works fine with http (non-secure) CONNECT request ...

curl -ix 192.168.1.1:8080 http://google.com

HTTP/1.1 302 Found
Server: squid/4.3
Mime-Version: 1.0
Date: Tue, 16 Oct 2018 11:02:05 GMT
Content-Type: text/html;charset=utf-8
Content-Length: 0
Location: http://192.168.1.1/blocked.html
X-Squid-Error: 403 Access Denied
X-Cache: MISS from somehost
X-Cache-Lookup: NONE from somehost:8080
Connection: keep-alive
 ... but gives certificate error with https CONNECT request.

curl -ix 192.168.1.1:8080 https://google.com

HTTP/1.1 200 Connection established
curl: (60) SSL certificate problem: self signed certificate in
certificate chain
More details here: https://curl.haxx.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this
situation and
how to fix it, please visit the web page mentioned above.
I believe this is because squid first gives 200 status and hence
browser thinks connection is established expects Google's certificate.
Can squid not return with 302/307 on CONNECT HTTPs requests for deny_info?
HTTP/1.1 307 Redirect
Location: http://192.168.1.1/blocked.html
(...)
Any response other than a successful response
indicates that the tunnel has not yet been formed and that the
connection remains governed by HTTP.
which means HTTP/1.1 307 Redirect should make browser treat connection
as HTTP and hopefully also follow Location.
Any idea? Or any other workaround (except importing squid certificate)
Thanks and regards,
Amish

[Alex Rousskov]

Post by Amish
1) Browser sends CONNECT request
2) Squid sends status 200 Connection Established
3) Check ACL
4) If denied, bump the connection with squid certificate
5) If allowed, go ahead with tunneling / bumping as applicable

Kind of. The actual sequence in this particular case is a bit different
because Squid discovers a denied CONNECT _before_ Squid decides how to
respond to that CONNECT request:

1) Receive CONNECT request.
3) Check access controls. The request is denied!
2) Send HTTP 200 Connection Established (see below for "why").
4) Bump the connections to deny/redirect the first bumped request.

Post by Amish
1) Browser sends CONNECT request
2) Check ACL
3) If denied, return with 307 (or 302)
4) If allowed, go ahead with tunneling / bumping as applicable

Unfortunately, that ideal sequence does not work well in practice
because popular browsers ignore CONNECT responses other than HTTP 200
and 407. As a consequence, if you want to redirect "secure" browser
traffic, Squid has to bump it first.


HTH,

Alex.

[Amish]

Post by Alex Rousskov

Post by Amish
1) Browser sends CONNECT request
2) Check ACL
3) If denied, return with 307 (or 302)
4) If allowed, go ahead with tunneling / bumping as applicable

Unfortunately, that ideal sequence does not work well in practice
because popular browsers ignore CONNECT responses other than HTTP 200
and 407. As a consequence, if you want to redirect "secure" browser
traffic, Squid has to bump it first.
HTH,
Alex.

No thats not correct.

Thing is that squid behaves differently for 2 exactly same CONNECT
request with only difference being ssl-bump

Case 1:
http_port 8080 #no ssl-bump
acl denyit src all
deny_info http://192.168.1.1/blocked.html denyit
http_access deny denyit

Post by Alex Rousskov
curl -ix 192.168.1.1:8080 https://google.com

HTTP/1.1 307 Temporary Redirect
Server: squid/4.3
Mime-Version: 1.0
Date: Tue, 16 Oct 2018 12:01:41 GMT
Content-Type: text/html;charset=utf-8
Content-Length: 0
Location: http://192.168.1.1/blocked.html
X-Squid-Error: 403 Access Denied
X-Cache: MISS from somehost
X-Cache-Lookup: NONE from somehost:8080
Connection: keep-alive


Notice that squid is indeed responding with code other than 200 or 407
for CONNECT and HTTPS request.

So what you said does not seem to be correct.

Case 2:
http_port 8080 ssl-bump ...
acl denyit src all
deny_info http://192.168.1.1/blocked.html denyit
http_access deny denyit

Post by Alex Rousskov
curl -ix 192.168.1.1:8080 https://google.com

HTTP/1.1 200 Connection established

curl: (60) SSL certificate problem: self signed certificate in
certificate chain
...



Case 1: Browser gives "Proxy connection refused" (or similar error).
Case 2: Browser gives "SSL certificate error".

Case 1 - Browser atleast makes it clear to end user that this is
something that proxy is not allowing.
Case 2 - End user would be clueless on why SSL error? He will never know
that its blocked by proxy.

To me case 1 is more appropriate response.

Please give a thought,

Thank you,

Amish.

[Alex Rousskov]

Post by Amish

Post by Alex Rousskov

Post by Amish
1) Browser sends CONNECT request
2) Check ACL
3) If denied, return with 307 (or 302)
4) If allowed, go ahead with tunneling / bumping as applicable

Unfortunately, that ideal sequence does not work well in practice
because popular browsers ignore CONNECT responses other than HTTP 200
and 407. As a consequence, if you want to redirect "secure" browser
traffic, Squid has to bump it first.

Thing is that squid behaves differently for 2 exactly same CONNECT
request with only difference being ssl-bump

Yes, Squid behaves differently when configured differently.

* My original response was specific to SslBump-enabled Squid ports.
Today, those configurations assume that the admin wants to bump CONNECTs
on errors (and has given Squid the certificate to enable such bumping).

* For SslBump-disabled ports (which is the default), Squid has no choice
but to deny/redirect the CONNECT request itself. Denied/redirected
CONNECT requests are mishandled by popular browsers -- Squid denial
errors are not shown to the user, and redirects are not followed.

Please note that the difference is not in matching ssl_bump actions, but
in whether the corresponding http_port was configured to use SslBump. In
the former case, whether the ssl_bump rules are checked depends on the
SslBump step where the CONNECT request is denied/redirected. In the
second/default case, ssl_bump rules are never checked.


If you prefer non-SslBump behavior, you should use it, of course! Some
admins find that browser-generated errors are insufficiently detailed
and/or produce more support queries than Squid-generated errors. YMMV.

If you want to change SslBump behavior when denying or redirecting
CONNECT requests, please make a specific proposal, keeping in mind that
many existing Squid deployments depend on Squid error pages being
displayed to the user (and/or on Squid redirects followed). Your
proposal will need to either convince folks that the existing behavior
should change or add options to optionally enable some new behavior.

Alex.

[Amish]

Post by Alex Rousskov

Post by Amish
Thing is that squid behaves differently for 2 exactly same CONNECT
request with only difference being ssl-bump

Yes, Squid behaves differently when configured differently.
* My original response was specific to SslBump-enabled Squid ports.
Today, those configurations assume that the admin wants to bump CONNECTs
on errors (and has given Squid the certificate to enable such bumping).
* For SslBump-disabled ports (which is the default), Squid has no choice
but to deny/redirect the CONNECT request itself. Denied/redirected
CONNECT requests are mishandled by popular browsers -- Squid denial
errors are not shown to the user, and redirects are not followed.
Please note that the difference is not in matching ssl_bump actions, but
in whether the corresponding http_port was configured to use SslBump. In
the former case, whether the ssl_bump rules are checked depends on the
SslBump step where the CONNECT request is denied/redirected. In the
second/default case, ssl_bump rules are never checked.

So if I have following config:

http_port 8080 ssl-bump ...
acl denyit src all
deny_info http://192.168.1.1/blocked.html denyit
http_access deny denyit
ssl_bump splice all

i.e. ssl-bump enabled on port but splice everything. (test case)

In this case one would expect that squid would not bump the connection
and return with 307 instead of 200.

But since it already sent 200 Connection Established - there is no
returning back.

Post by Alex Rousskov
If you prefer non-SslBump behavior, you should use it, of course! Some
admins find that browser-generated errors are insufficiently detailed
and/or produce more support queries than Squid-generated errors. YMMV.
If you want to change SslBump behavior when denying or redirecting
CONNECT requests, please make a specific proposal, keeping in mind that
many existing Squid deployments depend on Squid error pages being
displayed to the user (and/or on Squid redirects followed). Your
proposal will need to either convince folks that the existing behavior
should change or add options to optionally enable some new behavior.

My proposal for would be to add "-n" (nobump) option to deny_info.

If -n is specified then squid will send 307 directly instead of 200.

Case 1)
deny_info http://192.168.1.1/blocked.html denyit

Return with 200 and bump it (existing behaviour)

Case 2)
deny_info 3xx:http://192.168.1.1/blocked.html denyit

Return with 200 and bump it (existing behaviour)

Case 3)
deny_info -n http://192.168.1.1/blocked.html denyit

Return with 307 Temporary Redirect and Location: header

Case 4)
deny_info -n 302:http://192.168.1.1/blocked.html denyit

Return with 302 Found and Location: header.

Case 1 and 2 above applicable only for sslbump cases.

For non-sslbump it already behaves as 3) and 4) above.


This would not change anything for existing users who want existing
behaviour.

But allow people like me to *NOT* bump connection when deny_info is
activated.

Please give a thought,

Thank you,

Amish

[Amos Jeffries]

Post by Amish
My proposal for would be to add "-n" (nobump) option to deny_info.
If -n is specified then squid will send 307 directly instead of 200.
Case 1)
deny_info http://192.168.1.1/blocked.html denyit
Return with 200 and bump it (existing behaviour)
Case 2)
deny_info 3xx:http://192.168.1.1/blocked.html denyit
Return with 200 and bump it (existing behaviour)
Case 3)
deny_info -n http://192.168.1.1/blocked.html denyit
Return with 307 Temporary Redirect and Location: header
Case 4)
deny_info -n 302:http://192.168.1.1/blocked.html denyit
Return with 302 Found and Location: header.
Case 1 and 2 above applicable only for sslbump cases.
For non-sslbump it already behaves as 3) and 4) above.
This would not change anything for existing users who want existing
behaviour.
But allow people like me to *NOT* bump connection when deny_info is
activated.

IMO the deny_info is very much the wrong place to be making such
decisions. Its purpose is to supply the *content* of the denial message
itself. Nothing about how that message gets delivered.

If anything this would be an additional ssl-bump option on the port line
to say that traffic is not really being ssl-bump'ed despite the presence
of the ssl-bump setting.

So think about that - why bother putting "ssl-bump" on the port in the
first place if the behaviour that option enables is not wanted to ever
happen?

If your purpose is simply to convert port 443 traffic into HTTP CONNECT
for upstream software to receive there are other far simpler and more
efficient software to be using for that. httptunnel being the popular one.

Amos

[Amish]

This post might be inappropriate. Click to display it.

[Amos Jeffries]

Post by Amish

Post by Amos Jeffries

Post by Amish
My proposal for would be to add "-n" (nobump) option to deny_info.
If -n is specified then squid will send 307 directly instead of 200.
Case 1)
deny_info http://192.168.1.1/blocked.html denyit
Return with 200 and bump it (existing behaviour)
Case 2)
deny_info 3xx:http://192.168.1.1/blocked.html denyit
Return with 200 and bump it (existing behaviour)
Case 3)
deny_info -n http://192.168.1.1/blocked.html denyit
Return with 307 Temporary Redirect and Location: header
Case 4)
deny_info -n 302:http://192.168.1.1/blocked.html denyit
Return with 302 Found and Location: header.
Case 1 and 2 above applicable only for sslbump cases.
For non-sslbump it already behaves as 3) and 4) above.
This would not change anything for existing users who want existing
behaviour.
But allow people like me to *NOT* bump connection when deny_info is
activated.

IMO the deny_info is very much the wrong place to be making such
decisions. Its purpose is to supply the *content* of the denial message
itself. Nothing about how that message gets delivered.
If anything this would be an additional ssl-bump option on the port line
to say that traffic is not really being ssl-bump'ed despite the presence
of the ssl-bump setting.

If its additional ssl-bump option then it would become a global thing.
If its on deny_info then for some ACLs I can select to bump and for
others not. (i.e. gives finer control)

Post by Amos Jeffries
So think about that - why bother putting "ssl-bump" on the port in the
first place if the behaviour that option enables is not wanted to ever
happen?

I need SSL bump because I am bumping few domains and splicing rest.

... and you have a third case:

Replace TLS ServerHello with a plain-text HTTP 3xx response.

Your intended redirect of the CONNECT tunnel includes traffic where the
client has already sent the 0-RTT TLS clientHello along with the CONNECT.

Post by Amish
But for blocked domains I prefer browser to show "Proxy refused the
connection" instead of SSL error.

When the proxy is *actually* refusing (403) the connection, that makes
sense. All the other non-200 status responses, including your 3xx
redirect also show that error page generated by the Browser.

Bumping is forced on us by that Browser behaviour. Without the bump your
redirect response has zero chance of working as you obviously want it
to. If you were happy with the Browsers error page you would not be
trying to redirect the CONNECT.

Post by Amish
Also browsers not respecting code other than 200 or 407 is actually a
browser bug. Hopefully corrected in future someday.

We call it a bug they call it intentionally designed behaviour.

Browsers used to support non-200 responses. That support has actively
and systematically been removed.

Post by Amish
May be browsers will start supporting 302/307 Location header someday,
for CONNECT requests too?

They used to support 301 until early 2010's. Then it was replaced with
that blanket "Proxy is refusing connections" error message on grounds
that MITM proxies were using 301 to display content to users.

Post by Amish
(ofcourse I am not RFC expert so I may be completely wrong to interpret it)

These issues have nothing to do with the RFCs (which mandate proxy
support), and everything to do with Browser folks interests and design
choices being different from other folks needs and interests.

Amos

[Alex Rousskov]

Post by Amish
http_port 8080 ssl-bump ...
http_access deny ...
ssl_bump splice all
In this case one would expect that squid would not bump the connection
and return with 307 instead of 200.

FWIW, I do not think "one would expect" can be the driving argument for
designing SslBump error handling:

* Most humans do not "expect" errors at all -- it is natural for humans
to _ignore_ errors when forming initial expectations. This is why error
handling is so difficult and/or error-prone in virtually all areas of
human endeavor.

* Most likely, it is impossible to make SslBump work correctly _and_
match the first-glance expectations of the majority of admins. One has
to either study the feature documentation (to understand how the feature
works) or use working snippets certified by someone they trust (and can
get support from). If Squid documentation is lacking, there are ways to
fix that, of course.

For example, explicit configuration of error handling via ssl_bump rules
have been attempted in the past and rejected because it makes writing
correct rules very difficult and makes the rules themselves unreadable.

Post by Amish
But since it already sent 200 Connection Established - there is no
returning back.

As I had tried to explain earlier, the steps are not done in the order
you continue to imply. First, Squid discovers the error and only then it
purposefully replies with "200 Connection Established" (to be able to
deliver the error message later, when the first bumped request comes).

The described (and implemented) behavior is what many of the admins
requesting and sponsoring SslBump improvements wanted based on their
deployment experience. Your needs may be different, of course, and the
Squid Project can often satisfy conflicting needs. However, please come
to this discussion not from the "I propose to fix this obvious Squid
bug" point of view, but from the "I propose to satisfy a new use case
while still addressing other reasonable admin needs" point of view.

Post by Amish
My proposal for would be to add "-n" (nobump) option to deny_info.

Whether forwarding errors should lead to bumped connections should not
be determined by deny_info because deny_info does not apply to many
errors. We need a mechanism that can be applied to all errors.

And again, please keep in mind that Squid decides to bump the connection
when it decides to block CONNECT. That happens _before_ Squid looks at
deny_info. Deny_info is an HTTP error adaptation mechanism. It is
orthogonal to whether the error is delivered securely to the user or
dumped on the ground by the browser.

Post by Amish
Also browsers not respecting code other than 200 or 407 is actually a
browser bug. Hopefully corrected in future someday.

Very true, but based on my interpretation of browser makers' feedback on
the HTTP WG mailing list, I doubt that will happen in the foreseeable
future: Adding a proxy "security context" (in addition to the existing
"insecure" and "origin" contexts) is not a priority for them, especially
since it is only required to accommodate the needs of hated proxies.


HTH,

Alex.

[Amish]

Post by Alex Rousskov
Very true, but based on my interpretation of browser makers' feedback on
the HTTP WG mailing list, I doubt that will happen in the foreseeable
future: Adding a proxy "security context" (in addition to the existing
"insecure" and "origin" contexts) is not a priority for them, especially
since it is only required to accommodate the needs of hated proxies.

While I slightly disagree with you for certain things but
I also understand the reasons that you have explained behind it.

So I will leave it at that.

Thank you very much for elaborate replies.

Amish.