[squid-users] URL Regex ACLs Don't Evaluate After Bumping

[Shane Poage]

Hello,

I have a proxy configured to bump all traffic in order to do traffic filtering to a target server (Artifactory, in my case) from a particular environment. The proxy needs to be able to allow or permit traffic based on the path part of the URL in order to only allow access to a certain set of Artifactory repositories. I have my proxy configured to bump all traffic so that the urlpath_regex ACL can be applied, but it appears to not have any effect post-bump. I have dug through the debug logging and it evaluates the ACL before the bump occurs, but not after. I know it's bumping and decrypting the request properly because the access logs shows the information I need, but it's not actually applying the ACL.

Can anybody provide recommendations as to what I might be doing wrong? My (highly simplified, to remove possible points of failure) squid.conf is provided below, as well as the debug logs from a couple of sample requests.

Thanks,
-Shane

---- squid.conf ----------------------------
# enable ACL debug logging
debug_options 28,3

# Configure a man-in-the-middle sslbumping strategy
http_port 3128 ssl-bump \
cert=/etc/squid/ssl_certs/artifactory_mitm_ca.pem \
generate-host-certificates=on \
dynamic_cert_mem_cache_size=4MB

sslcrtd_program /usr/lib64/squid/ssl_crtd -s /var/lib/ssl_db -M 4MB

ssl_bump bump all

# Additional custom ACLs
acl artifactory_repo_filter urlpath_regex ^/artifactory
http_access allow artifactory_repo_filter

# And finally deny all other access to this proxy
http_access deny all


---- access.log -----------------------------
1541104544.428 0 <CLIENT_IP> TCP_DENIED/200 0 CONNECT google.com:443 - HIER_NONE/- -
1541104544.433 0 <CLIENT_IP> TAG_NONE/403 3900 GET https://google.com/ - HIER_NONE/- text/html
1541104555.408 0 <CLIENT_IP> TCP_DENIED/200 0 CONNECT <ARTIFACTORY_SERVER_DOMAIN>:8443 - HIER_NONE/- -
1541104555.411 0 <CLIENT_IP> TAG_NONE/403 3975 GET https://<ARTIFACTORY_SERVER_DOMAIN>:8443/artifactory/path/matching/the_request.jar - HIER_NONE/- text/html


---- cache.log -----------------------------
2018/11/01 20:35:44.427 kid1| 28,3| Checklist.cc(70) preCheck: 0x55d5742e4918 checking slow rules
2018/11/01 20:35:44.427 kid1| 28,3| Acl.cc(158) matches: checked: artifactory_repo_filter = -1
2018/11/01 20:35:44.427 kid1| 28,3| Acl.cc(158) matches: checked: http_access#1 = 0
2018/11/01 20:35:44.427 kid1| 28,3| Ip.cc(539) match: aclIpMatchIp: '<CLIENT_IP>:41736' found
2018/11/01 20:35:44.428 kid1| 28,3| Acl.cc(158) matches: checked: all = 1
2018/11/01 20:35:44.428 kid1| 28,3| Acl.cc(158) matches: checked: http_access#2 = 1
2018/11/01 20:35:44.428 kid1| 28,3| Acl.cc(158) matches: checked: http_access = 1
2018/11/01 20:35:44.428 kid1| 28,3| Checklist.cc(63) markFinished: 0x55d5742e4918 answer DENIED for match
2018/11/01 20:35:44.428 kid1| 28,3| Checklist.cc(163) checkCallback: ACLChecklist::checkCallback: 0x55d5742e4918 answer=DENIED
2018/11/01 20:35:44.428 kid1| 28,3| Checklist.cc(70) preCheck: 0x55d5742e5378 checking slow rules
2018/11/01 20:35:44.428 kid1| 28,3| Ip.cc(539) match: aclIpMatchIp: '<CLIENT_IP>:41736' found
2018/11/01 20:35:44.428 kid1| 28,3| Acl.cc(158) matches: checked: all = 1
2018/11/01 20:35:44.428 kid1| 28,3| Acl.cc(158) matches: checked: (ssl_bump rule) = 1
2018/11/01 20:35:44.428 kid1| 28,3| Acl.cc(158) matches: checked: (ssl_bump rules) = 1
2018/11/01 20:35:44.428 kid1| 28,3| Checklist.cc(63) markFinished: 0x55d5742e5378 answer ALLOWED for match
2018/11/01 20:35:44.428 kid1| 28,3| Checklist.cc(163) checkCallback: ACLChecklist::checkCallback: 0x55d5742e5378 answer=ALLOWED
2018/11/01 20:35:44.428 kid1| 28,3| Checklist.cc(70) preCheck: 0x7ffebd1f0df0 checking fast ACLs
2018/11/01 20:35:44.428 kid1| 28,3| Acl.cc(158) matches: checked: (access_log daemon:/var/log/squid/access.log line) = 1
2018/11/01 20:35:44.428 kid1| 28,3| Acl.cc(158) matches: checked: access_log daemon:/var/log/squid/access.log = 1
2018/11/01 20:35:44.428 kid1| 28,3| Checklist.cc(63) markFinished: 0x7ffebd1f0df0 answer ALLOWED for match
2018/11/01 20:35:44.433 kid1| 28,3| Checklist.cc(70) preCheck: 0x7ffebd1f0ec0 checking fast ACLs
2018/11/01 20:35:44.433 kid1| 28,3| Acl.cc(158) matches: checked: (access_log daemon:/var/log/squid/access.log line) = 1
2018/11/01 20:35:44.433 kid1| 28,3| Acl.cc(158) matches: checked: access_log daemon:/var/log/squid/access.log = 1
2018/11/01 20:35:44.433 kid1| 28,3| Checklist.cc(63) markFinished: 0x7ffebd1f0ec0 answer ALLOWED for match
2018/11/01 20:35:55.408 kid1| 28,3| Checklist.cc(70) preCheck: 0x55d5742e4918 checking slow rules
2018/11/01 20:35:55.408 kid1| 28,3| Acl.cc(158) matches: checked: artifactory_repo_filter = -1
2018/11/01 20:35:55.408 kid1| 28,3| Acl.cc(158) matches: checked: http_access#1 = 0
2018/11/01 20:35:55.408 kid1| 28,3| Ip.cc(539) match: aclIpMatchIp: '<CLIENT_IP>:41738' found
2018/11/01 20:35:55.408 kid1| 28,3| Acl.cc(158) matches: checked: all = 1
2018/11/01 20:35:55.408 kid1| 28,3| Acl.cc(158) matches: checked: http_access#2 = 1
2018/11/01 20:35:55.408 kid1| 28,3| Acl.cc(158) matches: checked: http_access = 1
2018/11/01 20:35:55.408 kid1| 28,3| Checklist.cc(63) markFinished: 0x55d5742e4918 answer DENIED for match
2018/11/01 20:35:55.408 kid1| 28,3| Checklist.cc(163) checkCallback: ACLChecklist::checkCallback: 0x55d5742e4918 answer=DENIED
2018/11/01 20:35:55.408 kid1| 28,3| Checklist.cc(70) preCheck: 0x55d5742e5378 checking slow rules
2018/11/01 20:35:55.408 kid1| 28,3| Ip.cc(539) match: aclIpMatchIp: '<CLIENT_IP>:41738' found
2018/11/01 20:35:55.408 kid1| 28,3| Acl.cc(158) matches: checked: all = 1
2018/11/01 20:35:55.408 kid1| 28,3| Acl.cc(158) matches: checked: (ssl_bump rule) = 1
2018/11/01 20:35:55.408 kid1| 28,3| Acl.cc(158) matches: checked: (ssl_bump rules) = 1
2018/11/01 20:35:55.408 kid1| 28,3| Checklist.cc(63) markFinished: 0x55d5742e5378 answer ALLOWED for match
2018/11/01 20:35:55.408 kid1| 28,3| Checklist.cc(163) checkCallback: ACLChecklist::checkCallback: 0x55d5742e5378 answer=ALLOWED
2018/11/01 20:35:55.408 kid1| 28,3| Checklist.cc(70) preCheck: 0x7ffebd1f0df0 checking fast ACLs
2018/11/01 20:35:55.408 kid1| 28,3| Acl.cc(158) matches: checked: (access_log daemon:/var/log/squid/access.log line) = 1
2018/11/01 20:35:55.408 kid1| 28,3| Acl.cc(158) matches: checked: access_log daemon:/var/log/squid/access.log = 1
2018/11/01 20:35:55.408 kid1| 28,3| Checklist.cc(63) markFinished: 0x7ffebd1f0df0 answer ALLOWED for match
2018/11/01 20:35:55.411 kid1| 28,3| Checklist.cc(70) preCheck: 0x7ffebd1f0ec0 checking fast ACLs
2018/11/01 20:35:55.411 kid1| 28,3| Acl.cc(158) matches: checked: (access_log daemon:/var/log/squid/access.log line) = 1
2018/11/01 20:35:55.411 kid1| 28,3| Acl.cc(158) matches: checked: access_log daemon:/var/log/squid/access.log = 1
2018/11/01 20:35:55.411 kid1| 28,3| Checklist.cc(63) markFinished: 0x7ffebd1f0ec0 answer ALLOWED for match

[Alex Rousskov]

Post by Shane Poage
I have my proxy configured to bump all traffic so that the
urlpath_regex ACL can be applied, but it appears to not have any
effect post-bump.

Your proxy will deny any first post-bump request and close the tunnel
because you deny all CONNECT requests that initiate tunnels. CONNECT
requests do not have a URL path so they will never match your
"http_access allow" rule.

When a CONNECT request is denied by a bumping Squid, that Squid bumps
the tunnel and then denies the very first bumped request on that tunnel,
whatever that request is. This delayed error return is done to deliver
the "access denied" error page to the client -- browsers ignore CONNECT
error responses.

Post by Shane Poage
http_port 3128 ssl-bump \
cert=/etc/squid/ssl_certs/artifactory_mitm_ca.pem \
generate-host-certificates=on \
dynamic_cert_mem_cache_size=4MB
ssl_bump bump all
acl artifactory_repo_filter urlpath_regex ^/artifactory
http_access allow artifactory_repo_filter
# And finally deny all other access to this proxy
http_access deny all

Insert an http_access rule to allow all safe CONNECT requests before you
deny everything else. IIRC, squid.conf.default has an example of how to
do that.

Alex.

[Shane Poage]

If I understand you correctly, my original squid.conf had an ACL directive corresponding to the functionality in question:

acl CONNECT method CONNECT

Regardless, I added that back to my config and re-tested both with it inline and stand-alone. My results as follows:

http_access allow CONNECT artifactory_repo_filter
This resulted in no traffic being passed, just like without the
CONNECT ACL present. I assume this is what you were telling
me to do because it's what made the most sense based off
of your description of the problem, but it did not have the
desired effect.

http_access allow CONNECT
http_access allow artifactory_repo_filter
This resulted in all SSL traffic being permitted and passed through
the proxy. This makes sense to me because the allow CONNECT
would whitelist all CONNECT traffic, which is what all SSL traffic is
by my understanding. I didn't think this is what you meant, but I
tried it anyway to be sure.

Have I misunderstood you somehow, or is it not behaving as expected?

Post by Shane Poage
I have my proxy configured to bump all traffic so that the
urlpath_regex ACL can be applied, but it appears to not have any
effect post-bump.

Your proxy will deny any first post-bump request and close the tunnel
because you deny all CONNECT requests that initiate tunnels. CONNECT
requests do not have a URL path so they will never match your
"http_access allow" rule.

When a CONNECT request is denied by a bumping Squid, that Squid bumps
the tunnel and then denies the very first bumped request on that tunnel,
whatever that request is. This delayed error return is done to deliver
the "access denied" error page to the client -- browsers ignore CONNECT
error responses.

Post by Shane Poage
http_port 3128 ssl-bump \
cert=/etc/squid/ssl_certs/artifactory_mitm_ca.pem \
generate-host-certificates=on \
dynamic_cert_mem_cache_size=4MB
ssl_bump bump all
acl artifactory_repo_filter urlpath_regex ^/artifactory
http_access allow artifactory_repo_filter
# And finally deny all other access to this proxy
http_access deny all

Insert an http_access rule to allow all safe CONNECT requests before you
deny everything else. IIRC, squid.conf.default has an example of how to
do that.

Alex.
_______________________________________________
squid-users mailing list
squid-***@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[Alex Rousskov]

Post by Shane Poage
my original squid.conf had an ACL directive corresponding to the
acl CONNECT method CONNECT

That CONNECT ACL declaration is OK, but what matters is how you _use_
that declared ACL. The configuration you posted earlier did not use it
at all.

Post by Shane Poage
http_access allow CONNECT artifactory_repo_filter

This "CONNECT and artifactory_repo_filter" rule does not make sense. As
I said earlier, your artifactory_repo_filter cannot match a CONNECT
request. Thus, you are joining two conditions that can never be
satisfied for the same request. For any request, you will get either
(false and true) or (true and false), which is, of course, always false.

Post by Shane Poage
http_access allow CONNECT
http_access allow artifactory_repo_filter

This "CONNECT or artifactory_repo_filter" combination makes sense, but
the first part is dangerous -- you probably should not allow CONNECT
request to arbitrary port numbers. If you look at how CONNECT requests
are handled in squid.conf.default, then you will probably come up with
something like this:

http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow CONNECT
http_access allow artifactory_repo_filter
http_access deny all

or a bit shorter but arguably less safe (long-term) version:

http_access deny !Safe_ports
http_access allow CONNECT SSL_ports
http_access allow artifactory_repo_filter
http_access deny all

or an even shorter but arguably even less safe (long-term) version:

http_access allow SSL_ports CONNECT
http_access allow Safe_ports artifactory_repo_filter
http_access deny all


N.B. The above configurations allow plain text traffic matching
artifactory_repo_filter. I do not know whether that is what you want.

Post by Shane Poage
This resulted in all SSL traffic being permitted and passed through
the proxy.

If you are still bumping all SSL traffic, and your
artifactory_repo_filter ACL is working, then all of the above reasonable
configurations should still block bumped GET requests that match
artifactory_repo_filter.

Post by Shane Poage
This makes sense to me because the allow CONNECT
would whitelist all CONNECT traffic, which is what all SSL traffic is
by my understanding.

Your understanding is incorrect. Since you are bumping all CONNECT
tunnels, your http_access traffic consists of:

* CONNECT requests
* decrypted requests (e.g., GET) inside bumped CONNECT tunnels
* plain requests (e.g., GET) outside CONNECT tunnels


HTH,

Alex.

Post by Shane Poage

Post by Shane Poage
I have my proxy configured to bump all traffic so that the
urlpath_regex ACL can be applied, but it appears to not have any
effect post-bump.

Your proxy will deny any first post-bump request and close the tunnel
because you deny all CONNECT requests that initiate tunnels. CONNECT
requests do not have a URL path so they will never match your
"http_access allow" rule.
When a CONNECT request is denied by a bumping Squid, that Squid bumps
the tunnel and then denies the very first bumped request on that tunnel,
whatever that request is. This delayed error return is done to deliver
the "access denied" error page to the client -- browsers ignore CONNECT
error responses.

Post by Shane Poage
http_port 3128 ssl-bump \
cert=/etc/squid/ssl_certs/artifactory_mitm_ca.pem \
generate-host-certificates=on \
dynamic_cert_mem_cache_size=4MB
ssl_bump bump all
acl artifactory_repo_filter urlpath_regex ^/artifactory
http_access allow artifactory_repo_filter
# And finally deny all other access to this proxy
http_access deny all

Insert an http_access rule to allow all safe CONNECT requests before you
deny everything else. IIRC, squid.conf.default has an example of how to
do that.
Alex.
_______________________________________________
squid-users mailing list
http://lists.squid-cache.org/listinfo/squid-users

[Shane Poage]

Alex,

Thanks for all your help so far - I really appreciate it. However, I'm not sure
we're on the same page for some reason because as I said previously, the
suggestions you have made do not appear to be having the effect I'm looking
for. Let me back up a bit and clarify a few things to make sure I am
understanding properly, because there seem to be a few things that I have
thus far fundamentally misunderstood.

My understanding of how SSL bumping works is as follows, based on what
I have read in all of the documentation I could possibly find:

1. A CONNECT request comes in to the proxy from a client.
2. The proxy does access control on that CONNECT request and decides
whether or not to permit the tunnel. For the sake of moving on, let's
assume that it decides to allow.
3. The proxy then begins processing the rules for SSL bumping. Again, for the
sake of simplicity, we'll say that all traffic is getting bumped.
4. The proxy begins bumping traffic and individually processing the requests
for that session.
5. Each request within the session has the http_access rules applied to them
as if they were normal HTTP requests made through the proxy, until the
session is terminated.

Is this understanding correct?

Thanks again,
-Shane

Post by Shane Poage
my original squid.conf had an ACL directive corresponding to the
acl CONNECT method CONNECT

That CONNECT ACL declaration is OK, but what matters is how you _use_
that declared ACL. The configuration you posted earlier did not use it
at all.

Post by Shane Poage
http_access allow CONNECT artifactory_repo_filter

This "CONNECT and artifactory_repo_filter" rule does not make sense. As
I said earlier, your artifactory_repo_filter cannot match a CONNECT
request. Thus, you are joining two conditions that can never be
satisfied for the same request. For any request, you will get either
(false and true) or (true and false), which is, of course, always false.

Post by Shane Poage
http_access allow CONNECT
http_access allow artifactory_repo_filter

This "CONNECT or artifactory_repo_filter" combination makes sense, but
the first part is dangerous -- you probably should not allow CONNECT
request to arbitrary port numbers. If you look at how CONNECT requests
are handled in squid.conf.default, then you will probably come up with
something like this:

http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow CONNECT
http_access allow artifactory_repo_filter
http_access deny all

or a bit shorter but arguably less safe (long-term) version:

http_access deny !Safe_ports
http_access allow CONNECT SSL_ports
http_access allow artifactory_repo_filter
http_access deny all

or an even shorter but arguably even less safe (long-term) version:

http_access allow SSL_ports CONNECT
http_access allow Safe_ports artifactory_repo_filter
http_access deny all


N.B. The above configurations allow plain text traffic matching
artifactory_repo_filter. I do not know whether that is what you want.

Post by Shane Poage
This resulted in all SSL traffic being permitted and passed through
the proxy.

If you are still bumping all SSL traffic, and your
artifactory_repo_filter ACL is working, then all of the above reasonable
configurations should still block bumped GET requests that match
artifactory_repo_filter.

Post by Shane Poage
This makes sense to me because the allow CONNECT
would whitelist all CONNECT traffic, which is what all SSL traffic is
by my understanding.

Your understanding is incorrect. Since you are bumping all CONNECT
tunnels, your http_access traffic consists of:

* CONNECT requests
* decrypted requests (e.g., GET) inside bumped CONNECT tunnels
* plain requests (e.g., GET) outside CONNECT tunnels


HTH,

Alex.

Post by Shane Poage

Post by Shane Poage
I have my proxy configured to bump all traffic so that the
urlpath_regex ACL can be applied, but it appears to not have any
effect post-bump.

Your proxy will deny any first post-bump request and close the tunnel
because you deny all CONNECT requests that initiate tunnels. CONNECT
requests do not have a URL path so they will never match your
"http_access allow" rule.
When a CONNECT request is denied by a bumping Squid, that Squid bumps
the tunnel and then denies the very first bumped request on that tunnel,
whatever that request is. This delayed error return is done to deliver
the "access denied" error page to the client -- browsers ignore CONNECT
error responses.

Post by Shane Poage
http_port 3128 ssl-bump \
cert=/etc/squid/ssl_certs/artifactory_mitm_ca.pem \
generate-host-certificates=on \
dynamic_cert_mem_cache_size=4MB
ssl_bump bump all
acl artifactory_repo_filter urlpath_regex ^/artifactory
http_access allow artifactory_repo_filter
# And finally deny all other access to this proxy
http_access deny all

Insert an http_access rule to allow all safe CONNECT requests before you
deny everything else. IIRC, squid.conf.default has an example of how to
do that.
Alex.
_______________________________________________
squid-users mailing list
http://lists.squid-cache.org/listinfo/squid-users

[Alex Rousskov]

Post by Shane Poage
My understanding of how SSL bumping works is as follows, based on what

To simplify, I will interpret the statements below in SslBump context.
That is, I will interpret them as they are applied to traffic received
on an http_port with ssl-bump configured.

I will also assume a bug-free implementation. In other words, the entire
text below is under the "Bugs notwithstanding..." precondition.

Post by Shane Poage
1. A CONNECT request comes in to the proxy from a client.
2. The proxy does access control on that CONNECT request and decides
whether or not to permit the tunnel. For the sake of moving on, let's
assume that it decides to allow.

Correct. This decision is a part of SslBump step1.

[ The decision to allow or deny CONNECT is done again during SslBump
step2 if a peek or stare action matches during step1. IIRC, those
additional checks do not happen in your configuration because your
ssl_bump rules match a bump action at step1. Thus, this additional
caveat does not apply to you. ]

Post by Shane Poage
3. The proxy then begins processing the rules for SSL bumping. Again, for the
sake of simplicity, we'll say that all traffic is getting bumped.

OK. This processing is a part of SslBump step1.

Post by Shane Poage
4. The proxy begins bumping traffic and individually processing the requests
for that session.

OK. This processing is a part of SslBump step1.

[ Again, there could be more SslBump steps in general, but, in your
specific case, the client connection is bumped here. ]

Post by Shane Poage
5. Each request within the session has the http_access rules applied to them
as if they were normal HTTP requests made through the proxy, until the
session is terminated.

These bumped requests have "https" scheme and some special properties
specific to bumped requests but, overall, they are indeed processed as
usual, including application of http_access rules to each of the bumped
request.


To make progress, I recommend identifying a specific Squid action that
contradicts either your understanding of how things should work or my
remarks and then posting the relevant details of that action along with
a clear explanation of where you see a contradiction.

Alex.

Post by Shane Poage

Post by Shane Poage
my original squid.conf had an ACL directive corresponding to the
acl CONNECT method CONNECT

That CONNECT ACL declaration is OK, but what matters is how you _use_
that declared ACL. The configuration you posted earlier did not use it
at all.

Post by Shane Poage
http_access allow CONNECT artifactory_repo_filter

This "CONNECT and artifactory_repo_filter" rule does not make sense. As
I said earlier, your artifactory_repo_filter cannot match a CONNECT
request. Thus, you are joining two conditions that can never be
satisfied for the same request. For any request, you will get either
(false and true) or (true and false), which is, of course, always false.

Post by Shane Poage
http_access allow CONNECT
http_access allow artifactory_repo_filter

This "CONNECT or artifactory_repo_filter" combination makes sense, but
the first part is dangerous -- you probably should not allow CONNECT
request to arbitrary port numbers. If you look at how CONNECT requests
are handled in squid.conf.default, then you will probably come up with
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow CONNECT
http_access allow artifactory_repo_filter
http_access deny all
http_access deny !Safe_ports
http_access allow CONNECT SSL_ports
http_access allow artifactory_repo_filter
http_access deny all
http_access allow SSL_ports CONNECT
http_access allow Safe_ports artifactory_repo_filter
http_access deny all
N.B. The above configurations allow plain text traffic matching
artifactory_repo_filter. I do not know whether that is what you want.

Post by Shane Poage
This resulted in all SSL traffic being permitted and passed through
the proxy.

If you are still bumping all SSL traffic, and your
artifactory_repo_filter ACL is working, then all of the above reasonable
configurations should still block bumped GET requests that match
artifactory_repo_filter.

Post by Shane Poage
This makes sense to me because the allow CONNECT
would whitelist all CONNECT traffic, which is what all SSL traffic is
by my understanding.

Your understanding is incorrect. Since you are bumping all CONNECT
* CONNECT requests
* decrypted requests (e.g., GET) inside bumped CONNECT tunnels
* plain requests (e.g., GET) outside CONNECT tunnels
HTH,
Alex.

Post by Shane Poage

Post by Shane Poage
I have my proxy configured to bump all traffic so that the
urlpath_regex ACL can be applied, but it appears to not have any
effect post-bump.

Your proxy will deny any first post-bump request and close the tunnel
because you deny all CONNECT requests that initiate tunnels. CONNECT
requests do not have a URL path so they will never match your
"http_access allow" rule.
When a CONNECT request is denied by a bumping Squid, that Squid bumps
the tunnel and then denies the very first bumped request on that tunnel,
whatever that request is. This delayed error return is done to deliver
the "access denied" error page to the client -- browsers ignore CONNECT
error responses.

Post by Shane Poage
http_port 3128 ssl-bump \
cert=/etc/squid/ssl_certs/artifactory_mitm_ca.pem \
generate-host-certificates=on \
dynamic_cert_mem_cache_size=4MB
ssl_bump bump all
acl artifactory_repo_filter urlpath_regex ^/artifactory
http_access allow artifactory_repo_filter
# And finally deny all other access to this proxy
http_access deny all

Insert an http_access rule to allow all safe CONNECT requests before you
deny everything else. IIRC, squid.conf.default has an example of how to
do that.
Alex.
_______________________________________________
squid-users mailing list
http://lists.squid-cache.org/listinfo/squid-users

[Shane Poage]

Alex,

Great, it sounds like I'm pretty much on-track in terms of understanding how bumping
works under the hood. Thanks for taking the time to step through that and confirm my
understanding.

So here's the problem: the initial solution you gave for my urlpath_regex ACL not
permitting the traffic I expected was that I needed to permit the initial CONNECT request.
That made sense, but did not appear to behave exactly as I expected. Adding the
'method CONNECT' ACL as an allow http_accept directive appears to let ALL HTTPS
traffic through the proxy and ignore the other ACLs that should apply to the subsequent
requests in the CONNECT tunnel. This doesn't make sense to me based on how I thought
I understood the way bumping functions.

So, here's the question: if I'm not misunderstanding how bumping works, have I uncovered
a bug in Squid? Either in the form of the CONNECT ACL not being applied as expected, or
in the form of the urlpath_regex not being applied... Or is there something additional that
I'm fundamentally missing?

Thanks for your continued assistance,
-Shane

On 11/2/18, 3:15 PM, "Alex Rousskov" <***@measurement-factory.com> wrote:

WARNING: This email originated outside of Entrust Datacard.
DO NOT CLICK links or attachments unless you trust the sender and know the content is safe.

Post by Shane Poage
My understanding of how SSL bumping works is as follows, based on what

To simplify, I will interpret the statements below in SslBump context.
That is, I will interpret them as they are applied to traffic received
on an http_port with ssl-bump configured.

I will also assume a bug-free implementation. In other words, the entire
text below is under the "Bugs notwithstanding..." precondition.

Post by Shane Poage
1. A CONNECT request comes in to the proxy from a client.
2. The proxy does access control on that CONNECT request and decides
whether or not to permit the tunnel. For the sake of moving on, let's
assume that it decides to allow.

Correct. This decision is a part of SslBump step1.

[ The decision to allow or deny CONNECT is done again during SslBump
step2 if a peek or stare action matches during step1. IIRC, those
additional checks do not happen in your configuration because your
ssl_bump rules match a bump action at step1. Thus, this additional
caveat does not apply to you. ]

Post by Shane Poage
3. The proxy then begins processing the rules for SSL bumping. Again, for the
sake of simplicity, we'll say that all traffic is getting bumped.

OK. This processing is a part of SslBump step1.

Post by Shane Poage
4. The proxy begins bumping traffic and individually processing the requests
for that session.

OK. This processing is a part of SslBump step1.

[ Again, there could be more SslBump steps in general, but, in your
specific case, the client connection is bumped here. ]

Post by Shane Poage
5. Each request within the session has the http_access rules applied to them
as if they were normal HTTP requests made through the proxy, until the
session is terminated.

These bumped requests have "https" scheme and some special properties
specific to bumped requests but, overall, they are indeed processed as
usual, including application of http_access rules to each of the bumped
request.


To make progress, I recommend identifying a specific Squid action that
contradicts either your understanding of how things should work or my
remarks and then posting the relevant details of that action along with
a clear explanation of where you see a contradiction.

Alex.

Post by Shane Poage

Post by Shane Poage
my original squid.conf had an ACL directive corresponding to the
acl CONNECT method CONNECT

That CONNECT ACL declaration is OK, but what matters is how you _use_
that declared ACL. The configuration you posted earlier did not use it
at all.

Post by Shane Poage
http_access allow CONNECT artifactory_repo_filter

This "CONNECT and artifactory_repo_filter" rule does not make sense. As
I said earlier, your artifactory_repo_filter cannot match a CONNECT
request. Thus, you are joining two conditions that can never be
satisfied for the same request. For any request, you will get either
(false and true) or (true and false), which is, of course, always false.

Post by Shane Poage
http_access allow CONNECT
http_access allow artifactory_repo_filter

This "CONNECT or artifactory_repo_filter" combination makes sense, but
the first part is dangerous -- you probably should not allow CONNECT
request to arbitrary port numbers. If you look at how CONNECT requests
are handled in squid.conf.default, then you will probably come up with
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow CONNECT
http_access allow artifactory_repo_filter
http_access deny all
http_access deny !Safe_ports
http_access allow CONNECT SSL_ports
http_access allow artifactory_repo_filter
http_access deny all
http_access allow SSL_ports CONNECT
http_access allow Safe_ports artifactory_repo_filter
http_access deny all
N.B. The above configurations allow plain text traffic matching
artifactory_repo_filter. I do not know whether that is what you want.

Post by Shane Poage
This resulted in all SSL traffic being permitted and passed through
the proxy.

If you are still bumping all SSL traffic, and your
artifactory_repo_filter ACL is working, then all of the above reasonable
configurations should still block bumped GET requests that match
artifactory_repo_filter.

Post by Shane Poage
This makes sense to me because the allow CONNECT
would whitelist all CONNECT traffic, which is what all SSL traffic is
by my understanding.

Your understanding is incorrect. Since you are bumping all CONNECT
* CONNECT requests
* decrypted requests (e.g., GET) inside bumped CONNECT tunnels
* plain requests (e.g., GET) outside CONNECT tunnels
HTH,
Alex.

Post by Shane Poage

Post by Shane Poage
I have my proxy configured to bump all traffic so that the
urlpath_regex ACL can be applied, but it appears to not have any
effect post-bump.

Your proxy will deny any first post-bump request and close the tunnel
because you deny all CONNECT requests that initiate tunnels. CONNECT
requests do not have a URL path so they will never match your
"http_access allow" rule.
When a CONNECT request is denied by a bumping Squid, that Squid bumps
the tunnel and then denies the very first bumped request on that tunnel,
whatever that request is. This delayed error return is done to deliver
the "access denied" error page to the client -- browsers ignore CONNECT
error responses.

Post by Shane Poage
http_port 3128 ssl-bump \
cert=/etc/squid/ssl_certs/artifactory_mitm_ca.pem \
generate-host-certificates=on \
dynamic_cert_mem_cache_size=4MB
ssl_bump bump all
acl artifactory_repo_filter urlpath_regex ^/artifactory
http_access allow artifactory_repo_filter
# And finally deny all other access to this proxy
http_access deny all

Insert an http_access rule to allow all safe CONNECT requests before you
deny everything else. IIRC, squid.conf.default has an example of how to
do that.
Alex.
_______________________________________________
squid-users mailing list
http://lists.squid-cache.org/listinfo/squid-users

[Alex Rousskov]

if I'm not misunderstanding how bumping works, have I uncovered a bug
in Squid? Either in the form of the CONNECT ACL not being applied as
expected, or in the form of the urlpath_regex not being applied... Or
is there something additional that I'm fundamentally missing?

It is difficult for me to say why a correct configuration does not work
in your environment because you have not supplied any relevant details
(yet). You have supplied details for some other configurations IIRC, but
not the one you should be using.

As a starting point, I recommend posting your current/correct
configuration and the corresponding compressed ALL,3 cache.log while
reproducing a problem on an isolated Squid handling a single problematic
transaction. If possible, please use curl, wget, or another
single-request HTTP client to drive that transaction, not a browser.
Please also post the corresponding access.log lines.


Thank you,

Alex.

WARNING: This email originated outside of Entrust Datacard.
DO NOT CLICK links or attachments unless you trust the sender and know the content is safe.

Post by Shane Poage
My understanding of how SSL bumping works is as follows, based on what

To simplify, I will interpret the statements below in SslBump context.
That is, I will interpret them as they are applied to traffic received
on an http_port with ssl-bump configured.
I will also assume a bug-free implementation. In other words, the entire
text below is under the "Bugs notwithstanding..." precondition.

Post by Shane Poage
1. A CONNECT request comes in to the proxy from a client.
2. The proxy does access control on that CONNECT request and decides
whether or not to permit the tunnel. For the sake of moving on, let's
assume that it decides to allow.

Correct. This decision is a part of SslBump step1.
[ The decision to allow or deny CONNECT is done again during SslBump
step2 if a peek or stare action matches during step1. IIRC, those
additional checks do not happen in your configuration because your
ssl_bump rules match a bump action at step1. Thus, this additional
caveat does not apply to you. ]

Post by Shane Poage
3. The proxy then begins processing the rules for SSL bumping. Again, for the
sake of simplicity, we'll say that all traffic is getting bumped.

OK. This processing is a part of SslBump step1.

Post by Shane Poage
4. The proxy begins bumping traffic and individually processing the requests
for that session.

OK. This processing is a part of SslBump step1.
[ Again, there could be more SslBump steps in general, but, in your
specific case, the client connection is bumped here. ]

Post by Shane Poage
5. Each request within the session has the http_access rules applied to them
as if they were normal HTTP requests made through the proxy, until the
session is terminated.

These bumped requests have "https" scheme and some special properties
specific to bumped requests but, overall, they are indeed processed as
usual, including application of http_access rules to each of the bumped
request.
To make progress, I recommend identifying a specific Squid action that
contradicts either your understanding of how things should work or my
remarks and then posting the relevant details of that action along with
a clear explanation of where you see a contradiction.
Alex.

Post by Shane Poage

Post by Shane Poage
my original squid.conf had an ACL directive corresponding to the
acl CONNECT method CONNECT

That CONNECT ACL declaration is OK, but what matters is how you _use_
that declared ACL. The configuration you posted earlier did not use it
at all.

Post by Shane Poage
http_access allow CONNECT artifactory_repo_filter

This "CONNECT and artifactory_repo_filter" rule does not make sense. As
I said earlier, your artifactory_repo_filter cannot match a CONNECT
request. Thus, you are joining two conditions that can never be
satisfied for the same request. For any request, you will get either
(false and true) or (true and false), which is, of course, always false.

Post by Shane Poage
http_access allow CONNECT
http_access allow artifactory_repo_filter

This "CONNECT or artifactory_repo_filter" combination makes sense, but
the first part is dangerous -- you probably should not allow CONNECT
request to arbitrary port numbers. If you look at how CONNECT requests
are handled in squid.conf.default, then you will probably come up with
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow CONNECT
http_access allow artifactory_repo_filter
http_access deny all
http_access deny !Safe_ports
http_access allow CONNECT SSL_ports
http_access allow artifactory_repo_filter
http_access deny all
http_access allow SSL_ports CONNECT
http_access allow Safe_ports artifactory_repo_filter
http_access deny all
N.B. The above configurations allow plain text traffic matching
artifactory_repo_filter. I do not know whether that is what you want.

Post by Shane Poage
This resulted in all SSL traffic being permitted and passed through
the proxy.

If you are still bumping all SSL traffic, and your
artifactory_repo_filter ACL is working, then all of the above reasonable
configurations should still block bumped GET requests that match
artifactory_repo_filter.

Post by Shane Poage
This makes sense to me because the allow CONNECT
would whitelist all CONNECT traffic, which is what all SSL traffic is
by my understanding.

Your understanding is incorrect. Since you are bumping all CONNECT
* CONNECT requests
* decrypted requests (e.g., GET) inside bumped CONNECT tunnels
* plain requests (e.g., GET) outside CONNECT tunnels
HTH,
Alex.

Post by Shane Poage

Post by Shane Poage
I have my proxy configured to bump all traffic so that the
urlpath_regex ACL can be applied, but it appears to not have any
effect post-bump.

Your proxy will deny any first post-bump request and close the tunnel
because you deny all CONNECT requests that initiate tunnels. CONNECT
requests do not have a URL path so they will never match your
"http_access allow" rule.
When a CONNECT request is denied by a bumping Squid, that Squid bumps
the tunnel and then denies the very first bumped request on that tunnel,
whatever that request is. This delayed error return is done to deliver
the "access denied" error page to the client -- browsers ignore CONNECT
error responses.

Post by Shane Poage
http_port 3128 ssl-bump \
cert=/etc/squid/ssl_certs/artifactory_mitm_ca.pem \
generate-host-certificates=on \
dynamic_cert_mem_cache_size=4MB
ssl_bump bump all
acl artifactory_repo_filter urlpath_regex ^/artifactory
http_access allow artifactory_repo_filter
# And finally deny all other access to this proxy
http_access deny all

Insert an http_access rule to allow all safe CONNECT requests before you
deny everything else. IIRC, squid.conf.default has an example of how to
do that.
Alex.
_______________________________________________
squid-users mailing list
http://lists.squid-cache.org/listinfo/squid-users

[Shane Poage]

Alex,

I have attached a gzipped tarball that includes three items:
1. The squid.conf for the configuration that is not behaving as I expected
2. The access.log for two requests (which I will describe in a moment)
3. The cache.log for startup, those same two requests, and shutdown of the squid instance.

The two requests made through the proxy were made using wget on another machine configured to route traffic through the proxy:
1) https://www.google.com/ - Squid permitted the traffic through. According to my understanding, it should have denied it due to the urlpath_regex ACL in the config.
2) https://sottartifactory.corporate.datacard.com:8443/artifactory/some/path/to/artifact.jar - This one was allowed through, which is expected in this case.

Please let me know if there's any more information I can provide that would be helpful, and thanks again for sticking with me on this.
-Shane


On 11/5/18, 1:23 PM, "Alex Rousskov" <***@measurement-factory.com> wrote:

WARNING: This email originated outside of Entrust Datacard.
DO NOT CLICK links or attachments unless you trust the sender and know the content is safe.

if I'm not misunderstanding how bumping works, have I uncovered a bug
in Squid? Either in the form of the CONNECT ACL not being applied as
expected, or in the form of the urlpath_regex not being applied... Or
is there something additional that I'm fundamentally missing?

It is difficult for me to say why a correct configuration does not work
in your environment because you have not supplied any relevant details
(yet). You have supplied details for some other configurations IIRC, but
not the one you should be using.

As a starting point, I recommend posting your current/correct
configuration and the corresponding compressed ALL,3 cache.log while
reproducing a problem on an isolated Squid handling a single problematic
transaction. If possible, please use curl, wget, or another
single-request HTTP client to drive that transaction, not a browser.
Please also post the corresponding access.log lines.


Thank you,

Alex.

WARNING: This email originated outside of Entrust Datacard.
DO NOT CLICK links or attachments unless you trust the sender and know the content is safe.

Post by Shane Poage
My understanding of how SSL bumping works is as follows, based on what

To simplify, I will interpret the statements below in SslBump context.
That is, I will interpret them as they are applied to traffic received
on an http_port with ssl-bump configured.
I will also assume a bug-free implementation. In other words, the entire
text below is under the "Bugs notwithstanding..." precondition.

Post by Shane Poage
1. A CONNECT request comes in to the proxy from a client.
2. The proxy does access control on that CONNECT request and decides
whether or not to permit the tunnel. For the sake of moving on, let's
assume that it decides to allow.

Correct. This decision is a part of SslBump step1.
[ The decision to allow or deny CONNECT is done again during SslBump
step2 if a peek or stare action matches during step1. IIRC, those
additional checks do not happen in your configuration because your
ssl_bump rules match a bump action at step1. Thus, this additional
caveat does not apply to you. ]

Post by Shane Poage
3. The proxy then begins processing the rules for SSL bumping. Again, for the
sake of simplicity, we'll say that all traffic is getting bumped.

OK. This processing is a part of SslBump step1.

Post by Shane Poage
4. The proxy begins bumping traffic and individually processing the requests
for that session.

OK. This processing is a part of SslBump step1.
[ Again, there could be more SslBump steps in general, but, in your
specific case, the client connection is bumped here. ]

Post by Shane Poage
5. Each request within the session has the http_access rules applied to them
as if they were normal HTTP requests made through the proxy, until the
session is terminated.

These bumped requests have "https" scheme and some special properties
specific to bumped requests but, overall, they are indeed processed as
usual, including application of http_access rules to each of the bumped
request.
To make progress, I recommend identifying a specific Squid action that
contradicts either your understanding of how things should work or my
remarks and then posting the relevant details of that action along with
a clear explanation of where you see a contradiction.
Alex.

Post by Shane Poage

Post by Shane Poage
my original squid.conf had an ACL directive corresponding to the
acl CONNECT method CONNECT

That CONNECT ACL declaration is OK, but what matters is how you _use_
that declared ACL. The configuration you posted earlier did not use it
at all.

Post by Shane Poage
http_access allow CONNECT artifactory_repo_filter

This "CONNECT and artifactory_repo_filter" rule does not make sense. As
I said earlier, your artifactory_repo_filter cannot match a CONNECT
request. Thus, you are joining two conditions that can never be
satisfied for the same request. For any request, you will get either
(false and true) or (true and false), which is, of course, always false.

Post by Shane Poage
http_access allow CONNECT
http_access allow artifactory_repo_filter

This "CONNECT or artifactory_repo_filter" combination makes sense, but
the first part is dangerous -- you probably should not allow CONNECT
request to arbitrary port numbers. If you look at how CONNECT requests
are handled in squid.conf.default, then you will probably come up with
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow CONNECT
http_access allow artifactory_repo_filter
http_access deny all
http_access deny !Safe_ports
http_access allow CONNECT SSL_ports
http_access allow artifactory_repo_filter
http_access deny all
http_access allow SSL_ports CONNECT
http_access allow Safe_ports artifactory_repo_filter
http_access deny all
N.B. The above configurations allow plain text traffic matching
artifactory_repo_filter. I do not know whether that is what you want.

Post by Shane Poage
This resulted in all SSL traffic being permitted and passed through
the proxy.

If you are still bumping all SSL traffic, and your
artifactory_repo_filter ACL is working, then all of the above reasonable
configurations should still block bumped GET requests that match
artifactory_repo_filter.

Post by Shane Poage
This makes sense to me because the allow CONNECT
would whitelist all CONNECT traffic, which is what all SSL traffic is
by my understanding.

Your understanding is incorrect. Since you are bumping all CONNECT
* CONNECT requests
* decrypted requests (e.g., GET) inside bumped CONNECT tunnels
* plain requests (e.g., GET) outside CONNECT tunnels
HTH,
Alex.

Post by Shane Poage

Post by Shane Poage
I have my proxy configured to bump all traffic so that the
urlpath_regex ACL can be applied, but it appears to not have any
effect post-bump.

Your proxy will deny any first post-bump request and close the tunnel
because you deny all CONNECT requests that initiate tunnels. CONNECT
requests do not have a URL path so they will never match your
"http_access allow" rule.
When a CONNECT request is denied by a bumping Squid, that Squid bumps
the tunnel and then denies the very first bumped request on that tunnel,
whatever that request is. This delayed error return is done to deliver
the "access denied" error page to the client -- browsers ignore CONNECT
error responses.

Post by Shane Poage
http_port 3128 ssl-bump \
cert=/etc/squid/ssl_certs/artifactory_mitm_ca.pem \
generate-host-certificates=on \
dynamic_cert_mem_cache_size=4MB
ssl_bump bump all
acl artifactory_repo_filter urlpath_regex ^/artifactory
http_access allow artifactory_repo_filter
# And finally deny all other access to this proxy
http_access deny all

Insert an http_access rule to allow all safe CONNECT requests before you
deny everything else. IIRC, squid.conf.default has an example of how to
do that.
Alex.
_______________________________________________
squid-users mailing list
http://lists.squid-cache.org/listinfo/squid-users

[Amos Jeffries]

Post by Shane Poage
Alex,
1. The squid.conf for the configuration that is not behaving as I expected
2. The access.log for two requests (which I will describe in a moment)
3. The cache.log for startup, those same two requests, and shutdown of the squid instance.
1) https://www.google.com/ - Squid permitted the traffic through. According to my understanding, it should have denied it due to the urlpath_regex ACL in the config.
2) https://sottartifactory.corporate.datacard.com:8443/artifactory/some/path/to/artifact.jar - This one was allowed through, which is expected in this case.
Please let me know if there's any more information I can provide that would be helpful, and thanks again for sticking with me on this.
-Shane

This is bug 4682 which was fixed last year in 3.5.26.

Amos