[squid-users] Squid, Gmail.com and HSTS.

Discussion:

Michael Monette

2015-05-27 16:15:37 UTC

Has anyone been able to configure Squid in a way so that if you type https://gmail.com in your browser, you are NOT presented with the "OMG HSTS I refuse to load anything" page? When I go to https://gmail.com, I get an invalid certificate because the cert is for mail.google.com, issued by my CA. If I go to https://mail.google.com, the cert is beautifully green. Why can't squid detect that gmail.com is redirecting my browser to mail.google.com and generate the cert accordingly?

Even configuring an acl for gmail.com doesn't work. It seems like even though I am punching https://gmail.com in my browser, Squid detects it as though I am typing "https://mail.google.com" in my browser and is ignoring any ACLs I have setup specifically for "gmail.com".

I can't be the only one with this issue?

I've also attempted to do:

acl bl1 gmail.com moz.com
always_direct allow bl1 <- from what I understand this bypasses squid and tells my browser to get the cert right from the site. Maybe I am wrong.

But certificates still come from Squid, so I don't see any effect from that line.

Here's my config, lots of garbage in there since I have been trying everything i can think of to get this working. I want to add that for my acl called BL1, the only one that works is moz.com . They are part of the same ACL line, so if one works, they should all work. Except they do not.

Thanks in advance.

cat /etc/squid/squid.conf

~~

debug_options ALL,9

acl localnet src 10.0.0.0/8 # RFC1918 possible internal network
acl localnet src 172.16.0.0/12 # RFC1918 possible internal network
acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
acl localnet src fc00::/7 # RFC 4193 local private network range
acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines

acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT

http_access deny !Safe_ports

http_access deny CONNECT !SSL_ports

http_access allow localhost manager
http_access deny manager

acl step1 at_step SslBump1
acl step2 at_step SslBump2
acl step3 at_step SslBump3

ssl_bump peek step1 all
ssl_bump bump step2 all
ssl_bump bump step3 all

acl bl1 dstdomain gmail.com mail.google.com accounts.google.com moz.com
#acl bl1 url_regex -i ^http(s)?://gmail.com
#acl bl2 url_regex -i ^http(s)?://([a-zA-Z]+).gmail.com.*
#acl bl3 url_regex -i ^http(s)?://moz.com.*
#acl bl4 url_regex -i moz.com
deny_info http://ask.com bl1 # I was testing redirecting stuff, but since the acl is not even picked up, this stuff is useless.
http_reply_access deny bl1 # useless
#http_access deny bl1
#http_access deny bl1 CONNECT

http_access allow localnet
http_access allow localhost

http_access allow all

http_port 3128 accel vhost allow-direct

#https_port 3129 transparent ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=4MB cert=/etc/squid/ssl_cert/myca.pem key=/etc/squid/ssl_cert/myca.pem options=NO_SSLv3
https_port 3129 intercept ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=4MB cert=/etc/squid/ssl_cert/myca.pem key=/etc/squid/ssl_cert/myca.pem options=NO_SSLv3

sslproxy_cert_error allow all
sslproxy_flags DONT_VERIFY_PEER

sslproxy_options NO_SSLv2
sslproxy_options NO_SSLv3

sslcrtd_program /usr/lib/squid/ssl_crtd -s /var/lib/ssl_db -M 4MB
sslcrtd_children 8 startup=1 idle=1

#cache_dir ufs /var/spool/squid 100 16 256
coredump_dir /var/spool/squid

refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320

Mike

Michael Monette

2015-05-27 16:23:33 UTC

Permalink

I just thought of something else. First of all I'm new to squid and I am not aware of 10% of the things its capable of yet so I will ask.

Is squid capable of adding custom SNIs? Like could I have it so gmail.com is added to the certificate as a subject alternate name EVEN though the original certificate doesn't contain it? If such a thing is possible I would love to know the term for it so I can do some searches.

Appreciate it!

Post by Michael Monette
Has anyone been able to configure Squid in a way so that if you type
https://gmail.com in your browser, you are NOT presented with the "OMG
HSTS I refuse to load anything" page? When I go to https://gmail.com, I
get an invalid certificate because the cert is for mail.google.com,
issued by my CA. If I go to https://mail.google.com, the cert is
beautifully green. Why can't squid detect that gmail.com is redirecting
my browser to mail.google.com and generate the cert accordingly?
Even configuring an acl for gmail.com doesn't work. It seems like even
though I am punching https://gmail.com in my browser, Squid detects it
as though I am typing "https://mail.google.com" in my browser and is
ignoring any ACLs I have setup specifically for "gmail.com".
I can't be the only one with this issue?
acl bl1 gmail.com moz.com
always_direct allow bl1 <- from what I understand this bypasses squid
and tells my browser to get the cert right from the site. Maybe I am
wrong.
But certificates still come from Squid, so I don't see any effect from that line.
Here's my config, lots of garbage in there since I have been trying
everything i can think of to get this working. I want to add that for
my acl called BL1, the only one that works is moz.com . They are part
of the same ACL line, so if one works, they should all work. Except
they do not.
Thanks in advance.
cat /etc/squid/squid.conf
~~
debug_options ALL,9
acl localnet src 10.0.0.0/8 # RFC1918 possible internal network
acl localnet src 172.16.0.0/12 # RFC1918 possible internal network
acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
acl localnet src fc00::/7 # RFC 4193 local private network range
acl localnet src fe80::/10 # RFC 4291 link-local (directly
plugged) machines
acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost manager
http_access deny manager
acl step1 at_step SslBump1
acl step2 at_step SslBump2
acl step3 at_step SslBump3
ssl_bump peek step1 all
ssl_bump bump step2 all
ssl_bump bump step3 all
acl bl1 dstdomain gmail.com mail.google.com accounts.google.com moz.com
#acl bl1 url_regex -i ^http(s)?://gmail.com
#acl bl2 url_regex -i ^http(s)?://([a-zA-Z]+).gmail.com.*
#acl bl3 url_regex -i ^http(s)?://moz.com.*
#acl bl4 url_regex -i moz.com
deny_info http://ask.com bl1 # I was testing redirecting stuff, but
since the acl is not even picked up, this stuff is useless.
http_reply_access deny bl1 # useless
#http_access deny bl1
#http_access deny bl1 CONNECT
http_access allow localnet
http_access allow localhost
http_access allow all
http_port 3128 accel vhost allow-direct
#https_port 3129 transparent ssl-bump generate-host-certificates=on
dynamic_cert_mem_cache_size=4MB cert=/etc/squid/ssl_cert/myca.pem
key=/etc/squid/ssl_cert/myca.pem options=NO_SSLv3
https_port 3129 intercept ssl-bump generate-host-certificates=on
dynamic_cert_mem_cache_size=4MB cert=/etc/squid/ssl_cert/myca.pem
key=/etc/squid/ssl_cert/myca.pem options=NO_SSLv3
sslproxy_cert_error allow all
sslproxy_flags DONT_VERIFY_PEER
sslproxy_options NO_SSLv2
sslproxy_options NO_SSLv3
sslcrtd_program /usr/lib/squid/ssl_crtd -s /var/lib/ssl_db -M 4MB
sslcrtd_children 8 startup=1 idle=1
#cache_dir ufs /var/spool/squid 100 16 256
coredump_dir /var/spool/squid
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320
Mike
_______________________________________________
squid-users mailing list
http://lists.squid-cache.org/listinfo/squid-users

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Amos Jeffries

2015-05-27 17:20:33 UTC

Permalink

Post by Michael Monette
Has anyone been able to configure Squid in a way so that if you type

https://gmail.com in your browser, you are NOT presented with the "OMG
HSTS I refuse to load anything" page? When I go to https://gmail.com, I
get an invalid certificate because the cert is for mail.google.com,
issued by my CA. If I go to https://mail.google.com, the cert is
beautifully green. Why can't squid detect that gmail.com is redirecting
my browser to mail.google.com and generate the cert accordingly?

That is *actually* what their server certificate contains. Ironic isn't
it that their own certs do not comply with the restrictions they require
of all others.

Squid actually does obey HSTS requirements for secure handling of the
reqeust. Its just the browser is incapable of detecting that, notices
the custom CA and assumes the worst.

Post by Michael Monette
Even configuring an acl for gmail.com doesn't work. It seems like
even

though I am punching https://gmail.com in my browser, Squid detects it
as though I am typing "https://mail.google.com" in my browser and is
ignoring any ACLs I have setup specifically for "gmail.com".

Post by Michael Monette
I can't be the only one with this issue?
acl bl1 gmail.com moz.com
always_direct allow bl1 <- from what I understand this bypasses squid and tells my browser to get the cert right from the site. Maybe I am wrong.

You are. squid.conf has nothing to do with your browser.

That line tells Squid not to use any cache_peer connections when serving
a request that matches ACL "bl1".

In the very first implementation way, way back in 3.1 decrypted requests
could leak out over insecure cache_peer. So people were advised to use
"always_direct allow all" to force it to work correctly. That bug was
fixed long ago but the config still persists in the web.

Post by Michael Monette
But certificates still come from Squid, so I don't see any effect from that line.
Here's my config, lots of garbage in there since I have been trying everything i can think of to get this working. I want to add that for my acl called BL1, the only one that works is moz.com . They are part of the same ACL line, so if one works, they should all work. Except they do not.
Thanks in advance.
cat /etc/squid/squid.conf
~~
debug_options ALL,9
acl localnet src 10.0.0.0/8 # RFC1918 possible internal network
acl localnet src 172.16.0.0/12 # RFC1918 possible internal network
acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
acl localnet src fc00::/7 # RFC 4193 local private network range
acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines
acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost manager
http_access deny manager
acl step1 at_step SslBump1
acl step2 at_step SslBump2
acl step3 at_step SslBump3
ssl_bump peek step1 all
ssl_bump bump step2 all
ssl_bump bump step3 all

"all" at the end of ACL lines has no meaning unless there is an
authentication ACL that would otherwise be on the end and you dont want
to trigger auth popups.

Post by Michael Monette
acl bl1 dstdomain gmail.com mail.google.com accounts.google.com moz.com
#acl bl1 url_regex -i ^http(s)?://gmail.com
#acl bl2 url_regex -i ^http(s)?://([a-zA-Z]+).gmail.com.*
#acl bl3 url_regex -i ^http(s)?://moz.com.*
#acl bl4 url_regex -i moz.com
deny_info http://ask.com bl1 # I was testing redirecting stuff, but since the acl is not even picked up, this stuff is useless.
http_reply_access deny bl1 # useless

Yes, why bother testing for request *URL* domain and blocking on the
*reply*.

Post by Michael Monette
#http_access deny bl1
#http_access deny bl1 CONNECT
http_access allow localnet
http_access allow localhost
http_access allow all
http_port 3128 accel vhost allow-direct
#https_port 3129 transparent ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=4MB cert=/etc/squid/ssl_cert/myca.pem key=/etc/squid/ssl_cert/myca.pem options=NO_SSLv3
https_port 3129 intercept ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=4MB cert=/etc/squid/ssl_cert/myca.pem key=/etc/squid/ssl_cert/myca.pem options=NO_SSLv3
sslproxy_cert_error allow all
sslproxy_flags DONT_VERIFY_PEER
sslproxy_options NO_SSLv2
sslproxy_options NO_SSLv3

NOTE: Only the latest sslproxy_options line has any effect. So the
NO_SSLv2 line is not obeyed.

Use this instead:
sslproxy_options NO_SSLv2,NO_SSLv3

Although that said. SSLv2 support was dropped earlier so the latest
releases will

Anyhow, back to your ACL problem. Be aware that the SNI and cert fields
are not the HTTP request URL.

* Ensure you are using the latest Squid, today that is a 3.5.4 snapshot
(and in ~48hrs will be 3.5.5).

* Use the "ssl::server_name" ACL type if you need to block using SNI or
server certificate SubjectName.

What Squid does for intercepted port 443 traffic is generate a fake
CONNECT request using the *IP:port* of the server from TCP. http_access
is checked for that. You can block if ou wish or pass it through,
ssl_bump may decide not to bump and it can safely be passed through
peers, or acted on directly to tunnel the intercepted bytes to the server.

If you do SSL-bump with peeking at client and/or server cert the
ssl_server_name ACL matches the data found there about the server
name/alias.

After the decryption SSL-bump process has been completed will you start
to get the HTTP requests that were encrypted as HTTPS. The https_access
etc are all run normally on those since they are just regular HTTP
traffic but for https:// URLs.

Amos

Michael Monette

2015-05-27 18:33:30 UTC

Permalink

Yeah I don't know what I am doing wrong but I don't have these ACL types..Or I am somehow not copy & pasting properly:

FATAL: Invalid ACL type 'ssl::server_name'
FATAL: Bungled /etc/squid/squid.conf line 54: acl nobumpsites ssl::server_name .google.com
Squid Cache (Version 3.5.4): Terminated abnormally.
CPU Usage: 0.005 seconds = 0.003 user + 0.002 sys
Maximum Resident Size: 24096 KB
Page faults with physical i/o: 0
Squid restarted
[***@ottt-corp-paz-squid-1 squid-3.5.4]# squid -v
Squid Cache: Version 3.5.4
Service Name: squid
configure options: '--prefix=/usr' '--includedir=/usr/include' '--datadir=/usr/share' '--bindir=/usr/sbin' '--libexecdir=/usr/lib/squid' '--localstatedir=/var' '--sysconfdir=/etc/squid' '--with-included-ltdl' --enable-ltdl-convenience

There are also issues with "at_step" now:

2015/05/27 14:32:17| FATAL: Invalid ACL type 'at_step'
FATAL: Bungled /etc/squid/squid.conf line 52: acl step1 at_step SslBump1
Squid Cache (Version 3.5.4): Terminated abnormally.
CPU Usage: 0.005 seconds = 0.003 user + 0.002 sys
Maximum Resident Size: 24080 KB
Page faults with physical i/o: 0

Did I miss something when compiling? I just followed what was on the Squid wiki.

I am all out of ideas..

Thanks,

Mike

----- Original Message -----
From: "Amos Jeffries" <***@treenet.co.nz>
To: "squid-users" <squid-***@lists.squid-cache.org>
Sent: Wednesday, May 27, 2015 1:20:33 PM
Subject: Re: [squid-users] Squid, Gmail.com and HSTS.

Post by Michael Monette
Has anyone been able to configure Squid in a way so that if you type

Post by Michael Monette
Even configuring an acl for gmail.com doesn't work. It seems like
even

"all" at the end of ACL lines has no meaning unless there is an
authentication ACL that would otherwise be on the end and you dont want
to trigger auth popups.

Yes, why bother testing for request *URL* domain and blocking on the
*reply*.

Amos Jeffries

2015-05-27 23:14:57 UTC

Permalink

Post by Michael Monette
FATAL: Invalid ACL type 'ssl::server_name'
FATAL: Bungled /etc/squid/squid.conf line 54: acl nobumpsites ssl::server_name .google.com
Squid Cache (Version 3.5.4): Terminated abnormally.
CPU Usage: 0.005 seconds = 0.003 user + 0.002 sys
Maximum Resident Size: 24096 KB
Page faults with physical i/o: 0
Squid restarted
Squid Cache: Version 3.5.4
Service Name: squid
configure options: '--prefix=/usr' '--includedir=/usr/include' '--datadir=/usr/share' '--bindir=/usr/sbin' '--libexecdir=/usr/lib/squid' '--localstatedir=/var' '--sysconfdir=/etc/squid' '--with-included-ltdl' --enable-ltdl-convenience

You are missing the --with-openssl --enable-ssl-crtd options on this build.

Amos

Michael Monette

2015-06-10 14:25:21 UTC

Permalink

Hi again,

I finally had some time to get back into this, been a busy couple weeks. I compiled squid with the "--with-openssl --enable-ssl-crtd" you mentioned, and now things seem to be working better with ssl::servername. But for some reason I can't get HTTPS traffic to get a cert from squid. All HTTPS traffic is getting their certificate from the real sites and I don't really know why because it's the same config as before.

Here's a small capture of the logs:

1433945978.888 95 10.117.67.157 TCP_MISS/302 694 GET http://a.tribalfusion.com/z/i.match? - HIER_DIRECT/204.11.109.68 text/html
1433945978.918 306 10.117.67.157 TCP_MISS/302 658 GET http://pixel.advertising.com/ups/50/sync? - HIER_DIRECT/149.174.67.72 -
1433945978.994 72 10.117.67.157 TCP_MISS/204 737 GET http://su.addthis.com/red/usync? - HIER_DIRECT/104.16.24.235 image/png
1433945979.147 65 10.117.67.157 TAG_NONE/200 0 CONNECT 104.236.7.74:443 - ORIGINAL_DST/104.236.7.74 -
1433945979.152 58 10.117.67.157 TAG_NONE/200 0 CONNECT 104.236.7.74:443 - ORIGINAL_DST/104.236.7.74 -
1433945979.972 1068 10.117.67.157 TCP_MISS/204 719 GET http://su.addthis.com/red/usync? - HIER_DIRECT/104.16.24.235 image/png
1433945981.527 50 10.117.67.157 TAG_NONE/200 0 CONNECT 104.236.7.74:443 - ORIGINAL_DST/104.236.7.74 -
1433945981.753 52 10.117.67.157 TAG_NONE/200 0 CONNECT 104.236.7.74:443 - ORIGINAL_DST/104.236.7.74 -
1433945982.006 100 10.117.67.157 TCP_MISS/200 546 GET http://www.google.ca/ads/user-lists/1072396910/? - HIER_DIRECT/216.254.140.45 text/html
1433945983.769 55 10.117.67.157 TCP_MISS/200 546 GET http://www.google.ca/ads/user-lists/1072396910/? - HIER_DIRECT/216.254.140.45 text/html

All the HTTPS traffic are just CONNECT's. I feel like I ran into this problem when I had been working on this a couple weeks and I was able to get myself out of it by messing with the bump steps, but I can't seem to figure it out this time(or I just can't remember). Hoping for some guidance or hints.

Here's my log again:

# cat /etc/squid/squid.conf
~
debug_options ALL,9

acl localnet src 10.0.0.0/8 # RFC1918 possible internal network
acl localnet src 172.16.0.0/12 # RFC1918 possible internal network
acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
acl localnet src fc00::/7 # RFC 4193 local private network range
acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines

acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT

http_access deny !Safe_ports

http_access deny CONNECT !SSL_ports

http_access allow localhost manager
http_access deny manager

acl step1 at_step SslBump1
acl step2 at_step SslBump2
acl step3 at_step SslBump3

ssl_bump peek step1 all
ssl_bump bump step2 all
ssl_bump bump step3 all

acl bl1 dstdomain gmail.com mail.google.com accounts.google.com moz.com
#acl bl1 url_regex -i ^http(s)?://gmail.com
#acl bl2 url_regex -i ^http(s)?://([a-zA-Z]+).gmail.com.*
#acl bl3 url_regex -i ^http(s)?://moz.com.*
#acl bl4 url_regex -i moz.com
deny_info http://ask.com bl1 # I was testing redirecting stuff, but since the acl is not even picked up, this stuff is useless.
http_reply_access deny bl1 # useless
#http_access deny bl1
#http_access deny bl1 CONNECT

http_access allow localnet
http_access allow localhost

http_access allow all

http_port 3128 accel vhost allow-direct

#https_port 3129 transparent ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=4MB cert=/etc/squid/ssl_cert/myca.pem key=/etc/squid/ssl_cert/myca.pem options=NO_SSLv3
https_port 3129 intercept ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=4MB cert=/etc/squid/ssl_cert/myca.pem key=/etc/squid/ssl_cert/myca.pem options=NO_SSLv3

sslproxy_cert_error allow all
sslproxy_flags DONT_VERIFY_PEER

sslproxy_options NO_SSLv2
sslproxy_options NO_SSLv3

sslcrtd_program /usr/lib/squid/ssl_crtd -s /var/lib/ssl_db -M 4MB
sslcrtd_children 8 startup=1 idle=1

#cache_dir ufs /var/spool/squid 100 16 256
coredump_dir /var/spool/squid

refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320

----- Original Message -----
From: "Amos Jeffries" <***@treenet.co.nz>
To: "Michael Monette" <***@2keys.ca>
Cc: "squid-users" <squid-***@lists.squid-cache.org>
Sent: Wednesday, May 27, 2015 7:14:57 PM
Subject: Re: [squid-users] Squid, Gmail.com and HSTS.

You are missing the --with-openssl --enable-ssl-crtd options on this build.

Amos

Michael Monette

2015-06-10 14:42:35 UTC

Permalink

Sorry for the noise - I figured it out.

HTTPS was completely dead which made me wonder if squid was working properly. It turns out I had some folder permission issues. I needed to chmod -R 777 /var/lib/ssl_db. I guess lack of permissions to that directory caused cert generation to fail and HTTPS to break..Thanks for reading

----- Original Message -----
From: "Michael Monette" <***@2keys.ca>
To: "Amos Jeffries" <***@treenet.co.nz>
Cc: "squid-users" <squid-***@lists.squid-cache.org>
Sent: Wednesday, June 10, 2015 10:25:21 AM
Subject: Re: [squid-users] Squid, Gmail.com and HSTS.

Hi again,

I finally had some time to get back into this, been a busy couple weeks. I compiled squid with the "--with-openssl --enable-ssl-crtd" you mentioned, and now things seem to be working better with ssl::servername. But for some reason I can't get HTTPS traffic to get a cert from squid. All HTTPS traffic is getting their certificate from the real sites and I don't really know why because it's the same config as before.

Here's a small capture of the logs:

1433945978.888 95 10.117.67.157 TCP_MISS/302 694 GET http://a.tribalfusion.com/z/i.match? - HIER_DIRECT/204.11.109.68 text/html
1433945978.918 306 10.117.67.157 TCP_MISS/302 658 GET http://pixel.advertising.com/ups/50/sync? - HIER_DIRECT/149.174.67.72 -
1433945978.994 72 10.117.67.157 TCP_MISS/204 737 GET http://su.addthis.com/red/usync? - HIER_DIRECT/104.16.24.235 image/png
1433945979.147 65 10.117.67.157 TAG_NONE/200 0 CONNECT 104.236.7.74:443 - ORIGINAL_DST/104.236.7.74 -
1433945979.152 58 10.117.67.157 TAG_NONE/200 0 CONNECT 104.236.7.74:443 - ORIGINAL_DST/104.236.7.74 -
1433945979.972 1068 10.117.67.157 TCP_MISS/204 719 GET http://su.addthis.com/red/usync? - HIER_DIRECT/104.16.24.235 image/png
1433945981.527 50 10.117.67.157 TAG_NONE/200 0 CONNECT 104.236.7.74:443 - ORIGINAL_DST/104.236.7.74 -
1433945981.753 52 10.117.67.157 TAG_NONE/200 0 CONNECT 104.236.7.74:443 - ORIGINAL_DST/104.236.7.74 -
1433945982.006 100 10.117.67.157 TCP_MISS/200 546 GET http://www.google.ca/ads/user-lists/1072396910/? - HIER_DIRECT/216.254.140.45 text/html
1433945983.769 55 10.117.67.157 TCP_MISS/200 546 GET http://www.google.ca/ads/user-lists/1072396910/? - HIER_DIRECT/216.254.140.45 text/html

All the HTTPS traffic are just CONNECT's. I feel like I ran into this problem when I had been working on this a couple weeks and I was able to get myself out of it by messing with the bump steps, but I can't seem to figure it out this time(or I just can't remember). Hoping for some guidance or hints.

Here's my log again:

# cat /etc/squid/squid.conf
~
debug_options ALL,9

acl localnet src 10.0.0.0/8 # RFC1918 possible internal network
acl localnet src 172.16.0.0/12 # RFC1918 possible internal network
acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
acl localnet src fc00::/7 # RFC 4193 local private network range
acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines

acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT

http_access deny !Safe_ports

http_access deny CONNECT !SSL_ports

http_access allow localhost manager
http_access deny manager

acl step1 at_step SslBump1
acl step2 at_step SslBump2
acl step3 at_step SslBump3

ssl_bump peek step1 all
ssl_bump bump step2 all
ssl_bump bump step3 all

acl bl1 dstdomain gmail.com mail.google.com accounts.google.com moz.com
#acl bl1 url_regex -i ^http(s)?://gmail.com
#acl bl2 url_regex -i ^http(s)?://([a-zA-Z]+).gmail.com.*
#acl bl3 url_regex -i ^http(s)?://moz.com.*
#acl bl4 url_regex -i moz.com
deny_info http://ask.com bl1 # I was testing redirecting stuff, but since the acl is not even picked up, this stuff is useless.
http_reply_access deny bl1 # useless
#http_access deny bl1
#http_access deny bl1 CONNECT

http_access allow localnet
http_access allow localhost

http_access allow all

http_port 3128 accel vhost allow-direct

#https_port 3129 transparent ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=4MB cert=/etc/squid/ssl_cert/myca.pem key=/etc/squid/ssl_cert/myca.pem options=NO_SSLv3
https_port 3129 intercept ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=4MB cert=/etc/squid/ssl_cert/myca.pem key=/etc/squid/ssl_cert/myca.pem options=NO_SSLv3

sslproxy_cert_error allow all
sslproxy_flags DONT_VERIFY_PEER

sslproxy_options NO_SSLv2
sslproxy_options NO_SSLv3

sslcrtd_program /usr/lib/squid/ssl_crtd -s /var/lib/ssl_db -M 4MB
sslcrtd_children 8 startup=1 idle=1

#cache_dir ufs /var/spool/squid 100 16 256
coredump_dir /var/spool/squid

refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320

----- Original Message -----
From: "Amos Jeffries" <***@treenet.co.nz>
To: "Michael Monette" <***@2keys.ca>
Cc: "squid-users" <squid-***@lists.squid-cache.org>
Sent: Wednesday, May 27, 2015 7:14:57 PM
Subject: Re: [squid-users] Squid, Gmail.com and HSTS.

You are missing the --with-openssl --enable-ssl-crtd options on this build.

Amos
_______________________________________________
squid-users mailing list
squid-***@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Amos Jeffries

2015-05-27 23:12:57 UTC

Permalink

Thanks for the reply. I am compiling the latest squid now and going to give that a shot using ssl::server_name. The precompiled binary for CentOS was only at 3.5.04 and didn't have ssl::server_name support I guess.
While I have you here, I was thinking of something else I could do. Tell me if you think it would work. You seem like an expert.

I'm the Squid maintainer. On this feature I'm just an interested party,
the real expert is Christos Tsantilas who authored it.

acl gml dst 1.2.3.4
# Redirect request for 1.2.3.4 to https://mail.google.com
deny_info https://mail.google.com gml
# Not even sure what this does, found it on a website, didn't really question it since things were working as they were supposed to.
http_reply_access deny bl1

This acts on the HTTP messages inside the TLS wrapper after SSL-Bump has
decrypted the connection and the request has been sent through to the
server. By replacing whatever reply the server actually sent with a 302
status diverting the client to that URL.

It has no effect on the bumping process itself or any of the TLS layer
details. It may have an effect on HSTS since any headers the server
might send are dropped by the 302 replacement.

This way it doesn't have to monitor for a dstdomain, it watches for an IP.

It wastes bandwidth sending a request to the server and fetching the
reply. Given that the dst-IP address is what you have from the very
first TCP packet onwards.

About the only utility it has over http_access is that its positioning
naturally restricts it to the decrypted traffic where the 302 will
always work. But by that time you have the domain name from the
encrypted HTTP message anyway.

Amos