Discussion:
[squid-users] ERR_CANNOT_FORWARD with Squid + Privoxy
Stepan Bujnak
8 years ago
Permalink
Hi,

I've been trying to configure intercepting proxy with privoxy as a
cache_peer. This is my Squid configuration:

acl all src all

acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT

#http_access deny !Safe_ports
#http_access deny CONNECT !SSL_ports
http_access allow all

# stop squid taking forever to restart.
shutdown_lifetime 3 second

client_dst_passthru off
host_verify_strict off

# IMPORTANT! squid requires at least one forward-proxy port configured
# http://wiki.squid-cache.org/KnowledgeBase/NoForwardProxyPorts
http_port 0.0.0.0:3127
http_port 0.0.0.0:3128 intercept
https_port 0.0.0.0:3129 intercept ssl-bump
generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
cert=/etc/squid/ssl_certs/squid.pem

sslcrtd_program /usr/lib/squid/ssl_crtd -s /var/lib/squid/ssl_db -M
4MB sslcrtd_children 8 startup=1 idle=1
sslproxy_capath /etc/ssl/certs

acl step1 at_step SslBump1
ssl_bump peek step1
ssl_bump bump all

cache_peer 127.0.0.1 parent 8118 7 no-query default no-digest
no-netdb-exchange proxy-only ssl
never_direct allow all

cache_mem 8 MB
maximum_object_size_in_memory 32 KB

# Disable the Via and X-Forwarded-For field from the request header to avoid
# leaking the use of a proxy and client ip address
via off
forwarded_for off
follow_x_forwarded_for deny all
request_header_access X-Forwarded-For deny all

#cache_dir ufs /var/spool/squid 1024 16 256
#coredump_dir /var/cache/squid
cache deny all

refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320


Now when making a request, privoxy prints out following:

2017-01-11 00:36:51.420 7fe4872a4700 Connect: Accepted connection from
127.0.0.1 on socket 4
2017-01-11 00:36:51.421 7fe4872a4700 Received: from socket 4:
\x16\x03\x01\x010\x01\x00\x01,\x03\x03xfOz\xc3\xc2\xf8\xf6\xc4\x972Y\xe5w\xf0\xd7\x98\xb5\xd3\x99\xfb\x97P%\x0aX\x1f\xefs\x91\xc6d\x00\x00\xaa\xc00\xc0,\xc0(\xc0$\xc0\x14\xc0\x0a\x00\xa5\x00\xa3\x00\xa1\x00\x9f\x00k\x00j\x00i\x00h\x009\x008\x007\x006\x00\x88\x00\x87\x00\x86\x00\x85\xc02\xc0.\xc0*\xc0&\xc0\x0f\xc0\x05\x00\x9d\x00=\x005\x00\x84\xc0/\xc0+\xc0'\xc0#\xc0\x13\xc0\x09\x00\xa4\x00\xa2\x00\xa0\x00\x9e\x00g\x00@\x00?\x00>\x003\x002\x001\x000\x00\x9a\x00\x99\x00\x98\x00\x97\x00E\x00D\x00C\x00B\xc01\xc0-\xc0)\xc0%\xc0\x0e\xc0\x04\x00\x9c\x00<\x00/\x00\x96\x00A\xc0\x11\xc0\x07\xc0\x0c\xc0\x02\x00\x05\x00\x04\xc0\x12\xc0\x08\x00\x16\x00\x13\x00\x10\x00\x0d\xc0\x0d\xc0\x03\x00\x0a\x00\xff\x01\x00\x00Y\x00\x0b\x00\x04\x03\x00\x01\x02\x00\x0a\x00\x1c\x00\x1a\x00\x17\x00\x19\x00\x1c\x00\x1b\x00\x18\x00\x1a\x00\x16\x00\x0e\x00\x0d\x00\x0b\x00\x0c\x00\x09\x00\x0a\x00#\x00\x00\x00\x0d\x00
\x00\x1e\x06\x01\x06\x02\x06\x03\x05\x01\x05\x02\x05\x03\x04\x01\x04\x02\x04\x03\x03\x01\x03\x02\x03\x03\x02\x01\x02\x02\x02\x03\x00\x0f\x00\x01\x013t\x00\x00
2017-01-11 00:37:21.450 7fe4872a4700 Connect: The client side of the
connection on socket 4 got closed without sending a complete request
line.


It seems like the bumped request is missing the CONNECT line and
privoxy gets confused.

Squid version:

Squid Cache: Version 3.5.23
Service Name: squid
configure options: 'CHOST=x86_64-pc-linux-gnu' 'CFLAGS=-march=core2
-O2 -pipe' 'CXXFLAGS=' '--build=x86_64-linux-gnu' '--prefix=/usr'
'--exec-prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin'
'--libdir=/usr/lib' '--sharedstatedir=/usr/com'
'--includedir=/usr/include' '--localstatedir=/var'
'--libexecdir=/usr/lib/squid' '--srcdir=.'
'--datadir=/usr/share/squid' '--sysconfdir=/etc/squid'
'--infodir=/usr/share/info' '--mandir=/usr/share/man'
'--x-includes=/usr/include' '--x-libraries=/usr/lib'
'--with-default-user=proxy' '--with-logdir=/var/log/squid'
'--with-pidfile=/var/run/squid.pid' '--enable-storeio=ufs,aufs,diskd'
'--enable-linux-netfilter' '--enable-removal-policies=lru,heap'
'--enable-gnuregex' '--enable-follow-x-forwarded-for'
'--enable-x-accelerator-vary' '--enable-zph-qos'
'--enable-delay-pools' '--enable-snmp' '--enable-underscores'
'--with-openssl' '--enable-ssl-crtd' '--enable-http-violations'
'--enable-async-io=24' '--enable-storeid-rewrite-helpers'
'--with-large-files' '--with-libcap' '--with-netfilter-conntrack'
'--with-included-ltdl' '--with-maxfd=65536'
'--with-filedescriptors=65536' '--with-pthreads' '--without-gnutls'
'--without-mit-krb5' '--without-heimdal-krb5' '--without-gnugss'
'--disable-icap-client' '--disable-wccp' '--disable-wccpv2'
'--disable-dependency-tracking' '--disable-auth' '--disable-epoll'
'--disable-ident-lookups' '--disable-icmp'
'build_alias=x86_64-linux-gnu' --enable-ltdl-convenience


As a result, the client receives ERR_CANNOT_FORWARD. Could someone
point me to the right direction? Thank you.
Amos Jeffries
8 years ago
Permalink
...
Please pay attention to the docs for these options. Specifically how it
says host_verify_strict has no effect on intercepted traffic. Also how
it says client_dst_passthru has no effect when the Host verify process
detects an origin mismatch (eg 'fails').
...
So what you have configured is Server-first bumping.

All clients will be presented with the Privoxy SSL certificate as if
Privoxy (at 127.0.0.1:8118) was the authoritative web server for the
HTTPS website being fetched.

"What could go wrong?" as the saying goes. A better question would be
what could possibly go _right_ in that setup. Very few websites will
work, and only where the TLS was completely broken in the first place.
Post by Stepan Bujnak
cache_peer 127.0.0.1 parent 8118 7 no-query default no-digest
no-netdb-exchange proxy-only ssl
never_direct allow all
cache_mem 8 MB
maximum_object_size_in_memory 32 KB
# Disable the Via and X-Forwarded-For field from the request header to avoid
# leaking the use of a proxy and client ip address
via off
forwarded_for off
The above injects "X-Forwarded-For: unknown" into the traffic. Squid has
long ago moved past the point where that was the only choice.

Use "forwarded_for transparent" instead for privacy. Then you can remove
the following two lines as well...
...
There is no CONNECT.

You configured Squid to SSL-Bump the HTTPS traffic. That means the
CONNECT gets absorbed by Squid and the TLS protocol to the client gets
terminated, and a new TLS connection is opened to the Privoxy to fetch
the server certificate (or not as this case shows).

The connection is native TLS between Squid and privoxy. As configured by
the 'ssl' option on cache_peer. Since Privoxy cannot handle TLS that
fails, and you get the forwarding error.

And before you ask, no Squid-3 will not send SSL-Bump'ed traffic through
a peer without that 'ssl' option. Squid-4 has some improvements for
non-HTTPS traffic, but still not to the point of Squid generating peer
CONNECT messages after bumping HTTPS.
Post by Stepan Bujnak
As a result, the client receives ERR_CANNOT_FORWARD. Could someone
point me to the right direction? Thank you.
Your best hope is to recreate in squid.conf settings the privacy
operations you are using privoxy for. Then remove privoxy from the chain
of proxies.

Amos
Stepan Bujnak
8 years ago
Permalink
Thank you very much for the reply!
...
I figured out the origin mismatch part. Unfortunately, this is very
important to me so I had to dig into the code turn the check off.
...
Would better solution be client-first configuration where client would
be presented with squid's self generated certificate, read the traffic
and then send it to the actual destination through privoxy using
CONNECT? How could that be configured?
...
I thought about this solution, but it seems that squid cannot use
socks parent yet.
Post by Amos Jeffries
Amos
_______________________________________________
squid-users mailing list
http://lists.squid-cache.org/listinfo/squid-users
Amos Jeffries
8 years ago
Permalink
...
Sigh. Please don't. That makes a short web script (eg a web advert) able
to hijack your proxy and use it as a base to hijack your whole network -
hiding the attacker while doing so.

There are details and some mitigations to reduce the pain listed at
<http://wiki.squid-cache.org/KnowledgeBase/HostHeaderForgery>
...
That can be done by making the ssl_bump directive decide to bump on
step1. It still has all the same problems as the existing config, the
issues all center around the fact that the client is not presented with
anything that looks even remotely like the real servers certificate or
TLS options. So any TLS/SSL security the client may be using is useless.
Post by Stepan Bujnak
using CONNECT?
No Squid is currently able to do that. Once decrypted the traffic has to
go to an HTTPS server directly (ORIGINAL_DST) or to a proxy which is
connected to using a secure channel, ie. TLS/SSL.
Post by Stepan Bujnak
Post by Amos Jeffries
Post by Stepan Bujnak
As a result, the client receives ERR_CANNOT_FORWARD. Could someone
point me to the right direction? Thank you.
Your best hope is to recreate in squid.conf settings the privacy
operations you are using privoxy for. Then remove privoxy from the chain
of proxies.
I thought about this solution, but it seems that squid cannot use
socks parent yet.
You should be able to send Squids outbound traffic through a regular
SOCKS tunnel/gateway if you need to. It is just configured at the OS
routing level rather than anything in squid.conf.

Amos

Loading...