[squid-users] https_port Listen on different IP

[houheming]

Hi squid-users,

I have to use squid in a very uncommon way, here it is:

1. Configure squid to be a https tproxy proxy;

2. For some https server IPs , configure squid to send the client
browser certificates which signed by some specific root CAs;

3. For other https server IPs, configure squid to send client browser
certificates which signed by default root CA;



Here is part of my squid.conf:



#default root CA for default https servers

https_port 443 tproxy ssl-bump cert=/usr/local/squid/server.crt
key=/usr/local/squid/server.key version=1 generate-host-certificates=on

#for special server baidu.com

https_port 180.97.33.107:443 tproxy ssl-bump
cert=/usr/local/squid/server_baidu.crt key=/usr/local/squid/server_baidu.key
version=1 generate-host-certificates=on

https_port 180.97.33.108:443 tproxy ssl-bump
cert=/usr/local/squid//server_baidu.crt
key=/usr/local/squid/server_baidu.key version=1
generate-host-certificates=on





The problem is if I configure squid like this, the line2 and line3 will
never work; If I switch line1 with line2 and line3(put line2 and line3 at
the above of line 1 in squid.conf), then only line2 and line3 will get its
chance to work, line1 will not work.





From the screenshot , you can see that when I use "squid -k parse" to check
squid.conf, it shows that it will initialize three SSL context, but if you
check the port, only the first SSL context is initialized.



Any help will be appreciated!



Horise Hou

[Alex Rousskov]

Post by houheming
Configure squid to be a https tproxy proxy

Terminology clarification: You are configuring an transparent proxy for
intercepting TLS/HTTPS traffic, not an (explicit) HTTPS proxy.

Post by houheming
configure squid to send the client browser certificates which signed by X

This phrase can be (mis)interpreted many ways:

1. Configure Squid to automatically generate origin server certificates
(signed by a configured CA X) and send them to browsers/clients that go
to those origin servers.

2. Configure Squid to use a configured client certificate (signed by
some CA X) and send it to origin servers that request client certificates.

Does any of the above match what you want to do?

Post by houheming
https_port 443 ...
https_port 180.97.33.107:443 ...
https_port 180.97.33.108:443 ...

I am not sure, but perhaps the first https_port line (the one without an
explicit IP address) should come _last_ so that Squid can listen on the
addresses that remain after 180.97.33.107 and 180.97.33.108 are taken by
the other two ports?

Also, if your Squid, when started without "-k parse", reports any
warnings or errors, please share them.

Alex.

[Amos Jeffries]

Post by Alex Rousskov

Post by houheming
https_port 443 ...
https_port 180.97.33.107:443 ...
https_port 180.97.33.108:443 ...

I am not sure, but perhaps the first https_port line (the one without an
explicit IP address) should come _last_ so that Squid can listen on the
addresses that remain after 180.97.33.107 and 180.97.33.108 are taken by
the other two ports?

I think that is what was meant by "If I switch line1 with line2 and
line3 ..., then only line2 and line3 will get its chance to work, line1
will not work. "

The problem is that TCP does not permit any IP:port combination to have
two simultaneous listening sockets with different parameters. These
configuration lines differ in both address and protocol they are receiving.


houheming:
you have to use different ports to receive the traffic into Squid.

Since you are using TPROXY there is no requirement for the proxy
listening port to be 443. Squid can listen on any port you want.

This problem should disappear if you set the wildcard port to another
number and update the TPROXY rule which is sending traffic to it.

Amos

[houheming]

Hi Alex & Amos,

Thanks for your replies, sorry for my poor English, I will add more information.



#1

1. Configure Squid to automatically generate origin server certificates

(signed by a configured CA X) and send them to browsers/clients that go

to those origin servers.



This is my situation.



#2

If I configure squid like the following:

https_port 443 ...

https_port 180.97.33.107:443 ...

https_port 180.97.33.108:443 ...



when I reconfigure squid, no errors come out, but when I check the tcp port listening using "netstat -tln", the line2 and line3 with specific IP do not work:





#3

If I configure squid like:

https_port 180.97.33.107:443 ...

https_port 180.97.33.108:443

https_port 443



when I use “squid –k reconfigure” to reconfigure squid, no error message. But when I check the tcp ports listening:







#4

I also tried to specific a different port for IP 0.0.0.0, like this:

https_port 180.97.33.107:443 ...

https_port 180.97.33.108:443

https_port 4433 #here specify a different port from above



and it works.





https_port 0.0.0.0:443

https_port 1.1.1.1:443

I was just curious that the above two types of rules cannot exist together, whichever you put the first in squid.conf, it will overwrite the second one.



Using a different port for wildcard, it is a good solution in this case, it is just that ,if I could use the same tcp port for IP-0.0.0.0, it will be easier for my situation.

In my case, the destination server IP and root CA are dynamically send to me by anther configure server, so I choose to use default https port 443 to receive all https traffic.

If I have to use different ports for IP-0.0.0.0, I think I have to write one iptables rule for one https server IP, in normal case this is ok, in my case I have to do it dynamically, in another way of saying, whenever I receive a configure rule, I have to write an iptables rule for it. Anyway, that is my problem, thank you guys!





Heming Hou

-----邮件原件-----
发件人: Amos Jeffries [mailto:***@treenet.co.nz]
发送时闎: 2018幎10月20日 12:10
收件人: squid-***@lists.squid-cache.org
䞻题: Re: [squid-users] https_port Listen on different IP

Post by Alex Rousskov

Post by houheming
https_port 443 ...
https_port 180.97.33.107:443 ...
https_port 180.97.33.108:443 ...

I am not sure, but perhaps the first https_port line (the one without an
explicit IP address) should come _last_ so that Squid can listen on the
addresses that remain after 180.97.33.107 and 180.97.33.108 are taken by
the other two ports?

I think that is what was meant by "If I switch line1 with line2 and

line3 ..., then only line2 and line3 will get its chance to work, line1

will not work. "



The problem is that TCP does not permit any IP:port combination to have

two simultaneous listening sockets with different parameters. These

configuration lines differ in both address and protocol they are receiving.





houheming:

you have to use different ports to receive the traffic into Squid.



Since you are using TPROXY there is no requirement for the proxy

listening port to be 443. Squid can listen on any port you want.



This problem should disappear if you set the wildcard port to another

number and update the TPROXY rule which is sending traffic to it.



Amos

_______________________________________________

squid-users mailing list

squid-***@lists.squid-cache.org

http://lists.squid-cache.org/listinfo/squid-users

[Alex Rousskov]

Post by Alex Rousskov
1. Configure Squid to automatically generate origin server certificates
(signed by a configured CA X) and send them to browsers/clients that go
to those origin servers.

 
OK.

Post by Alex Rousskov
https_port 180.97.33.107:443 ...
https_port 180.97.33.108:443
https_port 443
when I use “squid –k reconfigure” to reconfigure squid, no error message.

In general, avoid using "-k anything" as the primary configuration test.
It just complicates matters by introducing a different error checking
context. Use a clean start. I am _not_ saying that a clean start would
have solved your problem in this particular case.
[image showing the first two out of three configured :443 ports]

If Squid did not complain about anything but did not start listening on
one of the configured ports, then there is a Squid error reporting bug
somewhere. Feel free to report it to Squid bugzilla.

My suggestion to reorder those https_port lines was wrong. The wildcard
bind(INADDR_ANY) system call does not bind to "any available" address.
It binds to "all" addresses and, hence, fails if one of the addresses is
not available.

As Amos said and you have confirmed, using a different port for the
third https_port avoids these problems. Using a different/specific IP
address for the third https_port is another option.

Post by Alex Rousskov
I was just curious that the above two types of rules cannot exist
together, whichever you put the first in squid.conf, it will overwrite
the second one.

https_port lines do not overwrite each other. The ports are independent,
but since they all consume a common resource (system IPs and ports),
they may conflict with each other. Use unique IPs and/or unique ports to
avoid conflicts.


Cheers,

Alex.

Post by Alex Rousskov
-----邮件原件-----
发送时间: 2018年10月20日12:10
主题: Re: [squid-users] https_port Listen on different IP
 

Post by Alex Rousskov

Post by houheming
https_port 443 ...
https_port 180.97.33.107:443 ...
https_port 180.97.33.108:443 ...

I am not sure, but perhaps the first https_port line (the one without an
explicit IP address) should come _last_ so that Squid can listen on the
addresses that remain after 180.97.33.107 and 180.97.33.108 are taken by
the other two ports?

 
I think that is what was meant by "If I switch line1 with line2 and
line3 ..., then only line2 and line3 will get its chance to work, line1
will not work. "
 
The problem is that TCP does not permit any IP:port combination to have
two simultaneous listening sockets with different parameters. These
configuration lines differ in both address and protocol they are receiving.
 
 
you have to use different ports to receive the traffic into Squid.
 
Since you are using TPROXY there is no requirement for the proxy
listening port to be 443. Squid can listen on any port you want.
 
This problem should disappear if you set the wildcard port to another
number and update the TPROXY rule which is sending traffic to it.
 
Amos
_______________________________________________
squid-users mailing list
http://lists.squid-cache.org/listinfo/squid-users

[Amos Jeffries]

Post by Alex Rousskov

Post by Alex Rousskov
1. Configure Squid to automatically generate origin server certificates
(signed by a configured CA X) and send them to browsers/clients that go
to those origin servers.

 
OK.

Post by Alex Rousskov
https_port 180.97.33.107:443 ...
https_port 180.97.33.108:443
https_port 443
when I use “squid –k reconfigure” to reconfigure squid, no error message.

In general, avoid using "-k anything" as the primary configuration test.
It just complicates matters by introducing a different error checking
context. Use a clean start. I am _not_ saying that a clean start would
have solved your problem in this particular case.
[image showing the first two out of three configured :443 ports]
If Squid did not complain about anything but did not start listening on
one of the configured ports, then there is a Squid error reporting bug
somewhere. Feel free to report it to Squid bugzilla.
My suggestion to reorder those https_port lines was wrong. The wildcard
bind(INADDR_ANY) system call does not bind to "any available" address.
It binds to "all" addresses and, hence, fails if one of the addresses is
not available.

Aye.

Just double-checked, Squid does report this problem:

"2018/10/21 19:12:30 kid1| ERROR: listen( FD 21, [::] [ job2], 256):
(98) Address already in use"

... but only only for -k start / restart / reconfigure.
The -k parse does not check it.

Amos