Discussion:
[squid-users] Squid 3.5.1 NTLM and LDAP
Rich549
2015-02-10 13:39:54 UTC
Permalink
Hi,

After running into plenty of issue with my Linux install of Squid 3.5.1 and
eventually solving those, my company has now got me to do some work for
another client that wants to use Squid. The issue with this one though is
that they will only use Windows, completely anti-Linux...

Anyway, I got a pre-compiled version of Squid 3.5.1 from here
http://squid.diladele.com/. It has been compiled using Cygwin and works
perfectly in Server 2008 and above. Until you want to use LDAP group
lookups.

Now, I'm getting really confused, everything seemed to be a lot simpler in
Squid 2.7 for Windows (the only reason I'm not using this is because video
buffering is slow).

So...would someone mind having a look at my attached config and tell me
where I have gone wrong please? My users only seem to be able to access
whitelisted sites, which leads me to believe that something is wrong with
the LDAP query for external_acl_type internet_domain_group.

A lot of the config is cannibalised from previous SquidNT 2.7 and Linux
Squid 3.5.1 configs.

example-config.txt
<http://squid-web-proxy-cache.1019090.n4.nabble.com/file/n4669661/example-config.txt>

Thanks,

Rich



--
View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-3-5-1-NTLM-and-LDAP-tp4669661.html
Sent from the Squid - Users mailing list archive at Nabble.com.
Amos Jeffries
2015-02-10 19:56:05 UTC
Permalink
Post by Rich549
Hi,
After running into plenty of issue with my Linux install of Squid 3.5.1 and
eventually solving those, my company has now got me to do some work for
another client that wants to use Squid. The issue with this one though is
that they will only use Windows, completely anti-Linux...
And yet they use Squid :-P lol.
Post by Rich549
Anyway, I got a pre-compiled version of Squid 3.5.1 from here
http://squid.diladele.com/. It has been compiled using Cygwin and works
perfectly in Server 2008 and above. Until you want to use LDAP group
lookups.
Please keep in mind that these builds are semi-experimental at present.
Squid-3.x Cygwin builds have been available longer than others for
Windows, but the user base for Windows in total is not very big. It may
just be that nobody before you used or tested this particular helper setup.
Post by Rich549
Now, I'm getting really confused, everything seemed to be a lot simpler in
Squid 2.7 for Windows (the only reason I'm not using this is because video
buffering is slow).
Are you finding 3.5 any faster on Windows?

The major bottleneck that makes Squid people avoid Windows is that they
are capped with a permanent absolute limit of 2048 sockets. At 2 sockets
per client connection (client+server or client+disk) and 8 connections
per user browser (just 'cause they do) thats a capacity of roughly
120-240 users that can be going through the proxy at any one time.
Post by Rich549
So...would someone mind having a look at my attached config and tell me
where I have gone wrong please? My users only seem to be able to access
whitelisted sites, which leads me to believe that something is wrong with
the LDAP query for external_acl_type internet_domain_group.
A lot of the config is cannibalised from previous SquidNT 2.7 and Linux
Squid 3.5.1 configs.
Please note the message about "SquidNT" at the top of
<http://wiki.squid-cache.org/KnowledgeBase/Windows>
Post by Rich549
http_port 3128
These ...
Post by Rich549
acl QUERY urlpath_regex cgi-bin \?
cache deny ALL
acl apache rep_header Server ^Apache
... should not be needed at all in 3.5.
Post by Rich549
cache_mem 1024 MB
#cache_dir ufs d:/squid/var/cache/squid/ 8000 16 256
access_log d:/squid/var/log/squid/access.log squid
cache_log d:/squid/var/cache/squid/cache.log
These...
Post by Rich549
cache_store_log d:/squid/var/log/store.log
mime_table d:/Squid/etc/squid/mime.conf
pid_filename d:/squid/var/log/squid/squid.pid
unlinkd_program d:/squid/lib/squid/unlinkd.exe
logfile_daemon d:/Squid/lib/squid/log_file_daemon.exe
icon_directory d:/squid/usr/share/squid/icons
... to here should also not be necessary in 3.5.
Post by Rich549
error_directory d:/squid/usr/share/squid/errors/en-uk
coredump_dir d:/squid/var/cache/squid/
dns_nameservers 172.30.12.9 172.31.12.10
### New NTLM Authentication Method
auth_param ntlm program d:/Squid/lib/squid/ntlm_fake_auth
auth_param ntlm children 80
auth_param ntlm keep_alive off
Note that all this helper does is check that the NTLM protocol is
syntactically accurate.
Post by Rich549
### Helper Processes
external_acl_type internet_domain_group %LOGIN d:/Squid/lib/squid/ext_ldap_group_acl.exe \
-b "ou=Domain_Groups,dc=domain-uk,dc=com" \
-f %v=Internet_Users -h srvham09.domain-uk.com
The documentation for -f option says that %u (not %v) will be replaced
with username and %g with group name.
Post by Rich549
# ------------------------------------------------
# ---- Declare domains for individual access ----
# ------------------------------------------------
# Blacklisted domains
acl BlacklistedSites dstdomain .yahoo.com .ebay.com .ebay.co.uk mail.google.com outlook.com hotmail.com hotmail.co.uk live.co.uk
# These domains will be reachable without authentication
acl OK_Unauthenticated dstdomain .domain-uk.com .stanford.edu
acl OK_Unauthenticated dstdomain .domainretail.local .everythingbedrooms.co.uk .canonical.com .sophos.com .ubuntu.com .oracle.com .bt.com
acl OK_Unauthenticated dstdomain .oanda.com .dell.com .launchpad.net
acl OK_Unauthenticated dstdomain .dashboards.my-tmac.co.uk
# ------------------------------------------------
# --- Map web access to AD groups via helpers ---
# ------------------------------------------------
# Allow Members of Internet Users To Anywhere Not Explicitly Denied
acl InetAllow external internet_domain_group Internet_Users
# Allow Store Access
#acl StoresAllow external stores_domain_group Stores_Internet_Access
# ------------------------------------------------
# --------------- Misc settings -----------------
# ------------------------------------------------
# These domains wont be cached - every request will be pulled directly from the web
acl do_not_cache dstdomain domain-uk.com youtube.com
cache deny do_not_cache
# Append domain-uk.com to hostnames without a dot in them
append_domain .domain-uk.com
The above will only have any effect if you also define:

dns_defnames on

<http://www.squid-cache.org/Doc/config/dns_defnames/>

If you need it fine, if not remove from the config.
Post by Rich549
# Allow these static IPs access to everything without authentication
acl StaticIPWhitelist src 172.31.12.* ....
# ------------------------------------------------
# ------ Permit/Deny access as appropriate -------
# ------------------------------------------------
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern . 0 20% 4320
shutdown_lifetime 10 seconds
acl SSL_ports port 443 563 21
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 22 # sftp
acl Safe_ports port 443 563 # https, snews
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl Safe_ports port 4004 # Radii website download site uses this port
acl Safe_ports port 10000 # Webmin
Note that the above two ports are within the 1024-65535 range. No need
to configurethem in.
Post by Rich549
acl Safe_ports port 900 # Swat
acl Safe_ports port 82 # Pacejet request - test site hosted on HTTP 82
acl Safe_ports port 81 # Image plus test server (hepplewhite)
acl CONNECT method CONNECT
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
#http_access deny BlacklistedSites StoresAllow
If the above were enabled any blacklisted site would get an auth popup
from the StoresAllow requiring %LOGIN. This may be part of your problem.
You need to sort out a policy logic order***.
Post by Rich549
http_access allow OK_Unauthenticated
http_access allow StaticIPWhitelist
acl auth proxy_auth REQUIRED
http_access deny !auth
http_access allow InetAllow
#http_access allow StoresAllow
http_access allow localhost manager
http_access deny all
This block...
Post by Rich549
acl ftp proto FTP
#http_access allow ftp
#http_access allow CONNECT Safe_ports
http_access deny manager
http_reply_access allow all
icp_access allow all
.. to here can be erased in 3.5.
Post by Rich549
forwarded_for off
***

I recommend something like this:

http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports

# whitelists that dont need authenticating first
http_access allow OK_Unauthenticated
http_access allow StaticIPWhitelist

# followed by auth (a blacklist against un-authenticated people)
acl auth proxy_auth REQUIRED
http_access deny !auth

# blacklists applied even if they login
http_access deny BlacklistedSites StoresAllow

# then where authenticated users can go
http_access allow localhost manager
http_access allow InetAllow
http_access allow StoresAllow
http_access deny all



After the above changes, if you are still having issues please try
testing the group helper manually from the command line. The input it is
expecting a username followed by one space then the group name being
tested. It should return OK (user in group) ERR (user not in group) or
BH (internal error).

NOTE you are not using the -S helper option so the username part is
actually the full DOMAIN\user syntax from NTLM.

If it turns out to be not working you can also try with the -d option to
get a debug trace about what the helper is doing.


Differences since 2.7 that may be affecting the helper:
* LDAP version bumped from v2 to v3
* LDAP over TLS support added - it may or may not need using

Most importantly: Windows 7+ all use Kerberos auth by default and
Windows8+ have NTLM actually removed from the OS - it may actually be
the NTLM auth check which is failing.

HTH
Amos
Rich549
2015-02-11 12:35:19 UTC
Permalink
*Ok, I've made all of the advised changes and it still didn't work.

I've just tried pasting my helper command into command prompt and it just
seems to hang. I tried the following:*

-----
D:\Squid>d:/Squid/lib/squid/ext_ldap_group_acl.exe -d -S -b
"ou=Domain_Groups,dc=domain-uk,dc=com" -f %g=Internet_Users -h
srvham09.domain-uk.com

D:\Squid>d:/Squid/lib/squid/ext_ldap_group_acl.exe -d -S -b
"ou=Domain_Groups,dc=domain-uk,dc=com" -f Internet_Users -h
srvham09.domain-uk.com

D:\Squid>d:/Squid/lib/squid/ext_ldap_group_acl.exe -d -b
"ou=Domain_Groups,dc=domain-uk,dc=com" -f %u "Internet_Users" -h
srvham09.domain-uk.com

D:\Squid>d:/Squid/lib/squid/ext_ldap_group_acl.exe -d -b
ou=Domain_Groups,dc=domain-uk,dc=com -f %u %g=Internet_Users -h
srvham09.domain-uk.com

D:\Squid>d:/Squid/lib/squid/ext_ldap_group_acl.exe -v 2 -d -b
ou=Domain_Groups,dc=domain-uk,dc=com -f Internet_Users -h
srvham09.domain-uk.com
-----

*I'm also seeing the following in the cache.log each time I try to access a
webpage as a user who is a member of Internet_Users, is this anything to be
concerned about?*
----
2015/02/11 12:21:47 kid1| helperOpenServers: Starting 1/80 'ntlm_fake_auth'
processes
2015/02/11 12:21:47 kid1| WARNING: no_suid: setuid(0): (22) Invalid argument
----

*And now, just to top things off I have errors in the cache.log stating:*

d:/Squid/lib/squid/ext_ldap_group_acl.exe: (6) No such device or address

*Quickly losing faith :-(*



--
View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-3-5-1-NTLM-and-LDAP-tp4669661p4669727.html
Sent from the Squid - Users mailing list archive at Nabble.com.
Amos Jeffries
2015-02-12 08:49:56 UTC
Permalink
Post by Rich549
*Ok, I've made all of the advised changes and it still didn't work.
I've just tried pasting my helper command into command prompt and it just
seems to hang. I tried the following:*
The helper takes -b string as the base DN and appends the dynamic -f
string to it.

You have configured this as the ACL test:

# Allow Members of Internet Users To Anywhere Not Explicitly Denied
acl InetAllow external internet_domain_group Internet_Users


So the "Internet_Users" is the name of the group being checked for using
the internet_domain_group helper. %g will always be "Internet_Users"
when testing this ACL, %u will change with each user login as its their
username value.
Post by Rich549
-----
D:\Squid>d:/Squid/lib/squid/ext_ldap_group_acl.exe -d -S -b
"ou=Domain_Groups,dc=domain-uk,dc=com" -f %g=Internet_Users -h
srvham09.domain-uk.com
That queries server srvham09.domain-uk.com for:

ou=Domain_Groups,dc=domain-uk,dc=com,Internet_Users=Internet_Users
Post by Rich549
D:\Squid>d:/Squid/lib/squid/ext_ldap_group_acl.exe -d -S -b
"ou=Domain_Groups,dc=domain-uk,dc=com" -f Internet_Users -h
srvham09.domain-uk.com
That queries server srvham09.domain-uk.com for:

ou=Domain_Groups,dc=domain-uk,dc=com,Internet_Users
Post by Rich549
D:\Squid>d:/Squid/lib/squid/ext_ldap_group_acl.exe -d -b
"ou=Domain_Groups,dc=domain-uk,dc=com" -f %u "Internet_Users" -h
srvham09.domain-uk.com
-f only takes one argument so that queries servers Internet_Users and
srvham09.domain-uk.com for:

ou=Domain_Groups,dc=domain-uk,dc=com,<username>
Post by Rich549
D:\Squid>d:/Squid/lib/squid/ext_ldap_group_acl.exe -d -b
ou=Domain_Groups,dc=domain-uk,dc=com -f %u %g=Internet_Users -h
srvham09.domain-uk.com
-f only takes one argument so that queries servers %g=Internet_Users and
srvham09.domain-uk.com for:

ou=Domain_Groups,dc=domain-uk,dc=com,Internet_Users=Internet_Users
Post by Rich549
D:\Squid>d:/Squid/lib/squid/ext_ldap_group_acl.exe -v 2 -d -b
ou=Domain_Groups,dc=domain-uk,dc=com -f Internet_Users -h
srvham09.domain-uk.com
That queries server srvham09.domain-uk.com using LDAPv2 for:

ou=Domain_Groups,dc=domain-uk,dc=com,Internet_Users



Do any of the above LDAP syntaxes look right to you?
I suspect you want some code like ou=%g in the filter.


Only you know what the actual AD directory structure is, and I'm not
very clued up on what the LDAP kv-pairs mean sorry. So that is just a
guess that I hope will point you in the right direction.
Post by Rich549
-----
*I'm also seeing the following in the cache.log each time I try to access a
webpage as a user who is a member of Internet_Users, is this anything to be
concerned about?*
----
2015/02/11 12:21:47 kid1| helperOpenServers: Starting 1/80 'ntlm_fake_auth'
processes
Squid starts helpers incrementally as they are needed nowdays. You can
expect 80 of those mostly when you restart Squid. Messages about dying
helpers are the sign of problems.
Post by Rich549
2015/02/11 12:21:47 kid1| WARNING: no_suid: setuid(0): (22) Invalid argument
Thats okay, I think. Squid is attempting to drop security privileges but
can't. Some OS seem to do it automatically then complain when its done
explicitly.
Post by Rich549
----
*And now, just to top things off I have errors in the cache.log stating:*
d:/Squid/lib/squid/ext_ldap_group_acl.exe: (6) No such device or address
That is a OS error being hit by the helper. Probably when it tries to
contact the LDAP servers "Internet_Users" or "%g=Internet_Users" in your
tests above.


Amos

Loading...