[squid-users] Why does this proxy configuration ignore no-cache and no-store?

[Brett]

I'm having some trouble because my 4.0.24-VCS squid proxy is caching requests
that it shouldn't be, breaking the website I'm routing through it.

From the HAR output of the client using the proxy:

Response Headers
Cache-Control
no-cache;no-store
Content-Encoding
gzip
Content-Type
text/html;charset=utf-8
Date
Thu, 27 Sep 2018 22:27:17 GMT
Pragma
no-cache
Server
pache-Coyote/1.1
Vary
Accept-Encoding
Age
24
Warning
110 squid/4.0.24-VCS "Response is stale"
X-Cache
HIT from proxy
Via
1.1 proxy (squid/4.0.24-VCS)
Connection
keep-alive


Note the no-cache;no-store Cache Control headers and then the proxy
returning the result from the cache, and it's awareness of not following
HTTP rules, i.e. "Response is stale"

This would indicate that my configuration is telling the proxy to ignore
these rules. I do have some rules setup for images etc that do override
cache control, but not for html, text etc, which this request was for.
Following is my configuration:

http_port 3128 ssl-bump \
cert=/apps/server_crt.pem key=/apps/server_key.pem \
generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
sslcrtd_program /apps/squid/libexec/security_file_certgen -s
/apps/squid/var/lib/ssl_db -M 4MB sslcrtd_children 8 startup=1 idle=1

acl step1 at_step SslBump1
ssl_bump peek step1
ssl_bump bump all

acl localnet src 10.0.0.0/8 # RFC1918 possible internal network
acl localnet src 172.16.0.0/12 # RFC1918 possible internal network
acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
acl localnet src fc00::/7 # RFC 4193 local private network range
acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged)
machines
acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl Safe_ports port 1025-65535 # unregistered ports
acl CONNECT method CONNECT
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost manager
http_access deny manager
coredump_dir /apps/squid/var/cache
maximum_object_size 10 GB
cache_dir ufs /apps/squid/var/cache/squid 100 16 256
cache_mem 1024 MB
maximum_object_size_in_memory 512 KB
cache_replacement_policy heap LFUDA
range_offset_limit -1
quick_abort_min -1 KB
offline_mode on
http_access allow localnet
http_access allow localhost
http_access deny all
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i \.(gif|png|jpg|jpeg|ico|woff|woff2)$ 10080 90% 43200
override-expire ignore-no-cache ignore-no-store ignore-private
refresh_pattern -i \.(iso|avi|wav|mp3|mp4|mpeg|swf|flv|x-flv)$ 43200 90%
432000 override-expire ignore-no-cache ignore-no-store ignore-private
refresh_pattern -i \.(css|js)$ 1440 40% 40320

I've also tried deleting all of the refresh_pattern statements and I still
get the same outcome. What am I doing wrong?



--
Sent from: http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-Users-f1019091.html

[Amos Jeffries]

Post by Brett
I'm having some trouble because my 4.0.24-VCS squid proxy is caching requests
that it shouldn't be, breaking the website I'm routing through it.

NP: please upgrade your proxy that is a beta release. Squid-4 now has
several stable releases.

Post by Brett
Response Headers
Cache-Control
no-cache;no-store

Is the ';' above part of the HAR format or part of the actual headers
received?

...

Post by Brett
Vary
Accept-Encoding
Age
24
Warning
110 squid/4.0.24-VCS "Response is stale"
X-Cache
HIT from proxy
Via
1.1 proxy (squid/4.0.24-VCS)

...

Post by Brett
Note the no-cache;no-store Cache Control headers and then the proxy
returning the result from the cache, and it's awareness of not following
HTTP rules, i.e. "Response is stale"

If your response contains "Cache-Control: no-cache;no-store" then that
actually means just "Cache-Control: no-cache" which tells Squid it *can*
cache the response.

If the response contains "Cache-Control: no-cache, no-store" then that
actually means just "Cache-Control: no-store". More on that below.

Post by Brett
This would indicate that my configuration is telling the proxy to ignore
these rules. I do have some rules setup for images etc that do override
cache control, but not for html, text etc, which this request was for.

One thing to be aware of "html, text etc" has no meaning to Squid. It is
working strictly from whether the given regex pattern matches against
the full URL string. Including the query-string part.

That means URLs like "http://example.com/some.html?got=.iso" will match
your pattern.
...

Post by Brett
offline_mode on

...

Post by Brett
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i \.(gif|png|jpg|jpeg|ico|woff|woff2)$ 10080 90% 43200
override-expire ignore-no-cache ignore-no-store ignore-private
refresh_pattern -i \.(iso|avi|wav|mp3|mp4|mpeg|swf|flv|x-flv)$ 43200 90%
432000 override-expire ignore-no-cache ignore-no-store ignore-private
refresh_pattern -i \.(css|js)$ 1440 40% 40320

NP: ignore-no-cache is no longer supported since version ~3.2. In
HTTP/1.1 compliant proxies like Squid it actually *prevents* caching
which is counter to most intended uses and better done with
cache/store_miss/send_hit directives allow/deny rules instead.


You are also missing the default refresh_pattern line carefully crafted
to make broken CGI scripts and dynamic content behave according to
RFC2616 caching requirements. Without it such broken content will be
cached for very, very long times.

Please restore this line to the end of your refresh_pattern lines:

refresh_pattern -i (/cgi-bin/\?) 0 0% 0

Post by Brett
I've also tried deleting all of the refresh_pattern statements and I still
get the same outcome. What am I doing wrong?

Three things that I can see:

1) ignore-no-store tells Squid to ignore Cache-Control:no-store headers.

Since no-store overrides no-cache in HTTP semantics which means
"Cache-Control: no-cache, no-store" is just "Cache-Control: no-store"
... you have told Squid to ignore that header entirely.

Removing that option was the right thing to do. But not sufficient by
itself (or removing the whole line ) to immediately change Squid
behaviour because of the below...


2) offline_mode on tells squid not to revalidate anything.

This directive is badly named and does not do what most people think it
does. It is global in effect. Please see the documentation:
<http://www.squid-cache.org/Doc/config/offline_mode/>

My advice is do not use this directive unless you are in the process of
a live server migration between two proxies. In which case debugging
weird traffic behaviour is best left until the procedure is completed
and both the directive and defunct proxy removed from the traffic path.


3) override-expire with an age parameter of 43200 minutes.

The specific response you are asking about is not affected by this. But
others using Expires header will be broken in similar ways when this
setting is applied to them.


Amos

[Amos Jeffries]

Post by Amos Jeffries
You are also missing the default refresh_pattern line carefully crafted
to make broken CGI scripts and dynamic content behave according to
RFC2616 caching requirements. Without it such broken content will be
cached for very, very long times.
refresh_pattern -i (/cgi-bin/\?) 0 0% 0

Oops, paste error. That should have been:

refresh_pattern -i (/cgi-bin/|\?) 0 0% 0


Amos