[squid-users] Caching ChromeOS update files

Alex Rousskov

2018-10-03 23:50:39 UTC

Post by Kevin Byers
I am trying to setup squid to specifically cache ChromeOS updates for
Chromebooks but I am currently stuck and lacking the proper knowledge
to solve the issue.
Issue #1: The update files are not being cached by squid.
TCP_MISS_ABORTED/206 messages occur in the squid access.log. I haven't
seen a HIT yet.
Issue #2: The update process on the Chromebooks seems to stall when
they are using squid as the proxy. The Chromebook will show the update
progress as a percentage increasing in number, and eventually it stops
changing. After waiting a while with no changes, the Chromebook will
stop the update process.
A) Chromebooks update by fetching a single file from dl.google.com.
B) The update files can be up to about 500 megabytes.
C) Chromebooks always request a partial download (HTTP 206) even if
they are downloading the full file from the beginning.
D) Chromebooks will resume downloading the file from a prior partial download.
E) The HTTP response headers for the files have age:0 and
cache-control: public,max-age=86400. There is no "partial" attribute.
Do any of these above things make it impossible to cache the update files?

Not necessarily.

As you already know, Squid does not support caching for partial
responses to Range requests (HTTP 206), but you can tell Squid to
request (and cache) the whole update object using a non-Range request
(HTTP 200) and then dole out its ranges as needed (HTTP 206). You are
already using range_offset_limit for that.

If Chromebook range requests are sequential (and start from the
beginning of the update object), this may work well.

You need to figure out where exactly things fail. For example:

1. On the first Chromebook request, does Squid convert Chromebook GET
Range request into a regular GET request (for the whole object; no Range
header)?

2. On the second Chromebook request, does Squid find the previously
fetched whole object in cache? Does Squid consider that cached entry fresh?

If you cannot figure this out, paste HTTP request and response headers
for relevant messages received _and_ sent by Squid (at least 8 messages
for the above two items), and somebody here will probably spot the
problem. Your "debug_options ALL,2" should log these headers to cache.log.

HTH,

Alex.

Post by Kevin Byers
Here's the squid configuration that I am using: https://pastebin.com/m15g84xv
Essentially, I increased the cache file size limits to 1 GB and
allowed only Chromebook IP traffic to squid.
I am only caching the files in RAM so there is no hard disk cache.
I created a refresh pattern to keep the update files cached for 7 days.
The Chromebooks are using a PAC file which only sends dl.google.com
traffic to squid, so squid is only dealing with dl.google.com traffic.
I am using squid 3.5.27 on ubuntu 18.04 LTS server edition VM with 2
vCPUs and 8 GB RAM.