Discussion:
[squid-users] ICAP and Allow 204 Header
Gilles Bardouillet
2016-01-25 17:28:28 UTC
Permalink
Hi,

I'm using SQUID with CAS ICAP Server but I have one issue :

* for some images, squid receive icap error as ICAP_ERR_OTHER
* I noticed that for all these errors, Squid dont send the HTTP header
Allows 204
* I read the code and find the Allow 204 header _is only set when
preview is enabled_.

My icap conf activated preview and preview size as follow :
icap_preview_enable on
icap_preview_size 1024

I read that the preview size value can be overwritten by OPTIONS
requests, so can give me some details, hints in order to find why some
pictures dont offer preview and then fails ?

Thanks,
Gilles.
Alex Rousskov
2016-01-25 21:32:40 UTC
Permalink
Post by Gilles Bardouillet
* for some images, squid receive icap error as ICAP_ERR_OTHER
It may be useful to know more details about that ICAP error. What ICAP
response, if any, does Squid receive when it generates ICAP_ERR_OTHER?
Post by Gilles Bardouillet
* I noticed that for all these errors, Squid dont send the HTTP header
Allows 204
Allow:204 is not an HTTP header field. It is an ICAP header field.
Post by Gilles Bardouillet
* I read the code and find the Allow 204 header _is only set when
preview is enabled_.
Are you sure? Several factors affect ICAP Allow:204 request header
presence. Preview availability should not be one of them because
Allow:204 is about 204 responses _outside_ of Preview. See RFC 3507
Section 4.6.
Post by Gilles Bardouillet
icap_preview_enable on
icap_preview_size 1024
IIRC, Squid ignores icap_preview_size in squid.conf (a bug). The ICAP
service OPTIONS response determines the Preview size (subject to an
internal limit of 64KB).
Post by Gilles Bardouillet
I read that the preview size value can be overwritten by OPTIONS
requests, so can give me some details, hints in order to find why some
pictures dont offer preview and then fails ?
See RFC 3507 Section 4.5 for details on how Preview is negotiated. If
you think Squid violates the ICAP protocol, please file a bug report
with the corresponding capture of ICAP messages (from and to Squid).

As for ICAP 204 outside of Preview, I believe Squid can offer to support
that ICAP response if all of the checks below are successful:

* the origin server OPTIONS response includes Allow:204;
* the message content length is known at the ICAP request time; and
* the message content length does not exceed 64KB.

If you prefer to analyze the code, see
Adaptation::Icap::ModXact::shouldAllow204() and
Adaptation::Icap::ModXact::canBackupEverything().


HTH,

Alex.
Gilles Bardouillet
2016-01-26 09:46:29 UTC
Permalink
Hi,

I'm using SQUID with CAS ICAP Server but I have one issue :

* for some images, squid receive icap error as ICAP_ERR_OTHER
* I noticed that for all these errors, Squid dont send the HTTP header
Allows 204
* I read the code and find the Allow 204 header _is only set when
preview is enabled_.

My icap conf activated preview and preview size as follow :
icap_preview_enable on
icap_preview_size 1024

I read that the preview size value can be overwritten by OPTIONS
requests, so can give me some details in order to find why some pictures
dont offer preview and then fails ?

Thanks,
Gilles.
Gilles Bardouillet
2016-01-28 16:46:46 UTC
Permalink
Sorry for the response form but I dont received the Alex email, so I
tried below to recompose the thread discussion
Post by Alex Rousskov
/I'm using SQUID with CAS ICAP Server but I have one issue : />//>/* for some images, squid receive icap error as ICAP_ERR_OTHER /
It may be useful to know more details about that ICAP error. What ICAP
response, if any, does Squid receive when it generates ICAP_ERR_OTHER?
Here is some details from debug mode :

2015/12/09 11:32:11.786 kid3| 93,5| ModXact.cc(653) parseMore: have 182
bytes to parse [FD 32;Rr/w job924]
2015/12/09 11:32:11.786 kid3| 93,5| ModXact.cc(654) parseMore:
ICAP/1.0 200 OK
X-Apparent-Data-Types: JPG
Service: CAS 1.3.1.1(170722)
Service-ID: avscanner
ISTag: "56680096"
Encapsulated: req-body=0
Date: Wed, 09 Dec 2015 10:32:19 GMT


2015/12/09 11:32:11.786 kid3| 93,5| ModXact.cc(749) parseHeaders: parse
ICAP headers
2015/12/09 11:32:11.786 kid3| 93,5| ModXact.cc(1079) parseHead: have 182
head bytes to parse; state: 0
2015/12/09 11:32:11.786 kid3| 93,5| ModXact.cc(1094) parseHead: parse
success, consume 182 bytes, return true
2015/12/09 11:32:11.786 kid3| 93,3|
../../../src/base/AsyncJobCalls.h(177) dial:
Adaptation::Icap::Xaction::noteCommRead threw exception: Invalid ICAP
Response
2015/12/09 11:32:11.786 kid3| 93,4| Xaction.cc(514) setOutcome:
ICAP_ERR_OTHER

Do you need more ?
Post by Alex Rousskov
/* I noticed that for all these errors, Squid dont send the HTTP header />/Allows 204 /
Allow:204 is not an HTTP header field. It is an ICAP header field.
Right
Post by Alex Rousskov
/* I read the code and find the Allow 204 header _is only set when />/preview is enabled_. /
Are you sure? Several factors affect ICAP Allow:204 request header
presence. Preview availability should not be one of them because
Allow:204 is about 204 responses _outside_ of Preview. See RFC 3507
Section 4.6.
Right, preview is only used for Allow 204 In and not Out
My case is about Allow 204 out.


here is the source code from 3.5.13 fromModXact.cc:
const bool allow204in = preview.enabled(); // TODO: add shouldAllow204in()
const bool allow204out = state.allowedPostview204 = shouldAllow204();
....
else if (allow204out)
allowHeader = "Allow: 204\r\n";
Post by Alex Rousskov
/My icap conf activated preview and preview size as follow : />/icap_preview_enable on />/icap_preview_size 1024 /
IIRC, Squid ignores icap_preview_size in squid.conf (a bug). The ICAP
service OPTIONS response determines the Preview size (subject to an
internal limit of 64KB).
My ICAP server (CAS) dont send any Preview size in OPTIONS response :-(
Post by Alex Rousskov
/I read that the preview size value can be overwritten by OPTIONS />/requests, so can give me some details, hints in order to find why some />/pictures dont offer preview and then fails ? /
See RFC 3507 Section 4.5 for details on how Preview is negotiated. If
you think Squid violates the ICAP protocol, please file a bug report
with the corresponding capture of ICAP messages (from and to Squid).
As for ICAP 204 outside of Preview, I believe Squid can offer to support
* the origin server OPTIONS response includes Allow:204;
* the message content length is known at the ICAP request time; and
* the message content length does not exceed 64KB.
Thanks, I will check theses things.
Post by Alex Rousskov
If you prefer to analyze the code, see
Adaptation::Icap::ModXact::shouldAllow204() and
Adaptation::Icap::ModXact::canBackupEverything().
HTH,
Alex.
Regards,
Gilles.
Alex Rousskov
2016-01-28 22:54:28 UTC
Permalink
Post by Gilles Bardouillet
Sorry for the response form but I dont received the Alex email,
You may want to check your email server. It is rejecting my emails.
Post by Gilles Bardouillet
2015/12/09 11:32:11.786 kid3| 93,5| ModXact.cc(653) parseMore: have 182
bytes to parse [FD 32;Rr/w job924]
ICAP/1.0 200 OK
X-Apparent-Data-Types: JPG
Service: CAS 1.3.1.1(170722)
Service-ID: avscanner
ISTag: "56680096"
Encapsulated: req-body=0
Date: Wed, 09 Dec 2015 10:32:19 GMT
Adaptation::Icap::Xaction::noteCommRead threw exception: Invalid ICAP
Response
The Encapsulated header is invalid because it indicates an adapted HTTP
request without headers. Your ICAP service appears to be broken.

Alex.
Gilles Bardouillet
2016-02-08 17:01:49 UTC
Permalink
OK,

I located my issue in Adaptation::Icap::ModXact::canBackupEverything()
with the TheBackupLimit (64k)

I tried to change the value of SQUID_TCP_SO_RCVBUF by without success :-(

This value is set with BodyPipe::MaxCapacity;
How can I set this MaxCapacity ?

Regards,
Gilles.
Post by Gilles Bardouillet
Sorry for the response form but I dont received the Alex email, so I
tried below to recompose the thread discussion
/I'm using SQUID with CAS ICAP Server but I have one issue : />//>/*
for some images, squid receive icap error as ICAP_ERR_OTHER /
It may be useful to know more details about that ICAP error. What ICAP
response, if any, does Squid receive when it generates ICAP_ERR_OTHER?
2015/12/09 11:32:11.786 kid3| 93,5| ModXact.cc(653) parseMore: have
182 bytes to parse [FD 32;Rr/w job924]
ICAP/1.0 200 OK
X-Apparent-Data-Types: JPG
Service: CAS 1.3.1.1(170722)
Service-ID: avscanner
ISTag: "56680096"
Encapsulated: req-body=0
Date: Wed, 09 Dec 2015 10:32:19 GMT
parse ICAP headers
2015/12/09 11:32:11.786 kid3| 93,5| ModXact.cc(1079) parseHead: have
182 head bytes to parse; state: 0
2015/12/09 11:32:11.786 kid3| 93,5| ModXact.cc(1094) parseHead: parse
success, consume 182 bytes, return true
2015/12/09 11:32:11.786 kid3| 93,3|
Adaptation::Icap::Xaction::noteCommRead threw exception: Invalid ICAP
Response
ICAP_ERR_OTHER
Do you need more ?
/* I noticed that for all these errors, Squid dont send the HTTP
header />/Allows 204 /
Allow:204 is not an HTTP header field. It is an ICAP header field.
Right
/* I read the code and find the Allow 204 header _is only set when
/>/preview is enabled_. /
Are you sure? Several factors affect ICAP Allow:204 request header
presence. Preview availability should not be one of them because
Allow:204 is about 204 responses _outside_ of Preview. See RFC 3507
Section 4.6.
Right, preview is only used for Allow 204 In and not Out
My case is about Allow 204 out.
const bool allow204in = preview.enabled(); // TODO: add
shouldAllow204in()
const bool allow204out = state.allowedPostview204 = shouldAllow204();
....
else if (allow204out)
allowHeader = "Allow: 204\r\n";
/>/icap_preview_enable on />/icap_preview_size 1024 /
IIRC, Squid ignores icap_preview_size in squid.conf (a bug). The ICAP
service OPTIONS response determines the Preview size (subject to an
internal limit of 64KB).
My ICAP server (CAS) dont send any Preview size in OPTIONS response :-(
/I read that the preview size value can be overwritten by OPTIONS
/>/requests, so can give me some details, hints in order to find why
some />/pictures dont offer preview and then fails ? /
See RFC 3507 Section 4.5 for details on how Preview is negotiated. If
you think Squid violates the ICAP protocol, please file a bug report
with the corresponding capture of ICAP messages (from and to Squid).
As for ICAP 204 outside of Preview, I believe Squid can offer to support
* the origin server OPTIONS response includes Allow:204;
* the message content length is known at the ICAP request time; and
* the message content length does not exceed 64KB.
Thanks, I will check theses things.
If you prefer to analyze the code, see
Adaptation::Icap::ModXact::shouldAllow204() and
Adaptation::Icap::ModXact::canBackupEverything().
HTH,
Alex.
Regards,
Gilles.
Continue reading on narkive:
Loading...