Bug 10445 - Squid 3.3.9 in testing branch delivers some Problems
Summary: Squid 3.3.9 in testing branch delivers some Problems
Status: CLOSED FIXED
Alias: None
Product: IPFire
Classification: Unclassified
Component: squid (show other bugs)
Version: 2
Hardware: unspecified Unspecified
: - Unknown - - Unknown -
Assignee: Michael Tremer
QA Contact:
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-11-07 07:25 UTC by Erik Kapfer
Modified: 2014-01-12 11:00 UTC (History)
2 users (show)

See Also:


Attachments
Patch for X509 errors (4.68 KB, application/octet-stream)
2013-11-07 07:25 UTC, Erik Kapfer
Details
setrlimit patch (296 bytes, patch)
2013-11-21 22:11 UTC, Michael Tremer
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Erik Kapfer 2013-11-07 07:25:11 UTC
Created attachment 156 [details]
Patch for X509 errors

Hi all,
Squid 3.3.9 which is at this time in testing branch deliveres some problems with the old building instructions.

1)
Invalid error detail name:  X509_V_ERR_DIFFERENT_CRL_SCOPE:

It seems that Squid have problems to parse the X509 errors that are unsupported by the current OpenSSL library.

Warnings displayed in cache.log are:
kid1| WARNING! invalid error detail name: X509_V_ERR_DIFFERENT_CRL_SCOPE
kid1|  parse error while reading template file: /usr/lib/squid/errors/de/error-details.txt
kid1| '/usr/lib/squid/errors/templates/error-details.txt': (2) No such file or directory
kid1| WARNING: failed to find or read error text file error-details.txt
kid1| WARNING! invalid error detail name: X509_V_ERR_DIFFERENT_CRL_SCOPE
kid1| WARNING! invalid error detail name: X509_V_ERR_DIFFERENT_CRL_SCOPE

This is a Bug which was also reported in Squids Bugzilla section --> http://bugs.squid-cache.org/show_bug.cgi?id=3936 .
Meanwhile i changed this Patch for usage in IPFire and the warnings and parse error disappears. The patch can be find in the attachment. 

May there are other possibilities without hardcode the X509 error list but i haven´t find out one at this time.

2) 
WARNING: max_filedescriptors disabled. Operating System setrlimit(RLIMIT_NOFILE) is missing :

The filedescritpors can´t be adjusted over the webinterface, the defined settings are printed into the squid.conf but the cache.log shows 

kid1| With 65536 file descriptors available

no matter which FD maximum was edited in the WUI. A "squid -k parse" brings up the foolowing warnings

| WARNING: max_filedescriptors disabled. Operating System setrlimit(RLIMIT_NOFILE) is missing.

I have found this problem also in the squid users mailinglist --> http://www.squid-cache.org/mail-archive/squid-users/201210/0249.html with a hint that possibly something went wrong while compile time.

3)
WARNING: HTTP requires the use of Via:

a "squid -k parse" prints also the point 3) warning out. But this already happens in 3.1.23, it looks like a configuration warning.

So for the first feedback this is all. Otherwise the updated Squid version works without bigger problems or fatal errors also in "Intercepted Mode". A forum discussion and some other testes can be found in here --> http://forum.ipfire.org/index.php?topic=9237.0 , surley not enough tests but may some others will come soon (hopefully).

Greetings 


Erik
Comment 1 Michael Tremer 2013-11-08 12:50:31 UTC
Hey Erik,

indeed I am very concerned about the problems we are currently experiencing with the new proxy in Core Update 73.

I am currently *not* aware of what the problems actually are. We provided fixes for some of these, but I am not sure if these fixes actually fix the bugs. So please lets try to sort out what is going wrong here...

1) I think the patch you provided solves the problem. This may be caused by our version of openssl. Note: there is still a spelling mistake.

I will apply the patch to our version as soon as possible and we will need to push out an other version of the update.

2) I still don't understand what is going wrong here. What did you configure? Does this message show up all the time? Even if you set the no. of open file descriptors to a very low one (like 4096)?

Thanks for you efforts. I appreciate it very much.
Comment 2 Michael Tremer 2013-11-08 14:53:13 UTC
I downloaded the version of the patch that has been applied to trunk from the squid BZR.

While at it, I updated to squid 3.3.10 as well.

http://git.ipfire.org/?p=ipfire-2.x.git;a=commitdiff;h=36b1c19138f9936ae97fac4f94c443593702f22d
Comment 3 Erik Kapfer 2013-11-08 16:07:33 UTC
Hi Michael,
the tested configurations was a varity between 4096 to 16384, tested on 3 systems (three different users) where the message "kid1| With 65536 file descriptors available" in cache.log attracts the attention. Testing around i see that parsing the configuration file brings up this warning

2013/11/08 15:39:53| Processing: max_filedescriptors 4096
2013/11/08 15:39:53| WARNING: max_filedescriptors disabled. Operating System setrlimit(RLIMIT_NOFILE) is missing.

There are several sites in the web findable which includes this warning (only squid 3.x) but most of them doesn´t makes depper sense for me whats problably causing this problem.

At this time i am searching in several release notes for deprecated compile and configure options, starting from Squid 3.0 --> ftp://ftp.fu-berlin.de/unix/www/squid/archive/3.0/squid-3.0.STABLE21-RELEASENOTES.html to the actual one and i have found a number of possible compile optimizations but nothing which leads seriously to the problem from point 2.

The corresponding line in squid source is in cache_cf.cc line 709

#if !HAVE_SETRLIMIT || !defined(RLIMIT_NOFILE)
    if (Config.max_filedescriptors > 0) {
        debugs(0, DBG_IMPORTANT, "WARNING: max_filedescriptors disabled. Operating System setrlimit(RLIMIT_NOFILE) is missing.");
    }
#elif USE_SELECT || USE_SELECT_WIN32
    if (Config.max_filedescriptors > FD_SETSIZE) {
        debugs(0, DBG_IMPORTANT, "WARNING: max_filedescriptors limited to " << FD_SETSIZE << " by select() algorithm.");
    }
#endif


Make now a clean build with an updated LFS file for squid. Will go then for further tests.

Greetings

Erik
Comment 4 Erik Kapfer 2013-11-11 09:02:43 UTC
I have made now a couple of tests with 3.3.10, 3.3.9 up to 3.2.13 and was playing around with some sugguestions from the changelogs with the compile options but unfortunatly the problem 2) persists. An interessting thing was if i delete the entries '--with-maxfd="65536"' '--with-filedescriptors=65536' the build_log shows 

checking Default FD_SETSIZE value... 1024
checking for setrlimit... yes
checking Maximum number of filedescriptors we can open... 4096

so the check for setrlimit works but Squid doesn´t work probperly without the option "--with-filedescriptors=", may the size of 65536 is too high ? Will compile it again with e.g. "32768" and check it then again.

Some other points:
the build_log gives out this warnings,

WARNING: Samba smbclient not found in default location. basic_smb_auth may not work on this machine
WARNING: Samba wbinfo not found in default location. ext_wbinfo_group_acl may not work on this machine
configure: WARNING: cppunit does not appear to be installed. squid does not require this, but code testing with 'make check' will fail.
WARNING: Translation toolkit was not detected.

they are not relevant for the actual problem but i wanted to list them here anyway. 

Also some compile options have changed, a list from ftp://ftp.fu-berlin.de/unix/www/squid/archive/3.0/squid-3.0.STABLE21-RELEASENOTES.html#s7 .

--enable-htcp

    HTCP protocol support is now built by default. Use --disable-htcp to build without it.

--enable-carp

    CARP support is now built by default. --disable-carp can be used to build without it.

     
--enable-epoll

    Deprecated. Automatic checks will enable best I/O loop method available.

--enable-poll

    Deprecated. Automatic checks will enable best I/O loop method available.

--with-maxfd=N

    Replaced by --with-filedescriptors=N

    Override maximum number of filedescriptors. Useful if you build as another user who is not privileged to use the number of filedescriptors you want the resulting binary to support .

So far from here.

Erik

P.S. The patch http://git.ipfire.org/?p=ipfire-2.x.git;a=blob;f=src/patches/squid-3.3.10-optional-ssl-options.patch;h=f6a108c5f1dbb00da599ab1c92891f68c7c3abb2;hb=36b1c19138f9936ae97fac4f94c443593702f22d differes with the lines for 3.3.9 but also for 3.3.10 .
Comment 5 Michael Tremer 2013-11-21 21:54:01 UTC
Regarding the file descriptor issue, please check out #10449.

It has been reported to me, that the SSL warning issue has been resolved. There are still no other reports about the "filedescriptors not configurable" issue and we are using 3.x already, so this is not a new regression and should not block Core Update 73 any longer.
Comment 6 Michael Tremer 2013-11-21 22:11:14 UTC
Created attachment 157 [details]
setrlimit patch

Erik, please have a look at the attached patch file.

It appears that the setrlimit function is not checked in the configure script.

Don't forget to run ./bootstrap.sh after applying the patch.

If this patch works for you, please send it to the squid developers with reference to this bug report.
Comment 7 Erik Kapfer 2013-11-22 18:03:11 UTC
Hi Michael,
this patch solves it for me, great work.

I have submit also an bugreport with your patch to Squid´s Bugzilla --> http://bugs.squid-cache.org/show_bug.cgi?id=3970 .

Erik
Comment 8 Erik Kapfer 2013-11-23 10:39:34 UTC
In here --> http://bazaar.launchpad.net/~squid/squid/3-trunk/revision/13144 the diff from Amos Jeffries causing the delivered bugreport of the FD problem can be found. 

Erik
Comment 10 Stefan Schantl 2014-01-12 11:00:40 UTC
Fixed with Core Update 74.