Bug 10445

Hey Erik,

indeed I am very concerned about the problems we are currently experiencing with the new proxy in Core Update 73.

I am currently *not* aware of what the problems actually are. We provided fixes for some of these, but I am not sure if these fixes actually fix the bugs. So please lets try to sort out what is going wrong here...

1) I think the patch you provided solves the problem. This may be caused by our version of openssl. Note: there is still a spelling mistake.

I will apply the patch to our version as soon as possible and we will need to push out an other version of the update.

2) I still don't understand what is going wrong here. What did you configure? Does this message show up all the time? Even if you set the no. of open file descriptors to a very low one (like 4096)?

Thanks for you efforts. I appreciate it very much.

I downloaded the version of the patch that has been applied to trunk from the squid BZR.

While at it, I updated to squid 3.3.10 as well.

http://git.ipfire.org/?p=ipfire-2.x.git;a=commitdiff;h=36b1c19138f9936ae97fac4f94c443593702f22d

Hi Michael,
the tested configurations was a varity between 4096 to 16384, tested on 3 systems (three different users) where the message "kid1| With 65536 file descriptors available" in cache.log attracts the attention. Testing around i see that parsing the configuration file brings up this warning

2013/11/08 15:39:53| Processing: max_filedescriptors 4096
2013/11/08 15:39:53| WARNING: max_filedescriptors disabled. Operating System setrlimit(RLIMIT_NOFILE) is missing.

There are several sites in the web findable which includes this warning (only squid 3.x) but most of them doesn´t makes depper sense for me whats problably causing this problem.

At this time i am searching in several release notes for deprecated compile and configure options, starting from Squid 3.0 --> ftp://ftp.fu-berlin.de/unix/www/squid/archive/3.0/squid-3.0.STABLE21-RELEASENOTES.html to the actual one and i have found a number of possible compile optimizations but nothing which leads seriously to the problem from point 2.

The corresponding line in squid source is in cache_cf.cc line 709

#if !HAVE_SETRLIMIT || !defined(RLIMIT_NOFILE)
    if (Config.max_filedescriptors > 0) {
        debugs(0, DBG_IMPORTANT, "WARNING: max_filedescriptors disabled. Operating System setrlimit(RLIMIT_NOFILE) is missing.");
    }
#elif USE_SELECT || USE_SELECT_WIN32
    if (Config.max_filedescriptors > FD_SETSIZE) {
        debugs(0, DBG_IMPORTANT, "WARNING: max_filedescriptors limited to " << FD_SETSIZE << " by select() algorithm.");
    }
#endif


Make now a clean build with an updated LFS file for squid. Will go then for further tests.

Greetings

Erik

I have made now a couple of tests with 3.3.10, 3.3.9 up to 3.2.13 and was playing around with some sugguestions from the changelogs with the compile options but unfortunatly the problem 2) persists. An interessting thing was if i delete the entries '--with-maxfd="65536"' '--with-filedescriptors=65536' the build_log shows 

checking Default FD_SETSIZE value... 1024
checking for setrlimit... yes
checking Maximum number of filedescriptors we can open... 4096

so the check for setrlimit works but Squid doesn´t work probperly without the option "--with-filedescriptors=", may the size of 65536 is too high ? Will compile it again with e.g. "32768" and check it then again.

Some other points:
the build_log gives out this warnings,

WARNING: Samba smbclient not found in default location. basic_smb_auth may not work on this machine
WARNING: Samba wbinfo not found in default location. ext_wbinfo_group_acl may not work on this machine
configure: WARNING: cppunit does not appear to be installed. squid does not require this, but code testing with 'make check' will fail.
WARNING: Translation toolkit was not detected.

they are not relevant for the actual problem but i wanted to list them here anyway. 

Also some compile options have changed, a list from ftp://ftp.fu-berlin.de/unix/www/squid/archive/3.0/squid-3.0.STABLE21-RELEASENOTES.html#s7 .

--enable-htcp

    HTCP protocol support is now built by default. Use --disable-htcp to build without it.

--enable-carp

    CARP support is now built by default. --disable-carp can be used to build without it.

     
--enable-epoll

    Deprecated. Automatic checks will enable best I/O loop method available.

--enable-poll

    Deprecated. Automatic checks will enable best I/O loop method available.

--with-maxfd=N

    Replaced by --with-filedescriptors=N

    Override maximum number of filedescriptors. Useful if you build as another user who is not privileged to use the number of filedescriptors you want the resulting binary to support .

So far from here.

Erik

P.S. The patch http://git.ipfire.org/?p=ipfire-2.x.git;a=blob;f=src/patches/squid-3.3.10-optional-ssl-options.patch;h=f6a108c5f1dbb00da599ab1c92891f68c7c3abb2;hb=36b1c19138f9936ae97fac4f94c443593702f22d differes with the lines for 3.3.9 but also for 3.3.10 .

Regarding the file descriptor issue, please check out #10449.

It has been reported to me, that the SSL warning issue has been resolved. There are still no other reports about the "filedescriptors not configurable" issue and we are using 3.x already, so this is not a new regression and should not block Core Update 73 any longer.

Created attachment 157 [details]
setrlimit patch

Erik, please have a look at the attached patch file.

It appears that the setrlimit function is not checked in the configure script.

Don't forget to run ./bootstrap.sh after applying the patch.

If this patch works for you, please send it to the squid developers with reference to this bug report.

Hi Michael,
this patch solves it for me, great work.

I have submit also an bugreport with your patch to Squid´s Bugzilla --> http://bugs.squid-cache.org/show_bug.cgi?id=3970 .

Erik

In here --> http://bazaar.launchpad.net/~squid/squid/3-trunk/revision/13144 the diff from Amos Jeffries causing the delivered bugreport of the FD problem can be found. 

Erik

Applied: http://git.ipfire.org/?p=ipfire-2.x.git;a=commitdiff;h=78c2b230d42ba20858d7f4ce115a0c7669aca8e1

Fixed with Core Update 74.