Bug 12623

Summary: Proxy problem since core 155 - The proxy server is refusing connections at machine startup
Product: IPFire Reporter: Panos Il <panosst3>
Component: ---Assignee: Matthias Fischer <matthias.fischer>
Status: CLOSED FIXED QA Contact:
Severity: Major Usability    
Priority: - Unknown - CC: hsienhold, leo.zanon, michael.tremer, niva, peter.mueller
Version: 2   
Hardware: x86_64   
OS: All   
Attachments: var-log-messages
var-log-squid-cache_log
Workaround: check and start squid via fcron.hourly
Workaround: check and start squid via fcron.hourly
Workaround: check and start squid via fcron.hourly
Workaround: check and start squid via fcron.hourly

Description Panos Il 2021-05-18 19:14:06 UTC
Since core 155, both on upgraded machines and on a new ipfire machine that was setup using core 155 iso, the proxy is refusing connections at startup. 

If I go to the IPFire web gui, Advanced web proxy configuration and click Save and Restart, then the proxy works normally and accepts connections.

I need to do this after every reboot/ startup.

All of the above are x64 physical machines.

I started a community thread here:
https://community.ipfire.org/t/proxy-problem-after-upgrade-to-core-155-the-proxy-server-is-refusing-connections/5020

It is also mentioned by another user in the thread that the issue is also present in core 156.
Comment 1 Michael Tremer 2021-05-25 11:23:43 UTC
Could you please provide log files? squid will probably tell us why it crashed.
Comment 2 Panos Il 2021-06-01 09:46:50 UTC
Created attachment 902 [details]
var-log-messages

I rebooted and after some time restarted the proxy service for web gui. I attach the /var/log/messages
Comment 3 Michael Tremer 2021-06-01 11:28:38 UTC
Could you please upload /var/log/squid/cache.log, too?
Comment 4 Panos Il 2021-06-01 11:38:53 UTC
Created attachment 903 [details]
var-log-squid-cache_log
Comment 5 Leonardo Zanon 2021-07-22 17:32:49 UTC
Hi,
I got this bug upgrading ipfire from 155 to 157.
(in the version 155 everything worked fine)

In my case, i noticed that enabling log management in GUI (before it was disabled) the service starts automatically during the boot phase, otherwise it doesnt.

Leonardo
Comment 6 Niva 2021-11-18 09:01:20 UTC
I have the same problem for a few months with an installation where several core updates have already been installed. Therefore I installed a system from scratch with Core 160 (without importing a backup). Unfortunately, I have to say that the problem occurs again with the new installation.
After restarting with "netstat -npl" I checked the open ports and determined that the proxy port (8080) is not open.
After "Save and restart" via the GUI, the port 8080 is open and the proxy works again.
I also have the entry:
Making directories in /var/log/cache/01
FATAL: Squid is already running: Found fresh instance PID file (/var/run/squid.pid) with PID 15433
    exception location: Instance.cc(121) ThrowIfAlreadyRunningWith


If this error is present, the missing in the cache log:
(what is available after the manual "save and restart")
Accepting HTTP Socket connections at local=x.x.x.x:8080

It should also be noted that the error occurs after an automatic restart via "Connection Scheduler" has taken place.
Comment 7 Michael Tremer 2021-11-18 11:16:19 UTC
@Matthias: Do you want to have a look at this?
Comment 8 Niva 2021-12-02 07:24:11 UTC
Created attachment 958 [details]
Workaround: check and start squid via fcron.hourly

I tested different settings.
Deactivate cache manager, hard disk cache size set to zero, URL filter deactivated. All without success, after a few days (1-3) the proxy service was not active during the night after an automatic restart (restart via the connection planner).

As a workaround I have now written myself a small script "check-squid" and run it every hour via fcron. This checks whether the proxy is running and starts it again if necessary.
At least that night the script worked. 

I copied the "check-squid" file to /etc/fcron.hourly and set the rights to rwxr-xr-x. After that there should be an entry "CheckSquid: ..." every hour under Logs -> System Logs -> Section:IPFire.
Perhaps this will also help others users as a temporary solution until the problem in the IPFire can be solved itself.
Comment 9 Niva 2021-12-02 07:25:43 UTC
Created attachment 959 [details]
Workaround: check and start squid via fcron.hourly

I tested different settings.
Deactivate cache manager, hard disk cache size set to zero, URL filter deactivated. All without success, after a few days (1-3) the proxy service was not active during the night after an automatic restart (restart via the connection planner).

As a workaround I have now written myself a small script "check-squid" and run it every hour via fcron. This checks whether the proxy is running and starts it again if necessary.
At least that night the script worked. 

I copied the "check-squid" file to /etc/fcron.hourly and set the rights to rwxr-xr-x. After that there should be an entry "CheckSquid: ..." every hour under Logs -> System Logs -> Section:IPFire.
Perhaps this will also help others users as a temporary solution until the problem in the IPFire can be solved itself.
Comment 10 Niva 2021-12-02 07:32:09 UTC
Created attachment 960 [details]
Workaround: check and start squid via fcron.hourly

I tested different settings.
Deactivate cache manager, hard disk cache size set to zero, URL filter deactivated. All without success, after a few days (1-3) the proxy service was not active during the night after an automatic restart (restart via the connection planner).

As a workaround I have now written myself a small script "check-squid" and run it every hour via fcron. This checks whether the proxy is running and starts it again if necessary.
At least that night the script worked.

I copied the "check-squid" file to /etc/fcron.hourly and set the rights to rwxr-xr-x. After that there should be an entry "CheckSquid: ..." every hour under Logs -> System Logs -> Section:IPFire.
Perhaps this will also help others users as a temporary solution until the problem in the IPFire can be solved itself.
Comment 11 Niva 2021-12-02 07:33:01 UTC
Created attachment 961 [details]
Workaround: check and start squid via fcron.hourly

I tested different settings.
Deactivate cache manager, hard disk cache size set to zero, URL filter deactivated. All without success, after a few days (1-3) the proxy service was not active during the night after an automatic restart (restart via the connection planner).

As a workaround I have now written myself a small script "check-squid" and run it every hour via fcron. This checks whether the proxy is running and starts it again if necessary.
At least that night the script worked.

I copied the "check-squid" file to /etc/fcron.hourly and set the rights to rwxr-xr-x. After that there should be an entry "CheckSquid: ..." every hour under Logs -> System Logs -> Section:IPFire.
Perhaps this will also help others users as a temporary solution until the problem in the IPFire can be solved itself.
Comment 12 Niva 2021-12-02 07:35:35 UTC
Sorry for the many entries.
I always got internal server error and tried several times as a result.
Comment 13 Peter Müller 2022-03-20 10:45:43 UTC
For the records: I am heavily using the web proxy, and did not experience this issue so far. In https://community.ipfire.org/t/proxy-problem-after-upgrade-to-core-155-the-proxy-server-is-refusing-connections/5020/34, I posted a proxy configuration I know is working fine, but unfortunately did not get a feedback back then if this error is also appearing with that configuration.

Therefore, may I ask you to try the configuration out, if possible, and report back if that changes the behaviour. That would help us to narrow-down the root cause for this issue. Thanks in advance.
Comment 14 Matthias Fischer 2022-08-22 08:06:01 UTC
In short:

I tried but sorry: I can't reproduce this behaviour.

I'm monitoring the latest 'squid' with 'monit' and never had this issue as well. I never got any messages from 'monit' that 'squid' wasn't running and needed to be restarted.
Comment 15 Hagen Weidlich 2023-01-02 10:58:25 UTC
I‘m not sure whether my problem is really related but after update 172 squid stopped (again). For me the solution was a change in squid.conf regarding cache location. Is it really intended that squid should create the cache under /var/log/cache?

I changed that to /var/cache/squid let it recreate its directory structure and it runs flawlessly again.
Comment 16 Matthias Fischer 2023-01-02 17:43:33 UTC
(In reply to Hagen Weidlich from comment #15)
> I‘m not sure whether my problem is really related but after update 172 squid
> stopped (again). For me the solution was a change in squid.conf regarding
> cache location. Is it really intended that squid should create the cache
> under /var/log/cache?

Yes. This is hardcoded in '/srv/web/ipfire/cgi-bin/proxy.cgi', lines 3250ff.:

...
	if ($proxysettings{'CACHE_SIZE'} > 0) {
		print FILE <<END
maximum_object_size $proxysettings{'MAX_SIZE'} KB
minimum_object_size $proxysettings{'MIN_SIZE'} KB

cache_dir aufs /var/log/cache $proxysettings{'CACHE_SIZE'} $proxysettings{'L1_DIRS'} 256
END
...

> I changed that to /var/cache/squid let it recreate its directory structure
> and it runs flawlessly again.

The next time you save your settings through the GUI, your changes will be overwritten.

jm2c
Comment 17 Hagen Weidlich 2023-01-02 18:43:06 UTC
Thanks for the clarification. I raise a separate request then as it can‘t be the problem here.
Comment 18 Matthias Fischer 2024-01-06 10:29:31 UTC
I'm closing this ticket - no further feedback since last comment "I raise a separate request then..."

I hope this is ok for the OP.