Hello, although we have talked about this at length, I am disappointed to find out that this is not working very well. First of all, the update is lacking the new ruleset file. After copying that into my test system, I now have a ruleset provider highlighted in red. Its name has disappeared. The only thing that is left is its handle. In red. It is still enabled, I can activate and deactivate it. This is obviously not at all what we want when we intend to remove a provider. What we need is: * The entry on the web UI has to go * If the removed providers were the only providers left, the IPS needs to be disabled * Any files in the filesystem have to be removed so that we won't waste any disk space on dead rules Otherwise, I don't quite understand what the red line is supposed to tell people?! This is a blocker bug that prevents us from releasing Core Update 185 into testing and therefore needs to be fixed urgently.
Testing out the suricata changes in CU185 testing highlighted a problem. Doing an upgrade with my existing suricata providers (abuse.ch and emerging threats) updated without any problem. I then cloned a new CU184 vm with the above existing rules and added PT Attack into it and selected its rules. Then I ran the CU185 Testing update and the WUI page basically stuck at the running the CU185 and post scripts section. I checked via the console and the update had completed with no errors in the update logfile. However the WUI page fails to connect. It eventually times out if I try to refresh it. I tried this twice more with and without PT Attack in the suricata providers. Whenever PT Attack was included the WUI page stops functioning in the update. Then on a vm that was timing out for the WUI page I stopped suricata via the console. The WUI page then became accessible again. I have tested this a second time and confirmed that a non responding WUI becomes responding again when suricata is stopped. I have no idea what is causing the problem but I have been able to reproduce it several times now with my vm systems.
After further checking I have found the following result. This is on a vm machine that had normal providers (abuse.ch and emerging threats) and after update to CU185 the IPS WUI page was working. However if the suricata service is running and I then go to another WUI page such as the DNS server page and add another entry then when saving the entry the screen stops responding and an no response - timeout message comes back. I tried leaving it for a few minutes and then trying to refresh the screen again but it stayed with the not responding - timeout message. I then stopped suricata from the console and the DNS Server WUI page came back again immediately on refresh. I tried this a few times and it is reproducible. It looks like something in the suricata updates that were done (removing old providers or the suricata-7 install) is stopping other WUI pages from getting a response. Has anyone else been able to reproduce this effect?
What happens when you run "update-ids-ruleset" on the console? Any output?
Created attachment 1491 [details] This is the output I get from running update-ids-ruleset on the console This is the output I get from running update-ids-ruleset. In terms of a timed out WUI page it doesn't change anything. I have found that if I have a WUI page showing the no response - timed out message then if I run the IPFire WUI URL link from fresh then I get the WUI main page. I can then change to the page I was on and it displays okay. However doing an action causes the same no response - timed out message. Will put some screenshots together for what I am finding and attach them in this bug.
This has already been reported here: > https://bugzilla.ipfire.org/show_bug.cgi?id=13632 So I am right in considering this a blocker for the c185 update. I however don't get this on my machine. We might not even have introduced this recently.
*** This bug has been marked as a duplicate of bug 13632 ***
I am re-opening this bug as I don't believe it is the same as bug13632. Bug13632 occurred in CU184 and CU185 and is releted to the emerging threats ruleset. This bug occurs for the first time in CU185. It occurs also with a fresh install of CU185 and does not require the emerging threats ruleset to be activated. I will provide screenshots and a sequence of how I get this bug to occur.
Created attachment 1496 [details] Initial suricata screen after installing CU185 from nightly iso The problem I am experiencing is not related to the upgrade process from 184 to 185 as it occurs when I do a fresh install of CU185 from nightly iso. Default install with red and green. No restore or other changes made to the fresh install. This is the initial IPS screen after installation.
Created attachment 1497 [details] IPS screen after adding provider and enabling IPS on red & green I then added the abuse.ch SSLBL Blacklist Rules, selected the single rule option, then selected the enable IPS, enable Red & enable Green then pressed the Save button.
Created attachment 1498 [details] Browser screen after about 15 seconds About 15 seconds after pressing the Save button the browser screen changed to this. Pressing the Try Again button came back with the same screen after some time.
Created attachment 1499 [details] Main screen shown after re-issuing url for IPFire If I entered my IPFire bookmark for https://ipfire.domain.name:444 then the main screen is shown again.
Created attachment 1500 [details] IPS screen showing that it was started Selecting the IPS screen then shows that it was successfully started, even though the WUI screen had the no response timeout message.
@Stefan: Anything on this? As far as I can see this is blocking the release and further testing.
Created attachment 1501 [details] After adding a second IPS provider When I selected a second provider and pressed the Save button the no response - timeout message was not shown. After that first time with that no response - timeout message on the IPS page it worked fine. However going back and repeating the previous steps caused the same thing.
Created attachment 1502 [details] DNS screen after the previous IPS screens I then changed to the DN provider screen. Without changing anything I pressed the Save button.
Created attachment 1503 [details] About 15 secs aftre pressing DNS Save button this message On the previous comment I said that I changed nothing and pressed the Save button. I actually disabled the ISP DNS servers checkbox and then pressed the Save button. About 15 seconds after pressing the Save button on the DNS page I got the no response - timeout message. Again trying the Try Again button did not work but re-issuing the IPFire URL caused the main page to be shown again and I could then change to the DNS page again.
Created attachment 1504 [details] DNS page after re-issuing IPFire URL The DNS page then shows that the change was successfully made, even though the WUI browser page showed the no response - timeout message. Any change I make to the DNS providers page, when I press the Save button the no response - timeout message comes up.
Created attachment 1505 [details] Disable the IPS screen If I disable the IPS screen and press Save then the change is made and the IPS stops. I can then go back to the DNS page and make any changes I want without ever seeing the no response - timeout message. If I go back and re-enable the IPS and press the Save button then the IPS is started and if I try to make any changes on the DNS page I again get the no response - timeout message again.
Hello Adolf, thanks for all your testing and detailed report. I'm totally able to reproduce this issue with the timing out request after starting up suricata. The strange thing is, if you are also logged in via SSH this session is also terminated, so it seems that suricata cuts off all active connections during startup. I did some further testing with "rolling back" to suricata 6 (I'v multiple suricata binaries on my IPFire system) and there is no such behavior - So this definitely can be nailed down to suricata 7! Next step would be to check if there is any kind of new option in the suricata.yaml config file for this or a bug report in the suricata bug tracker.
I did a diff of the default suricata 6 and suricata 7 config: https://nopaste.ipfire.org/view/mKfhrhSu A very interesting part is the following one: # Exception Policies # # Define a common behavior for all exception policies. -# Default is ignore. +# In IPS mode, the default is drop-flow. For cases when that's not possible, the +# engine will fall to drop-packet. To fallback to old behavior (setting each of +# them individually, or ignoring all), set this to ignore. # All values available for exception policies can be used, and there is one -# extra option: auto - which means ignore (in Suricata 7.0 this changes to drop -# in IPS mode). -# -# Exception policy values are: drop-packet, drop-flow, reject, bypass, -# pass-packet, pass-flow, auto or ignore (disable). +# extra option: auto - which means drop-flow or drop-packet (as explained above) +# in IPS mode, and ignore in IDS mode. Exception policy values are: drop-packet, +# drop-flow, reject, bypass, pass-packet, pass-flow, ignore (disable). exception-policy: auto When checking the IPFire suricata configuration there is no option "exception-policy" set, so this defaulted to "ignore" in suricata 6 and to "drop-flow" since suricata 7 which causes the changed/new behavior during startup. When adding a line "exception-policy: ignore" the page timeout is gone, also my SSH session keeps alive. So this could be a possible solution, but when reading trough the comment there might be a better one - any thoughts? I also noticed a lot of other differences, when reading trough the diff, which also needs to be checked in case something other important (new parsers etc.) is missing in our current IPFire suricata config.
Hi Stefan, I can also confirm that adding exception-policy: ignore into the suricata.yaml file stops the problem happening. Looking through the suricata.yaml.in diff from 6 to 7 I came to the conclusion that I can't help to identify if there is a better way than the ignore option. I don't understand enough of what all the options are doing.
(In reply to Stefan Schantl from comment #20) > I did a diff of the default suricata 6 and suricata 7 config: > > https://nopaste.ipfire.org/view/mKfhrhSu Wow, that is a lot of things that we are not using in Suricata 7 then. I suggest we update the configuration before we push out the update. Please apply the following changes: * Enable all decoders, please keep the SIP decoder enabled, too. This just looks confusing in the diff. * Please enable Landlock, but don't blanco-allow /etc and /usr. That perverts the entire reason. It should be possible to figure out what files we need to list here. * Disable SWF Decompression (https://redmine.openinfosecfoundation.org/issues/5632) * Update all the rest, so that next time we diff the configuration we won't see any changed comments and so on again. * I don't we want to copy the AF-XDP and DPDK sections to keep our configuration as short as possible. > A very interesting part is the following one: > > # Exception Policies > # > # Define a common behavior for all exception policies. > -# Default is ignore. > +# In IPS mode, the default is drop-flow. For cases when that's not > possible, the > +# engine will fall to drop-packet. To fallback to old behavior (setting > each of > +# them individually, or ignoring all), set this to ignore. > # All values available for exception policies can be used, and there is one > -# extra option: auto - which means ignore (in Suricata 7.0 this changes to > drop > -# in IPS mode). > -# > -# Exception policy values are: drop-packet, drop-flow, reject, bypass, > -# pass-packet, pass-flow, auto or ignore (disable). > +# extra option: auto - which means drop-flow or drop-packet (as explained > above) > +# in IPS mode, and ignore in IDS mode. Exception policy values are: > drop-packet, > +# drop-flow, reject, bypass, pass-packet, pass-flow, ignore (disable). > exception-policy: auto I believe that we should try it with "pass-packet" here. That will simply skip processing that one packet that caused the exception and will allow Suricata to process all following packets of a flow. pass-flow might also be an option, but I think that we should try pass-packet first. If we ever were to look at defaulting to pass-flow, we might also want to use "bypass" here to take the load off of Suricata. (In reply to Adolf Belka from comment #21) > I can also confirm that adding Thank you for confirming :)
I just tried out my vm with pass-packet and pass-flow. In both cases the original bug I raised is fixed. Looking in the suricata logs with pass-packet defined I found the following suggesting some further modification to the .yaml config file is required. Error parsing stream.midstream-policy from config file. "(null)" is not valid for this exception policy. See our documentation for a list of all possible values Additionally I saw the following which doesn't require fixing now but will need to be addressed when the move from suricata-7 to suricata-8 occurs. Multipline "include" fields at the same level are deprecated and will not work in Suricata 8, please move to an array of include files: line: 14 With pass-flow in place then the following messages are shown:- flow actions not supported for flow.memcap-policy, defaulting to "pass-packet" flow actions not supported for defrag.memcap-policy, defaulting to "pass-packet"
CU186 Testing has been issued. https://www.ipfire.org/blog/ipfire-2-29-core-update-186-is-available-for-testing
My comment 24 was incorrect. It should have said:- CU185 Testing has been issued. https://www.ipfire.org/blog/ipfire-2-29-core-update-185-is-available-for-testing
CU185 has been released. https://www.ipfire.org/blog/ipfire-2-29-core-update-185-released