Bug 12838 - rrd files in /var/log/rrd/collectd/localhost/iptables-filter-HOSTILE/ not being updated
Summary: rrd files in /var/log/rrd/collectd/localhost/iptables-filter-HOSTILE/ not bei...
Status: CLOSED FIXED
Alias: None
Product: IPFire
Classification: Unclassified
Component: --- (show other bugs)
Version: 2
Hardware: unspecified Unspecified
: - Unknown - Minor Usability
Assignee: Adolf Belka
QA Contact:
URL:
Keywords:
: 12823 (view as bug list)
Depends on:
Blocks:
 
Reported: 2022-04-03 12:13 UTC by Adolf Belka
Modified: 2022-04-27 18:14 UTC (History)
4 users (show)

See Also:


Attachments
repair script (545 bytes, application/x-shellscript)
2022-04-04 08:10 UTC, Bernhard Bitsch
Details
corrected repair script (557 bytes, application/x-shellscript)
2022-04-04 11:43 UTC, Bernhard Bitsch
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Adolf Belka 2022-04-03 12:13:33 UTC
I found that my Hostile Networks line in the Firewallhits graph was showing -nan

Another person on the forum had a problem where the graphs was showing an error of invalid argument for ipt_bytes-DROP_HOSTILE.rrd

Checking my rrd directories I found that all were up to date and being regularly updated. However the entries in the /var/log/rrd/collectd/localhost/iptables-filter-HOSTILE/ directory had a date time of Mar 29 15:13, which is when I did my Core Update from 164 to 165.

I tried a reboot and the status us the same.

I stopped collectd and vnstat and renamed the files in the iptables-filter-HOSTILE directory to append .bak to them. Then I restarted collectd and vnstat.

Then I rebooted and I had an error message on the graph saying no such file or directory related to ipt_bytes-DROP_HOSTILE.rrd

Then I renamed the .bak files back to .rrd and rebooted and had the graph again but still with -nan and with the date of Mar 29 15:13 so it looks like these graphs are not being updated.

I am not sure if this is just my system or if other people have this problem but have not noticed.

Would be good to know if it can be reproduced.
Comment 1 Bernhard Bitsch 2022-04-03 20:26:31 UTC
I can reproduce the error.

I can also give an explanation.
collectd.conf tries to collect entries in chain HOSTILE, but the check is made in chain HOSTILE_DROP.
Changing this in collectd.conf and /var/ipfire/graphs.pl brings up the data again.

Affected files 
/etc/collectd.conf:
<plugin iptables>
...
	Chain filter HOSTILE DROP_HOSTILE ->
	Chain filter HOSTILE_DROP DROP_HOSTILE

/var/ipfire/graphs.pl:
in function updatefwhitsgraph
		"DEF:hostile=".$mainsettings{'RRDLOG'}."/collectd/localhost/iptables-filter-HOSTILE/ipt_bytes-DROP_HOSTILE.rrd:value:AVERAGE", ->
		"DEF:hostile=".$mainsettings{'RRDLOG'}."/collectd/localhost/iptables-filter-HOSTILE_DROP/ipt_bytes-DROP_HOSTILE.rrd:value:AVERAGE",


Didn't check whether the graphs/values for week/month/year were present.
Also didn't search for a transport of the old values in xxx-HOSTILE RRDS into the xxx-HOSTILE_DROP RRDs.
Comment 2 Adolf Belka 2022-04-03 20:56:13 UTC
I will give that a try. Had a look through the commits and not sure I can see where that has been changed but will look further tomorrow when I am not feeling so tired.

Thanks for the guidance.
Comment 3 Bernhard Bitsch 2022-04-04 08:10:45 UTC
Created attachment 1030 [details]
repair script
Comment 4 Bernhard Bitsch 2022-04-04 08:12:44 UTC
Comment on attachment 1030 [details]
repair script

A first attempt to solve the problem.
Hope it is complete, correct and helpful.
Comment 5 Adolf Belka 2022-04-04 11:32:33 UTC
I just tried the script on a vm testbed clone. It didn't work for me.

I had read through it beforehand but had missed that the line doing sed on graphs.pl presumes you are in the /var/ipfire/ directory or at least a directory that has a path that knows where graphs.pl is.


Anyway, I manually edited the files on my vm clone system and the graph then worked. It shows 0Bps for the hour, day and week entry as the vm does not get any Hostile Networks through to it because my boundary production IPFire is set to drop them all. The month and year have -nan as the entry.

Anyway this looks to be working so I will no look at creating a patch to fix this and confirm with an install that it actually works before submitting.
Comment 6 Bernhard Bitsch 2022-04-04 11:42:01 UTC
Sorry for the incompleteness. :(

Yes, the sed command must work on /var/ipfire/graphs.pl
I should look closer if I make a resume of my manually changes.
Comment 7 Bernhard Bitsch 2022-04-04 11:43:26 UTC
Created attachment 1031 [details]
corrected repair script
Comment 8 Adolf Belka 2022-04-04 17:49:26 UTC
I have done a build of the changes and installed it on a vm in my testbed system. The graph was created as expected.

I will submit a patch of the changes
Comment 9 Adolf Belka 2022-04-04 18:02:04 UTC
One thing I have noticed is that any backup done with CU165/166 will have the directory iptables-filter-HOSTILE backed up.
Once the directory iptables-filter-HOSTILE_DROP is present then that will be backed up. It could leave some people with an orphaned directory and files. Could always be removed if present by the update.sh script for the CU that this fix is implemented with.
Comment 10 Adolf Belka 2022-04-04 20:36:59 UTC
Patch submitted to dev mailing list and to patchwork.

https://patchwork.ipfire.org/project/ipfire/list/?series=2711
Comment 12 Peter Müller 2022-04-06 17:21:12 UTC
*** Bug 12823 has been marked as a duplicate of this bug. ***
Comment 14 Adolf Belka 2022-04-12 11:15:00 UTC
Upgraded a CU166 vm to CU167 and confirmed that the graphs now work correctly after a short while to fill some data.

The directory
/var/log/rrd/collectd/localhost/iptables-filter-HOSTILE/

should be deleted in the CU update.sh script