Bug 12515

Summary: libloc produces memory leaks ( in perl interface )
Product: Location Database Reporter: Bernhard Bitsch <bbitsch>
Component: liblocAssignee: Stefan Schantl <stefan.schantl>
Status: CLOSED FIXED QA Contact: Peter Müller <peter.mueller>
Severity: Crash    
Priority: Will affect an average number of users CC: agcanum, michael.tremer, m_kaya13, rolf
Version: unspecified   
Hardware: i686   
OS: Unspecified   
Attachments: a temporary work-around ( /var/ipfire/location-functions.pl )

Description Bernhard Bitsch 2020-10-28 20:25:12 UTC
Using "small" HW for IPfire ( ALIX ), I noticed odd behaviour in the WUI ( location-block.cgi ). There were listed only first ~70 countries.
Enabling the debug output showed "database file ... can not be read".
Analyzing the permissions and updating the database didn't help.

For investigation of the problem I just copied the beginning of the .cgi to a separate perl script and changed it just to print lines of country codes and names.
This script also threw the error, but before there was an error "libloc: loc_database_free: Could not unmap network nodes section: Invalid argument". First I thought of a "usual" memory limitation error of my system.
Further investigations about the source location where the error is generated showed, my limitations only raised the probability of the error.

What I've found exactly:
- most functions in perl module location-functions call init() first.
- there is no "deinit" called/processed.
- this leaves the amount of the database handle

I haven't found (yet) the reason for this situation, neither the reason for the loc_database_free error. Maybe it is urged by the memory management of perl/perl_XS or by the C libloc functions.

What I've done so far:
- I reduced the init() calls in location-functions to one. The database handle is a variable of the module, which is initialised by a call to init() with 'require "${General::swroot}/location-functions.pl";'.
- all functions use this module wide handle.
- this doesn't cure the missing 'deinit', but the memory block for the handle is allocated only once in the program. The memory block is deallocated with normal program end.

Hope this helps for deeper investigations.
Comment 1 Bernhard Bitsch 2020-10-30 09:19:25 UTC
Created attachment 792 [details]
a temporary work-around
( /var/ipfire/location-functions.pl )
Comment 2 Michael Tremer 2020-11-04 14:35:22 UTC
(In reply to Bernhard Bitsch from comment #1)
> Created attachment 792 [details]
> a temporary work-around

Bebima, you should absolutely post these things as a patch to the list. Attaching a changed file does not help because the changes cannot be seen without downloading the file, finding the correct version you used and changed, and nobody can reply to this.

You can avoid generating double work and get bugs fixed quicker.

I am increasing the priority of this ticket, because it seems to affect a large-is number of users and it crashes using the firewall GUI.
Comment 3 Michael Tremer 2020-11-04 14:35:52 UTC
*** Bug 12526 has been marked as a duplicate of this bug. ***
Comment 4 Michael Tremer 2020-11-04 14:36:26 UTC
*** Bug 12517 has been marked as a duplicate of this bug. ***
Comment 5 Bernhard Bitsch 2020-11-04 22:50:43 UTC
(In reply to Michael Tremer from comment #2)
> (In reply to Bernhard Bitsch from comment #1)
> > Created attachment 792 [details]
> > a temporary work-around
> 
> Bebima, you should absolutely post these things as a patch to the list.
> Attaching a changed file does not help because the changes cannot be seen
> without downloading the file, finding the correct version you used and
> changed, and nobody can reply to this.

Michael, I have described the bug as deep as I could do, without reading the whole source of the sources affected. I did not find the real error. Thus I cannot send a patch to correct it.
What I did is a workaround to survive 'til the issue is corrected.
I posted it here for all affected users. It is just easier to cite the bugzilla ticket than repost the work-around every time an user reports this or similiar errors in the community.

The cause of the issue seems to be a memory leak ( as far as I can see ), my changed script ( /var/ipfire/location-functions.pl of core 151 ) reduces the init() calls and thus the memory usage.

- Bernhard
Comment 6 Michael Tremer 2020-11-05 10:30:24 UTC
(In reply to Bernhard Bitsch from comment #5)
> Michael, I have described the bug as deep as I could do, without reading the
> whole source of the sources affected. I did not find the real error. Thus I
> cannot send a patch to correct it.

This is an assumption, but isn't a workaround already a good solution?

The final solution would indeed be what you implemented there. The software itself is not buggy. Some functions are just being called at the wrong time.

> What I did is a workaround to survive 'til the issue is corrected.

Which could have been shipped to users already. Workarounds are acceptable. Obviously if there is a better solution possible, it would be worth shipping that, too.

> I posted it here for all affected users. It is just easier to cite the
> bugzilla ticket than repost the work-around every time an user reports this
> or similiar errors in the community.

The is a bug *tracker*. It tracks the status of things. It is not a place for discussion about proposed patches. It is as simple as that.

I would also *strongly* discourage any user from applying random patches. This is not specifically about this patch, but we have loads of people posting their own software, which will never be patched on Community and people basically ruin their firewalls. We might even consider to ban these users at some point. It is simply dangerous for people to install random software if they do not know what they are doing.
Comment 7 Bernhard Bitsch 2020-11-13 09:03:00 UTC
I've used now some time my 'work-around' as the patches Stephan posted in the devel list.
It works. 

This for the records.
Comment 8 Michael Tremer 2020-11-13 11:03:33 UTC
Thank you for testing & letting us know.

I consider this the solution and not only a work-around.
Comment 9 Peter Müller 2020-11-19 23:49:00 UTC
Resetting this back to ASSIGNED since a bug is MODIFIED only as soon as some commits made it into a repository (please see https://wiki.ipfire.org/devel/bugzilla/workflow for details on this).

Anyway, the patchset in question is tracked here: https://patchwork.ipfire.org/project/ipfire/list/?series=1594
Comment 11 Michael Tremer 2020-11-29 17:35:19 UTC
*** Bug 12542 has been marked as a duplicate of this bug. ***
Comment 12 Rolf Ackermann 2020-12-15 09:19:17 UTC
I can confirm that the work-around solves the problem I had with creating new firewall rules (tried with core 152)
Comment 13 Michael Tremer 2020-12-15 10:03:55 UTC
Great. Thank you for your feedback.