Bug 12769

This is probably caused by https://git.ipfire.org/?p=ipfire-2.x.git;a=commit;h=976b84ccb6303a172416e904b7dc431c7b8c46c4 .

Do you think it is related to the name change that I made or to the version change?

Having looked more closely at your error message it looks like it is missing the http-message perl module which has the http::request and http:response modules in it which look like they are now needed.

I could have a go at picking this up and adding in the http-message module and testing the built iso on my vm and see if it solves the problem, unless anyone thinks it is not as simple as adding the extra module but requires changes in the pakfire coding itself.

I have had a look through the Makefile.pl for libwww-perl-6.60 and libwww-perl-5.803 and I have found a section on runtime prerequisites which is hugely different between the two in terms of number of modules required.

The old version had 6 requirements listed. The new one has 27 requirements listed. Some of those may already come from existing modules but some don't.

I will pick this up and have a go at adding in any required runtime packages and rebuilding it and see how pakfire then works when I install it in my vm testbed.

The previous patch I created should be reverted.

After a bit of further investigation I have figured it out.

The following modules have always been used in the pakfire functions.pl

HTTP::Response;
HTTP::Headers;
HTTP::Message;
HTTP::Request;

and they were a part of libwww-perl

From version 6.0 (2011) a number of bundled modules were separated out from libwww

Extract from Changelog for 6.0
"    - Unbundled all modules not in the LWP:: namespace from the distribution.
        The new broken out CPAN distributions are
        File-Listing, HTML-Form, HTTP-Cookies, HTTP-Daemon, HTTP-Date,
        HTTP-Message, HTTP-Negotiate, Net-HTTP, and WWW-RobotRules.
        libwww-perl-6 require these to be installed."

HTTP-Message now provides the Response, Headers, Message and Request modules and therefore needs to be installed.

I will create a new patch for libwww-perl together with HTTP-message.

It is not clear to me if the other broken out modules are required for IPFire's use of libwww-perl or not. I will try initially with only adding in HTTP-Message.

From my check I only found the pakfire functions.pl file having the HTTP modules in it so I believe that libwww-perl is only used for that. If I have that wrong and it is used elsewhere, let me know so I can include that in my testing but also check for any other modules that might be needed.

Further update, I have found that libwww-perl's LWP::UserAgent is also used in ids-functions.pl and in squid accounting

So I will also test out the ids with an updated patch. I will see how easy it is to test out the squid accounting, it is an addon that I have not used before.


I have also see that the HTTP-Date module needs to be installed as it is used in the Update Accelerator.  I don't use that as all my distribution updates are done via https, so someone else will need to test that when I have my updated patch available.

Hello Adolf,
hello Arne,

I reverted the changes regarding libwww-perl in my temporary branch:

https://git.ipfire.org/?p=people/pmueller/ipfire-2.x.git;a=commit;h=12f867444d5e19844dfa2e67752cd5720870294d
https://git.ipfire.org/?p=people/pmueller/ipfire-2.x.git;a=commit;h=15de7a78feecdfe2fed7ab27d3d8182d8ffa27b3

@Arne: Please merge this branch into next. It also contains Bind updated by Matthias, which is somewhat security sensitive, so it should go into Core Update 164 as well IMHO.

@Adolf: Thank you in advance for working on this! :-)

Thanks, and best regards,
Peter Müller

Did anyone try to remove the lines starting with "use HTTP::*"? I do not find that this is being used anywhere in the code, so this might fix the problem already.

Some of the HTTP::? modules are used inside the LWP:UserAgent module from libwww-perl.

I added in HTTP-Message based on the original error message and that solved that one but then pakfire failed on another place with a missing module. So I then added that and pakfire then failed on the next missing module.
I then decided to add in all the broken out modules and the build is going at the moment and I will then see if that works successfully. It should.

If someone can identify any of the additional modules that aren't needed they can always be removed later.

I didn't fancy doing a complete build and IPFire installation and pakfire check nine times to incrementally add each module that has been broken out.

Additionally to those broken out ones there is also now a run requirement for Try-Tiny because that came up as an error of a missing module in one of my build/installation/test steps. Probably another change somewhere in the long history of this module. Just found it in the Changelog. It came in as a requirement in version 6.17 in 2017 related to a bug fix.

Hopefully later today I will be able to give good news on pakfire fully working again. Will also have a look at IPS and try and have a look at testing out squid-accounting.

@Peter, glad to help. I felt honour bound anyway to work on it as I raised the original patch that caused the problem. :-)

It took longer than I had thought as after adding in all the additional modules it still came up with an error message.

Eventually figured out that it was failing related to https.

The LWP::Protocol::https module was broken out of libwww-perl in version 6.02
I don't know how it worked in the past but now this module either has to have the ssl_opts option for LWP::UserAgent set to define the ca certs director or file or the Mozilla-CA module needs to be installed.

It seemed sensible to use the CA certs already within IPFire, so I tried that but my coding was obviously not good as I still ended up with an https failure.

Then I installed Mozilla-CA module and then I was able to end up with a working pakfire with libwww-perl yesterday.

I have had some further thoughts on the ssl_opts coding and I think I may understand the error I made. So I am trying again with coding adjustments to the pakfire functions.pl and ids-functions.pl files and with Mozilla-CA not installed.
If this doesn't work then I will come back for further help.

Created attachment 976 [details]
diff of pakfire functions.pl

I am coming back for help.

Everything I have tried so far except for having the module Mozilla-CA installed results in the following error message in the IPFire pakfire logs.

pakfire: DOWNLOAD INFO: HTTP-Status-Code: 500 - 500 Can't verify SSL peers without knowing which Certificate Authorities to trust

This error message comes from LWP::Protocol::https at line 33

The attached diff shows the changes I made to the pakfire functions.pl file

I can't tell if the problem is my coding to tell LWP::UserAgent where to find the ca certs file or if it just doesn't work with the way that cerst are stored in IPFire in the ca-bundle.crt which is what cert.pem links to.

Any help gratefully accepted on how to deal with this.

It feels like I am so near yet so far.

Hello Adolf,

I've sent a patchset to the development mailing list to fix the issue with the new libwww-perl (LWP::UserAgent).

As they have written on their CPAN site, the author has moved all the shipped dependencies with 6.02 into own perl modules or dropped them entirely in favour of the original modules of their developers.

So a lot of new packages have to be added to get them all back (litte site-note, libwww does not provider support for HTTPS by default anymore and requires therfore an own module).

Next I also strugled a bit with the SSL verification issue you posted here and found a solution without using the Mozilla::CA module.

LWP::UserAgent allows to directly specify the CA file and if done so everything worked well again.

I only had to adjust the code of two perl scripts (pakfire functions file and ids functions file).

https://patchwork.ipfire.org/project/ipfire/patch/20220203111613.13632-7-stefan.schantl@ipfire.org/

Best regards,

-Stefan

Oh forgot to post the fix for pakfire:

https://patchwork.ipfire.org/project/ipfire/patch/20220203111613.13632-8-stefan.schantl@ipfire.org/

@Arne, please check the patchset and merge into the next.

Thanks in advance,

-Stefan

Hi Stefan,

Yes, I just saw your patch set.

Your code changes were very similar to mine but it looks like I had them in the incorrect place and missing one option.

As @Stefan has provided the patchset to fix this bug I have changed the assignee to him.

See: https://git.ipfire.org/?p=ipfire-2.x.git;a=commit;h=53f7dc7620bfb51e8b1daa8c727206425e06d87a et al.

Resetting this back to MODIFIED, since the patches have already been merged for Core Update 165, and once there is a testing announcement for it, it will go to ON_QA.

Thanks to all who contributed here.

https://blog.ipfire.org/post/ipfire-2-27-core-update-165-is-available-for-testing

https://blog.ipfire.org/post/ipfire-2-27-core-update-165-released