Bug 13032

Summary: BorgBackups: error with python msgpack version
Product: IPFire Reporter: Jon <jon.murphy>
Component: ---Assignee: Adolf Belka <adolf.belka>
Status: CLOSED FIXED QA Contact:
Severity: Major Usability    
Priority: - Unknown - CC: michael.tremer, peter.mueller
Version: 2   
Hardware: unspecified   
OS: Unspecified   

Description Jon 2023-01-18 03:05:01 UTC
I get this error when trying to run Borg:

```
[root@ipfire BorgBackups] # borg init --encryption=none /mnt/ssd/BorgBackups/ipfireBU
You do not have a supported version of the msgpack python package installed. Terminating.
This should never happen as specific, supported versions are required by our setup.py.
Do not contact borgbackup support about this.
```
Comment 1 Jon 2023-01-18 03:48:02 UTC
as a guess, this may be related to python msgpack being at 1.0.4 and Borg being at 1.2.0.

I think Borg needs to be at 1.2.1 (or later) to work with msgpack 1.0.4.

see:
https://github.com/borgbackup/borg/blob/1.2.3/docs/changes.rst#version-123-2022-12-24

recommend searching for "msgpack".
Comment 2 Michael Tremer 2023-01-18 10:46:31 UTC
Good catch. This might tell us how many users this package has :)

@Adolf: Would you like to pick this one up?
Comment 3 Adolf Belka 2023-01-18 12:30:16 UTC
I will pick this up.

I was a bit surprised initially to see this bug because I know that borg requires a certain version of msgpack and usually test that borg works okay in my vm testbed systems after doing a version update.

I realised what happened here. I did the update of python to 3.10.8 and updated a huge number of python packages that were older at the same time. This went into CU172.
So msgpack was just one of something like 80 python packages and I forgot about testing msgpack with borg. Everything just built successfully and I stopped at that point, other than testing a fresh install of IPFire to test all the WUI pages.

Apologies for missing that and I will do an update of borg to put it back in line with the version status.

Good that someone caught this :-))
Comment 4 Michael Tremer 2023-01-18 13:04:06 UTC
No worries. It happens.

What might be useful (and we have started doing this in some packages now) is adding a note to either LFS file so that next time, you will remember - or somebody else will see the note that there is a relationship between those two packages.
Comment 5 Adolf Belka 2023-01-20 20:00:10 UTC
I built borgbackup-1.2.3 and installed the .ipfire package into my vm testbed and ran borg -V and got an error message.

Did the build a second time to ensure that there had not been any hiccup and the same error message occurred which was

-bash-5.2$ borg -V
Traceback (most recent call last):
  File "/usr/bin/borg", line 33, in <module>
    sys.exit(load_entry_point('borgbackup==1.2.3', 'console_scripts', 'borg')())
  File "/usr/bin/borg", line 25, in importlib_load_entry_point
    return next(matches).load()
StopIteration

Checked the issues section of borgbackup Github and there is nothing there with a similar message.

Tried building borgbackup-1.2.2 and this worked normally after installing the .ipfire package.


Checked out the various python3 packages used for borgbackup and found python3-packaging has an updated version. python3-packaging is a required package for borgbackup but no specific version numbe3r is specified. I thought that maybe the newer borgbackup version might need a newer version of python3-packaging so I tried building borgbackup-1.2.3 with python-packaging-23.0 and installed both .ipfire packages in my vm testbed.

Based on the above I will do a final build again with borgbackup-1.2.2 and submit that as a patch while in parallel I try and figure out what needs to be changed for borgbackup-1.2.3 (maybe I need to raise it as an issue in the borgbackup Github issues section unless anyone else has any ideas why I am getting the above error message.

I tried running borg --debug but the error message was the same - no extra information.
Comment 6 Jon 2023-01-20 22:19:44 UTC
found this (it might help):

Is there more than one ".egg-info" in a Borg directory on the ipfire device?

https://github.com/pypa/setuptools/issues/2390
"The StopIteration occurs when importlib_load_entry_point fails to find the relevant package."

"It turns out where was a directory ".egg-info" in the package's directory, probably from a previous install."
Comment 7 Jon 2023-01-20 22:45:08 UTC
the eggs I found:

```
[root@ipfire ~] # find / -iname "*egg-info*" 2>/dev/null
/usr/lib/python3.10/site-packages/borgbackup-1.2.0-py3.10.egg-info
/usr/lib/python3.10/site-packages/speedtest_cli-2.1.3-py3.10.egg-info
[root@ipfire ~] # 
```
Comment 8 Adolf Belka 2023-01-21 21:43:09 UTC
Thanks @Jon for the hint. That helped me find the solution.

The rootfile for borgbackup had the directory for the egg-info commented out so it is not removed when an uninstall is run.

For borgbackup-1.2.2 and earlier this made no difference as the code found the correct egg-info directory.

However in borgbackup-1.2.3 it no longer likes there being more than one egg-info directory even if the directory has the version number in the name and the old versions are empty.

Clearing all the old versions out enabled the 1.2.3 version to work correctly.

So patch created for the update. I have also made borgbackup specific versions of the pak files (install.sh etc) and have added a line to the install.sh file to remove all borgbackup egg-info directories before extracting the files into place.

Tested this with and without old egg-info directories and it works successfully. The old directories are removed and then the install puts in place only the current egg-info directory.

I also added comments to both borgbackup and python3-msgpack lfs files about the version requirements of borgbackup and did an update of the python3-packaging file.

Patch submitted to dev mailing list and patchwork.

https://lists.ipfire.org/pipermail/development/2023-January/015282.html
https://patchwork.ipfire.org/project/ipfire/list/?series=3329
Comment 9 Jon 2023-01-22 16:52:36 UTC
Excellent!  I am glad it helped.  Thank you!
Comment 10 Adolf Belka 2023-01-27 11:29:15 UTC
Patch has been committed into next.

https://git.ipfire.org/?p=ipfire-2.x.git;a=commit;h=b8a598fc92afa86f03212398f21b64efa427236c
Comment 11 Adolf Belka 2023-02-06 14:05:52 UTC
Core Update 173 Testing released

https://blog.ipfire.org/post/ipfire-2-27-core-update-173-is-available-for-testing
Comment 12 Adolf Belka 2023-02-06 14:12:45 UTC
Tested out borg list path-to-repo and it now works in CU173 Testing whereas it didn't in CU172

Also tested out the command that didn't work for @jon with CU172

borg init --encryption=none /mnt/ipfireBU

Confirmed that fix works.
Comment 13 Jon 2023-02-06 19:50:12 UTC
All works A-OK!  Yay!  Thank you!

```
[root@ipfireAPU ~] # borg init -v --encryption=none /mnt/ssd/borgbackup1
Initializing repository at "/mnt/ssd/borgbackup1"
Encryption NOT enabled.
Use the "--encryption=repokey|keyfile" to enable encryption.
Synchronizing chunks cache...
Archives: 0, w/ cached Idx: 0, w/ outdated Idx: 0, w/o cached Idx: 0.
Done.
[root@ipfireAPU ~] # echo $?
0
[root@ipfireAPU ~] # 
```

Tested with:
APU4d4
IPFire 2.27 (x86_64) - Core-Update 173 Development Build: master/fa2f6cb6