Bug 12438

Summary: Borgbackup - No module named 'borg.testsuite'
Product: IPFire Reporter: charette mtb <charette.mtb>
Component: ---Assignee: Jonatan Schlag <jonatan.schlag>
Status: CLOSED FIXED QA Contact:
Severity: Crash    
Priority: - Unknown - CC: admin, michael.tremer, peter.mueller, p
Version: 2   
Hardware: x86_64   
OS: Linux   
Attachments: detail

Description charette mtb 2020-06-11 19:05:26 UTC 
Created attachment 765 [details]
detail

Hi,

I got this error when running Borg (below).

IPFire 2.25 (x86_64) - Core Update 145 Development Build: master/fead20a9.
Testing version cause previous version of borg was already buggy ( https://bugzilla.ipfire.org/show_bug.cgi?id=12356 )

---

[root@ipfire ~]# borg
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/borg/archiver.py", line 81, in <module>
    from .selftest import selftest
  File "/usr/lib/python3.8/site-packages/borg/selftest.py", line 21, in <module>
    from .testsuite.hashindex import HashIndexDataTestCase, HashIndexRefcountingTestCase, HashIndexTestCase
ModuleNotFoundError: No module named 'borg.testsuite'

---

I've tried to reinstall borgbackup, but same error.
If think "testsuite" folder is missing inside /usr/lib/python3.8/site-packages/borg/.

Thanks
Comment 1 charette mtb 2020-07-26 09:38:22 UTC 
Fixed with Core Update 147, who include borgbackup 1.1.13.
Thanks
Comment 2 Patrick Useldinger 2020-09-09 09:40:58 UTC 
I still get this error on IPFire 2.25 (x86_64) - Core Update 148, cannot initialize a remote repository:

[root@ipfire ~]# borg init --encryption=repokey-blake2 blabla.repo.borgbase.com:repo
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/borg/archiver.py", line 81, in <module>
    from .selftest import selftest
  File "/usr/lib/python3.8/site-packages/borg/selftest.py", line 21, in <module>
    from .testsuite.hashindex import HashIndexDataTestCase, HashIndexRefcountingTestCase, HashIndexTestCase
ModuleNotFoundError: No module named 'borg.testsuite'

[root@ipfire ~]# uname -a
Linux ipfire.localdomain 4.14.184-ipfire #1 SMP Tue Jul 14 13:43:23 GMT 2020 x86_64 AMD GX-412TC SOC AuthenticAMD GNU/Linux
Comment 3 Michael Tremer 2020-09-10 09:02:41 UTC 
Jonatan, can you check if you can reproduce this?
Comment 4 Jonatan Schlag 2020-09-14 11:13:29 UTC 
(In reply to charette mtb from comment #1)
> Fixed with Core Update 147, who include borgbackup 1.1.13.
> Thanks

This version is commited in git, but I currently only get 1.1.11? So where did you get this version?

Jonatan
Comment 5 Jonatan Schlag 2020-09-14 11:16:42 UTC 
(In reply to Patrick Useldinger from comment #2)
> I still get this error on IPFire 2.25 (x86_64) - Core Update 148, cannot
> initialize a remote repository:
> 
> [root@ipfire ~]# borg init --encryption=repokey-blake2
> blabla.repo.borgbase.com:repo
> Traceback (most recent call last):
>   File "/usr/lib/python3.8/site-packages/borg/archiver.py", line 81, in
> <module>
>     from .selftest import selftest
>   File "/usr/lib/python3.8/site-packages/borg/selftest.py", line 21, in
> <module>
>     from .testsuite.hashindex import HashIndexDataTestCase,
> HashIndexRefcountingTestCase, HashIndexTestCase
> ModuleNotFoundError: No module named 'borg.testsuite'
> 
> [root@ipfire ~]# uname -a
> Linux ipfire.localdomain 4.14.184-ipfire #1 SMP Tue Jul 14 13:43:23 GMT 2020
> x86_64 AMD GX-412TC SOC AuthenticAMD GNU/Linux

Works for me with version 1.1.11 on Core 148. 

Command:

cat  /etc/system-release
borg --version
mkdir -p borg-test                                                                                                                     
borg init --encryption=repokey-blake2 borg-test/ 


Please post your version of borg. Thank you
Comment 6 Patrick Useldinger 2020-09-14 11:25:45 UTC 
```
[root@ipfire ~]# cat  /etc/system-release
IPFire 2.25 (x86_64) - core148

[root@ipfire ~]# borg --version
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/borg/archiver.py", line 81, in <module>
    from .selftest import selftest
  File "/usr/lib/python3.8/site-packages/borg/selftest.py", line 21, in <module>
    from .testsuite.hashindex import HashIndexDataTestCase, HashIndexRefcountingTestCase, HashIndexTestCase
ModuleNotFoundError: No module named 'borg.testsuite'

[root@ipfire ~]# mkdir -p borg-test

[root@ipfire ~]# borg init --encryption=repokey-blake2 borg-test/
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/borg/archiver.py", line 81, in <module>
    from .selftest import selftest
  File "/usr/lib/python3.8/site-packages/borg/selftest.py", line 21, in <module>
    from .testsuite.hashindex import HashIndexDataTestCase, HashIndexRefcountingTestCase, HashIndexTestCase
ModuleNotFoundError: No module named 'borg.testsuite'

[root@ipfire ~]# pakfire list|grep borg
Name: borgbackup

[root@ipfire ~]# pakfire status
core-list.db         100.00% |=============================>|   903.00 B
Core-Version: 2.25-x86_64
Core-Update-Level: 148
Last update: 9d 39m 46s ago
Last core-list update: 10h 59m 55s ago
Last server-list update: 10h 59m 57s ago
Last packages-list update: 10h 59m 56s ago
Core-Update available: no
Package-Updates available: 0
Reboot required: no

[root@ipfire ~]# head $(which borg)
#!/usr/bin/python3
# EASY-INSTALL-ENTRY-SCRIPT: 'borgbackup==1.1.13','console_scripts','borg'
__requires__ = 'borgbackup==1.1.13'
```
Comment 7 Patrick Useldinger 2020-09-14 11:32:55 UTC 
Not sure if this matters (core 147 vs core 148):

```
[root@ipfire borg]# cat /etc/*release*
NAME="IPFire"
VERSION="2.25"
ID=ipfire
VERSION_ID=2
PRETTY_NAME="IPFire 2.25 (x86_64) - core147"
ANSI_COLOR="0:31"
IPFire 2.25 (x86_64) - core148
```

Also module `testsuite` is there, `selftest` is not:

```
[root@ipfire borg]# ls -a1 /usr/lib/python3.8/site-packages/borg
.
..
algorithms
archive.py
archiver.py
cache.py
chunker.cpython-38-x86_64-linux-gnu.so
compress.cpython-38-x86_64-linux-gnu.so
constants.py
crypto
fuse.py
hashindex.cpython-38-x86_64-linux-gnu.so
helpers.py
__init__.py
item.cpython-38-x86_64-linux-gnu.so
locking.py
logger.py
lrucache.py
__main__.py
nanorst.py
paperkey.html
patterns.py
platform
__pycache__
remote.py
repository.py
selftest.py
shellpattern.py
upgrader.py
_version.py
version.py
xattr.py
```
Comment 8 Patrick Useldinger 2020-09-16 13:13:00 UTC 
FWIW, I cloned the testsuite from the github repository since it only contains python files - but then I got other errors:
```
[root@ipfire borg]# borg
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/borg/archiver.py", line 81, in <module>
    from .selftest import selftest
  File "/usr/lib/python3.8/site-packages/borg/selftest.py", line 22, in <module>
    from .testsuite.crypto import CryptoTestCase
  File "/usr/lib/python3.8/site-packages/borg/testsuite/crypto.py", line 3, in <module>
    from ..crypto.low_level import AES256_CTR_HMAC_SHA256, AES256_OCB, CHACHA20_POLY1305, UNENCRYPTED, \
ImportError: cannot import name 'AES256_CTR_HMAC_SHA256' from 'borg.crypto.low_level' (/usr/lib/python3.8/site-packages/borg/crypto/low_level.cpython-38-x86_64-linux-gnu.so)
```
Comment 9 Thomas Void 2020-10-09 22:34:47 UTC 
Possible fix:
-------------

As I wrote here
https://community.ipfire.org/t/borgbackup-no-module-named-borg-testsuite/2412/6
I also tried to copy the testsuite from the git repo but I got the same error as Patrick (Comment 8)

I then tried to copy the testsuite directory from my working x86 box via scp to the test bench x64 and it's working fine.
Comment 10 Patrick Useldinger 2020-10-10 07:24:09 UTC 
Still doesn't work for me (core 150 in the meantime)

[root@ipfire ~]# /usr/local/bin/pakfire remove borgbackup


PAKFIRE INFO: Packages to remove:
PAKFIRE INFO: borgbackup 	 - 790.00 KB
PAKFIRE INFO: Is this okay? [y/N]
y
PAKFIRE REMV: borgbackup: Decrypting...
PAKFIRE REMV: borgbackup: Removing files and running post-removing scripts...
PAKFIRE REMV: borgbackup: Finished.

[root@ipfire ~]# /usr/local/bin/pakfire install borgbackup
PAKFIRE RESV: borgbackup: Resolving dependencies...
PAKFIRE RESV: borgbackup: Dependency is already installed: python3-llfuse
PAKFIRE RESV: borgbackup: Dependency is already installed: python3-msgpack


PAKFIRE INFO: Packages to install:
PAKFIRE INFO: borgbackup 	 - 790.00 KB

PAKFIRE INFO: Total size: 	 ~ 790.00 KB

PAKFIRE INFO: Is this okay? [y/N]
y

PAKFIRE INST: borgbackup: Decrypting...
PAKFIRE INST: borgbackup: Copying files and running post-installation scripts...
PAKFIRE INST: borgbackup: Finished.

[root@ipfire ~]# borg -h
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/borg/archiver.py", line 81, in <module>
    from .selftest import selftest
  File "/usr/lib/python3.8/site-packages/borg/selftest.py", line 22, in <module>
    from .testsuite.crypto import CryptoTestCase
  File "/usr/lib/python3.8/site-packages/borg/testsuite/crypto.py", line 3, in <module>
    from ..crypto.low_level import AES256_CTR_HMAC_SHA256, AES256_OCB, CHACHA20_POLY1305, UNENCRYPTED, \
ImportError: cannot import name 'AES256_CTR_HMAC_SHA256' from 'borg.crypto.low_level' (/usr/lib/python3.8/site-packages/borg/crypto/low_level.cpython-38-x86_64-linux-gnu.so)
Comment 12 Michael Tremer 2020-10-10 11:50:54 UTC 
Merged into next.