Bug 13969 - CU201 : restore URLFilter with Toulouse list
Summary: CU201 : restore URLFilter with Toulouse list
Status: ON_QA
Alias: None
Product: IPFire
Classification: Unclassified
Component: --- (show other bugs)
Version: 2
Hardware: all All
: - Unknown - Minor Usability
Assignee: pscar13
QA Contact:
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2026-05-12 13:29 UTC by pscar13
Modified: 2026-09-13 20:34 UTC (History)
1 user (show)

See Also:


Attachments
Screenshot crash install backup iso (277.99 KB, image/jpeg)
2026-08-18 18:39 UTC, pscar13
Details
urlfilter.cgi (54.91 KB, image/jpeg)
2026-08-24 17:42 UTC, pscar13
Details

Note You need to log in before you can comment on or make changes to this bug.
Description pscar13 2026-05-12 13:29:31 UTC
Between March 2nd and 12th, 2026, the University of Toulouse modified its lists by replacing the directories
- aggressive
- proxy
- ads
- mail
- drugs
- porn
- violence

Through symbolic links
Therefore, any attempt to restore IPFire containing this new list Before an update of the list has been done

will fail :

[root@ipfireTest bin]# ./backup.pl restore /var/ipfire/backup/2026-03-12-1612.ipf
...
tar: var/ipfire/urlfilter/blacklists/aggressive: Cannot create symlink to ‘agressif’: File exists
tar: var/ipfire/urlfilter/blacklists/proxy: Cannot create symlink to ‘redirector’: File exists
tar: var/ipfire/urlfilter/blacklists/ads: Cannot create symlink to ‘publicite’: File exists
tar: var/ipfire/urlfilter/blacklists/mail: Cannot create symlink to ‘forums’: File exists
tar: var/ipfire/urlfilter/blacklists/drugs: Cannot create symlink to ‘drogue’: File exists
tar: var/ipfire/urlfilter/blacklists/porn: Cannot create symlink to ‘adult’: File exists
tar: var/ipfire/urlfilter/blacklists/violence: Cannot create symlink to ‘agressif’: File exists
...
tar: Exiting with failure status due to previous errors
[backup.pl:93] echo 'Could not extract backup'
Could not extract backup
[backup.pl:94] return 1
Comment 1 pscar13 2026-05-12 13:38:15 UTC
can be added
rm -f /var/ipfire/urlfilter/blacklists/*
Before extract
Comment 2 pscar13 2026-05-12 13:41:02 UTC
rm -rf /var/ipfire/urlfilter/blacklists/*
Comment 5 pscar13 2026-08-18 18:39:47 UTC
Created attachment 1746 [details]
Screenshot crash install backup iso

I have just tested the CU203 backup ISO.

And it fail  because of this URLFilter bug not taken into account when restoring the backup from the ISO:

In the file 

// src/installer/hw.c
1441: int hw_restore_backup(const char* output, const char* backup_path, const char* destination) {
1442:     char command[STRING_SIZE];
1443:
1444:     snprintf(command, sizeof(command), "/bin/tar xzpf %s -C %s "
1445:         "--exclude-from=%s/var/ipfire/backup/exclude --exclude-from=%s/var/ipfire/backup/exclude.user",
1446:         backup_path, destination, destination, destination);
1447:     int rc = mysystem(output, command);
1448:
1449:     if (rc)
1450:         return -1;
1451:
1452:     return 0;
1453: }
Comment 6 Adolf Belka 2026-08-20 15:41:38 UTC
The issue with the backup iso is that the default install includes a very okd version of the blacklists.tar.gz 

The fix will be to update the blacklists file in the urlfilter config for a fresh install so that the updated symlinked directories no longer exist in a fresh install and therefore will no longer give a problem with a backup restore.
Comment 7 pscar13 2026-08-21 08:18:22 UTC
(In reply to Adolf Belka from comment #6)
> The issue with the backup iso is that the default install includes a very
> okd version of the blacklists.tar.gz 
> 
> The fix will be to update the blacklists file in the urlfilter config for a
> fresh install so that the updated symlinked directories no longer exist in a
> fresh install and therefore will no longer give a problem with a backup
> restore.

I agree that this is the case for restoring a backup after installation. I had indeed suggested this workaround here:

https://community.ipfire.org/t/ipfire-202-test-restoring-a-201-backup-after-upgrading-to-202/15775/14?u=pscar13

However, when restoring from the backup ISO, we don't have this option.

The ISO contains directories, while the `.ipf` archive contains symlinks with the same names. As a result, `tar` fails and the restoration cannot be completed (see the screenshot).

There is no workaround in this case.
Comment 8 pscar13 2026-08-21 09:59:45 UTC
Voici la version en texte brut, adaptée à Bugzilla :

Steps to reproduce the bug

1. Install CU203.

2. Update the Toulouse blacklist:
   Network -> URL Filter -> URL filter maintenance
   Select "Univ. Toulouse" as the download source.
   Click "Update now".
   Wait for the list download to complete.

3. Create an ISO backup:
   System -> Backup -> Generate ISO
   Click the "Backup" icon.

4. Export the generated backup file:
   ipfire-2.29-core203-x86_64-YYYY-MM-DD-HHMM.iso

5. Reinstall IPFire using this ISO.
   When prompted "Do you want to restore the backup?", select "Yes".

6. The following error message is displayed:
   "An error occurred ..."

   This is followed by:
   "Setup has failed ..."

7. Press Alt+F2 to switch to the second console and view the output.
Comment 9 Adolf Belka 2026-08-21 19:38:26 UTC
You have misunderstood what i have written in my fix proposal.

My proposal was not a workaround.

The current install has a very old version of blacklists.tar.gz which is installed and has the fixed old directories that are now symlinked.

I will be adding a new version of blacklists.tar.gz into the build repo in the config/urlfilter/ directory.
The build will then put this newer version of blacklists.tar.gz into the /var/ipfire/urlfilter/ directory.
Therefore when a new install is done with this updated CU version (probably/hopefully CU205) then the install will automatically have the symlinked directories and then the existing restore should then work.

Testing and confirming that will be something i will do as part of my patch submission of the fix of the updated blacklists.tar.gz in config/urlfilter/
Comment 10 pscar13 2026-08-21 20:31:28 UTC
Sorry, I hadn't understood.

Wouldn't it be better not to include any list in the installation ISO and instead load it only when URLFilter is activated?
That would avoid any restoration issues.
Comment 11 Adolf Belka 2026-08-24 17:08:03 UTC
(In reply to pscar13 from comment #10)
> 
> Wouldn't it be better not to include any list in the installation ISO and
> instead load it only when URLFilter is activated?
> That would avoid any restoration issues.

I am not sure how the url filter would react if there was no blocklist available and it was started before having run the update button in urlfilter.

I will check this when I have some time and then decide on the best approach.
Comment 12 pscar13 2026-08-24 17:42:50 UTC
Created attachment 1749 [details]
urlfilter.cgi

Here is a screenshot of urlfilter.cgi page when /var/ipfire/urlfilter/blacklists/ is empty.
Comment 13 pscar13 2026-08-24 18:31:03 UTC
(In reply to pscar13 from comment #12)
> Created attachment 1749 [details]
> urlfilter.cgi
> 
> Here is a screenshot of urlfilter.cgi page when
> /var/ipfire/urlfilter/blacklists/ is empty.

And squidGuard start without error before update blacklist, I can even add domain in "Custom blacklist" and it works fine (URLFilter block this domain)
Comment 14 pscar13 2026-08-26 16:24:19 UTC
How should I submit a patch that deletes a binary file?

I have prepared and tested a patch on MASTER (CU204) for the URL Filter blacklist changes.

The patch includes the deletion of:
config/urlfilter/blacklists.tar.gz

My Git commit is clean and includes the deletion of this binary file.

However, when I run "git format-patch -1", Git includes the complete binary data in the patch as a GIT binary patch.

This makes the patch much larger than necessary for a file that is simply being deleted.

What is the preferred way to submit this patch to the IPFire development mailing list in this case?

Should I submit the patch generated by "git format-patch" as it is, or should the binary file deletion be handled differently?

I would like to follow the correct procedure before sending the patch with "git send-email".