Bug 11761 - firewall: add ability to use local blocklists
Summary: firewall: add ability to use local blocklists
Status: CLOSED WONTFIX
Alias: None
Product: IPFire
Classification: Unclassified
Component: --- (show other bugs)
Version: 2
Hardware: all All
: Will affect an average number of users Balancing
Assignee: Assigned to nobody - feel free to grab it and work on it
QA Contact: Peter Müller
URL: https://www.spamhaus.org/drop/
Keywords: NewFeature, Security
Depends on:
Blocks:
 
Reported: 2018-06-07 06:12 UTC by Peter Müller
Modified: 2021-12-05 16:56 UTC (History)
4 users (show)

See Also:


Attachments
attachment-2020-0.html (673 bytes, text/html)
2018-06-18 21:05 UTC, Michael Tremer
Details
attachment-17738-0.html (287 bytes, text/html)
2018-06-19 20:36 UTC, Michael Tremer
Details
Script to implement and update firewall blocklists (17.53 KB, application/x-perl)
2018-06-28 20:13 UTC, Tim
Details
Example of blocklist source list file (6.54 KB, text/plain)
2018-06-28 20:14 UTC, Tim
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Peter Müller 2018-06-07 06:12:41 UTC
My use case is to block traffic to networks listed in Spamhaus DROP (https://www.spamhaus.org/drop/), since there should be no traffic to these. Instead of adding all the networks to the firewall GUI on my own, could there be a possibility to upload a text file which is parsed automatically?

This would make using blocklists more easy.
Comment 1 Michael Tremer 2018-06-17 23:10:41 UTC
What is the license of that list? Would it make sense to integrate this
similar to the blacklists of the URL filter?
Comment 2 Tim 2018-06-18 20:50:28 UTC
The emerging threats ruleset includes the Spamhaus DROP list (emerging-drop.rules).

There are some other of other lists of IP addresses that can be dropped:
emerging-ciarmy.rules, emerging-compromised.rules, emerging-dshield.rules

They do make the point somewhere (I can't find it at the moment) that Snort is not particularly efficient at blocking IP addresses, so they provide a consolidated list of IP addresses and a script to regularly update the IPTables at http://doc.emergingthreats.net/bin/view/Main/EmergingFirewallRules.

I would personally prefer to blobk these addresses using the firewall since that ensures that you can't bypass the block by encrypting the link.
Comment 3 Michael Tremer 2018-06-18 21:05:54 UTC
Created attachment 593 [details]
attachment-2020-0.html

Indeed snort is not a good way to block a number of IP addresses. This is done best by iptables. We even have ipset.

So the logical thing would be a script that puts that rule set into the firewall if the user wishes to. Ideally REJECT.

Since this is under a free license we can redistribute this because having another download script is not what we want.

Who is up for implementing this?
Comment 4 Peter Müller 2018-06-19 20:33:21 UTC
Snort/Guardian are the wrong tools here ineed. Worse, there is a bug (#11532) preventing access to bad remote IPs to be blocked.
Comment 5 Michael Tremer 2018-06-19 20:36:27 UTC
Created attachment 597 [details]
attachment-17738-0.html

Please remind Stefan to have a look at it or submit patches to help him out...
Comment 6 Alexander Marx 2018-06-21 09:22:02 UTC
What is your idea of implementing this list?

My first idea would be to make a kind of Group and name it spamhaus or something like that.
With this in mind i could build an upload button for this list (or others with the right format) which can be uploaded, checked and when all ip-adresses are ok, a new group could be build and firewall has to be reloaded manually like when you create a custom group.
Comment 7 Michael Tremer 2018-06-21 16:20:35 UTC
(In reply to Alexander Marx from comment #6)
> What is your idea of implementing this list?
> 
> My first idea would be to make a kind of Group and name it spamhaus or
> something like that.
> With this in mind i could build an upload button for this list (or others
> with the right format) which can be uploaded, checked and when all
> ip-adresses are ok, a new group could be build and firewall has to be
> reloaded manually like when you create a custom group.

Is it not better to update the lists automatically?
Comment 8 Alexander Marx 2018-06-22 07:04:26 UTC
Not sure what you mean here. 

Just a checkbox "Enable Spamhaus lists" and then download the lists, generate a group and let cron watch every day for new lists?

I need some more input for implementing that.

Alex
Comment 9 Michael Tremer 2018-06-22 16:53:38 UTC
Let's put this on the agenda for the next monthly developer call then...
Comment 10 Tim 2018-06-26 19:02:04 UTC
I've implemented a script that downloads blocklists from the internet and installs them into the firewall.  It's successfully implemented the Emerging Threats FWRule list.  I'll test it for a few more days to check that it's handling updates correctly and then attach it for discussion.
Comment 11 Peter Müller 2018-06-28 19:21:13 UTC
Sorry for the delay.

Some thoughts about this:
(a) Since we already have a list of the "baddest of the bad" (Spamhaus DROP + Dshield Top 20 + Shadowserver C&C), we could just include this list as a network group into the firewall engine. That way, users are more flexible (although dropping the stuff seems to be the only useful option for me :-) ).
(b) A possbility to integrate own blocklists would be useful for some, but is somewhat optional for the first step.

What do you think of dropping traffic to the bad networks in (a) by default? In my opinion, it might be a good idea since there is almost no legitimate application there.
Comment 12 Tim 2018-06-28 20:12:28 UTC
I've attached my script to implement a blocklist.  When run it will, by default, check the Emerging Threats FWRule block list for changes and implement them.  It uses a header request to get the time of last change which it stores in /var/ipfire/blocklist/status.

You can also give it a sources file in the directory containing a list of blocklist sources and a settings file with enables.  A WUI would read the first and write the second.

It's got a few issues:

- Doesn't keep rules over a reboot.
- Handling of overlap between lists is limited.
- Probably should allow for a whitelist (including red i/f address).
- Each blocklist produces an additional list on the log summary, but there's nothing to distinguish between the blocklists.

I also thing it would probably be better written using ipset, but it's a working example that you can try out.
Comment 13 Tim 2018-06-28 20:13:44 UTC
Created attachment 602 [details]
Script to implement and update firewall blocklists
Comment 14 Tim 2018-06-28 20:14:20 UTC
Created attachment 603 [details]
Example of blocklist source list file
Comment 15 Tim 2018-06-28 20:15:50 UTC
Comment on attachment 603 [details]
Example of blocklist source list file

>############################################################################
>#                                                                          #
># IP Address blocklists for IPFire                                         #
>#                                                                          #
># This file contains a list of blocklist sources that will replace the one #
># internal to the updated if it is found at /var/ipfire/blocklist/sources. #
># The intention is to provide a common source of information for both the  #
># updater and a possible WUI.                                              #
>#                                                                          #
># The chains created in the packet filter will be named by the top level   #
># key and this will also be used in the log message to identify the reason #
># for the dropped packet.                                                  #
>#                                                                          #
># The fields are:                                                          #
>#                                                                          #
># name    The blocklist's full name                                        #
># url     URL of the file containing the list                              #
># info    URL giving information about the source                          #
># parser  The parser function used to extract IP addresses from the        #
>#         downloaded list                                                  #
># enable  Set to 1 to enable the list by default.  Overridden by the       #
>#         settings file                                                    #
># safe    'yes' if the list is unlikely to contain addresses that can be   #
>#         used for legitimate traffic, or 'no' otherwise                   #
>#                                                                          #
># The info and safe fields are purely for documentation.                   #
>#                                                                          #
># Note that the Emerging Threats blocklist is a composite list containing  #
># addresses from some of the other lists.  It is unnecessary to enable     #
># this list if the other lists are enabled.                                #
>#                                                                          #
>############################################################################
>
>%sources = ( 'EMERGING_FWRULE' => { 'name'    => 'Emerging Threats Blocklist',
>                                    'url'     => 'https://rules.emergingthreats.net/fwrules/emerging-Block-IPs.txt',
>                                    'info'    => 'http://doc.emergingthreats.net/bin/view/Main/EmergingFirewallRules',
>                                    'parser'  => 'text_with_hash_comments',
>                                    'enable'  => 1,
>                                    'safe'    => 'no' },
>             'SPAMHAUS_DROP'   => { 'name'    => "Spamhaus Don't Route or Peer List",
>                                    'url'     => 'https://www.spamhaus.org/drop/drop.txt',
>                                    'info'    => 'https://www.spamhaus.org/drop/',
>                                    'parser'  => 'text_with_semicolon_comments',
>                                    'enable'  => 0,
>                                    'safe'    => 'yes' },
>             'SPAMHAUS_EDROP'  => { 'name'    => "Spamhaus Extended Don't Route or Peer List",
>                                    'url'     => 'https://www.spamhaus.org/drop/edrop.txt',
>                                    'info'    => 'https://www.spamhaus.org/drop/',
>                                    'parser'  => 'text_with_semicolon_comments',
>                                    'enable'  => 0,
>                                    'safe'    => 'no' },
>             'DSHIELD'         => { 'name'    => 'Dshield.org Recommended Block List',
>                                    'url'     => 'https://www.dshield.org/block.txt',
>                                    'info'    => 'https://dshield.org/',
>                                    'parser'  => 'dshield',
>                                    'enable'  => 0,
>                                    'safe'    => 'no' },
>             'FEODO_IP'        => { 'name'    => 'Feodo Trojan IP Blocklist (Versions A, C, D)',
>                                    'url'     => 'https://feodotracker.abuse.ch/blocklist/?download=ipblocklist',
>                                    'info'    => 'https://feodotracker.abuse.ch/blocklist',
>                                    'parser'  => 'text_with_hash_comments',
>                                    'enable'  => 0,
>                                    'safe'    => 'no' },
>             'FEODO_BAD_IP'    => { 'name'    => 'Feodo Trojan BadIP Blocklist (Version B)',
>                                    'url'     => 'https://feodotracker.abuse.ch/blocklist/?download=badips',
>                                    'info'    => 'https://feodotracker.abuse.ch/blocklist',
>                                    'parser'  => 'text_with_hash_comments',
>                                    'enable'  => 0,
>                                    'safe'    => 'yes' },
>             'ABUSE_CH'        => { 'name'    => 'Abuse.ch Ransomware C&C Blocklist',
>                                    'url'     => 'https://ransomwaretracker.abuse.ch/downloads/RW_IPBL.txt',
>                                    'info'    => 'https://ransomwaretracker.abuse.ch/blocklist/',
>                                    'parser'  => 'text_with_hash_comments',
>                                    'enable'  => 0,
>                                    'safe'    => 'no' },
>             'ZEUS_IP'         => { 'name'    => 'Abuse.ch Zeus IP Blocklist',
>                                    'url'     => 'https://zeustracker.abuse.ch/blocklist.php?download=badips',
>                                    'info'    => 'https://zeustracker.abuse.ch/blocklist.php',
>                                    'parse'   => 'text_with_hash_comments',
>                                    'enable'  => 0,
>                                    'safe'    => 'no' },
>             'ZEUS_BAD_IP'     => { 'name'    => 'Abuse.ch Zeus BadIP Blocklist',
>                                    'url'     => 'https://zeustracker.abuse.ch/blocklist.php?download=ipblocklist',
>                                    'info'    => 'https://zeustracker.abuse.ch/blocklist.php',
>                                    'parse'   => 'text_with_hash_comments',
>                                    'enable'  => 0,
>                                    'safe'    => 'yes' }
>           );
Comment 16 Michael Tremer 2018-06-29 00:04:59 UTC
I think there is a point is just logging traffic (-> honeypot).

We are building another thing that downloads blacklists from somewhere
and leaking that someone is using IPFire. We have been trying to
eliminate this in the past.
Comment 17 Peter Müller 2021-12-05 16:56:07 UTC
Closing this as being WONTFIX, since the "IP Blocklist" patchset never landed into the distribution for various reasons, its maintainer does not respond to various mails, and Core Update 162 comes with a solution being sufficient enough to close this.

https://blog.ipfire.org/post/ipfire-2-27-core-update-162-is-available-for-testing