After today's update of the location database, if the option "Deny access to destinations hosted on selectively announced networks" is enabled, at least the following domains are blocked by squid: *.amazon.com *.amazon.de community.intel.com static.twitchcdn.net and other smaller domains I tested. After deselecting the option, these domains work as expected. Others like youtube.com are working without changing it. So it seems that with the latest update a lot of AWS-Servers are blocked. # location --version location 0.9.16 # location version Thu, 03 Aug 2023 05:22:02 GMT # location update Already on the latest version # update-location-database The database has been updated recently The firewall is running on CU176.
Just FYI: I've tested 2 firewalls running CU175, none of them were affected before the update to the latest CU177 even with an updated location database. Afterwards, they were both affected and the option needs to be disabled to have a fully functional proxy. And I digged deeper into the logs and these error messages started right after the update of the location database (Thu, 03 Aug 2023 05:22:02 GMT). Aug 07 08:13:03 squid-asnbl-helper[3454] WARN: Destination 'www.amazon.de' resolves to IP addresses '2600:9000:222e:dc00:e:13a1:b913:5f61' without corresponding ASN, probably selectively announced Aug 07 08:13:03 squid-asnbl-helper[3454] WARN: Destination 'www.amazon.de' resolves to IP addresses '2600:9000:222e:8200:e:13a1:b913:5f61' without corresponding ASN, probably selectively announced ... (repeated another 6 times) ... Aug 07 08:13:03 squid-asnbl-helper[3454] INFO: Denying access to destination 'www.amazon.de' due to suspected selective announcements As it seems the helper script gets multiple IPV6 destinations for the domain instead of only supported IPV4's. Or did I missed that IPFire should now support IPV6? And somethimes it also gets a mixed IPV4 and IPV6 return before it denies the access: Aug 07 06:57:17 squid-asnbl-helper[31110] WARN: Destination 'html5.gamedistribution.com' resolves to IP addresses '2600:9000:211e:200:5:4275:8dc0:93a1' without corresponding ASN, probably selectively announced Aug 07 06:57:17 squid-asnbl-helper[31110] WARN: Destination 'html5.gamedistribution.com' resolves to IP addresses '13.32.27.64' without corresponding ASN, probably selectively announced Aug 07 06:57:17 squid-asnbl-helper[31110] WARN: Destination 'html5.gamedistribution.com' resolves to IP addresses '2600:9000:211e:e600:5:4275:8dc0:93a1' without corresponding ASN, probably selectively announced Aug 07 06:57:17 squid-asnbl-helper[31110] WARN: Destination 'html5.gamedistribution.com' resolves to IP addresses '13.32.27.15' without corresponding ASN, probably selectively announced Aug 07 06:57:17 squid-asnbl-helper[31110] WARN: Destination 'html5.gamedistribution.com' resolves to IP addresses '2600:9000:211e:5800:5:4275:8dc0:93a1' without corresponding ASN, probably selectively announced Aug 07 06:57:17 squid-asnbl-helper[31110] WARN: Destination 'html5.gamedistribution.com' resolves to IP addresses '13.32.27.5' without corresponding ASN, probably selectively announced Aug 07 06:57:17 squid-asnbl-helper[31110] WARN: Destination 'html5.gamedistribution.com' resolves to IP addresses '2600:9000:211e:9200:5:4275:8dc0:93a1' without corresponding ASN, probably selectively announced Aug 07 06:57:17 squid-asnbl-helper[31110] WARN: Destination 'html5.gamedistribution.com' resolves to IP addresses '2600:9000:211e:0:5:4275:8dc0:93a1' without corresponding ASN, probably selectively announced Aug 07 06:57:17 squid-asnbl-helper[31110] WARN: Destination 'html5.gamedistribution.com' resolves to IP addresses '2600:9000:211e:d800:5:4275:8dc0:93a1' without corresponding ASN, probably selectively announced Aug 07 06:57:17 squid-asnbl-helper[31110] WARN: Destination 'html5.gamedistribution.com' resolves to IP addresses '13.32.27.83' without corresponding ASN, probably selectively announced Aug 07 06:57:17 squid-asnbl-helper[31110] WARN: Destination 'html5.gamedistribution.com' resolves to IP addresses '2600:9000:211e:6200:5:4275:8dc0:93a1' without corresponding ASN, probably selectively announced Aug 07 06:57:17 squid-asnbl-helper[31110] WARN: Destination 'html5.gamedistribution.com' resolves to IP addresses '2600:9000:211e:d600:5:4275:8dc0:93a1' without corresponding ASN, probably selectively announced
Just FYI, here are manual lookups for an affected IPV6 (www.amazon.de and community.intel.com). # location --debug lookup '2600:9000:222e:dc00:e:13a1:b913:5f61' 2600:9000:222e:dc00:e:13a1:b913:5f61: Network : 2600:9000::/34 Country : United States of America Anycast : yes # location --debug lookup '2600:9000:223f:3600:1e:3b87:9a00:93a1' 2600:9000:223f:3600:1e:3b87:9a00:93a1: Network : 2600:9000::/34 Country : United States of America Anycast : yes Or a lookup for an affected IPV4 (html5.gamedistribution.com). # location --debug lookup '13.32.27.15' 13.32.27.15: Network : 13.32.0.0/15 Country : United States of America Anycast : yes Currently, it seems that only Amazon hosted domains are affected when this option is active.
I am having the problem also with www.kernel.org. If Deny Selective Announcements option is disabled then the website opens with no problems otherwise it comes back with The proxy server is refusing connections. An error occurred during a connection to www.kernel.org I am running CU177.
The message in my log is Aug 11 14:50:56 squid-asnbl-helper[7822] WARN: Destination 'www.kernel.org' resolves to IP addresses '2604:1380:4601:e00::1' without corresponding ASN, probably selectively announced
I setup a vm machine and downloaded and installed the databases from 2023-08-03 back to 2023-07-30 and the problem stayed in effect until I got to 2023-07-30 when the www.kernel.org website opened with the Deny Selective Announcements option enabled. So the problem came in with database version 2023-07-31
Created attachment 1251 [details] squid logs showing the errors accessing kagi.com I have the same problem with the search engine https://kagi.com Please find in attachment the relevant part of /var/log/squid/cache.log # location --version location 0.9.16 # location version Sun, 06 Aug 2023 05:23:29 GMT # location update Downloaded new database from Fri, 11 Aug 2023 05:26:26 GMT # update-location-database The database has been updated recently
I forgot to mention that I am on CU177
DeepL Translator also is flagged for Selective Announcements Aug 20 15:10:34 squid-asnbl-helper[14011] WARN: Destination 'www2.deepl.com' resolves to IP addresses '172.65.212.243' without corresponding ASN, probably selectively announced Aug 20 15:10:34 squid-asnbl-helper[14011] INFO: Denying access to destination 'www2.deepl.com' due to suspected selective announcements
wikipedia also gets blocked for selective announcements
A quick update: Since we rolled back to the database being generated by the previous libloc release, this works again. I will have to investigate.
This issue is back again. tplink websites, kernel.org and amazon.co.uk all fail due to selective announcements. I am sure others do also. There have also been other reports in the forum of sites failing to be accessed. If selective announcements option is disabled in the web proxy then the above sites open with no problems. [root@ipfire user]# location --version location 0.9.17 [root@ipfire user]# location version Sat, 09 Mar 2024 06:05:24 GMT [root@ipfirer user]# location update https://location.ipfire.org/databases/ is serving an outdated database. Trying next mirror... Could not download a new database
Here is the squid cache log showing the selective announcements Mar 17 22:04:41 squid-asnbl-helper[2743] WARN: Destination 'www.amazon.co.uk' resolves to IP addresses '2600:9000:275b:aa00:15:c9dc:593:6781' without corresponding ASN, probably selectively announced Mar 17 22:04:41 squid-asnbl-helper[2743] WARN: Destination 'www.amazon.co.uk' resolves to IP addresses '2600:9000:275b:3800:15:c9dc:593:6781' without corresponding ASN, probably selectively announced Mar 17 22:04:41 squid-asnbl-helper[2743] WARN: Destination 'www.amazon.co.uk' resolves to IP addresses '2600:9000:275b:6400:15:c9dc:593:6781' without corresponding ASN, probably selectively announced Mar 17 22:06:14 squid-asnbl-helper[21281] WARN: Destination 'static.siege-amazon.com' resolves to IP addresses '2600:9000:269a:4a00:1a:f1e1:7cc0:93a1' without corresponding ASN, probably selectively announced Mar 17 22:06:14 squid-asnbl-helper[21281] WARN: Destination 'static.siege-amazon.com' resolves to IP addresses '2600:9000:269a:800:1a:f1e1:7cc0:93a1' without corresponding ASN, probably selectively announced Mar 17 22:06:14 squid-asnbl-helper[21281] WARN: Destination 'static.siege-amazon.com' resolves to IP addresses '2600:9000:269a:9000:1a:f1e1:7cc0:93a1' without corresponding ASN, probably selectively announced Mar 17 22:06:14 squid-asnbl-helper[21281] WARN: Destination 'static.siege-amazon.com' resolves to IP addresses '2600:9000:269a:d200:1a:f1e1:7cc0:93a1' without corresponding ASN, probably selectively announced Mar 17 22:11:41 squid-asnbl-helper[21786] WARN: Destination 'www.kernel.org' resolves to IP addresses '2604:1380:4601:e00::1' without corresponding ASN, probably selectively announced Mar 17 22:11:41 squid-asnbl-helper[21786] INFO: Denying access to destination 'www.kernel.org' due to suspected selective announcements Mar 17 22:27:43 squid-asnbl-helper[21786] WARN: Destination 'hfc-fs.s3-eu-west-1.amazonaws.com' resolves to IP addresses '52.218.100.32' without corresponding ASN, probably selectively announced Mar 17 22:27:43 squid-asnbl-helper[21786] WARN: Destination 'hfc-fs.s3-eu-west-1.amazonaws.com' resolves to IP addresses '52.218.46.74' without corresponding ASN, probably selectively announced Mar 17 22:27:43 squid-asnbl-helper[21786] INFO: Denying access to destination 'hfc-fs.s3-eu-west-1.amazonaws.com' due to suspected selective announcements This is just a few lines from the logs. There are more entries for a range of web site locations.
Is this all still reproducible with the location database from today or later? Lots of changes have been rolled which should hopefully fix this.
I updated to the latest database (21st March) but I am still having the problems. location version Thu, 21 Mar 2024 04:40:05 GMT Here is the squid cache.log when trying to access amazon.co.uk and amazon.de although amazon.nl does not show the problem. Mar 21 20:25:10 squid-asnbl-helper[2577] WARN: Destination 'www.amazon.co.uk' resolves to IP addresses '2600:9000:275b:ba00:15:c9dc:593:6781' without corresponding ASN, probably selectively announced Mar 21 20:25:10 squid-asnbl-helper[2577] WARN: Destination 'www.amazon.co.uk' resolves to IP addresses '2600:9000:275b:a200:15:c9dc:593:6781' without corresponding ASN, probably selectively announced Mar 21 20:25:10 squid-asnbl-helper[2577] WARN: Destination 'www.amazon.co.uk' resolves to IP addresses '2600:9000:275b:b200:15:c9dc:593:6781' without corresponding ASN, probably selectively announced Mar 21 20:25:10 squid-asnbl-helper[2577] WARN: Destination 'www.amazon.co.uk' resolves to IP addresses '2600:9000:275b:5e00:15:c9dc:593:6781' without corresponding ASN, probably selectively announced Mar 21 20:25:10 squid-asnbl-helper[2577] WARN: Destination 'www.amazon.co.uk' resolves to IP addresses '2600:9000:275b:5600:15:c9dc:593:6781' without corresponding ASN, probably selectively announced Mar 21 20:25:10 squid-asnbl-helper[2577] WARN: Destination 'www.amazon.co.uk' resolves to IP addresses '2600:9000:275b:4a00:15:c9dc:593:6781' without corresponding ASN, probably selectively announced Mar 21 20:25:10 squid-asnbl-helper[2577] WARN: Destination 'www.amazon.co.uk' resolves to IP addresses '2600:9000:275b:5200:15:c9dc:593:6781' without corresponding ASN, probably selectively announced Mar 21 20:25:10 squid-asnbl-helper[2577] WARN: Destination 'www.amazon.co.uk' resolves to IP addresses '2600:9000:275b:9c00:15:c9dc:593:6781' without corresponding ASN, probably selectively announced Mar 21 20:25:10 squid-asnbl-helper[2577] INFO: Denying access to destination 'www.amazon.co.uk' due to suspected selective announcements Mar 21 20:25:13 squid-asnbl-helper[2577] WARN: Destination 'www.amazon.de' resolves to IP addresses '52.85.240.204' without corresponding ASN, probably selectively announced Mar 21 20:25:13 squid-asnbl-helper[2577] INFO: Denying access to destination 'www.amazon.de' due to suspected selective announcements Disabling the selective announcements option in the web proxy allows the above two sites to be accessed. With the latest database the kernel.org site no longer gets blocked due to selective announcement.
Thank you for testing this. I believe the problem is now resolved: > root@michael:~# location lookup 2600:9000:275b:4a00:15:c9dc:593:6781 52.85.240.204 > 2600:9000:275b:4a00:15:c9dc:593:6781: > Network : 2600:9000:2750::/44 > Country : United States of America > Autonomous System : AS16509 - Amazon.com, Inc. > Anycast : yes > 52.85.240.204: > Network : 52.85.240.0/22 > Country : United States of America > Autonomous System : AS16509 - Amazon.com, Inc. > Anycast : yes I have written more about this here: > https://lists.ipfire.org/hyperkitty/list/location@lists.ipfire.org/thread/GXVEVONEYWQOIAZM5MBX7YAUCWYNH6ZA/
I updated the database but had to then clear the web proxy cache before it would work properly. All the website urls that were causing a problem for me before, are now working with no problems.
Thanks for the feedback. I think we can close this one straight away then since there is nothing to ship in an update. The new database has already been released.