When adding IPSec tunnels in IPFire 2.13 Core 67, the new tunnel will not connect unless I restart StrongSWAN, either by disabling IPSec in the GUI and then re-enabling it, or by issuing "/etc/rc.d/init.d/ipsec restart. Once StrongSWAN has been restarted, the tunnel comes right up.
It is NOT required to restart the charon daemon, but to reload the configuration. This is done by the web interface and the newly created connection will be brought up by running "ipsec up <connection name>". Any error in the logs?
Created attachment 125 [details] Log entries related to new IPSec tunnels requiring a restart. Output from 'grep -v DROP /var/log/messages'.
Michael, I presumed that what you describe is what should happen, but I can confirm that my experience has been that newly added tunnels will not come up until I issue '/etc/rc.d/init.d/ipsec restart' from the CLI. Perhaps a reload instead of a restart would suffice, but I cannot confirm. This is true even if you add multiple tunnels. None come up until charon is restarted. Perhaps I have done something wrong, but whatever that might be is beyond me. I posted up an attachment containing log entries, but did not redact the IP addresses. What is the best way to do that after-the-fact? Tom
Could you please try to figure out if strongswan knows of your connection, that you just created (ipsec status)? Maybe try to run "ipsec reload" first and see if this gives us some more information. I do not have this problem on my machines...
I will try adding a new connection when I have a chance and post back with the results of an "ipsec status" and "ipsec reload" If you view the log snippet, you can see that the tunnel is there, and failing to come up, then a restart and BOOM! It comes up.
Created attachment 126 [details] IPSec General Settings I have tried this again on another 2.13 Core67 machine, connecting to an Endian 2.51 firewall (strongSwan 4.5.3). I created the tunnel on the remote end and then on the IPFire side. The attached file shows the general settings. After editing the advanced settings and saving, the same behavior is exhibited: the tunnel fails to come up. 'ipsec status' shows Security Associations (0 up, 0 connecting): none Additionally, traffic across the router to the internet stops flowing for about 15-30 seconds. Eventually, it comes back up, but the result of 'ipsec status' does not change. Issuing 'ipsec reload' results in the status changing briefly to Security Associations (1 up, 0 connecting): none before reverting back. Issuing the 'ipsec restart' command results in the tunnel coming right up.
Created attachment 127 [details] Advanced settings screenshot.
Could you try using IKEv2 instead of IKEv1? It may be possible that the system is unable to get enough entropy - but that's me just guessing...
Created attachment 128 [details] strongswan 5.0.3rc1 I compiled strongswan 5.0.3rc1. Could you please install this on a test system and check if the problem still exists? Just extract the tarball to /.
I downloaded the 5.0.3rc1, copied it to / and ran the command 'tar -xvf strongswan 5.0.3rc1' to install it. I rebooted for good measure, and when I run 'ipsec version', I get the result 'Linux strongSwan U5.0.3rc1/K3.2.38-ipfire-pae' as a result. So far, so good. The existing tunnel on the machine came up and seemed to be working properly. However, the first thing I noticed was that, after deleting the existing tunnel in the GUI, I was still seeing messages such as 'Mar 29 12:37:10 lebanon charon: 16[IKE] initiating Main Mode IKE_SA Keenegas[2] to xxx.xxx.xxx.xxx' in the kernel log. So, I ran 'ipsec status' and got this: Security Associations (1 up, 0 connecting): Keenegas[2]: CONNECTING, xxx.xxx.xxx.xxx[%any]...yyy.yyy.yyy.yyy[%any] Then, I ran: [root@lebanon ~]# ipsec restart Stopping strongSwan IPsec... Starting strongSwan 5.0.3rc1 IPsec [starter]... [root@lebanon ~]# ipsec status Security Associations (0 up, 0 connecting): none I will continue to re-create the tunnels and let you know what I get.
More info: this is still acting in the same manner. 1.) Added tunnel to remote machine (not-IPFire). 2.) Added tunnel to IPFire. 3.) CLI output after saving tunnel configuration: [root@lebanon ~]# ipsec status Security Associations (0 up, 0 connecting): none [root@lebanon ~]# ipsec reload Reloading strongSwan IPsec configuration... [root@lebanon ~]# ipsec status Security Associations (0 up, 0 connecting): none [root@lebanon ~]# ipsec restart Stopping strongSwan IPsec... Starting strongSwan 5.0.3rc1 IPsec [starter]... [root@lebanon ~]# ipsec status Security Associations (1 up, 0 connecting): Endian[1]: ESTABLISHED 3 seconds ago, xxx.xxx.xxx.xxx[ipfire]...yyy.yyy.yyy.yyy[endian] Endian{1}: INSTALLED, TUNNEL, ESP SPIs: c7682795_i cbe998a9_o, IPCOMP CPIs: 1c14_i b3c3_o Endian{1}: 10.8.0.0/16 === 10.7.0.0/16 Output from 'tail -f /var/log/messages |grep -v DROP': Mar 29 12:52:15 lebanon charon: 15[IKE] initiating Main Mode IKE_SA Endian[6] to yyy.yyy.yyy.yyy Mar 29 12:52:15 lebanon charon: 15[IKE] initiating Main Mode IKE_SA Endian[6] to yyy.yyy.yyy.yyy Mar 29 12:52:36 lebanon charon: 13[IKE] initiating Main Mode IKE_SA Endian[7] to yyy.yyy.yyy.yyy Mar 29 12:52:36 lebanon charon: 13[IKE] initiating Main Mode IKE_SA Endian[7] to yyy.yyy.yyy.yyy Mar 29 12:52:42 lebanon charon: 13[IKE] yyy.yyy.yyy.yyy is initiating a Main Mode IKE_SA Mar 29 12:52:42 lebanon charon: 13[IKE] yyy.yyy.yyy.yyy is initiating a Main Mode IKE_SA Mar 29 12:53:52 lebanon charon: 14[IKE] yyy.yyy.yyy.yyy is initiating a Main Mode IKE_SA Mar 29 12:53:52 lebanon charon: 14[IKE] yyy.yyy.yyy.yyy is initiating a Main Mode IKE_SA Mar 29 12:54:41 lebanon charon: 09[IKE] initiating Main Mode IKE_SA Endian[10] to yyy.yyy.yyy.yyy Mar 29 12:54:41 lebanon charon: 09[IKE] initiating Main Mode IKE_SA Endian[10] to yyy.yyy.yyy.yyy Mar 29 12:54:47 lebanon charon: 12[IKE] initiating Main Mode IKE_SA Endian[11] to yyy.yyy.yyy.yyy Mar 29 12:54:47 lebanon charon: 12[IKE] initiating Main Mode IKE_SA Endian[11] to yyy.yyy.yyy.yyy Mar 29 12:55:02 lebanon charon: 12[IKE] yyy.yyy.yyy.yyy is initiating a Main Mode IKE_SA Mar 29 12:55:02 lebanon charon: 12[IKE] yyy.yyy.yyy.yyy is initiating a Main Mode IKE_SA Mar 29 12:55:18 lebanon ipsec_starter[7304]: charon stopped after 200 ms Mar 29 12:55:18 lebanon ipsec_starter[7304]: ipsec starter stopped Mar 29 12:55:21 lebanon ipsec_starter[8128]: Starting strongSwan 5.0.3rc1 IPsec [starter]... Mar 29 12:55:21 lebanon ipsec_starter[8149]: charon (8150) started after 20 ms Mar 29 12:55:21 lebanon charon: 12[IKE] initiating Main Mode IKE_SA Endian[1] to yyy.yyy.yyy.yyy Mar 29 12:55:21 lebanon charon: 12[IKE] initiating Main Mode IKE_SA Endian[1] to yyy.yyy.yyy.yyy Mar 29 12:55:21 lebanon charon: 15[IKE] IKE_SA Endian[1] established between xxx.xxx.xxx.xxx[ipfire]...yyy.yyy.yyy.yyy[endian] Mar 29 12:55:21 lebanon charon: 15[IKE] IKE_SA Endian[1] established between xxx.xxx.xxx.xxx[ipfire]...yyy.yyy.yyy.yyy[endian] Mar 29 12:55:22 lebanon charon: 01[IKE] CHILD_SA Endian{1} established with SPIs c7682795_i cbe998a9_o and TS 10.8.0.0/16 === 10.7.0.0/16 Mar 29 12:55:22 lebanon charon: 01[IKE] CHILD_SA Endian{1} established with SPIs c7682795_i cbe998a9_o and TS 10.8.0.0/16 === 10.7.0.0/16 Mar 29 12:55:22 lebanon vpn: client+ endian 10.7.0.0/16 == yyy.yyy.yyy.yyy -- xxx.xxx.xxx.xxx == 10.8.0.0/16 Mar 29 12:55:22 lebanon vpn: tunnel+ yyy.yyy.yyy.yyy -- xxx.xxx.xxx.xxx Mar 29 12:55:22 lebanon vpn: snat+ red0-xxx.xxx.xxx.xxx : 10.7.0.0/16 - 10.8.0.1 Mar 29 12:56:13 lebanon charon: 09[IKE] yyy.yyy.yyy.yyy is initiating a Main Mode IKE_SA Mar 29 12:56:13 lebanon charon: 09[IKE] yyy.yyy.yyy.yyy is initiating a Main Mode IKE_SA Mar 29 12:56:14 lebanon charon: 10[IKE] deleting IKE_SA Endian[1] between xxx.xxx.xxx.xxx[ipfire]...yyy.yyy.yyy.yyy[endian] Mar 29 12:56:14 lebanon charon: 10[IKE] deleting IKE_SA Endian[1] between xxx.xxx.xxx.xxx[ipfire]...yyy.yyy.yyy.yyy[endian] Mar 29 12:56:14 lebanon charon: 10[IKE] IKE_SA Endian[2] established between xxx.xxx.xxx.xxx[ipfire]...yyy.yyy.yyy.yyy[endian] Mar 29 12:56:14 lebanon charon: 10[IKE] IKE_SA Endian[2] established between xxx.xxx.xxx.xxx[ipfire]...yyy.yyy.yyy.yyy[endian] Mar 29 12:56:14 lebanon charon: 14[IKE] CHILD_SA Endian{1} established with SPIs c4441854_i cdd7b148_o and TS 10.8.0.0/16 === 10.7.0.0/16 Please let me know if there is anything I can do to help figure this out.
(In reply to comment #8) > Could you try using IKEv2 instead of IKEv1? I just spent a fair amount of time trying to make a tunnel work via ike2, including a fresh install and upgrade to core67, all to no avail. That is, until I remembered that the machine on the far end was also running IPFire core67. Once I logged in to the remote machine and issued an 'ipsec restart', the tunnel came right up. So, I guess we can say that using ikev2 does not help. I am not able to try strongswan 5.0.3rc1, as the second machine is a production machine.
OK, so I have tracked this down a bit further, and I can now get a tunnel to come up without restarting ipsec: 1.) Create tunnel, check box for "edit advanced settings" 2.) There is a significant delay before the "advanced settings" page is displayed (40 seconds, perhaps?). During this delay, "ipsec status" will briefly show a keying attempt: ------------------------------------------------------------------- [root@ipfire ~]# ipsec status Security Associations (2 up, 0 connecting): ikev2v5[2]: CONNECTING, xxx.xxx.xxx.xxx[%any]...yyy.yyy.yyy.yyy[%any] ikev2v5[1]: CONNECTING, xxx.xxx.xxx.xxx[%any]...yyy.yyy.yyy.yyy[%any] ------------------------------------------------------------------- 3.) Eventually, "ipsec status" will revert to showing nothing, and the "advanced settings" page will finally display: ------------------------------------------------------------------- [root@ipfire ~]# ipsec status Security Associations (0 up, 0 connecting): none ------------------------------------------------------------------- 4.) Complete the "Advanced Settings" page, and click Save. The tunnel will still be down, and running "ipsec reload" and "ipsec update" have no effect: ------------------------------------------------------------------- [root@ipfire ~]# ipsec reload Reloading strongSwan IPsec configuration... [root@ipfire ~]# ipsec status Security Associations (0 up, 0 connecting): none [root@ipfire ~]# ipsec update Updating strongSwan IPsec configuration... [root@ipfire ~]# ipsec status Security Associations (0 up, 0 connecting): none ------------------------------------------------------------------- 5.) If you then run "ipsec up <connectionname>", you get this error, and the tunnel still does not come up: ------------------------------------------------------------------- [root@ipfire ~]# ipsec up ikev2v5 initiating IKE_SA ikev2v5[7] to yyy.yyy.yyy.yyy generating IKE_SA_INIT request 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) ] sending packet: from xxx.xxx.xxx.xxx[500] to yyy.yyy.yyy.yyy[500] (768 bytes) received packet: from yyy.yyy.yyy.yyy[500] to xxx.xxx.xxx.xxx[500] (401 bytes) parsed IKE_SA_INIT response 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) CERTREQ N(MULT_AUTH) ] received 1 cert requests for an unknown ca authentication of 'foo' (myself) with pre-shared key no shared key found for 'foo' - 'bar' [root@ipfire ~]# ipsec status Security Associations (0 up, 0 connecting): none ------------------------------------------------------------------- 6.) If you then run "ipsec rereadsecrets", followed by "ipsec up <connectioname>", the tunnel comes up fine: ------------------------------------------------------------------- [root@ipfire ~]# ipsec rereadsecrets [root@ipfire ~]# ipsec up ikev2v5 initiating IKE_SA ikev2v5[8] to yyy.yyy.yyy.yyy generating IKE_SA_INIT request 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) ] sending packet: from xxx.xxx.xxx.xxx[500] to yyy.yyy.yyy.yyy[500] (768 bytes) received packet: from yyy.yyy.yyy.yyy[500] to xxx.xxx.xxx.xxx[500] (401 bytes) parsed IKE_SA_INIT response 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) CERTREQ N(MULT_AUTH) ] received 1 cert requests for an unknown ca authentication of 'foo' (myself) with pre-shared key establishing CHILD_SA ikev2v5 generating IKE_AUTH request 1 [ IDi N(INIT_CONTACT) IDr AUTH N(IPCOMP_SUP) SA TSi TSr N(MOBIKE_SUP) N(ADD_4_ADDR) N(MULT_AUTH) N(EAP_ONLY) ] sending packet: from xxx.xxx.xxx.xxx[4500] to yyy.yyy.yyy.yyy[4500] (476 bytes) received packet: from yyy.yyy.yyy.yyy[4500] to xxx.xxx.xxx.xxx[4500] (236 bytes) parsed IKE_AUTH response 1 [ IDr AUTH N(IPCOMP_SUP) SA TSi TSr N(AUTH_LFT) N(MOBIKE_SUP) N(ADD_4_ADDR) ] authentication of 'bar' with pre-shared key successful IKE_SA ikev2v5[8] established between xxx.xxx.xxx.xxx[foo]...yyy.yyy.yyy.yyy[bar] scheduling reauthentication in 28121s maximum IKE_SA lifetime 28661s CHILD_SA ikev2v5{1} established with SPIs c8d628a4_i cde00ef4_o and TS 10.8.0.0/16 === 10.6.0.0/16 [root@ipfire ~]# ipsec status Security Associations (1 up, 0 connecting): ikev2v5[8]: ESTABLISHED 4 seconds ago, xxx.xxx.xxx.xxx[foo]...yyy.yyy.yyy.yyy[bar] ikev2v5{1}: INSTALLED, TUNNEL, ESP SPIs: c8d628a4_i cde00ef4_o, IPCOMP CPIs: 7b46_i 77e3_o ikev2v5{1}: 10.8.0.0/16 === 10.6.0.0/16 ------------------------------------------------------------------- So, I don't know what this means from a development point of view, but perhaps the calls made by the GUI need to be modified to accomodate changes made in StrongSwan 5. Also, it seems like the GUI actually creates and tries to start the tunnel when you click "save", even if you select the "Edit Advanced Settings when done" checkbox. I think it would make sense to wait and only create/start the tunnel until AFTER you have clicked "Save" on the "Advanced Settings" page in this scenario. Lastly, I have also noticed a similar problem where a tunnel stays up after it has been deleted, and traffic will still travel across it: ------------------------------------------------------------------- [root@ipfire ~]# ipsec status Security Associations (1 up, 0 connecting): ikev2v5[6]: ESTABLISHED 5 minutes ago, xxx.xxx.xxx.xxx[foo]...yyy.yyy.yyy.yyy[bar] ikev2v5{2}: INSTALLED, TUNNEL, ESP SPIs: cfdce2f9_i c1f1fd3a_o, IPCOMP CPIs: 8538_i c65c_o ikev2v5{2}: 10.8.0.0/16 === 10.6.0.0/16 [root@ipfire ~]# ping 10.6.0.1 PING 10.6.0.1 (10.6.0.1): 56 data bytes 64 bytes from 10.6.0.1: icmp_seq=0 ttl=64 time=39.375 ms 64 bytes from 10.6.0.1: icmp_seq=1 ttl=64 time=35.887 ms 64 bytes from 10.6.0.1: icmp_seq=2 ttl=64 time=33.759 ms --- 10.6.0.1 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max/stddev = 31.780/35.200/39.375/2.814 ms ------------------------------------------------------------------- Running the command "ipsec down <connectionname>" will successfully bring the tunnel down: ------------------------------------------------------------------- [root@ipfire ~]# ipsec down ikev2v5 deleting IKE_SA ikev2v5[6] between xxx.xxx.xxx.xxx[foo]...yyy.yyy.yyy.yyy[bar] sending DELETE for IKE_SA ikev2v5[6] generating INFORMATIONAL request 2 [ D ] sending packet: from xxx.xxx.xxx.xxx[4500] to yyy.yyy.yyy.yyy[4500] (76 bytes) received packet: from yyy.yyy.yyy.yyy[4500] to xxx.xxx.xxx.xxx[4500] (76 bytes) parsed INFORMATIONAL response 2 [ ] IKE_SA deleted [root@ipfire ~]# ipsec status Security Associations (0 up, 0 connecting): none ------------------------------------------------------------------- I do hope this narrows down the problem a bit. Let me know if there is any more information I can provide. Tom
A little more browsing of the docs leads me to believe that "ipsec rereadall" would be a better command to use than "ipsec rereadsecrets", as the former includes the latter, and the latter only works with secrets defined in ipsec.secrets.
More information: Creating a tunnel via the GUI and then issuing these commands will also bring the tunnel up: ipsec rereadall ipsec reload
A few more observations: First: If one does the following, the tunnel will come right up with no interaction at the shell: 1.) Deletes a working tunnel without issuing 'ipsec rereadall' or 'ipsec restart'. 2.) Recreates another tunnel with the same endpoint IDs and the same PSK. That new tunnel will work as expected. Presumably, this is because the running process still has the PSK for those endpoint IDs cached, again pointing to my initial problem being because the secrets need to be reread before a newly created tunnel is started. Second: Some potentially scary behavior related to the issue with tunnels not being brought down when they are deleted. I just inadvertently confirmed that a deleted tunnel can be brought back up if one does the following: 1.) Delete a tunnel from both ends of a tunnel (IPFire Core 67 on both ends). The tunnel will continue running as reported before. 2.) Manually bring the tunnel down on IPFire #1 by issuing "ipsec down <connectionname>" 3.) DO NOT manually bring the connection down on IPFire #2. 4.) Issue "ipsec restart" on IPFire #1. 5.) Recreate the tunnel on IPFire #1. The tunnel will then come back up on IPFire #2, even though it has been deleted! This is somewhat scary, as you might think that that the remote side no longer has access to your network because you have deleted the tunnel. However, if the remote side recreates that tunnel, IPFire will allow them back in! Perhaps the issues with deleting tunnels merit their own bug? Tom
Well, this is a lot of text you posted here... If you hit the advanced check box, the connection will be established with the default values and the page will be displayed after that. There is a slight delay on my box, but not more than a second or two. The code that is executed when a new tunnel is created or an other one is deleted is over here: http://git.ipfire.org/?p=ipfire-2.x.git;a=blob;f=src/misc-progs/ipsecctrl.c;h=633004e2331d4d7044c77f4cf9d5f1dc82938a30;hb=HEAD The binary explicitly runs "ipsec down <conn>", so the connection should be brought down. After that, the entire ipsec stuff is reloaded so the daemons will forget the connection that has just been deleted. Running "ipsec reload" should implicitly run "ipsec rereadall". I cannot figure out what the difference is.
Created attachment 131 [details] strongswan 5.0.3 This is a tarball with the final version of strongswan 5.0.3.
Michael, Sorry about the excessive text, but I thought it would be a case of more is better. I wasn't certain if an attachment would be better, but I doubt it matters. Anyhow, I will download the tarball when I have a chance, though I am away from my test machine for at least a week. As for the "reread secrets" and "reload" commands, I am not certain that reload calls rereadsecrets. The "Introduction to strongSwan" (http://wiki.strongswan.org/projects/strongswan/wiki/IntroductionTostrongSwan) seems to imply that reload only looks for changes to ipsec.conf: "Whenever the ipsec.conf file is changed it may be reloaded with ipsec update or ipsec reload. Already established connections are not affected by these commands, if that is required ipsec restart must be used. If ipsec.secrets or the files in ipsec.d have been changed the ipsec reread... commands may be used to reload these files." Regarding the delay on tunnel creation, it seems to be the worst when creating the very first tunnel (i.e.: when ipsec was started with no defined tunnels). On a freshly installed machine, this delay is quite long. Lastly, what can I do to provide further information about the tunnels that persist after deletion? I don't think it's user error on my end, but it could be, I suppose. Either way, I'd sure like to get to the bottom of it. Tom
Created attachment 135 [details] strongswan 5.1.0dr1
Created attachment 137 [details] strongswan 5.1.0dr2
Michael: I downloaded and installed 5.1.0dr2 today, and the behavior is the same when creating a new tunnel. It fails to come up until I issue "ipsec rereadall" and then "ipsec reload". If I may, I think you are barking up the wrong tree here. IIRC, the GUI is still just issuing an "ipsec reload", which only reloads the ipsec.conf file, but does not load the newly created PSK. Supporting that theory, I see this in the kernel log, despite the fact that the PSK is properly written to /etc/ipsec.secrets: "no shared key found for 'bar' - 'foo'" My theory is that issuing "ipsec reload" was sufficient for StrongSWAN version 4, because pluto restarted every time you reloaded, causing the secrets to be read at that time, while charon does not. In version 5, "ipsec reload" does not work, and the GUI must be rewritten to explicitly reread the secrets. I would recommend (as I did earlier) that the command "ipsec rereadall" be used, instead of "ipsec rereadsecrects", as the latter only rereads PSKs, as I understand it. Thank you for your continued work on this. Tom
Created attachment 138 [details] ipsecctrl binary I just attached a modified version of the ipsecctrl binary that is used by the web user interface to strongswan. Could you please test that? Copy it to /usr/local/bin/ipsecctrl, chmod 4750 /usr/local/bin/ipsecctrl
Michael: Thank you, this does seem to resolve the issues when creating a new tunnel. I am not currently able to test the situations previously mentioned when deleting a tunnel, but I would suggest that the ipsecctrl program also reread the secrets when deleting a tunnel, such that it forces ipsec to forget the now-seleted password. Thanks again! Tom
You can check out the source code changes over here: http://git.ipfire.org/?p=people/ms/ipfire-2.x.git;a=commitdiff;h=e0cdf670a3d79b6d607f7eade6d99743f5cd5769 Please let me know when you have tested your configuration. Thanks for the help.
I can confirm that the issue I previously had with tunnels not coming down is not present at this point, using two Core 70 machines (my original report was on Core 67). I can't say why that was occurring before, but it was likely due to user error on my part, I am guessing. Having said that, I think it would be wise to modify ipsecctrl to issue a "ipsec rereadall" command when deleting a connection. If that is not done, the running process will still keep the PSK cached in memory, even though it has been deleted from the ipsec.secrets file. Other thanthat suggested change, everything seems to be working here with the modified ipsecctrl.
I guess we can close this then. Thank you.