Reconnecting probe 3508 - troubleshooting
Hi I've just reconnected probe 3508 to the network after a number of years offline (I think it was last online in November 2012?). Under the SOS history, I can see that it's making DNS queries successfully (using the DNS servers it receives by DHCP), but it's not connecting correctly to the RIPE network. The following is also displayed under SOS history: "This probe cannot connect on port 443 (HTTPS)". Any thoughts on where I can start troubleshooting? For test purposes I've connected the probe to my LAN, with full internet access. Cheers, MJ -- Michael-John Turner * mj@mjturner.net * http://mjturner.net/
Hi Michael-John, Check if you can connect from you network to woolsey.atlas.ripe.net and/or oneill.atlas.ripe.net using port 443 wbr /vty On 8/21/16 7:43 PM, Michael-John Turner wrote:
Hi
I've just reconnected probe 3508 to the network after a number of years offline (I think it was last online in November 2012?).
Under the SOS history, I can see that it's making DNS queries successfully (using the DNS servers it receives by DHCP), but it's not connecting correctly to the RIPE network. The following is also displayed under SOS history: "This probe cannot connect on port 443 (HTTPS)".
Any thoughts on where I can start troubleshooting? For test purposes I've connected the probe to my LAN, with full internet access.
Cheers, MJ
Hi Viktor, On Mon, Aug 22, 2016 at 08:09:51AM +0200, Viktor Naumov wrote:
Check if you can connect from you network to woolsey.atlas.ripe.net and/or oneill.atlas.ripe.net using port 443
Yes, I can connect to both, using both IPv4 and IPv6 (I tested using the OpenSSH client from a machine on the same network). The reason why the probe was offline originally was that it stopped accepting DHCPOFFERs for some reason, with the result that it would continually send DHCPDISCOVERs every few seconds but never ACK an OFFER and never connect properly to the network. That seemed to be fixed when I powered it on again yesterday but now I see the problem is occurring again (it started again as soon as the initial lease it received yesterday expired): ... Aug 22 21:10:18 dreamland dhcpd: DHCPOFFER on 192.168.1.247 to 00:80:a3:91:38:ac via br0 Aug 22 21:10:21 dreamland dhcpd: DHCPDISCOVER from 00:80:a3:91:38:ac via br0 Aug 22 21:10:21 dreamland dhcpd: DHCPOFFER on 192.168.1.247 to 00:80:a3:91:38:ac via br0 Aug 22 21:10:24 dreamland dhcpd: DHCPDISCOVER from 00:80:a3:91:38:ac via br0 Aug 22 21:10:24 dreamland dhcpd: DHCPOFFER on 192.168.1.247 to 00:80:a3:91:38:ac via br0 Aug 22 21:10:27 dreamland dhcpd: DHCPDISCOVER from 00:80:a3:91:38:ac via br0 Aug 22 21:10:27 dreamland dhcpd: DHCPOFFER on 192.168.1.247 to 00:80:a3:91:38:ac via br0 Aug 22 21:10:30 dreamland dhcpd: DHCPDISCOVER from 00:80:a3:91:38:ac via br0 Aug 22 21:10:30 dreamland dhcpd: DHCPOFFER on 192.168.1.247 to 00:80:a3:91:38:ac via br0 Aug 22 21:10:33 dreamland dhcpd: DHCPDISCOVER from 00:80:a3:91:38:ac via br0 Aug 22 21:10:33 dreamland dhcpd: DHCPOFFER on 192.168.1.247 to 00:80:a3:91:38:ac via br0 Aug 22 21:10:36 dreamland dhcpd: DHCPDISCOVER from 00:80:a3:91:38:ac via br0 ... This poor probe seems cursed :( Is there any way to reset it (it's a v1 probe, according to the Atlas website on firmware 4480)? Cheers, MJ -- Michael-John Turner * mj@mjturner.net * http://mjturner.net/
On 2016/08/22 22:14 , Michael-John Turner wrote:
The reason why the probe was offline originally was that it stopped accepting DHCPOFFERs for some reason, with the result that it would continually send DHCPDISCOVERs every few seconds but never ACK an OFFER and never connect properly to the network. That seemed to be fixed when I powered it on again yesterday but now I see the problem is occurring again (it started again as soon as the initial lease it received yesterday expired):
... Aug 22 21:10:18 dreamland dhcpd: DHCPOFFER on 192.168.1.247 to 00:80:a3:91:38:ac via br0 Aug 22 21:10:21 dreamland dhcpd: DHCPDISCOVER from 00:80:a3:91:38:ac via br0 Aug 22 21:10:21 dreamland dhcpd: DHCPOFFER on 192.168.1.247 to 00:80:a3:91:38:ac via br0 ...
This poor probe seems cursed :( Is there any way to reset it (it's a v1 probe, according to the Atlas website on firmware 4480)?
One possible explanation is that the probe doesn't actually receive any packets (i.e. the receive circuitry at the ethernet level is broken). The only way to know for sure is to open up the probe and attach a serial console. But I'm not aware of any software failure mode that leads to this behavior. Philip
On 23.08.16 10:52 , Philip Homburg wrote:
On 2016/08/22 22:14 , Michael-John Turner wrote:
The reason why the probe was offline originally was that it stopped accepting DHCPOFFERs for some reason, with the result that it would continually send DHCPDISCOVERs every few seconds but never ACK an OFFER and never connect properly to the network. That seemed to be fixed when I powered it on again yesterday but now I see the problem is occurring again (it started again as soon as the initial lease it received yesterday expired):
... Aug 22 21:10:18 dreamland dhcpd: DHCPOFFER on 192.168.1.247 to 00:80:a3:91:38:ac via br0 Aug 22 21:10:21 dreamland dhcpd: DHCPDISCOVER from 00:80:a3:91:38:ac via br0 Aug 22 21:10:21 dreamland dhcpd: DHCPOFFER on 192.168.1.247 to 00:80:a3:91:38:ac via br0 ...
This poor probe seems cursed :( Is there any way to reset it (it's a v1 probe, according to the Atlas website on firmware 4480)?
One possible explanation is that the probe doesn't actually receive any packets (i.e. the receive circuitry at the ethernet level is broken). The only way to know for sure is to open up the probe and attach a serial console. But I'm not aware of any software failure mode that leads to this behavior.
Try a different ethernet cable and/or switch port. Daniel
On Tue, Aug 23, 2016 at 12:02:46PM +0200, Daniel Karrenberg wrote:
Try a different ethernet cable and/or switch port.
I have done both of those (a completely different switch, in fact), but to no avail. Cheers, MJ -- Michael-John Turner * mj@mjturner.net * http://mjturner.net/
On 2016-08-23 21:20, Michael-John Turner wrote:
On Tue, Aug 23, 2016 at 12:02:46PM +0200, Daniel Karrenberg wrote:
Try a different ethernet cable and/or switch port.
I have done both of those (a completely different switch, in fact), but to no avail.
Cheers, MJ
Hi, It is possible that the probe is in some irrecoverable state. However, before requesting a new one, please try plugging it in into a different network (friend's home, work or such) and leave it for a bit. If this works then you can rule out the probe being faulty. (We had many cases where the probe was sent back to us, and simply plugging it in on our network "just worked".) Regards, Robert
On Wed, Aug 24, 2016 at 10:17:14AM +0200, Robert Kisteleki wrote:
It is possible that the probe is in some irrecoverable state. However, before requesting a new one, please try plugging it in into a different network (friend's home, work or such) and leave it for a bit. If this works then you can rule out the probe being faulty.
Good thinking. I'll do that in the next few days and report back my findings.
(We had many cases where the probe was sent back to us, and simply plugging it in on our network "just worked".)
If I don't get the probe working, what is the process to request a replacement? Cheers, MJ -- Michael-John Turner * mj@mjturner.net * http://mjturner.net/
On Tue, Aug 23, 2016 at 10:52:58AM +0200, Philip Homburg wrote:
One possible explanation is that the probe doesn't actually receive any packets (i.e. the receive circuitry at the ethernet level is broken). The only way to know for sure is to open up the probe and attach a serial console. But I'm not aware of any software failure mode that leads to this behavior.
Hmm. The odd thing is that the first time I powered it on (after it had been powered off for a long time), it received the lease correctly. When that expired and it needed to renew, the problems started gain (and exactly the same DHCP problem I had that caused me to disconnect it some years back). Oddly, even after receiving an IP the first time, it was able to do DNS requests (they showed up in the SOS log on the Atlas site), but it wasn't able to report via ssh on 443 (which works fine from other hosts on the same network). Argh, this is so frustrating :( Is it possible to open a V1 probe and connect a serial console without damaging it? Cheers, MJ -- Michael-John Turner * mj@mjturner.net * http://mjturner.net/
Hi, On Tue, Aug 23, 2016 at 08:24:17PM +0100, Michael-John Turner wrote:
Oddly, even after receiving an IP the first time, it was able to do DNS requests (they showed up in the SOS log on the Atlas site), but it wasn't able to report via ssh on 443 (which works fine from other hosts on the same network).
Well, it can *send*. It might not be able to *hear* the answer to the DNS requests, so the fact that the SOS messages are seen at Atlas Central doesn't really say anything about receiving of packets after the initial DHCP packet... So, it could just be borderline broken... if cold enough, it works, if the room is warming up, it no longer receives packets... Gert Doering -- NetMaster -- have you enabled IPv6 on something today...? SpaceNet AG Vorstand: Sebastian v. Bomhard Joseph-Dollinger-Bogen 14 Aufsichtsratsvors.: A. Grundner-Culemann D-80807 Muenchen HRB: 136055 (AG Muenchen) Tel: +49 (0)89/32356-444 USt-IdNr.: DE813185279
The best way I think is to request newer probe. Thank you budiwijaya On Wed, Aug 24, 2016 at 2:51 AM, Gert Doering <gert@space.net> wrote:
Hi,
On Tue, Aug 23, 2016 at 08:24:17PM +0100, Michael-John Turner wrote:
Oddly, even after receiving an IP the first time, it was able to do DNS requests (they showed up in the SOS log on the Atlas site), but it wasn't able to report via ssh on 443 (which works fine from other hosts on the same network).
Well, it can *send*. It might not be able to *hear* the answer to the DNS requests, so the fact that the SOS messages are seen at Atlas Central doesn't really say anything about receiving of packets after the initial DHCP packet...
So, it could just be borderline broken... if cold enough, it works, if the room is warming up, it no longer receives packets...
Gert Doering -- NetMaster -- have you enabled IPv6 on something today...?
SpaceNet AG Vorstand: Sebastian v. Bomhard Joseph-Dollinger-Bogen 14 Aufsichtsratsvors.: A. Grundner-Culemann D-80807 Muenchen HRB: 136055 (AG Muenchen) Tel: +49 (0)89/32356-444 USt-IdNr.: DE813185279
participants (7)
-
Budiwijaya -
Daniel Karrenberg -
Gert Doering -
Michael-John Turner -
Philip Homburg -
Robert Kisteleki -
Viktor Naumov