Robert Glover via Outages | 30 Jan 23:58 2015

[outages] Charter Voice outage

Just got told by a charter rep that there is an issue affecting Charter 
Voice in multiple US states.

We are hearing from multiple Charter cable customers in California that 
their Charter Voice service is not working at all.  A sub-set of those 
customers also report no internet as well.

Anyone have any more information?

-Robert
Bill Wichers | 30 Jan 22:12 2015
Picon

Easy question for anyone with a Comcast BGP feed

If anyone has a BGP feed from Comcast, can you please check if you see 192.83.112.0/24 and 192.83.113.0/24 coming from them?

Comcast does not appear to run an internal looking glass and we don’t announce routes through them outside their own network so I can’t easily test this on my own.

 

Thanks in advance.

 

 -Bill

<div>
<div class="WordSection1">
<p class="MsoNormal">If anyone has a BGP feed from Comcast, can you please check if you see 192.83.112.0/24 and 192.83.113.0/24 coming from them?
<p></p></p>
<p class="MsoNormal">Comcast does not appear to run an internal looking glass and we don&rsquo;t announce routes through them outside their own network so I can&rsquo;t easily test this on my own.<p></p></p>
<p class="MsoNormal"><p>&nbsp;</p></p>
<p class="MsoNormal">Thanks in advance.<p></p></p>
<p class="MsoNormal"><p>&nbsp;</p></p>
<p class="MsoNormal">&nbsp;-Bill<p></p></p>
</div>
</div>
John Pittavino via Outages | 29 Jan 06:02 2015

[outages] Level 3 packet loss - Pittsburgh PA

Is anyone seeing packet loss on level3 in the Pittsburgh PA area? It has been on and off. Started around 8:30pm ET. 


I have a ticket with them on which they just responded


"The IP NOC advised that an issue was detected on a Border Gateway Protocol. The affected Border Gateway Protocol was shutdown, which resolved the issue."


However, I am still seeing it:


-----------------------------------------------------------------------------------------

Reply from 4.69.135.242: bytes=32 time=22ms TTL=250
Reply from 4.69.135.242: bytes=32 time=24ms TTL=250
Reply from 4.69.135.242: bytes=32 time=23ms TTL=250
Reply from 4.69.135.242: bytes=32 time=22ms TTL=250

Ping statistics for 4.69.135.242:
    Packets: Sent = 499, Received = 498, Lost = 1 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 20ms, Maximum = 27ms, Average = 22ms

------------------------------------------------------------------------------------------
Reply from 4.34.25.185: bytes=32 time=71ms TTL=250
Request timed out.
Request timed out.
Reply from 4.34.25.185: bytes=32 time=681ms TTL=250
Request timed out.

Ping statistics for 4.34.25.185:
    Packets: Sent = 386, Received = 356, Lost = 30 (7% loss),
Approximate round trip times in milli-seconds:
    Minimum = 23ms, Maximum = 1337ms, Average = 77ms​​


<div>
<div>
<p></p>
<p>Is anyone seeing packet loss on&nbsp;level3 in the Pittsburgh PA area? It has been on and off. Started around 8:30pm ET.&nbsp;<br></p>
<p><br></p>
<p>I have a ticket with them on which they just responded<br></p>
<p><br></p>
<p>"<span>The IP
 NOC advised that an issue was detected on a Border Gateway Protocol. The affected Border Gateway Protocol was shutdown, which resolved the issue."</span></p>
<p><br></p>
<p>However, I am still seeing it:<br></p>
<p>&#8203;<br></p>
<p>-----------------------------------------------------------------------------------------<br></p>
<div>Reply from 4.69.135.242: bytes=32 time=22ms TTL=250<br>
</div>
<div>Reply from 4.69.135.242: bytes=32 time=24ms TTL=250</div>
<div>Reply from 4.69.135.242: bytes=32 time=23ms TTL=250</div>
<div>Reply from 4.69.135.242: bytes=32 time=22ms TTL=250</div>
<div><br></div>
<div>Ping statistics for 4.69.135.242:</div>
<div>&nbsp; &nbsp; Packets: Sent = 499, Received = 498, Lost = 1 (0% loss),</div>
<div>Approximate round trip times in milli-seconds:</div>
<div>&nbsp; &nbsp; Minimum = 20ms, Maximum = 27ms, Average = 22ms<br>
</div>
<div><br></div>
<div>------------------------------------------------------------------------------------------<br>
</div>
<div>
<div>Reply from 4.34.25.185: bytes=32 time=71ms TTL=250</div>
<div>Request timed out.</div>
<div>Request timed out.</div>
<div>Reply from 4.34.25.185: bytes=32 time=681ms TTL=250</div>
<div>Request timed out.</div>
<div><br></div>
<div>Ping statistics for 4.34.25.185:</div>
<div>&nbsp; &nbsp; Packets: Sent = 386, Received = 356, Lost = 30 (7% loss),</div>
<div>Approximate round trip times in milli-seconds:</div>
<div>&nbsp; &nbsp; Minimum = 23ms, Maximum = 1337ms, Average = 77ms&#8203;&#8203;<br>
</div>
</div>
<p><br></p>
</div>
</div>
Norman Brown via Outages | 28 Jan 19:37 2015

[outages] East coast problems between TeliaSonera and Verizon

Is anyone experiencing problems between Verizon and TeliaSonera? I'm in Pittsburgh, PA and anything that goes to TeliaSonera is 100+ ms and throughput is incredibly low. I apologize for the poor formatting below:

|------------------------------------------------------------------------------------------|
|                                      WinMTR statistics                                   |
|                       Host              -   %  | Sent | Recv | Best | Avrg | Wrst | Last |
|------------------------------------------------|------|------|------|------|------|------|
|    L100.PITBPA-VFTTP-33.verizon-gni.net -    0 |    8 |    8 |    2 |    3 |    4 |    4 |
|  G0-4-1-3.PITBPA-LCR-21.verizon-gni.net -    0 |    8 |    8 |    6 |    7 |    9 |    8 |
|                   No response from host -  100 |    1 |    0 |    0 |    0 |    0 |    0 |
|           0.so-6-0-1.XL3.IAD8.ALTER.NET -    0 |    8 |    8 |   12 |   27 |   85 |   14 |
|       TenGigE0-6-0-1.GW1.IAD8.ALTER.NET -    0 |    8 |    8 |   13 |   17 |   25 |   14 |
|       teliasonera-gw.customer.alter.net -    0 |    8 |    8 |   97 |  108 |  116 |  109 |
|                  ash-bb3-link.telia.net -    0 |    8 |    8 |  100 |  109 |  117 |  111 |
|                  nyk-bb1-link.telia.net -    0 |    8 |    8 |  106 |  122 |  151 |  114 |
|                   nyk-b6-link.telia.net -    0 |    8 |    8 |  106 |  112 |  120 |  113 |
|digitalocean-ic-302450-nyk-b6.c.telia.net -   20 |    5 |    4 |  106 |  116 |  122 |  122 |
|                          198.211.112.36 -    0 |    8 |    8 |  106 |  114 |  124 |  117 |
|________________________________________________|______|______|______|______|______|______|
   WinMTR v0.92 GPL V2 by Appnor MSP - Fully Managed Hosting & Cloud Provider

                                                         Packets               Pings
 Host                                                  Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. 104.131.0.253                                       0.0%   121    0.6   1.8   0.3  12.5   2.5
 2. 162.243.188.221                                     0.0%   121    0.5   1.3   0.3  27.1   3.7
 3. nyk-b3-link.telia.net                               0.0%   120    1.2   2.0   0.9  48.1   5.5
 4. nyk-bb2-link.telia.net                              0.0%   120   59.3  12.6   1.3 102.4  23.2
 5. nyk-b5-link.telia.net                               0.0%   120    1.9   2.1   1.7  14.3   1.5
 6. GigabitEthernet1-0-0.GW1.NYC4.ALTER.NET             0.0%   120    1.7   1.8   1.5   2.6   0.0
 7. ???
 8. pool-74-111-114-111.pitbpa.fios.verizon.net         0.0%   120  116.2 115.5  95.9 124.2   6.4

<div><div dir="ltr">Is anyone experiencing problems between Verizon and TeliaSonera? I'm in Pittsburgh, PA and anything that goes to TeliaSonera is 100+ ms and throughput is incredibly low. I apologize for the poor formatting below:<div><br></div>
<div>
<div>|------------------------------------------------------------------------------------------|</div>
<div>| &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;WinMTR statistics &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; |</div>
<div>| &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Host &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;- &nbsp; % &nbsp;| Sent | Recv | Best | Avrg | Wrst | Last |</div>
<div>|------------------------------------------------|------|------|------|------|------|------|</div>
<div>| &nbsp; &nbsp;<a href="http://L100.PITBPA-VFTTP-33.verizon-gni.net">L100.PITBPA-VFTTP-33.verizon-gni.net</a> - &nbsp; &nbsp;0 | &nbsp; &nbsp;8 | &nbsp; &nbsp;8 | &nbsp; &nbsp;2 | &nbsp; &nbsp;3 | &nbsp; &nbsp;4 | &nbsp; &nbsp;4 |</div>
<div>| &nbsp;<a href="http://G0-4-1-3.PITBPA-LCR-21.verizon-gni.net">G0-4-1-3.PITBPA-LCR-21.verizon-gni.net</a> - &nbsp; &nbsp;0 | &nbsp; &nbsp;8 | &nbsp; &nbsp;8 | &nbsp; &nbsp;6 | &nbsp; &nbsp;7 | &nbsp; &nbsp;9 | &nbsp; &nbsp;8 |</div>
<div>| &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; No response from host - &nbsp;100 | &nbsp; &nbsp;1 | &nbsp; &nbsp;0 | &nbsp; &nbsp;0 | &nbsp; &nbsp;0 | &nbsp; &nbsp;0 | &nbsp; &nbsp;0 |</div>
<div>| &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <a href="http://0.so-6-0-1.XL3.IAD8.ALTER.NET">0.so-6-0-1.XL3.IAD8.ALTER.NET</a> - &nbsp; &nbsp;0 | &nbsp; &nbsp;8 | &nbsp; &nbsp;8 | &nbsp; 12 | &nbsp; 27 | &nbsp; 85 | &nbsp; 14 |</div>
<div>| &nbsp; &nbsp; &nbsp; <a href="http://TenGigE0-6-0-1.GW1.IAD8.ALTER.NET">TenGigE0-6-0-1.GW1.IAD8.ALTER.NET</a> - &nbsp; &nbsp;0 | &nbsp; &nbsp;8 | &nbsp; &nbsp;8 | &nbsp; 13 | &nbsp; 17 | &nbsp; 25 | &nbsp; 14 |</div>
<div>| &nbsp; &nbsp; &nbsp; <a href="http://teliasonera-gw.customer.alter.net">teliasonera-gw.customer.alter.net</a> - &nbsp; &nbsp;0 | &nbsp; &nbsp;8 | &nbsp; &nbsp;8 | &nbsp; 97 | &nbsp;108 | &nbsp;116 | &nbsp;109 |</div>
<div>| &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;<a href="http://ash-bb3-link.telia.net">ash-bb3-link.telia.net</a> - &nbsp; &nbsp;0 | &nbsp; &nbsp;8 | &nbsp; &nbsp;8 | &nbsp;100 | &nbsp;109 | &nbsp;117 | &nbsp;111 |</div>
<div>| &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;<a href="http://nyk-bb1-link.telia.net">nyk-bb1-link.telia.net</a> - &nbsp; &nbsp;0 | &nbsp; &nbsp;8 | &nbsp; &nbsp;8 | &nbsp;106 | &nbsp;122 | &nbsp;151 | &nbsp;114 |</div>
<div>| &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <a href="http://nyk-b6-link.telia.net">nyk-b6-link.telia.net</a> - &nbsp; &nbsp;0 | &nbsp; &nbsp;8 | &nbsp; &nbsp;8 | &nbsp;106 | &nbsp;112 | &nbsp;120 | &nbsp;113 |</div>
<div>|<a href="http://digitalocean-ic-302450-nyk-b6.c.telia.net">digitalocean-ic-302450-nyk-b6.c.telia.net</a> - &nbsp; 20 | &nbsp; &nbsp;5 | &nbsp; &nbsp;4 | &nbsp;106 | &nbsp;116 | &nbsp;122 | &nbsp;122 |</div>
<div>| &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;198.211.112.36 - &nbsp; &nbsp;0 | &nbsp; &nbsp;8 | &nbsp; &nbsp;8 | &nbsp;106 | &nbsp;114 | &nbsp;124 | &nbsp;117 |</div>
<div>|________________________________________________|______|______|______|______|______|______|</div>
<div>&nbsp; &nbsp;WinMTR v0.92 GPL V2 by Appnor MSP - Fully Managed Hosting &amp; Cloud Provider</div>
</div>
<div><br></div>
<div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;Packets &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Pings</div>
<div>&nbsp;Host &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;Loss% &nbsp; Snt &nbsp; Last &nbsp; Avg &nbsp;Best &nbsp;Wrst StDev</div>
<div>&nbsp;1. 104.131.0.253 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 0.0% &nbsp; 121 &nbsp; &nbsp;0.6 &nbsp; 1.8 &nbsp; 0.3 &nbsp;12.5 &nbsp; 2.5</div>
<div>&nbsp;2. 162.243.188.221 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 0.0% &nbsp; 121 &nbsp; &nbsp;0.5 &nbsp; 1.3 &nbsp; 0.3 &nbsp;27.1 &nbsp; 3.7</div>
<div>&nbsp;3. <a href="http://nyk-b3-link.telia.net">nyk-b3-link.telia.net</a> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 0.0% &nbsp; 120 &nbsp; &nbsp;1.2 &nbsp; 2.0 &nbsp; 0.9 &nbsp;48.1 &nbsp; 5.5</div>
<div>&nbsp;4. <a href="http://nyk-bb2-link.telia.net">nyk-bb2-link.telia.net</a> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;0.0% &nbsp; 120 &nbsp; 59.3 &nbsp;12.6 &nbsp; 1.3 102.4 &nbsp;23.2</div>
<div>&nbsp;5. <a href="http://nyk-b5-link.telia.net">nyk-b5-link.telia.net</a> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 0.0% &nbsp; 120 &nbsp; &nbsp;1.9 &nbsp; 2.1 &nbsp; 1.7 &nbsp;14.3 &nbsp; 1.5</div>
<div>&nbsp;6. <a href="http://GigabitEthernet1-0-0.GW1.NYC4.ALTER.NET">GigabitEthernet1-0-0.GW1.NYC4.ALTER.NET</a> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 0.0% &nbsp; 120 &nbsp; &nbsp;1.7 &nbsp; 1.8 &nbsp; 1.5 &nbsp; 2.6 &nbsp; 0.0</div>
<div>&nbsp;7. ???</div>
<div>&nbsp;8. <a href="http://pool-74-111-114-111.pitbpa.fios.verizon.net">pool-74-111-114-111.pitbpa.fios.verizon.net</a> &nbsp; &nbsp; &nbsp; &nbsp; 0.0% &nbsp; 120 &nbsp;116.2 115.5 &nbsp;95.9 124.2 &nbsp; 6.4</div>
</div>
<div><br></div>
</div></div>
Jason Lixfeld via Outages | 28 Jan 17:04 2015

[outages] Bell 100G card failure North Toronto

Happened around 03:15 GMT-5.  We lost a 10G wave between their Don Mills CO and their Sheppard Ave. E CO.  They
are currently dispatching transport folks.  Just trying to get clarification on exactly which CO the card
was in.
Frank Bulk (iname.com | 27 Jan 14:12 2015
Picon

Facebook were down for a while

http://www.reuters.com/article/2015/01/27/us-facebook-down-idUSKBN0L00GE2015
0127?feedType=RSS&feedName=topNews&utm_source=twitter
http://www.theverge.com/2015/1/27/7920161/facebook-instagram-down-lizard-squ
ad-hints-at-involvement

Our own monitoring system saw HTTP and ICMP access to www.facebook.com over
IPv6 down from 12:13 am to 1:07 am (U.S. Central).

Frank

Grant Ridder via Outages | 25 Jan 21:38 2015

[outages] AWS us-west-2 network issue?

Anyone else seeing issues to and from us-west-2?

Looks like routing between us-west-2 and us-west-1 are going though us-east-1 and eu-west-2 as well as a high packet loss from the SF bay area into us-west-2.

-Grant
<div><div dir="ltr">Anyone else seeing issues to and from us-west-2?<div><br></div>
<div>Looks like routing between us-west-2 and us-west-1 are going though us-east-1 and eu-west-2 as well as a high packet loss from the SF bay area into us-west-2.</div>
<div><br></div>
<div>-Grant</div>
</div></div>
Pete E via Outages | 23 Jan 14:23 2015

[outages] Qwest/Centurylink toll free

Anyone seeing an issue with Centurylink (legacy Qwest) toll free originations? Around 1:20AM EST we see very poor quality / one-way audio.
<div><div dir="ltr">Anyone seeing an issue with Centurylink (legacy Qwest) toll free originations? Around 1:20AM EST we see very poor quality / one-way audio.</div></div>
Mehmet Akcin via Outages | 16 Jan 18:37 2015

[outages] dns cache flush request

can you please flush cache for windowsmedia.com


mehmet

<div><div dir="ltr">
<div>can you please flush cache for&nbsp;<a href="http://windowsmedia.com/" target="_blank">windowsmedia.com</a> </div>
<div><div>
<p class="MsoNormal"><span></span><br></p>
<p class="MsoNormal"><span>mehmet</span></p>
</div></div>
</div></div>
Mauricio Lizano via Outages | 16 Jan 16:54 2015

[outages] Level 3 call cuality issues

Anybody else experiencing call quality issues with Level 3 SIP trunks? Termination and Orig?

Thanks,

--
Mauricio Lizano

<div><div dir="ltr">Anybody else experiencing call quality issues with Level 3 SIP trunks? Termination and Orig?<div><br></div>
<div>Thanks,<br clear="all"><div><br></div>-- <br><div class="gmail_signature">
<div>Mauricio Lizano</div>
<div><br></div>
</div>
</div>
</div></div>
Jay Ashworth via Outages | 15 Jan 19:09 2015

Re: [outages] EDGE: Anyone seeing 100% CPU on Fortigate edge routers?

----- Original Message -----
> From: "Jeremy Slater" <jeremy.slater@...>

> We have a large amount of 60D's deployed and we're not seeing any
> similar behavior.

Thanks, Jeremy; it's not the answer I hoped for, but it's helpful.  :-)

Cheers,
-- jra
--

-- 
Jay R. Ashworth                  Baylink                       jra@...
Designer                     The Things I Think                       RFC 2100
Ashworth & Associates       http://www.bcp38.info          2000 Land Rover DII
St Petersburg FL USA      BCP38: Ask For It By Name!           +1 727 647 1274

Gmane