Connection loss in Milan
Incident Report for SeFlow s.n.c.
Resolved
This incident has been resolved.
Posted 10 months ago. May 03, 2018 - 14:58 CEST
Monitoring
We monitored load in last 30 minutes and load is fine. New configuration seems to fix the issue.

We identified a customer with ton of /48 IPv6 blocks that is causing high cpu load with IPv6 process on distribution router in DC2 in Milan. Routers was online but lost advertisement on some network block to edge routers that stop advertising them outside our lan.

With new configuration, we rate-limited the customer fib usage. As we see he change thousand ips every second. We now rate limited it and routers are fine. We're testing this configuration to put globally to avoid that issue again
Posted 10 months ago. May 03, 2018 - 10:50 CEST
Update
we're rebooting routers last time to update new configuration
Posted 10 months ago. May 03, 2018 - 10:08 CEST
Update
we found the root cause and now the load on routers is lowering down. We will keep customer updated
Posted 10 months ago. May 03, 2018 - 10:02 CEST
Update
We still experiencing packet loss and routers are overloaded. We're investigating what cause the issue
Posted 10 months ago. May 03, 2018 - 09:43 CEST
Update
routers went offline again. we're still investigating
Posted 10 months ago. May 03, 2018 - 09:27 CEST
Update
connection has been restored. We're now investigating with routers went offline and vrrp protocol failed
Posted 10 months ago. May 03, 2018 - 09:16 CEST
Update
We not have any power outage and routers are all powered on. We're rebooting both distribution routers to see if connection will came back
Posted 10 months ago. May 03, 2018 - 09:07 CEST
Investigating
we're experiencing downtime in Milan Datacenter. we're investigating
Posted 10 months ago. May 03, 2018 - 08:56 CEST