Re: BGP over satellite link - update!!

[Geoff Huston <cidr-report at telstra.net>]

>1. It can help a lot to have routers that you administer on both sides of 
>the satellite circuit (i.e. don't just buy a satellite internet service; 
>buy a small amount of colo at the earthstation on the sending side and 
>uplink directly from there).


I did not have that config - the satellite connection used eBGP and the routers
at either end were operated by different entities. It did not worry me as much
as it did for Joe, evidently - I had no problems with the split responsibility.


>2. If possible, try and land your back-channel (whether it's narrowband 
>terrestrial capacity, or a carrier on a different satellite) in the same 
>place in the remote location. If you can do that, you can build a tunnel 
>which presents both send and receive channels as a single interface, which 
>can be a good thing.


I did neither! The eBGP loopback addresses were part of normal aggregated
announcements. All that was required was a static route entry on the
feeding routers pointing to the unidirectional satellite circuit interface
as the nexthop for the receiver's loopback address.

I didn't see the benefit of the added complexity of attempting to
fake out a bi-directional channel.



>3. If your solution involves tunnels, think long and hard before you run 
>production internet traffic over a link that has an MTU less than 1500 
>bytes. There are a lot of broken firewalls in the world, and you'll find 
>lots of web and mail breakage due to stalled TCP sessions and broken path 
>MTU discovery. [I tried this; it was not fun].


Exactly - tunnels are less than fun and probably well worth avoiding.


>Our satellite circuit was presented through SDM2020 [de]modulators as HSSI 
>to our routers. We couldn't find HSSI cables that were suitable to connect 
>to unidirectional circuits, so we wound up buying a handful of those 
>horribly expensive cisco HSSI cables and breaking them open, looping 
>various tx/rx/whatever lines back to other connectors to make the 
>interface appear up to the router. We had to try that a couple of times 
>since our HSSI interfaces were kind of sensitive to badly made cables 
>(e.g. cables made by me :-)
>
>I spent quite a bit of time worrying about how I could get traffic to 
>re-route in a sane way in the event of a satellite channel failure, or at 
>least get dropped in a way that could cause alerts. There were definitely 
>occasional failures where the sending side would merrily send frames 
>towards the modulator and the uplink, and nothing would be received at the 
>downlink; apart from the absense of traffic on the receiving side, there 
>was no "interface down" type event to tell us that anything was wrong.
>
>Constructing a tunnel and running OSPF over it worked, since a dead 
>satellite would lead to dropped OSPF hellos. This also had the side effect 
>of lowering the MTU on the NZ->US path, though, since there was ethernet 
>involved in it. Eventually we decided that the lower MTU was causing more 
>problems than the occasional satellite outage, and we ripped all the 
>tunnel stuff out.

And here is where eBGP shines - a dead satellite circuit leads to BGP
keepalive failure, leads to the feeder router dropping its eBGP session
to the receiver router, leads to the feeder router dropping its advertisements
of reachable networks to the world at large, causing your traffic to be sent
to you by less preferred backup paths (if such backup routes exist).

regards,

   Geoff




-----
This is the afnog mailing list, managed by Majordomo 1.94.5

To send a message to this list, e-mail afnog at afnog.org
To send a request to majordomo, e-mail majordomo at afnog.org and put
your request in the body of the message (i.e use "help" for help)

This list is maintained by owner-afnog at afnog.org