[afnog] Best Practices

[Geert Jan de Groot]

On Fri, 18 Dec 2009 10:59:28 +0100 Phil Regnauld wrote:
>     In terms of network architectures, you'll want to split out the
>     services, but also think how to place your servers with regards
>     to access levels -- sensitive servers such as authentication and
>     email storage probably should not be publicly visible.  

Ons line of thought these days - not sure if you should go this route,
but it is at least something to consider - is that people create
*dedicated*, (often virtual) machines for each of the services.
The thought is that by installing only a single service on each host,
maintenance and upgrading will be easier.

This in contrast to one or more "single boxes" that "do all".
If you ever need to upgrade one of those, then you need to verify
that *all* services on that box still work as expected after the
upgrade: what do you do if 7 out of 8 services do work, but 1 service
has a software issue and cannot (at this time) be upgraded?

People tend to virtualize these services on a pool of hardware,
so they can move virtual services from physical server to the next
as dictated by operational issues, by simply stopping the
virtual machine, moving the image to another box, and re-starting it.

Of course, if your application has high resource demands (such as
mail virus scanning), then you don't virtualize, but make a physical
box, perhaps more.

As an example, xs4all, the ISP I'm using now, when forced with the
ever-growing volume of mail (and growing volume of spam traffic, now 98%!)
re-engineered their mail subsystem some time ago. When I visited
their facilities, they had just installed the first rack of
mail relay / mail scan boxes, behind loadbalancing hardware.
They told me they had reserved the whole street of cabinets 
(30 cabinets at least, all of them empty when I was there),
for expansion and would meet higher volume demands by
simply filling the next rack, and, over time, the rest of the racks.

This approach does have side effects too - you need to carefully take
resources, such as I/O, traffic, memory into consideration, but it's
at least something to think about.

Phil's comments on proper network layout, proper network access policies
etc, very much applies in any case.

Geert Jan