home | list info | list archive | date index | thread index

Re: [OCLUG-Tech] Hybrid cloud+private virtualization for CentOS servers - recommendations?

On 12-08-14 12:39 PM, Steve La Rocque wrote:
Hi guys.

I'm at a nonprofit educational outfit here in Ottawa and for a few years
my organization (the Canadian Math Socity) has been running two CentOS
headless servers with Xen hypervisors on them.  Each of these supports
several virtual machines, which are all running CentOS as well.

Now we're looking for the next generation upgrades, both hardware and
platform and are looking for best current recommendations, with a heavy
leaning towards open source options.  Any comments or recommendations
would be very helpful!

First, we don't know if Xen (I'm talking about the freeware Xen) is
still the best approach.  Problems we've had are in terms of
migrate-ability (our servers are partly intended to back one another up
such that all the VMs could conceivably run under one metal box) and
provisioning new VMs.  There are a lot of other options I think, but we
do want something quite solid for headless servers running in a rack.
What would you do if you could re-start from scratch?

Second, we would like to use something that can blend with cloud hosting
as well (such that we could have VMs elsewhere, but managed by the same
software on our site -- we are interested in getting into selling cloud
hosting).

Have you looked at OpenStack[1]? Everything you have asked for can be done with it. Personally I like libvirt / KVM for all my VMs, mostly because it just works for almost everything we do. I do have a customer site using Xen for their VMs which is running the software I manage however I don't have any direct interactions with the VMs.

The bonus is, OpenStack is basically using the same API's as Amazon, so there are many existing tools (e2tools under Ubuntu) which allow you to provision both OpenStack and EC2 instances. This would allow you to extend your local infrastructure into Amazon or Rackspace if needed.

Third, a related but separate question: What's the right way to run say
a dozen CentOS VMs in production and yet keep all the patching up to
date?  We're always worried about updates and new versions to various
installed products that will conflict with other installed products, yet
not running the updates and patches would mean security holes that don't
get filled and new features remain on the greener side of the fence.
There's probably no magic bullet here, but what techniques do you guys
use to keep your various VMs up to date without worrying about breaking
systems?  Is extensive testing environments the only option?

We use Puppet[2] to do exactly this. Combined with OpenStack from above you get a very powerful toolset that allow you to dynamically create, provision, and destroy new environments very quickly.

For example, when I provision new Asterisk PBX's for clients, I'll first generate the work order with the customer (picking extensions, phones, voicemail, etc). Then I'll go back to my development environment and provision everything using Puppet. After a few days, I'll move the instance into testing, making sure everything works properly and fixing any bugs I run into. Lastly, when it is time to move the instance into production, I do so not in my lab but on the customers hardware. Because everything is written within puppet, deploying the customer server takes a short amount of time (OS install + 8mins for puppet to run). And BLAM, a working production installation working exactly as I expected it to.

Because of the way puppet works, I am guaranteed the same setup of the system each time I do a new install. This allow me to quickly duplicate my clients environments within my lab, apply any new software or OS updates and confirming nothing get broken. Once I am happy things are working well, I simply have puppet update my remote instances because everything was already tested and working in the lab environment.

[1] http://openstack.org
[2] http://puppetlabs.com

--
Paul Belanger | PolyBeacon, Inc.
Jabber: paul [ dot ] belanger [ at ] polybeacon [ dot ] com | IRC: pabelanger (Freenode)
Github: https://github.com/pabelanger | Twitter: https://twitter.com/pabelanger