[ceph-users] Live migration of VM using librbd and OpenStack
trhoden at gmail.com
Wed Mar 13 14:00:15 PDT 2013
Just for posterity, my ultimate solution was to patch nova on each
compute host to always return True in _check_shared_storage_test_file
This did make migration work with "nova live-migration", with one
caveat. Since Nova is assuming that /var/lib/nova/instances is on
shared storage (and since I hard coded the check to say "yes, it
really is"), it thinks the /var/lib/nova/instances/<domain> folder
will exist at both source and destination, and makes no attempt to
create it on the destination. So before I run live-migration, I pop
over to the source host, and rsync that folder to the destination.
A little dirty, but it allows me to move running VMs around just fine
in cases of maintenance on a host, which is exactly what I need.
Thanks for everyone's feedback.
On Tue, Mar 12, 2013 at 6:33 PM, Travis Rhoden <trhoden at gmail.com> wrote:
> On Tue, Mar 12, 2013 at 5:06 PM, Josh Durgin <josh.durgin at inktank.com>
>> On 03/12/2013 01:48 PM, Travis Rhoden wrote:
>>> Hi Josh,
>>> Thanks for the info. So if I want to do live migration with VMs that
>>> launched with boot-from-volume, I'll need to use virsh to do the
>>> rather than Nova. Okay, that should be doable. As an aside, I will
>>> probably want to look at the OpenStack DB and figure out how to tell it
>>> that the VM has moved to a different host. I'd rather there not be a
>>> disconnect between Nova and libvirt about where the VM lives. =)
>> It's probably not too hard to edit nova to skip the checks when the
>> instance is volume-backed, but if you don't want to do that, libvirt
>> should be fine, and a bit more flexible.
> After messing with it for a few hours, I'm thinking about doing just that.
> The nova edits should be easy. Looks like it tests for shared storage by
> writing a file on the migration destination, and trying to read it at the
> source. I should be able to just comment out the check entirely, or make
> the check always pass.
> The virsh migrate strategy has been surprisingly difficult. Since I'm
> migrating a Nova VM, I had to the following pre-requisites (so far).
> Define the /var/lib/nova/instance/<domain> dir on the destination
> Define/migrate the nova libvirt-nwfilter for the specific VM
> Then, when I try to do the actual migration, I always get (at the source):
> error: internal error Process exited while reading console log output:
> chardev: opening backend "file" failed: Permission denied
> So QEMU is bailing, saying it can't read the console.log file. When I go
> look at that file, it is created, but with owner root:root and perms 0600.
> However, libvirtd makes it libvirt-qemu:kvm, 0600 before KVM tries to
> actually start the VM. I've always found this dynamic file ownership bit in
> KVM/libvirt/qemu very confusing. Anyways, I tried a few different things,
> debug logging, etc. Even tried disabling apparmor. Still get permission
> denied each time.
> The commands I"m running manually should be identical to what OpenStack is
> doing, so I can't figure out why their migrate is working and mine wouldn't.
> Oh well, will edit Nova and give that shot.
> - Travis
More information about the ceph-users