Hi,
Recently I have been working on a project that requires me to migrate few multi-terabyte databases from physical to virtual machines.
Since we were lucky enough that the LUNs for those databases were hosting a LVM-backed filesystems I was able to present the LUNs as RDMs to the VMware virtual machines and then create new virtual hard disks and use the magical pvmove command to migrate the data.
The total downtime for each database is around 5-15 minutes and is mostly due to the fact that we have to present the LUNs to the virtual machine, mount the file systems and then chown the database files to a new uid/gid. After that is done the databases are started.
When the database has been verified to work as expected we created new virtual hard disks, ran pvcreate on them and import them to the volume group we were migrating.
After that we just fire up a trusty screen session (or tmux or whatever!) and run the mythical command: pvmove -i 10 -v /dev/oldlun /dev/newlun.
When that command finishes we remove the LUN from the volume group with vgreduce, run pvdestroy on the LUN and then remove the LUN from the virtual machine (you might want to run echo 1 >/sys/block/lunname/device/delete before you do that), unmap the LUN from the ESXi hosts and we are done!
The biggest reason for us not to use RDMs is that the flexibility we get by using native virtual disks kind of nulls all performance gains we might gain (with emphasis on might) by using RDMs (although I have yet to see any performance loss due to using VMFS). And when we finally make the jump over to vSphere 6.x I can migrate those virtual disks straight to VVols.
The only sad thing in our case is that by using this method we are stuck on EXT3 since the file systems are migrated over from old RHEL5 machines. I’m not sure I want to recommend anyone to run a migration from EXT3 to EXT4 on 6-16TB file systems đŸ˜€ (at least make sure you have a full backup available before testing this!).
Bgrds,
Finnur