Often customer get stuck with designs that were modern for the era of time they first were implemented, and then new features cannot be utilized or cannot be utilized to their fully potential. Security is one of those things, or where features are not used or cannot be used because the old design did not take any consideration to this. Now one or almost 2 decades after that initial implementation, your solution is still stuck in the network design that was “normal” in the beginning of 2000’s. Does this sounds familiar or can you relate to this? Well then this post might be the right post for you to read especially if you are planning to upgrade Vmware.
Primarily this blogpost focuses on cross vCenter vmotion, or where you move or migrate one VM from one vCenter to another vCenter without having any form of trust or connection in between them. Obviously, having a network connection among the vCenters is a pre-requisite so that you must at least have. We will also be focusing on the performance part and some general recommendations to get most performance. So lets starts talking about the scenario.
The picture below depicts to vCenters, one names A and another one named B. The goal is to move a VM from A-B or vice versa depending on your needs.
The feature has been available for some time, however, it is not being used as extensively as one might expect. The reasons might be many, however the use case of such a scenario might be
1 – Changing of vCenter core design
2 – Network segmentation
3 – Merger and acquisition scenarios
Change of vCenter core design/Network segmentation
If you find yourself in a scenario where there are major design changes involved and DNS and FQDN has not been properly used you are almost forced to go for a redesign of vCenter, in case you are trying to implement a segmented network design and where this does not suits with the existing vCenter you might end up redesigning vCenter. In such a case you should consider following not only the best practices for Vmware and vCenter implementation in general, but also recommendation regarding secure management. At a bare minimum you should differentiate between control and data planes. If you do not know what I am talking about, my fried you really need to focus on the security in your environment. Anyhow, while designing Vmware based solution or any other virtualization based platform there are I guess some key areas that needs your attention. These include compute, storage and network, in addition you have the security layer which is applicable to all of the previously mentioned areas. More than often, customers utilize SAN based storage. Now a days, things are moving in the direction called hyper converged implementation, where your hosts have local storage which is replicated amongst your hosts. One or even several failing hosts do not effect the health of your cluster and you have the time to either change/repair hardware on defect hosts or just completely remove them and introduce new ones. In this kind of scenario, or honestly speaking in any SMB customer design, you would have two networks which are dedicated to Vmware i.e. vMotion and vSAN. In addition to these you will have the data plane and control planes. vMotion and vSAN are only used by Vmware traffic for internal functions like migration of VMs or replicating their storage. In case you are using iscsi, FC or any form for NAS type storage, you will also have dedicated network for storage.
The dialog above was just a preset, to set the stage for what is coming next. When you need to move VMs from an existing vCenter to another vCenter, you will be moving the VMs logically, physically or both depending on the implementation that is applicable for you. In case you are using vSAN in a hyper converged scenario, you are definitely going to be moving the VMs physically from vCenter solution to another. In this scenario it becomes pivotal to know about which network paths will be utilized during migration and to what extent will this cause strain or bottleneck for your production during the migration. When using vSAN and migrating Vms, you will be utilizing the vSAN network. If you have designed your solution properly, this can be a journey or it can be a disaster all depending on your design.
Merger and acquisitions
In case your company has bough another business or you have been sold and are to be merged in another environment, it might be useful to use cross vCenter migrations. Either one being the case, this can save you a lot of time and effort.
Prerequisites for vCenter cross vMotion
An obvious requirement is the version of both vCenters that you are using. These must be 6.x or higher, and most preferably 7.x At once you are above level 7, you get the option for migrating VMs in the GUI. Vmware has a KB article describing the details and version requirements here: https://kb.vmware.com/s/article/2106952
Following bidirectional port communication is required between the vCenters where you are intending to migrate the VMs.
- TCP – 80 – Http
- TCP – 443 – Https
- TCP – 902 – Vmware port
- TCP/UDP 53 – DNS
In addition to this, the vSAN or SAN/Storage network must be the same, or I should say that it should be the same. If you choose to migrate a VM between two different networks where they traverse through a firewall, it can take some serious amount of time. Normally the storage will be on the same vlan and all hosts will be connected to it directly. If you still cannot have the storage on the same vlan, Vmware recommends at least 1 Gbps connection for vSAN replication. If you consider moving a VM on lets say 100 GB or larger you can think yourself how much time it will require passing through a firewall where the throughput is at most a couple of gigabytes per second, i.e. when advanced features have been enabled. My job is to provide a recommendation based on practice and experience, and you are the one who needs to decide which path you want to take.
The migration process or the actual move
Assuming that you have name resolution functioning and NTP configured at both source and destination along with symmetric network access we are now ready to perform the cross vCenter migrations.
The process starts in vCenter where you choose a VM and choose to migrate the VM. Right click the VM, then choose Migrate…
The wizard start up, and if you are running vSphere 7.x then you will have the option choose cross vCenter migration
In the next Windows we choose details about the target vCenter
When all the moving parts fit together adding credentials and pressing next will cause a little warning message, that is if the both vCenters do not trust each other.
This is a good sign, now you will be presented with a message confirming that you are connected to target vCenter. You can also choose to save the credentials, so that during next move you do not have to re-enter the same information. The information is saved per user i.e. saved for your user profile.
The next step is to choose the target destination folder or resource group/pool where you will place your VM.
It is important to pay attention to that if you have any resources actively being used from datastore these will cause a warning message to be displayed and at a later stage the migration process will fail. The best way to proceed is to remove and dependencies or anything else causing warnings.
In the next page you choose storage options for your VM in the target vCenter
If you are moving a VM from one type of storage to another you will get warnings. This can be a case where you are moving from hyper converged to converged or vice versa scenario. In either case the migration will be successful but will trigger a warning informing you about possible mismatches and performance degradation.
At the last page you are presented with a summary of the choices that you have made during the migration. Look through these and hit finish. Now the fun part starts, or should I say, the waiting part.
And the result might not be what you were expecting. If you are migrating a stopped VM or turned off VM, the migration process is done being considered a low priority task. The move/migration job can take quite some time. In my case here, this test VM is about 50GB of size on disk and the move took about 30 minutes on a 10Gbps interface or should I say on a pair of 10Gbps interfaces.
Cross vCenter vmotion – vSphere 7.x VM migration
I guess the recommended and more operational approach would be to migrate running VMs. When the migration wizard is run while a VM is running, you are presented with the option to perform the migration marking the task as highly prioritized task. In most environments, you will have necessary resources and reserves available. However, if you have over committed compute and storage, network utilization you might run into trouble. So be careful regarding amount of resources you have available.
The process starts and continues to be the same till you reach the wizard page where you can define the priority and schedule for the migration
Even the summary page is almost the same, except for the fact that it includes you guessed it, priority.
This time you really will see the difference, on the same test VM, now the cross vCenter vMotion or migration took less than 5 minutes, that is almost 10Gbps per minute, roughly speaking.
My mistake, it took even less than 4 minutes, to be honest we are speaking about 3,5 minutes or so.
Cross vCenter vMotion recommendation
As the post clearly shows, if you are planning to move VMs across vCenter, you should really consider moving them while they are running and so that you can give them highest possible priority. On the other hand if you have very limited resources you can choose to power off a VM and migrate it from one vCenter to the other. But in the later case you would be using a much longer time period.