Check Replica Status – Before deleting it – Part 1

Last month, a partner had to face-up a strange VBR behavior.

From VBR console he deleted a VM’s replica (Picture 1), and suddenly the production VM has been also erased (don’t worry, before doing any activity he tested the backups using sure backup technology).

Picture 1

The reason why it happened become clear to me once I read the logs.

To do it shortly, some weeks before someone started a Failover directly from vCENTER console without doing any communication to the internal IT team.

This article wants to explain how to avoid this common mistake.

The first step is understanding some basic concepts:

a) VMware identify any single VM with a number named MorefID and a UUID.

b) Any single operating system has an identifier named Instance UUID (Universal Unique IDentifier); in my lab, I set-up  more than one replica job for a single VM

Table 1. row 2. shows the name of production VM (Ubuntu-02), its morefID  (vm2270), where it is running (Milan), the UUID (…bcc12) and its VM UUID … f58b.

Table 1. row 3-4 shows the name of  VMs replicas,  morefID, instance UUID and its UUID.

All tables shown in these articles have been created using Veeam One

Name PCName morefID DataCenter Instance UUID UUID
Ubuntu-02 ubuntu-02 vm-2270 Milan 502d2405-cc8f-de73-1a19-57e8496bcc12 564d013a-7835-9d1b-841e-32855790f58b
Ubuntu-02_Rep_VC01 ubuntu-02 vm-2694 Milan 502d2d90-d08f-08aa-efcf-d9feaa1d13f8 564d013a-7835-9d1b-841e-32855790f58b
Ubuntu-02_Rep_VCDR ubuntu-02 vm-399 Venice 501d517b-672e-30e0-665a-fd4b4af7dcb6 564d013a-7835-9d1b-841e-32855790f58b

Table-1

Picture 2 shows the VM  source (highlighted in yellow) from vCENTER console.

Picture 2

After checking-up that VM source is switched off, it’s possible to start a Failover (Picture 3).

Picture 3

Next five pictures show the step by step wizard to complete the procedure correctly.  As you can see from picture 4 the VM that has been replicated with two different jobs (Picture 5) is always Ubuntu-02.

Picture 4

Picture 5

Pictures 6-8 show the result of the failover.

Picture 6

Picture 7

Picture 8

What happens when you complete the task with the Permanent failover? (Picture 9/10/11)

Picture 9

Picture 10

Picture 11

First of all, comparing picture 3 with 12 it is possible to see that one of the Replica Ready VM, and precisely the VM in permanent failover, has been deleted 

Picture 12

Picture 13 shows that now the replica job contains 0 objects. The right behavior is confirmed by pictures 14,15,16 and 17  where it is shown that the replica is not available anymore.

Picture 13

Picture 14

Picture 15

Picture 16

Picture 17

The cloning job option didn’t change the correct behavior (Pictures 18 and 19)

Picture 18

Picture 19

Let’s sum up. Following the right procedure, the Failover works as aspected

Now …. why the VM has been deleted? The next article will explain it in detail.

Veeam Availability Orchestrator – v.3.0 – DR from Netapp – Storage Replica

The fifth article will show how to use VAO version 3.0 to automatize the Disaster Recovery using the Netapp snap-mirror technology as an engine.

 

Netapp – SnapMirror

The article (quite long) is composed of 6 parts:

  1. Setting up Netapp Snap-mirror Protection
  2. Setting up the Recovery Location
  3. Setting up the Scope
  4. Creation of Orchestration Plan
  5. Starting the Plan
  6. Checking the Orchestration Plan status

1 Setting up Netapp Snap-mirror Protection

In my personal lab, I added two NetApp simulators 9.6 setted-up in peer relationships.

Pictures 1 to 5,  show how I set-up  a protection rule for a single Volume (named Vol_iScsi_N01)

Picture 1

Picture 2

Picture 3

Picture 4

Picture 5

Picture 6 shows the VMware console view of DR site; the replicated volume is presented as read-only volume.

Picture 6

Tips: My personal suggestion is asking your storage expert the right procedure to set-up a snap-mirror relation between two Netapp storages.

2. Setting up the Recovery Location

The recovery location wizard is shown in pictures 7, 8 and 9.

Picture 7

Picture 8

Picture 9

3. Setting up the Scope

The Scope wizard is shown in picture 10.

Picture 10

4. Creation of Orchestration Plan

Let’s go back to the main steps.

The wizard drives the users to select the right voices as shown in pictures 11, 12, 13 and 14.

Picture 11

Picture 12

Picture 13

Picture 14

The test scenarios are available in this configuration too

The Readiness Test is a low-impact and fast method one to confirm that configuration of an orchestration plan matches the DR environment.

The Data-lab test verifies the DR plan starting VM in a separate network.

 As shown in pictures 15,16,17 and 18. the steps are:

– Assigning  Datalab to VM Group (it is available from the admin menu)

– Setting up the Lab Group

Picture 15

Tips: for testing the Netapp integration select the option Restore.

It means the Replica option is just available for VBR Replica jobs.

Picture 16

Picture 17

Picture 18

5. Starting the Plan

Next pictures show how to run the just created Orchestration plan.

Picture 19

Picture 20

Picture 21

Picture 22 shows the restore points available.

Tips:  the shown Restore points are the replicated snapshot  (created by snap-mirror).

Picture 22

Picture 23 shows my favorite option available on VAO 3.0 in the Storage replica scenario.

Let’s imagine you run the orchestration plan.
While failover is running, the primary site is back normal operativity.

Your IT manager asks to revert the failover. Is it possible? Well, it can be. If you selected “reprotect storage volumes after failover” during the steps you can easily do it!

Picture 23

Picture 24

Picture 25

6. Checking the Orchestration Plan status

Time to run the orchestration plan and watch the steps performed.

I would like you to give your attention to the following three pre-plan steps:

a. Breaking the snapshot relationship

b. Putting the destination volumes on-line

c. Mounting the volumes

(Clicking on the picture you can enlarge the images)

Picture 26

From picture 27 you can see  the main steps:

d. VM registration

e. Network founding and connection

f. Booting VM

Picture 27

Pictures 28 and 29 show the result of post plan steps:

g. heart-beating test

h. Unmounting source Datastore

Picture 28

Picture 29

Tips: how to check in 5 points if everything is correctly working

1. VM in the DR site is running (from DR vCenter console connect remotely to VM as shown in picture 30 and 31).

Picture 30

Picture 31

2. Source VM has been deleted (from Production vCenter console check if VM is disappeared) (Picture 32).

Picture 32

3. The Orchestration plan launch button is grey and the plan has to be reset (picture 33)

Picture 33

4. Destination volumes on Netapp console have been set as read/write (Picture 34)

Picture 34

5. The Netapp relationship between source and destination volumes is broken (Picture 35)

Picture 35

– Readiness check report example dowload

That’s all folks for now.

Veeam Availability Orchestrator – v.3.0 – DR from Backup

Fourth article about VAO 3.0 and the second use case.

How to use a Backup Job to implement a DR scenario with VAO. In this article, I don’t cover the mandatory step related to VMs tag.

If you need a guide please refer to my previous writes (Baseline  1  & 2)

The main steps are:

  1. Creating a Failover location where the VMs can be restarted
  2. Setting up scope
  3. Creating the orchestration plan

Let’s start!

1 . Creating the Failover location

Pictures 1 to 3 show how easier it is to set Name, computers, and storage resources.

Picture 1

Picture 2

Picture 3

Picture 4 shows the maximum storage % that can be used by Restore task and picture 5 shows if the Instant VM recovery has to be enabled.

Picture 4

Picture 5

Picture 6 shows one of my favorites VAO options.

Why? Because it allows VAO to be implemented where the primary backup job has been copied to a second location.

To be clearer: the VBR backup copy job allows to copy backup data to the secondary site. If there are VMware hosts, it’s possible to create a DR site just restoring data from backup copy job.

Picture 6

Picture 7 shows how to twin the production with DR networks (the previous step sets-up the re-IP addressing just for windows VMs)

Picture 7

Picture 8 shows the GDPR option

Picture 8

2. Setting up scope

Pictures 9 to 12 show the main points:

To set-up a scope,  check-up the VM Groups,  select the Recovery Location and which plan steps have to be added.  (all details are available at Baseline articles).

Picture 9

Picture 10

Picture 11

Picture 12

3. Creating the Orchestration Plan

From Picture 13 to 16 the main points to set up the Restore Plan

Picture 13

Picture 14

Picture 15

Picture 16

Please find here an example of Readiness check Reports

Readiness check report example dowload