vxprint: NODEVICE

Today I had a broken Veritas mirror. In the middle of the night one path to a storage was missing and when I took look at the Veritas configuration I got the following (output truncated):

# vxprint -g oracledg
Disk group: oracledg
...
v  oravol03        fsgen       ENABLED 1570766848 - ACTIVE   - -
pl oravol03-01     oravol03    ENABLED 1570766848 - NODEVICE - -
sd stor1-vol003-01 oravol03-01 ENABLED 1570766848 0 NODEVICE - -
pl oravol03-02     oravol03    ENABLED 1570766848 - ACTIVE   - -
sd stor2-vol003-01 oravol03-02 ENABLED 1570766848 0 -        - -
...

A closer look at the disks gave me the following (output truncated):

# vxdisk list
...
eva44001_0 auto:cdsdisk -            -        online
...
-          -            stor1-vol003 oracledg failed nohotuse was:eva44001_0
...

In this case one path to one storage was missing a short time. That means that Veritas has lost the disk but detected a prior configured disk again. To reattach the disks again I had to run vxreattach:

# vxreattach

To recover the mirror configuration I had to run vxrecover. But be aware: recovering all mirror can produce heavy load on the server. In this case run vxtask and pause several jobs. First run vxrecover in background (-b):

# vxrecover -b

Then run vxtask to get all current task id's (output truncated):

# vxtask list
TASKID  PTID TYPE/STATE    PCT   PROGRESS
   223           PARENT/R  0.00% 9/0(5) VXRECOVER
   224   223     ATCOPY/R 00.03% ... oravol01 oravol01-01 oracledg
   225   223     ATCOPY/R 00.04% ... oravol02 oravol02-01 oracledg
   226   223     ATCOPY/R 00.03% ... oravol03 oravol03-01 oracledg
   227   223     ATCOPY/R 00.04% ... oravol04 oravol04-01 oracledg

Next pause as much as possible tasks:

# vxtask pause 225
# vxtask pause 226
# vxtask pause 227

Now only one mirror is recovering (output truncated):

# vxtask list
TASKID  PTID TYPE/STATE    PCT   PROGRESS
   223           PARENT/R  0.00% 9/0(5) VXRECOVER
   224   223     ATCOPY/R 00.15% ... oravol01 oravol01-01 oracledg
   225   223     ATCOPY/P 00.04% ... oravol02 oravol02-01 oracledg
   226   223     ATCOPY/P 00.03% ... oravol03 oravol03-01 oracledg
   227   223     ATCOPY/P 00.04% ... oravol04 oravol04-01 oracledg

When the recover process has finished resume the next one and so on:

# vxtask resume 225