Last night a customer had a blackout in his small datacenter. Luckily all machines came up again without any problem. The only problem he figured was his veritas environment. Volumes seemed to be mounted without content, disk groups could not be found etc.
The first thing I deceided to do was to unmount all volumes. First I listed all volumes in the Oracle disk group:
# mount | grep oracledg
/u04 on /dev/vx/dsk/oracledg/oraclevol01 ioerror=mwdisable,largefiles,delaylog,dev=2005dc0 on Wed Sep 26 07:38:46 2012
Then I unmounted the above volume (the disk group had only one volume which made my life easier):
# umount /u04
# mount | grep oracledg
After the volume was unmounted I needed to stop the volume:
# vxvol -g oracledg stopall
VxVM vxvol ERROR V-5-1-607 Diskgroup oracledg not found
And that was the funny part: the volume was mounted (the mountpoint was empty) and the diskgroup was not available. I couldn't look anymore but I'm pretty sure that /etc/mnttab (HP-UX) had an 'ghost'-entry that /dev/vx/dsk/oracledg/oraclevol01 was still mounted on /u04. Anyway, I deceided to continue with my stopping-and-restarting-everything-manually-action-plan. The next logical step was to deport the disk group:
# vxdg -g oracledg deport
To restart everything I imported the disk group again:
# vxdg -g oracledg import
And started all volumes:
# vxvol -g oracledg startall
After that the volume was up and running again and I was able to mount /u04 again (after a filesystem check):
# mount -F vxfs /dev/vx/dsk/oracledg/oraclevol01 /u04
UX:vxfs mount: ERROR: V-3-21268: /dev/vx/dsk/oracledg/oraclevol01 is corrupted. needs checking
# fsck /dev/vx/dsk/oracledg/oraclevol01
file system is clean - log replay is not required
# mount -F vxfs /dev/vx/dsk/oracledg/oraclevol01 /u04
No comments:
Post a Comment