** Myself
vxsvc - graphic UI daemon (must be running to use vea)
/opt/VRTSob/bin/vea #GUI
vea server, kill with /opt/VRTSob/bin/vxsvc -k
can remove lock file under /var/vx/isis/vxisis.lock if it says running
when it isn't.
vxconfigd - vol config daemon, enable with below command
vxiod - IO manager, reads and write to volumes
vxrelocd - Hot swap manager which uses hotswap if a disk fails.
vxdctl enable
#rescan for all attached disks
vxdisk -o alldgs list # show all dgs even if not imported
DEVICE is disk accessname, could be c2t4d0s2 alt. enclosure based ENC1_05
*NOTE: different hosts can have the different DEVICE names for the same disk,
even enclosure names can be different.
DISK is the disk media name (dm), like data01 for dg data, can exist the same name
in different dgs. Can use either name in disk operations.
vxdisk -s list ENC1_05 - lists info on 1 disk, only disk_id guaranteed unique
to each disk, use this to get detail info on a single disk
vxprint -htg my_dg
# view vols, plexs, sd for my_dg
dg diskgroup e.g. datadg
dm diskmedia name, ie. temp01
device c0t1d0s2 or enc. based SENA1
v volume vol01
pl plex vol01-01
sd subdisk i.e SENA1_13
disk access name is: c0t3d5s2
Length is disk capacity in sectors (divide by 2 to get kbytes)
length does not indicate how full the disk is.
vxdisksetup -g DATADG -i c1t2d0
#create private and public areas on new disk and put in new diskgroup
vxdg -g CASHIP01dg free
# check if any imported disks are free
vxdg init newdg c3t3d13
# initialise newdg with free disk , disk cannot be in a DG beforehand
# use vxdisk -o alldgs list to check that disks are free before initialising
# as vxdiskadm doesn't show this info. Also run all commands on the same box
# as different boxes can give different disks the same names
vxdg -g datadg adddisk c3t3d13
#put disk in dg
vxinfo -g datadg
# print volume state info, ie. started, offline etc.
vxtask list
# print current tasks , like remirroring etc.
Adding new disks
================
devfsadm -v (was drvconfig and disks)
use format to label disks
vxdctl enable
vxdiskadm # add new disks,adddisk,
vxassist -g ENTMTP01dg -U fsgen make bin92 14g #create 14Gb partition
#-U fsgen|gen fsgen= normal volume, gen=raw volume
# make a newfs
mkfs -F vxfs -o largefiles /dev/vx/rdsk/ENTMTP01dg/bin92
# mount it
mount -F vxfs /dev/vx/dsk/ENTMTP01dg/bin92 /oracle1/bin
to add manydisks do this:
/etc/vx/bin/vxdisksetup -i c5t5d160 # use for loop to add many
vxdg -g CCV00P02dg adddisk c5t5d163s2
#alternative to add new disks interactively
vxdiskadd c1 # lists available disks on controller c1, or give c1t5d0 etc
# now answer the questions about what diskgroup etc.
Replacing disks
===============
use vxevac to remove all subdisks from disk
---------- use either vxdiskadm or cli
1: vxdiskadm method
vxdiskadm -> option 4 (Remove a disk for replacement)
vxdiskadm -> option 11 (Disable (offline) a disk device)
disk should now appear as offline in vxdisk list, take the DEVICE name:
now swap the disks
vxdiskadm -> option 5 Replace a failed or removed disk
choose the failed disk to replace (use list to see the ones marked failed e.g. billingdg01)
replace with the disk you put in, use list to see name i.e. c1t2d0
it will then ask to bring online, initialize etc.
------------
2: using CLI
vxdg rmdisk rep03
(if you wish to clean the disk of veritas then do
vxdiskunsetup <DEVICE_NAME> before offlining
vxdisk offline <DEVICE_NAME> i.e. LOIS2_8 or c2t43d0s2
------------
renaming enclosure
from veritas
vxdmpadm setattr enclosure LOIS2 name=MELONS
from luxadm
luxadm enclosure_name MELONS LOIS2
don't know if this is correct order, there were no live disks when I did this
-------------
sccli to look at T3 enclosure
-------------
vxdisk list LOIS2_6
gives last line:
c2t41d0s2 state=enabled #this is your device access name
now find the wwn:
ls -l /dev/dsk |grep c2t41d0s2
lrwxrwxrwx 1 root other 76 May 25 08:19 c2t41d0s2 -> ../../devices/pci@8,700000/pci@4/SUNW,qlc@4/fp@0,0/ssd@w22000004cf4cc7e6,0:c
the wwn here is 4cf4cc7e6
now find the slot position from:
luxadm display LOIS2
luxadm remove_device enclosurename,[f|r]slot#
luxadm remove_device LOIS2,f3 # it then expects you to remove the disk
Moving individual subdisks
==========================
1. view volume subdisks
vxprint -htv u04-snapshots01
pl u04-snapshots01-04 u04-snapshots01 ENABLED ACTIVE LOGONLY CONCAT - RW
sd rep22-28 u04-snapshots01-04 rep22 1485 66 LOG PLUM_0 ENA
note: subdisk rep22-28 has length 66, this is the one we want to move
2. find space on disk to use
vxprint -st | grep report08
sd report08-21 u04-user01-04 report08 71131989 33 LOG SENA1_10 ENA
offset for next subdisk is 71131989 + 33 = 71132022
3. create new subdisk
vxmake -g rep sd report08-22 disk=report08 offset=71132022 len=66
subdiskname diskname from step 2. from step 1.
it should look like this:
sd report08-22 - report08 71132022 66 - SENA1_10 ENA
4. move subdisk rep22-28 to report08-22
/usr/sbin/vxsd -d -g rep -o rm mv rep22-28 report08-22
vxprint step 1. will show it moved.
to remove a subdisk use vxedit, example shown earlier.
Volumes
=======
Layered volumes:
a layered volume is a concat-mirror, stripe-mirror as opposed to a mirror-concat or mirror-stripe.
the difference being that for layered the subdisks are mirrored and built into a plex whereas normally the subdisks are built into plexes and mirrored.
This gives a 50% increase in redundancy for layered.
vxassist -g datadg make concat_vol 50m
#create a concat volume size is 50MB
vxassist -g datadg make strip1_vol 20m layout=stripe,mirror ncols=2 stwidth=128k c3t3d15 c3t3d14 c3t3d13 c3t3d12
#create stripe mirror over 2 columns, width 128k, using 4 disks
vxassist -g datadg make raid1_vol 50m layout=raid5 ncol=4 disk1 disk2 disk3 disk4 disk5
# create raid5 volume with 4 columns over 5 disks, you get a log plex with
# raid 5, so specify ncol 4 for 5 disks and the log will land
# on the last disk (disk5 in this case)
vxassist -g testdg mirror volume1 layout=stripe ncols=4 stwidth=64k MELONS_16 MELONS_17 MELONS_18 MELONS_19
# add a striped mirror to existing volume using specified disks and cols.
vxassist -g datadg remove volume strip1_vol
# delete/remove a volume
vxdisk -e list
#print device name and path, like this:
SENA0_18 sliced bostmp07 bos online c4t87d0s2
vxdisk rm ENC1_1
# remove a disk from veritas all together, note disk should not be
used, imported or have subdisks when done.
vxassist -g datadg mirror Vol_01 bostemp11
# add a mirror to a running volume vol_01, use disk bostemp11
vxassist -g datadg remove mirror strip1_vol
# remove a mirror from a volume
vxassist remove mirror rootvol !rootmirr
# split a root mirror from a rootvol
vxassist -g testdg remove mirror volume1 !disk01
# remove the mirror on disk01 - untested
#vxdg destroy datadg # this is very dangerous to cut and paste by accident!
#destroy diskgroup
vxvol -g testdg stop u01_mirror
# stop volume before trying to remove it
vxplex -g testdg -o rm -v volume01 dis plex01
#remove and destroy plex01 from volume01 - frees up subdisks
vxplex -g datadg -v u04-betssnap06 dis u04-betssnap06-03
#dis plex from volume, use det for detach
vxedit -g testdg -r rm plex01
# remove and destroy a stand alone plex
vxedit -g datadg rm volume1
# remove a volume
Moving Plexes
=============
first create mirror plex
then remove unwanted plex
1. vxassist -g datadg mirror <volume> <diskname>
2. vxplex -g rep -o rm -v <volume> dis <plexname>
Volume creation, different ways
===============================
#using vxassist:
vxassist -g rep -U gen make u05-games 32g layout=mirror rep24 report06
#using vxmake
# make plex pl-games-01 from vmdisk rep23
vxmake -g rep plex pl-games-01 sd=rep23 len=67108864
# make plex pl-games-02 from vmdisk report07
vxmake -g rep plex pl-games-02 sd=report07 len=67108864
# make volume vol-games using plex pl-games-01
vxmake -g rep -U gen vol vol-games plex=pl-games-01
# attach plex pl-games-02 to make mirrored volume
vxplex -g rep att vol-games pl-games-02
# change volume user and group id: so oracle can mount it
vxedit -g rep set user=oracle group=oinstall u03-games
#create link for raw device
ln -s /dev/vx/rdsk/rep/u03-games /GAMES/u03-games.dbf
Volume Operations
=================
vxdg split datadg splitdg vol01
source dest volume
# extract vol from a dg and put in another dg
vxrecover -g splitdg -sE vol01
# start a stopped volume, e.g after splitting it out
vxsd -g diskgroup -v vol2 mv c3t3d14-02 c3t3d15-02
/usr/sbin/vxsd -t taskid_119 -g rep -o rm mv trep0_8-20 rep26-01
# move subdisks
/usr/sbin/vxsd -g datadg -o rm dis trep0_8-2
#dissassociate the subdisk trep0_8-22, i.e. remove an unwanted drl
/usr/sbin/vxedit -g datadg -rf rm report08-22
# destroy the subdisk report08-22 entirely
#Warning!!!
# on using vxevac, if you use vxdisk list to find a free disk from
# one machine and then use vxevac on another machine, it might not
# be the same disk, but it can have the same DEVICE name. this is
# because each machine can rename enclosure named disks or have them
# on different controllers, always check with vxdisk list diskname
# and look at the disk id.
vxevac -g bos bosdrl02 PLUMSD_4
# evacuate subdisks from dm bosdrl02 to PLUMSD_4
vxvol -g datadg rdpol round strip1_vol
#change read policy to round robin
vxedit -g datadg set spare=off c3t3d11
vxedit -g datadg set nohotuse=on c3t3d11
#set spare flag on/off for a disk
# a disk set with the spare flag on is available for hot relocation
# setting the disk as spare=off makes it part of the free disk pool
# setting the nohotuse=on prevents it from being used for hot relocation
vxedit -g datadg rename c3t3d11 data-01
#use vxedit to rename disks, volumes, plexes etc
#reserve a disk (on/off)so vxassist won't grab it,
# specify by name to use it later
vxedit set reserve=on c3t3d11
#remove a subdisk
vxedit -g datadg -rf rm disk01-01
# start all vols in imported dg
vxvol -g testdg startall
Using vxmake
============
# use vxmake to create a subdisk (sd) of VMdisk disk01 with 0 offset
vxmake sd disk01-01 disk=disk01 offset=0 len=17678493
subdisk sd-name vmdiskname,offset,size
# make a plex called "vol01-01"
vxmake plex vol01-01
#put subdisks into a plex vol01-01
vxsd assoc vol01-01 sd1 sd2 sd3 ..
#alt. make plex and assign sd at once: vol01-01 using subdisk sd1
vxmake -g datadg plex vol01-01 sd=sd1
#create a volume vol01 using plex vol01-01
vxmake -U fsgen vol01 plex=vol01-01
# to make a raw vol use -U gen
#start volume, then do newfs and mount it.
vxvol start vol01
#to create striped plex use
vxmake plex vol01-01 layout=stripe ncolumn=4 stwidth=32k sd=sd1,sd2,sd3,sd4
Snapshot
========
vxassist -g datadg snapstart vol01
#start the snapshot process
vxassist snapwait
# the snapstart command exits before it's fully sync'd, so snapwait means
do not allow a snapshot until the sync is done.
vxassist -g datadg snapshot vol01 snap_vol01
#create snapshot volume snap_vol01
vxassist -g datadg -o resyncfromreplica snapback snap_vol01
# replace data in original vol with snapshot volume data
vxassist -g datadg snapclear snap_vol02
# disassociate snap_vol02 from original volume, after this the resyncfromreplica won't work anymore, this has the effect of creating a freestanding volume with
the snapshot data.
vxassist -g datadg snapabort vol01
#abort snapshot process
Fast resync
===========
vxvol -g datadg set fastresync=on fmron
#enable nonpersistent fast resync
vxprint -g datadg -F%fastresync fmron
# is fast resync enabled? rtns on else off
vxprint -g datadg -F%hasdcolog fmron
#whether volume has persistent fast resync (rtrns on) otherwise rtrns off
Recovery
========
vxassist -g datadg addlog strip1_vol logtype=drl c3t3d11
# add dirty log to volume strip1_vol
# Dirty Region Logging only speeds up volume start by ensuring mirrors are
#consistent with each other, it replays the log on the unsynced mirror.
#there is no data protection here.
vxdctl enable
#Rescan for all attached harddisks
vxdg -g datadg -k adddisk c3t3d13
#add new hardisk and use old hd name etc (-k option)
vxrecover -g datadg recvol
# recover vol, spin up disk, reattach plexes etc.
# fro plexes marked Detached stale do this
# set plex clean
vxmend -g bos fix clean plex
# reattach it to volume
vxplex -g bos att vol plex
use vxdiskadm to replace a failed disk or to mirror a root disk
or use
/etc/vx/bin/vxrootmir rootmirr # where rootmirr is the mirror disk name
to replace:
vxdiskadm
4 Remove a disk for replacement
11 Disable (offline) a disk device
use luxadm to quiesce bus and blink the led
e.g luxadm led /dev/rdsk/c1t3d0s2 # led status
led_blink # blink led
led_off # stop blinking
change disk
vxdiskadm
5 Replace a failed or removed disk
to use the new disk instead
Volume resize
=============
vxassist maxgrow vol_01 #returns the max a volume can grow by.
vxassist maxgrow vol_01 disk1 disk2 disk3 disk4
# this will grow the volume to fill the specified disks
# inc. vol_01 with 5Meg grows both FS and volume (use this)
# allegedly "-o verify" is used to test only, but did the operation anyway!
vxresize -F vxfs -g datadg vol_01 +5m
or
#resize a volume, but only use contiguous diskspace, here vol_01 by 1GB
vxassist growby vol_01 1g layout=nodiskalign
#specify size to grow to (here to 5g)
vxassist growto vol_01 5g
#resize filesystem, also known as fsadm_vxfs, not necessary with vxresize
fsadm -F vxfs -b 97656 /rezmnt
# shrinking can be done similarly with shrinkto and shrinkby
# resize keeping plexes on specific controllers (untested)
vxresize -x -F vxfs -g bos u08 <newlength> <disk1> <disk2> <disk3> <disk4> mirror=ctlr ctlr:c2 ctlr:c3
Trouble shooting
================
check support.veritas.com for help docs.
if a volume is listed as DISABLED ACTIVE or a disk group is online disabled from vxprint -ht or vxdg list
then you can deport/import it again.
Use vxrecover to recover and start the volumes.
+++
To fix a volume with stale plexes and 1 plexes marked RECOVER:
Ignore the LOGONLY plexes
Set the STALE plex to CLEAN
vxmend -g datadg fix clean <volumename>
start the volume
vxvol start <volumename>
fix a volume with 2 recover states:
vxmend off <plexname>
vxmend on <plexname>
vxmend fix active <plexname>
vxvol start <volumename>
Misc
=====
from http://seer.support.veritas.com/docs/237510.htm
get size of all vm diskspace in a dg
vxprint -g datadg -dF "%publen" | awk 'BEGIN {s = 0 } {s += $1} END {print s}'
this adds up the columns in vxprint labelled length (capacity)
convert sectors to meaningful sizes (on sun systems ):
1 sector = 512 bytes,
2 sectors = 2x512bytes = 1kbyte
divide:
sectors / 2 = kb
sectors / 2048 = Mb
sectors / 2097152 = Gb
RAID types:
raid0 striping
raid1 mirroring
raid 0+1 striping + mirroring
raid 5 striping + parity checked
DMP
===
Dynamic Multipathing - more than 1 path to diskbox.
to see what paths are available:
vxdisk list c3t3d0
advanced vxprint
================
use vxprint -F to format output,
see what fields are available:
vxprint -g datadg -m "subdisk01"
then
vxprint -g datadg -h -F "%vname %daname"
this gives a volume name (%vname column 2) and what subdisks (%daname column 8) it uses.
using vol_pattern to get what you want
.i.e I want to see volumes with 4 plexes only:
vxprint -g datadg -hv -e v_nplex=4
put it together:
vxprint -g datadg -hv -e v_nplex=4 -F "%vname %daname"
shows the subdisks on volumes with only 4 plexes
or a more complete example:
vxprint -g rep -htv -e v_nplex=3 -F "%vname %daname %dm %is_log"
gives list
vol vmdisk subdisk logging :
log_02 - - -
log_02 - - -
log_02 SENA0_19 trep0_9 off
log_02 - - -
log_02 SENA0_8 rep00 off
log_02 - - -
log_02 LOIS2_0 rep01 on
vxprint -g datadg -e '( pl_sd ~ /bos16/ )' #print plexes using subdisk bos16
vxprint -g bos -e '( pl_sd = "bos16-01" )' # exact match
#see what volumes are on a device, are they drl vols?
vxprint -g bos -e '( sd_device = "LOIS2_12" )' -F "%vname %daname %is_log"
volume device name
bos-stat LOIS2_12 off
u07-temp LOIS2_12 on
bos-stat LOIS2_12 on
to see if any volumes are using the same three disks:
vxprint -g bos -e '( sd_device = "LOIS2_12" ) || ( sd_device = "LOIS2_14" ) || ( sd_device = "LOIS2_17" ) ' -F "%vname %daname"
using && (and) would give the answer as those that are, or nothing.
find volumes which have only 2 plexes but ignore log plexes:
vxprint -g bos -htv -e '( v_nplex=4 ) && ( ! v_plex.pl_subdisk.sd_is_log ) ' -F "%vname %daname %dm %is_log" |grep off | awk '{ print $1 }' | uniq -c | sort
Disaster recovery
=================
backup configs with:
vxprint -mpvshr -g diskgroup_name > file1
vxprint -ht > file2
or
/etc/vx/bin/vxconfigbackup
Execute as required manually (if vxconfigbackupd is not running), when you require a backup of the configuration data
/etc/vx/bin/vxconfigrestore
Execute as required manually when you require to restore corrupted configuration data
The "-l" option allows you to specify an alternative directory for the location of the backup configuration files other than the default location, /etc/vx/cbr/bk.
## End
0 comments:
Post a Comment