Step by Step Single Node Cluster OpenAIS Suse 11 with Oracle11g & SAPInst

**Myself
OpenAIS Cluster Environment
Cluster information
host01 information :
-          hostname=host01
-          ip= 172.10.10.10
-          virtual IP=172.10.10.11
-          heartbeat ip=No
-          eth0=172.10.10.10 (data no heartbeat)
-          eth1=Not use ,
-          vgshare=qasdatavg
lv=lvol1,lvol2
-          Oracle sid = QAS [Mcod csd,csq,bcd,bcq,j2d,j2q,ecd,ecq]
-          SAP sid = j2d,j2q,trex_90,trex_92
-          disk on qasdatavg= md0/md1/md2/md3/md4

Create OpenAIS cluster  
  1. Create a shared volume group.
  2. Create a OpenAIS Cluster.
  3. Create an application package.
  4. Dependency or Constraints of Resource cluster startup.
  5. Maintain cluster.

1.     Create a shared volume group


1.       Install multipath and configuration mpio/lvm

 (SLES 9 only) Changing system configuration
Using an editor of your choice, within /etc/sysconfig/hotplug set this value:
HOTPLUG_USE_SUBFS=no
On SLES:
rpm -ivh multipath-tools-0.X.X-XX.X.i586.rpm
On Red Hat:
rpm -ivh device-mapper-multipath-0.X.X-XX.elX.x86_64.rpm
- Load driver for the HBA is added to INITRD_MODULES in /etc/sysconfig/kernel
INITRD_MODULES="cciss reiserfs piix megaraid_sas mptspi siimage processor thermal fan jbd ext3 dm_mod edd dm-multipath qla2xxx"
- Ensure that boot.multipath and multipathd are set to start on boot: MPIO Services
chkconfig boot.multipath on
chkconfig multipathd on

The services can be started immediately with: 
/etc/init.d/boot.multipath start
/etc/init.d/multipathd start

To verify 
chkconfig --list | grep multipath

- Configuring multipath-tools  /etc/multipath.conf.
“failback 60”  # incase san not stable
Suse if the /etc/multipath.conf file does not exist, copy the example to create the file:
cp /usr/share/doc/packages/multipathtools/multipath.conf.synthetic /etc/multipath.conf
          Edit Backlist
blacklist {
devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st|sda)[0-9]*"
devnode "^hd[a-z][0-9]*"
devnode "^cciss!c[0-9]d[0-9].*"
}

- MD RAID Preconfiguration
chkconfig boot.md off
It is still possible to use local MD devices. These can be configured in the file /etc/mdadm.conf.localdevices,
which uses the same syntax as the /etc/mdadm.conf. The cluster tools RPM package contains a new initscript called boot.md-localdevices. Copy this file to the /etc/init.d directory and enable it using the command
chkconfig boot.md-localdevices on
- Using LVM2 on top of the MPIO devices
           modify/etc/lvm/lvm.conf

filter = [ "a|/dev/sda[1-4]|", "a|/dev/md.*|", "r|/dev/.*|" ]

- Reboot


2 Create md Configuration

Check DM multipath
 multipath -l

600507680281816384000000000000dc dm-27 ABC,1234
size=200G features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=50 status=active
| |- 3:0:0:4 sdf 8:80  active ready running
| `- 4:0:0:4 sdp 8:240 active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
  |- 3:0:1:4 sdk 8:160 active ready running
  `- 4:0:1:4 sdu 65:64 active ready running
3600507680281816384000000000000cc dm-26 ABC,1234
size=200G features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=50 status=active
| |- 3:0:1:3 sdj 8:144 active ready running
| `- 4:0:1:3 sdt 65:48 active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
  |- 3:0:0:3 sde 8:64  active ready running
  `- 4:0:0:3 sdo 8:224 active ready running
3600507680281816384000000000000cb dm-25 ABC,1234
size=200G features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=50 status=active
| |- 3:0:0:2 sdd 8:48  active ready running
| `- 4:0:0:2 sdn 8:208 active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
  |- 3:0:1:2 sdi 8:128 active ready running
  `- 4:0:1:2 sds 65:32 active ready running
3600507680281816384000000000000ca dm-23 ABC,1234
size=200G features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=50 status=active
| |- 3:0:1:1 sdh 8:112 active ready running
| `- 4:0:1:1 sdr 65:16 active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
  |- 3:0:0:1 sdc 8:32  active ready running
  `- 4:0:0:1 sdm 8:192 active ready running
3600507680281816384000000000000c9 dm-24 ABC,1234
size=200G features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=50 status=active
| |- 3:0:0:0 sdb 8:16  active ready running
| `- 4:0:0:0 sdl 8:176 active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
  |- 3:0:1:0 sdg 8:96  active ready running
  `- 4:0:1:0 sdq 65:0  active ready runnin



#Create MD arrays using mdadm.


mdadm --create /dev/md0 --raid-devices=1 --level=0
--metadata=1.2 --force /dev/mapper/3600507680281816384000000000000c9

mdadm --create /dev/md1 --raid-devices=1 --level=0
--metadata=1.2 --force /dev/mapper/3600507680281816384000000000000ca

mdadm --create /dev/md2 --raid-devices=1 --level=0
--metadata=1.2 --force /dev/mapper/3600507680281816384000000000000cb
mdadm --create /dev/md3 --raid-devices=1 --level=0
--metadata=1.2 --force /dev/mapper/3600507680281816384000000000000cc
mdadm --create /dev/md4 --raid-devices=1 --level=0
--metadata=1.2 --force /dev/mapper/3600507680281816384000000000000dc

#Create md for reboot /clusterconf/QAS/mdadm.conf

ARRAY /dev/md0 UUID=cb6443ea:7a85f171:3dfca667:723eb9c9
ARRAY /dev/md1 UUID=fd640129:153c4dd5:0a7b2596:44c1584f
ARRAY /dev/md2 UUID=e8c6b9d2:14f71cee:f27dc725:c452ea1e
ARRAY /dev/md3 UUID=8158487c:5ea824ee:fe01fca9:32dc5cbd
ARRAY /dev/md4 UUID=2e152c68:2f72bfb8:4373b348:1164ddc6

#manually start and stop md devices like this:
# to stop md scan
mdadm --stop -scan 
# to verify stop md
more /proc/mdstat 

mdadm --detail /dev/md0



for DEVICE in /dev/md0 /dev/md1 /dev/md2 /dev/md3 /dev/md4; do
mdadm --assemble "${DEVICE}" --config=/clusterconf/QAS/mdadm.conf; done;
 
## to scan md ->  mdadm --assemble --scan
 
 

3. create physical volumn
for i in 0 1 2 3 4
do
pvcreate -f /dev/md$i
done
 
 

4. create vg
vgcreate /dev/qasdatavg /dev/md0 /dev/md1 /dev/md2 /dev/md3 /dev/md4
 

5. create lv
lvcreate -L 20G -n lvorahomeQAS /dev/qasdatavg
lvcreate -L 1G -n lvQASoriglogA /dev/qasdatavg
lvcreate -L 1G -n lvQASoriglogB /dev/qasdatavg
lvcreate -L 1G -n lvQASmirrlogA /dev/qasdatavg
lvcreate -L 1G -n lvQASmirrlogB /dev/qasdatavg
lvcreate -L 30G -n lvQASoraarch /dev/qasdatavg
lvcreate -L 130G -n lvQASsapdata1 /dev/qasdatavg
lvcreate -L 130G -n lvQASsapdata2 /dev/qasdatavg
lvcreate -L 130G -n lvQASsapdata3 /dev/qasdatavg
lvcreate -L 130G -n lvQASsapdata4 /dev/qasdatavg

6. create file system
mkfs.ext3  /dev/qasdatavg/lvorahomeQAS 
mkfs.ext3 -b 1024 /dev/qasdatavg/lvQASoriglogA 
mkfs.ext3 -b 1024 /dev/qasdatavg/lvQASoriglogB 
mkfs.ext3 -b 1024  /dev/qasdatavg/lvQASmirrlogA 
mkfs.ext3 -b 1024  /dev/qasdatavg/lvQASmirrlogB 
mkfs.ext3   /dev/qasdatavg/lvQASoraarch 
mkfs.ext3   /dev/qasdatavg/lvQASsapdata1 
mkfs.ext3   /dev/qasdatavg/lvQASsapdata2 
mkfs.ext3  /dev/qasdatavg/lvQASsapdata3 
mkfs.ext3  /dev/qasdatavg/lvQASsapdata4
 
#verify blocksize
dumpe2fs -h /dev/mapper/qasdatavg-lvQASmirrlogA |grep "Block size"
 
dumpe2fs -h /dev/mapper/qasdatavg-lvQASoraarch |grep "Block size"

7. create layout fs
mkdir -p /oracle/QAS
mkdir -p /oracle/QAS/sapdata1
mkdir -p /oracle/QAS/sapdata2
mkdir -p /oracle/QAS/sapdata3
mkdir -p /oracle/QAS/sapdata4
mkdir -p /oracle/QAS/origlogA
mkdir -p /oracle/QAS/origlogB
mkdir -p /oracle/QAS/mirrlogA
mkdir -p /oracle/QAS/mirrlogB
mkdir -p /oracle/QAS/oraarch
 


2.
   
Create a OpenAIS Cluster


SAP Version Naming Metrix


** Credit Web
Year
UI Type
Edition
Major
Version
Minor
Version
BASIS
WebAS
ECC
NetWeaver
Solution
 Manager
Remark

Terminal
R/2
-
-






1992
Desktop
R/3
1
-
1






Desktop
R/3
2.0, 2.1
-
2.0, 2.1





1997
Desktop
R/3
3.0, 3.1
3.1I
3.1I





1998
Desktop
R/3
4
4.0B
4.0B





2000
Desktop
R/3
4.5
4.5B
4.5B





2001
Desktop
R/3
4.6
4.6B, 4.6C
4.6B, 4.6C





2001
Desktop
R/3
4.6D*
4.6D
4.6D, 610
6.1




2003
Web
R/3 Enterprise
1.1
4.7×110
620
6.2

NetWeaver (2003)
2.1


Web
R/3 Enterprise
2
4.7×200
630
6.3

NetWeaver (2003)
2.2, 3.1,3.2

2005
Web
mySAP
2004
-
640
6.4
ECC 5.0
NetWeaver 04
4.0
* SAP appears to support 4.6D on
BASIS server and WebAS
2006
Web
mySAP
2005
-
700
7.0 (AS)
ECC 6.0
NetWeaver 04S
4.0
SAP Netweaver 7
(Previously called Netweaver 04S
Loading