Disks configured multipath on IBM ESS San are not discovered by ASM

posted Mar 8, 2011, 11:47 AM by Sachchida Ojha
The information in this document applies to: 
Oracle 10gR1 using ASM with disks IBM ESS San configured for multipath running on Linux 3.0


Symptoms

* Query from v$asm_disk does not return rows. In other words disks are not discovered by ASM
select path,header_status, state from v$asm_disk;
* DBCA fails as disks are not recognized.
* SDD 1.5.1.1-10 installed.
SDD (Subsystem Device Driver) is the pseudo device driver designed to support the multipath configuration environment
in IBM products.

a. cat /proc/modules to verify that SDD sdd-mod driver is loaded. loaded. If it is successfully loaded, output similar
to the following is displayed:

sdd-mod 233360 0 (unused) *************** SDD MODULE **************
qla2300 192000 0 (autoclean)
nls_iso8859-1 2880 1 (autoclean)
cs4232 3760 1 (autoclean)
ad1848 16752 0 (autoclean) [cs4232]
uart401 6352 0 (autoclean) [cs4232]
sound 56192 1 (autoclean) [cs4232 ad1848 uart401]
soundcore 4048 4 (autoclean) [sound]
nfsd 67664 4 (autoclean)
usb-uhci 20928 0 (unused)
usbcore 48320 1 [usb-uhci]
ipv6 131872 -1 (autoclean)
olympic 15856 1 (autoclean)
ipchains 34112 0 (unused)
lvm-mod 40880 0 (autoclean)

b. check cat/proc/scsi/scsi to verify that all expected devices (LUNs)
were detected and configured.
c. The SDD server was started
ps was | grep sddsrv

If processes are not listed, SDD server was not started.

* The vpath devices are configured with command cfgvpath. Check /etc/vpath.conf
* To review the existance of vpath devices use commands cfgvpath or lsfpcfg.

* ASMLIB allows to mark the disks:
/etc/init.d/oracleasm createdisk VOL2 /dev/vpatha1.

dd if=/dev/vpatha1 bs=4096 count=1 |od -c

0000000 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 V O L 2
0000020 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0
0000040 O R C L C L R D \0 \0 \0 \0 \0 \0 \0 \0
0000060 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0
*
0000760 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 U 252
0001000 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0
*
0010000

A NON ASM DISK wont display previous information.

* Same command executed as the owner of oracle software failed with
INPUT OUTPUT error

THAT WAS THE KEY FOR THE DIAGNOSTIC OF THE PROBLEM.
DD COMMAND FAILED WITH THE ORACLE USER

Changes
No changes. New installation of this environment.


Cause
When a non-root user of Linux SDD 1.5.1.1-10 or below attempts to open a vpath, the vpath open
command may fail. The situation described above may be a concern if non-root users are not using
filesystems with vpath devices.

For example, ORACLE,DB2 and other database applications may use rvpath devices (raw devices).

Fix
Install a higher version of SDD 1.5.1.1-10

Reference
For the explanation of this problem:
http://www-1.ibm.com/support/docview.wss?uid=ssg1S1002369

For documentation of SDD:
Publication No. SC26-7637-00
http://www-1.ibm.com/support/docview.wss?uid=ssg1S7000303&aid=1
Comments