SA changed the host name in addition to the IP address, so things were a little different to recover.  All the below are from the ASM home.

 1.       First, I had to run the following as root:   $GRID_HOME/crs/install/roothas.pl -deconfig -force -verbose

2.       Then, again as root, you have to run $GRID_HOME/root.sh .  you may run into a message saying that RAC was turned off, if so, then you’ll need to run make -f ins_rdbms.mk rac_off as root in $GRID_HOME/rdbms/lib  (if you get “make: 1254-004 The error code from the last command is 1.” follow 1290438.1 from metalink)

3.       After this, make sure you update the $GRID_HOME/network/bin listener with the correct host

4.       Start the listener

5.       Do steps 2-9 below.  The disk groups will auto-mount, but it will take a while.

6.       No need to do steps 10-15

   c) Add an entry (tnsalias) for your ASM instance (see example)

EXAMPLE

   NOTE: It may be required that an PFILE be created from an SPFILE in order to be able to edit the file properly ... once the line have been added ... the process can be reversed

Applies to:

Purpose

The purpose of this document is to provide the steps required to resize a physical disk
or LUN at the operating system level and subsequently resize the diskgroup using the new space.

Scope and Application

This document is created for any person involved with ASM administration. 


For this example the storage used is a SCSI storage on Linux Red Hat 3.0 using RAW DEVICES to 
simulate the LUN.

How to resize a physical disk or LUN and an ASM DISKGROUP

Resizing a physical disk or LUN and the ASM DISKGROUP

Today most of the storage technology used is SAN or NAS.  The disks in the storage are configured
as a unit (LUN) and presented to the host.  When the LUN run out of space, new
disks can be added to the LUN.

In all platforms a LUN is identified in the host as a physical device (disk).  The hole LUN can
be used  or it can be partitioned.  Each partition can be seen as an individual disk.
There are platforms  that require a LVM in order to partition a LUN/disk.  If a LVM is not used,
then the hole disk needs to be used.


When the space in the LUN is exhausted, in the storage side it is a simple task adding
new disks to the current LUN.  Now, it is required  the operating system recognize
the new space.

In Linux in order to work on a physical disk, it requires at least one partition.  The fdisk command is used
to display and manipulate the partitions.

fdisk -l /dev/sdg returns:

[root@arlnx2 root]# fdisk -l /dev/sdg

Disk /dev/sdg: 9105 MB, 9105018880 bytes
64 heads, 32 sectors/track, 8683 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes

   Device Boot    Start       End    Blocks   Id  System
/dev/sdg1             1      1908   1953776   83  Linux


For this particular example we are using a disk of 9g but only 2gb have been used.

Also a diskgroup has been created using this disk:



SQL> select name,path,total_mb,free_mb from v$asm_disk;



NAME

PATH

TOTAL_MB 

FREE_MB

DG1_0000

/dev/raw/raw4

1500

90

The previous output shows that most of the space has been allocated, so assuming that there are not ASM disks 

available, then the LUN will have to be expanded at the OS level and finally at the ASM level resize the diskgroup.

The first step will be adding space to the LUN.  In this exercise we are using a DAS (Direct Attached Storage),

using SCSI interface, but on SAN or NAS the steps will be similar.


As we saw in the output of fdisk command, the disk has a capacity of 9gb but only 2gb have been used

for the partition.  
In order to use more space, the partition has to be recreated.  This operation is at the partition table level,

which is stored in the first sectors of the disk.  Changing the partition table is not going to affect the data.

In other words, is extending the partition.



The general steps are:

1. resize LUN from storage
2. pick up new size from OS
3. in rolling fashion, shutdown ASM instances.
4. Only when all ASM have been recycled, you should resize the said disk on 1 ASM instance.

Now, some details/examples to resize the LUN in Linux:

A. Delete the current  partition.
B. Recreate the partition with the new size.

The device /dev/raw/raw4 is attached to the physical disk /dev/sdg.  fdisk command is used to modify the partition table.  Use d to delete the partition, n to create the partition and specify the new size.  Finish the operation using w to write the partition table..

At this point the partition has been configured but it requires the operating system recognize the new size.

When the LUN is ready at the OS level and the ASM instance has been recycled, the next step is resizing the diskgroup at the ASM level:

The information in this document applies to: 
Oracle 10gR1 using ASM with disks IBM ESS San configured for  multipath running on Linux 3.0


Symptoms

* Query from v$asm_disk does not return rows. In other words disks are not discovered by ASM
  select path,header_status, state from v$asm_disk;
* DBCA fails as disks are not recognized.
* SDD 1.5.1.1-10 installed.    
  SDD (Subsystem Device Driver) is the pseudo device driver designed to support the multipath configuration environment 
  in IBM products.
  
  a. cat /proc/modules to verify that SDD  sdd-mod driver is loaded. loaded. If it is successfully loaded, output similar 
     to the following is displayed: 
     
     sdd-mod 233360 0 (unused)    *************** SDD MODULE **************
     qla2300 192000 0 (autoclean) 
     nls_iso8859-1 2880 1 (autoclean) 
     cs4232 3760 1 (autoclean) 
     ad1848 16752 0 (autoclean) [cs4232] 
     uart401 6352 0 (autoclean) [cs4232] 
     sound 56192 1 (autoclean) [cs4232 ad1848 uart401] 
     soundcore 4048 4 (autoclean) [sound] 
     nfsd 67664 4 (autoclean) 
     usb-uhci 20928 0 (unused) 
     usbcore 48320 1 [usb-uhci] 
     ipv6 131872 -1 (autoclean) 
     olympic 15856 1 (autoclean) 
     ipchains 34112 0 (unused) 
     lvm-mod 40880 0 (autoclean)

  b. check cat/proc/scsi/scsi to verify that all expected devices (LUNs) 
     were detected and configured.
  c. The SDD server was started
     ps was | grep sddsrv

     If processes are not listed, SDD server was not started.

* The vpath devices are configured with command cfgvpath. Check /etc/vpath.conf
* To review the existance of vpath devices use commands cfgvpath or lsfpcfg.

* ASMLIB allows to mark the disks:
  /etc/init.d/oracleasm createdisk VOL2 /dev/vpatha1.

  dd if=/dev/vpatha1 bs=4096 count=1 |od -c    
 
  0000000  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0   V   O   L   2
  0000020  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
  0000040   O   R   C   L   C   L   R   D  \0  \0  \0  \0  \0  \0  \0  \0
  0000060  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
  *
  0000760  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0   U 252
  0001000  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
  *
  0010000

  A NON ASM DISK wont display previous information.

* Same command executed as the owner of oracle software failed with 
  INPUT OUTPUT error
 
  THAT WAS THE KEY FOR THE DIAGNOSTIC OF THE PROBLEM. 
  DD COMMAND FAILED WITH THE ORACLE USER  

Changes
No changes.  New installation of this environment.


Cause
When a non-root user of Linux SDD 1.5.1.1-10 or below attempts to open a vpath, the vpath open 
command may fail. The situation described above may be a concern if non-root users are not using 
filesystems with vpath devices. 

For example, ORACLE,DB2 and other database applications may use rvpath devices (raw devices).

Fix
Install a higher version of SDD 1.5.1.1-10

Reference
For the explanation of this problem:
 http://www-1.ibm.com/support/docview.wss?uid=ssg1S1002369

For documentation of SDD:
Publication No. SC26-7637-00
http://www-1.ibm.com/support/docview.wss?uid=ssg1S7000303&aid=1

To find the major, minor numbers of a device associated with a Diskgroup, e.g. PTTQA_BF
-------------------------------------------------------------------------------------------------

select path from v$asm_disk;

select * from v$asm_operation;

select * from V$ASM_CLIENT;

SQL> select path from v$asm_disk;

PATH
--------------------------------------------------------------------------------

/dev/rhdisk10
/dev/rhdisk11
/dev/rhdisk12
/dev/rhdisk13
/dev/rhdisk14
/dev/rhdisk15
/dev/rhdisk16
/dev/rhdisk17
/dev/rhdisk18
/dev/rhdisk19

PATH
--------------------------------------------------------------------------------
/dev/rhdisk20
/dev/rhdisk21
/dev/rhdisk22
/dev/rhdisk23
/dev/rhdisk24
/dev/rhdisk5
/dev/rhdisk6
/dev/rhdisk7
/dev/rhdisk8
/dev/rhdisk9

21 rows selected.

SQL> show parameter asm_disktring;
SQL> select * from v$asm_operation;

no rows selected

SQL> select * from V$ASM_CLIENT;

GROUP_NUMBER INSTANCE_NAME
------------ ----------------------------------------------------------------
DB_NAME  STATUS
-------- ------------
SOFTWARE_VERSION
------------------------------------------------------------
COMPATIBLE_VERSION
------------------------------------------------------------
           1 TXQADB2
TXQADB2  CONNECTED
11.2.0.1.0
10.2.0.4.0


GROUP_NUMBER INSTANCE_NAME
------------ ----------------------------------------------------------------
DB_NAME  STATUS
-------- ------------
SOFTWARE_VERSION
------------------------------------------------------------
COMPATIBLE_VERSION
------------------------------------------------------------
           1 TXQADB3
TXQADB3  CONNECTED
11.2.0.1.0
10.2.0.4.0


GROUP_NUMBER INSTANCE_NAME
------------ ----------------------------------------------------------------
DB_NAME  STATUS
-------- ------------
SOFTWARE_VERSION
------------------------------------------------------------
COMPATIBLE_VERSION
------------------------------------------------------------
           2 TXQADB3
TXQADB3  CONNECTED
11.2.0.1.0
10.2.0.4.0


GROUP_NUMBER INSTANCE_NAME
------------ ----------------------------------------------------------------
DB_NAME  STATUS
-------- ------------
SOFTWARE_VERSION
------------------------------------------------------------
COMPATIBLE_VERSION
------------------------------------------------------------
           1 TXQADB4
TXQADB4  CONNECTED
11.2.0.1.0
10.2.0.4.0


GROUP_NUMBER INSTANCE_NAME
------------ ----------------------------------------------------------------
DB_NAME  STATUS
-------- ------------
SOFTWARE_VERSION
------------------------------------------------------------
COMPATIBLE_VERSION
------------------------------------------------------------
           2 TXQADB4
TXQADB4  CONNECTED
11.2.0.1.0
10.2.0.4.0


GROUP_NUMBER INSTANCE_NAME
------------ ----------------------------------------------------------------
DB_NAME  STATUS
-------- ------------
SOFTWARE_VERSION
------------------------------------------------------------
COMPATIBLE_VERSION
------------------------------------------------------------
           1 TXQADB5
TXQADB5  CONNECTED
11.2.0.1.0
10.2.0.3.0


GROUP_NUMBER INSTANCE_NAME
------------ ----------------------------------------------------------------
DB_NAME  STATUS
-------- ------------
SOFTWARE_VERSION
------------------------------------------------------------
COMPATIBLE_VERSION
------------------------------------------------------------
           2 TXQADB5
TXQADB5  CONNECTED
11.2.0.1.0
10.2.0.3.0


GROUP_NUMBER INSTANCE_NAME
------------ ----------------------------------------------------------------
DB_NAME  STATUS
-------- ------------
SOFTWARE_VERSION
------------------------------------------------------------
COMPATIBLE_VERSION
------------------------------------------------------------
           1 STGRPTDB
STGRPTDB CONNECTED
11.2.0.1.0
11.0.0.0.0


8 rows selected.

SQL> show parameter ASM_DISKGROUPS;

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
asm_diskgroups                       string      DEV_DATA, QA_FLASH
SQL> show parameter ASM_DISKSTRING;

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
asm_diskstring                       string      /dev/rhdisk*



ASM_DISKSTRING

ASM_DISKSTRING specifies an operating system-dependent value used by Automatic Storage Management to limit the set of disks considered for discovery. When a new disk is added to a disk group, each Automatic Storage Management instance that has the disk group mounted must be able to discover the new disk using the value of ASM_DISKSTRING.

In most cases, the default value will be sufficient. Using a more restrictive value may reduce the time required for Automatic Storage Management to perform discovery, and thus improve disk group mount time or the time for adding a disk to a disk group. A "?" at the beginning of the string gets expanded to the Oracle home directory. Depending on the operating system, wildcard characters can be used. It may be necessary to dynamically change ASM_DISKSTRING before adding a disk so that the new disk will be discovered.

An attempt to dynamically modify ASM_DISKSTRING will be rejected and the old value retained if the new value cannot be used to discover a disk that is in a disk group that is already mounted.

If you’re more comfortable with GUI, all tasks here can be accomplished using the Enterprise Manager.

1. Add new disks to ASM diskgroups.

ALTER DISKGROUP PMDW_DG1 ADD DISK
  '/dev/rdsk/c4t60060E80056FB30000006FB300000823d0s6' NAME PMDW_DG1_0003,
  '/dev/rdsk/c4t60060E80056FB30000006FB300000826d0s6' NAME PMDW_DG1_0004,
  '/dev/rdsk/c4t60060E80056FB30000006FB300000829d0s6' NAME PMDW_DG1_0005
REBALANCE POWER 11;

We go with the rebalance power of 11 which is full throttle because it is planned maintenance.

2. Check rebalance status from Enterprise Manager or v$ASM_OPERATION.

3. When rebalance completes, drop the old disks.

ALTER DISKGROUP PMDW_DG1 DROP DISK
   PMDW_DG1_0000,
   PMDW_DG1_0001,
   PMDW_DG1_0002
REBALANCE POWER 11;