Oracle RAC handson Part 8 RAC on ASM

Summary:

ASM normal redundancy works but is as half of the IO speed as external redundancy. Adding or removing disk from ASM disk group will automatically start disk rebalance and will slower system IO to half of it before rebalancing. So the good way to add disk is to add small with small failure group. Stepped on few bugs related to ASM 10g release 2. ASM is clear the winner in terms fo performance, comparing with other file systems (OCFS, ext3), which is more than twice as fast as the OCFS for RAC configuration. It is 60% and 100% and faster than OCFS and ext3 for non-RAC configuration respectively

Test Case 16:

Name: ASM fail over to failure group (ASM normal mirroring)

Purpose: Test ASM mirroring

Detail:

1. Set up ASM disk group and failure group

2. Query a large table with full table scan and record the time for completion. (

select distinct species from sequences; ) It takes 74 seconds to completion on average.

Query the table again and introduce/simulate a disk failure and record the time for query to completion. (blade1:root@ca-test1 /]# /etc/init.d/oracleasm deletedisk DISK1) It takes 74 seconds to completion on average.

Result:

Mirroring disk failure is transparent to database.

Test Case 17:

Name: Impact of ASM add/remove disk operation

Purpose: Test ASM dynamic disk swapping

Detail:

1. Set up ASM disk group with normal mirroring.

2. Query a large table with full table scan (select distinct species from sequences;) and record the time for completion.

3. Query the table again and add disk to the disk group to ASM instance (ALTER DISKGROUP DISKGRP1 ADD FAILGROUP DISK5 DISK 'ORCL:DISK5' NAME DISK5 SIZE 102398 M REBALANCE POWER 5; ) at the same time. Record the completion of the query.

4. Query the table again and remove disk to the disk group (ALTER DISKGROUP DISKGRP1 DROP DISK DISK5 REBALANCE POWER 5; ) at the same time. Record the completion of the query.

Result:

The baseline query (with no add/drop disk activity) takes average 57 seconds. (56, 56, 59 seconds). With disk group rebalance activity, the query takes average of 106 seconds. (103, 109, 106 seconds.) By default, the add/drop disk is with rebalance power of 1. If disk does not have failure group available, it will hung. Bug 4747535 and Bug 4772979 make operation more complicate. It is good idea to make failure group small, such as 100 GB. One more bug in EM. When drop disk from disk group, you can select reblanace power level from “Advanced Options” button. However, it always use default (rebalance power 1) no matter what setting you select.

Test Case 18:

Name: Impact of ASM normal and external redundancy

Purpose: Test cost associated with ASM redundancy

Detail:

1. Set up ASM disk group with normal and external redundancy. (DISKGRP1 normal redundancy, DISKGRP2 external redundancy).

2. Query a large table with full table scan and record the time for completion on both external mirroring and normal redundancy. (select distinct species from sequences; select distinct species from sequences_e; )

Result:

Query identical table on normal redundancy disk group takes average of 70 seconds. (71,69,69 seconds.) Query identical table on external redundancy disk group takes average of 37 seconds. (38,36,37 seconds.), which is 53% of the time to query same table under normal redundancy.

Test Case 19:

Compare performance of ASM, OCFS2 and ext3.

Summary

1. ASM is much faster than OCFS for Oracle RAC configuration. (1.34 times faster)
2. ASM is faster than OCFS for Oracle non-RAC configuration. (0.59 times faster)
3. OCFS is faster than ext3 for non-RAC configuration (0.25 times faster)
4. OCFS for non-RAC configuration is much faster than RAC configuration (0.78 times faster)

Normalized IO comparison for Oracle Storage (time)

	ASM	OCFS	ext3
RAC	1.00	2.34	NA
NON-RAC	0.861	1.37	1.72

* Normalization based on time takes to read fixed amount of data. RAC-ASM as value 1. The lower the number, the better IO performance.

Details

database	storage	normalized IO	normalized IO
		(Reads)	(time)
RAC	ASM	1	1
RAC	OCFS	0.429	2.34
NORAC	ASM	1.161	0.86
NORAC	OCFS	0.729	1.37
NORAC	ext3	0.581	1.72