Configure RAW devices to be used by Oracle ASM using UDEV in RHEL 5 and RHEL 6

OBJECTIVE

The post describes on how to configure RAW devices to be used by Oracle ASM using UDEV in RHEL 5 and RHEL 6.

STEPS

Ø Configuring raw devices via UDEV rules for Oracle ASM is very helpful when you want to overrule ASM Lib packages due to OS dependency.

Ø ASM Lib packages are not easily available for RHEL kernel version greater than 6.3.

Ø Important point to consider is UDEV rules and ASM lib packages won’t work together.

I. Check all the allocated disks.

[root@raeydb]# fdisk -l

Disk /dev/xvda: 64.4 GB, 64424509440 bytes

255 heads, 63 sectors/track, 7832 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0x000187a6

Device Boot Start End Blocks Id System

/dev/xvda1 * 1 26 204800 83 Linux

Partition 1 does not end on cylinder boundary.

/dev/xvda2 26 3347 26672128 83 Linux

/dev/xvda3 3347 4391 8388608 82 Linux swap / Solaris

/dev/xvda4 4391 7833 27648000 5 Extended

/dev/xvda5 4391 7833 27646976 83 Linux

Disk /dev/xvdb: 21.5 GB, 21474836480 bytes

255 heads, 63 sectors/track, 2610 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0xc8c5039a

Device Boot Start End Blocks Id System

Disk /dev/xvdc: 21.5 GB, 21474836480 bytes

255 heads, 63 sectors/track, 2610 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0xbc71bf7e

Device Boot Start End Blocks Id System

[root@raeydb]#

II. Create partitions for the disk.

[root@raeydb ~]# fdisk /dev/xvdb

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to

switch off the mode (command 'c') and change display units to

sectors (command 'u').

Command (m for help): n

Command action

e extended

p primary partition (1-4)

Partition number (1-4):

Value out of range.

Partition number (1-4): 1

First cylinder (1-2610, default 1):

Using default value 1

Last cylinder, +cylinders or +size{K,M,G} (1-2610, default 2610):

Using default value 2610

Command (m for help): w

The partition table has been altered!

Calling ioctl() to re-read partition table.

Syncing disks.

[root@raeydb ~]#

[root@raeydb ~]# fdisk -l

Disk /dev/xvda: 64.4 GB, 64424509440 bytes

255 heads, 63 sectors/track, 7832 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0x000187a6

Device Boot Start End Blocks Id System

/dev/xvda1 * 1 26 204800 83 Linux

Partition 1 does not end on cylinder boundary.

/dev/xvda2 26 3347 26672128 83 Linux

/dev/xvda3 3347 4391 8388608 82 Linux swap / Solaris

/dev/xvda4 4391 7833 27648000 5 Extended

/dev/xvda5 4391 7833 27646976 83 Linux

Disk /dev/xvdb: 21.5 GB, 21474836480 bytes

255 heads, 63 sectors/track, 2610 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0xc8c5039a

Device Boot Start End Blocks Id System

/dev/xvdb1 1 2610 20964793+ 83 Linux

Disk /dev/xvdc: 21.5 GB, 21474836480 bytes

255 heads, 63 sectors/track, 2610 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0xbc71bf7e

Device Boot Start End Blocks Id System

[root@raeydb ~]# /sbin/partprobe

Warning: WARNING: the kernel failed to re-read the partition table on /dev/xvda (Device or resource busy). As a result, it may not reflect all of your changes until after reboot.

[root@raeydb ~]#

[root@raeydb rules.d]# cat /proc/partitions

major minor #blocks name

202 0 62914560 xvda

202 1 204800 xvda1

202 2 26672128 xvda2

202 3 8388608 xvda3

202 4 1 xvda4

202 5 27646976 xvda5

202 16 20971520 xvdb

202 17 20964793 xvdb1

202 32 20971520 xvdc

III. Repeat above steps for remaining disks.

IV. Add disks in file with name <anyname.rules> under rules.d directory

[root@raeydb ~]# cd /etc/udev/rules.d/

[root@raeydb rules.d]# vi 60-raw.rules

[root@raeydb rules.d]#

[root@raeydb rules.d]# cat 60-raw.rules

# Enter raw device bindings here.

# An example would be:

# ACTION=="add", KERNEL=="sda", RUN+="/bin/raw /dev/raw/raw1 %N"

# to bind /dev/raw/raw1 to /dev/sda, or

# ACTION=="add", ENV{MAJOR}=="8", ENV{MINOR}=="1", RUN+="/bin/raw /dev/raw/raw2 %M %m"

# to bind /dev/raw/raw2 to the device with major 8, minor 1.

ACTION=="add", KERNEL=="xvdb1", RUN+="/bin/raw /dev/raw/raw1 %N"

ACTION=="add", KERNEL=="raw*", OWNER=="grid", GROUP=="oinstall",MODE=="0660"

[root@raeydb rules.d]#

[root@raeydb rules.d]# /bin/raw /dev/raw/raw1 /dev/xvdb1

/dev/raw/raw1: bound to major 202, minor 17

[root@raeydb rules.d]#

[root@raeydb rules.d]# udevadm test /block/xvdb/xvdb1 | grep raw

For RHEL 5.X
[root@raeydb rules.d]# udevtest /block/sdb/sdb1 | grep raw

run_command: calling: test

udevadm_test: version 147

parse_file: reading '/lib/udev/rules.d/10-console.rules' as rules file

parse_file: reading '/lib/udev/rules.d/10-dm.rules' as rules file

parse_file: reading '/lib/udev/rules.d/11-dm-lraeydb.rules' as rules file

parse_file: reading '/lib/udev/rules.d/13-dm-disk.rules' as rules file

parse_file: reading '/etc/udev/rules.d/40-hplip.rules' as rules file

parse_file: reading '/lib/udev/rules.d/40-isdn.rules' as rules file

parse_file: reading '/lib/udev/rules.d/40-libgphoto2.rules' as rules file

parse_file: reading '/lib/udev/rules.d/40-redhat.rules' as rules file

parse_file: reading '/lib/udev/rules.d/40-usb-media-players.rules' as rules file

parse_file: reading '/lib/udev/rules.d/42-qemu-usb.rules' as rules file

parse_file: reading '/lib/udev/rules.d/50-firmware.rules' as rules file

parse_file: reading '/lib/udev/rules.d/50-udev-default.rules' as rules file

parse_file: reading '/etc/udev/rules.d/56-hpmud_support.rules' as rules file

parse_file: reading '/lib/udev/rules.d/60-cdrom_id.rules' as rules file

parse_file: reading '/etc/udev/rules.d/60-fprint-autosuspend.rules' as rules file

parse_file: reading '/lib/udev/rules.d/60-libmtp.rules' as rules file

parse_file: reading '/lib/udev/rules.d/60-net.rules' as rules file

parse_file: reading '/etc/udev/rules.d/60-pcmcia.rules' as rules file

parse_file: reading '/lib/udev/rules.d/60-persistent-alsa.rules' as rules file

parse_file: reading '/lib/udev/rules.d/60-persistent-input.rules' as rules file

parse_file: reading '/lib/udev/rules.d/60-persistent-serial.rules' as rules file

parse_file: reading '/lib/udev/rules.d/60-persistent-storage-tape.rules' as rules file

parse_file: reading '/lib/udev/rules.d/60-persistent-storage.rules' as rules file

parse_file: reading '/lib/udev/rules.d/60-persistent-v4l.rules' as rules file

parse_file: reading '/etc/udev/rules.d/60-raw.rules' as rules file

parse_file: reading '/lib/udev/rules.d/60-wacom.rules' as rules file

parse_file: reading '/lib/udev/rules.d/61-gnome-bluetooth-rfkill.rules' as rules file

parse_file: reading '/lib/udev/rules.d/61-mobile-action.rules' as rules file

parse_file: reading '/lib/udev/rules.d/61-option-modem-modeswitch.rules' as rules file

parse_file: reading '/lib/udev/rules.d/61-persistent-storage-edd.rules' as rules file

parse_file: reading '/lib/udev/rules.d/61-uinput-stddev.rules' as rules file

parse_file: reading '/lib/udev/rules.d/61-uinput-wacom.rules' as rules file

parse_file: reading '/lib/udev/rules.d/64-device-mapper.rules' as rules file

parse_file: reading '/lib/udev/rules.d/64-md-raid.rules' as rules file

parse_file: reading '/lib/udev/rules.d/65-libsane.rules' as rules file

parse_file: reading '/lib/udev/rules.d/65-libwacom.rules' as rules file

parse_file: reading '/lib/udev/rules.d/65-md-incremental.rules' as rules file

parse_file: reading '/lib/udev/rules.d/69-dm-lraeydb-metad.rules' as rules file

parse_file: reading '/lib/udev/rules.d/69-pilot-link.rules' as rules file

parse_file: reading '/lib/udev/rules.d/70-acl.rules' as rules file

parse_file: reading '/lib/udev/rules.d/70-cups-libusb.rules' as rules file

parse_file: reading '/lib/udev/rules.d/70-hid2hci.rules' as rules file

parse_file: reading '/etc/udev/rules.d/70-persistent-cd.rules' as rules file

parse_file: reading '/lib/udev/rules.d/70-printers.rules' as rules file

parse_file: reading '/lib/udev/rules.d/71-biosdevname.rules' as rules file

parse_file: reading '/lib/udev/rules.d/75-cd-aliases-generator.rules' as rules file

parse_file: reading '/lib/udev/rules.d/75-net-description.rules' as rules file

parse_file: reading '/lib/udev/rules.d/75-persistent-net-generator.rules' as rules file

parse_file: reading '/lib/udev/rules.d/75-tty-description.rules' as rules file

parse_file: reading '/lib/udev/rules.d/77-mm-ericsson-mbm.rules' as rules file

parse_file: reading '/lib/udev/rules.d/77-mm-longcheer-port-types.rules' as rules file

parse_file: reading '/lib/udev/rules.d/77-mm-pcmcia-device-blacklist.rules' as rules file

parse_file: reading '/lib/udev/rules.d/77-mm-platform-serial-whitelist.rules' as rules file

parse_file: reading '/lib/udev/rules.d/77-mm-simtech-port-types.rules' as rules file

parse_file: reading '/lib/udev/rules.d/77-mm-usb-device-blacklist.rules' as rules file

parse_file: reading '/lib/udev/rules.d/77-mm-zte-port-types.rules' as rules file

parse_file: reading '/lib/udev/rules.d/77-nm-olpc-mesh.rules' as rules file

parse_file: reading '/lib/udev/rules.d/78-sound-card.rules' as rules file

parse_file: reading '/lib/udev/rules.d/79-fstab_import.rules' as rules file

parse_file: reading '/lib/udev/rules.d/80-drivers.rules' as rules file

parse_file: reading '/lib/udev/rules.d/80-iosched.rules' as rules file

parse_file: reading '/lib/udev/rules.d/80-mpath-iosched.rules' as rules file

parse_file: reading '/lib/udev/rules.d/80-udisks.rules' as rules file

parse_file: reading '/lib/udev/rules.d/85-regulatory.rules' as rules file

parse_file: reading '/lib/udev/rules.d/85-usbmuxd.rules' as rules file

parse_file: reading '/lib/udev/rules.d/88-clock.rules' as rules file

parse_file: reading '/lib/udev/rules.d/89-microcode.rules' as rules file

parse_file: reading '/etc/udev/rules.d/90-alsa.rules' as rules file

parse_file: reading '/etc/udev/rules.d/90-hal.rules' as rules file

parse_file: reading '/lib/udev/rules.d/90-infiniband.rules' as rules file

parse_file: reading '/lib/udev/rules.d/90-pulseaudio.rules' as rules file

parse_file: reading '/etc/udev/rules.d/91-drm-modeset.rules' as rules file

parse_file: reading '/lib/udev/rules.d/95-devkit-power-battery-recall-dell.rules' as rules file

parse_file: reading '/lib/udev/rules.d/95-devkit-power-battery-recall-fujitsu.rules' as rules file

parse_file: reading '/lib/udev/rules.d/95-devkit-power-battery-recall-gateway.rules' as rules file

parse_file: reading '/lib/udev/rules.d/95-devkit-power-battery-recall-ibm.rules' as rules file

parse_file: reading '/lib/udev/rules.d/95-devkit-power-battery-recall-lenovo.rules' as rules file

parse_file: reading '/lib/udev/rules.d/95-devkit-power-battery-recall-toshiba.rules' as rules file

parse_file: reading '/lib/udev/rules.d/95-devkit-power-csr.rules' as rules file

parse_file: reading '/lib/udev/rules.d/95-devkit-power-hid.rules' as rules file

parse_file: reading '/lib/udev/rules.d/95-devkit-power-wup.rules' as rules file

parse_file: reading '/lib/udev/rules.d/95-dm-notify.rules' as rules file

parse_file: reading '/lib/udev/rules.d/95-keyboard-force-release.rules' as rules file

parse_file: reading '/lib/udev/rules.d/95-keymap.rules' as rules file

parse_file: reading '/lib/udev/rules.d/95-udev-late.rules' as rules file

parse_file: reading '/etc/udev/rules.d/97-bluetooth-serial.rules' as rules file

parse_file: reading '/lib/udev/rules.d/97-bluetooth.rules' as rules file

parse_file: reading '/etc/udev/rules.d/98-kexec.rules' as rules file

parse_file: reading '/etc/udev/rules.d/99-fuse.rules' as rules file

parse_file: reading '/dev/.udev/rules.d/99-root.rules' as rules file

parse_file: reading '/etc/udev/rules.d/z10-xen-vcpu-hotplug.rules' as rules file

udev_rules_new: rules use 191448 bytes tokens (15954 * 12 bytes), 39810 bytes buffer

udev_rules_new: temporary index used 60780 bytes (3039 * 20 bytes)

udev_device_new_from_syspath: device 0x7fa67ba6dfa0 has devpath '/devices/vbd-832/block/xvdb/xvdb1'

udev_device_new_from_syspath: device 0x7fa67ba6dcc0 has devpath '/devices/vbd-832/block/xvdb/xvdb1'

udev_device_read_db: device 0x7fa67ba6dcc0 filled with db file data

udev_device_new_from_syspath: device 0x7fa67ba6bae0 has devpath '/devices/vbd-832/block/xvdb'

udev_device_new_from_syspath: device 0x7fa67ba6c570 has devpath '/devices/vbd-832'

udev_rules_apply_to_event: LINK 'block/202:17' /lib/udev/rules.d/50-udev-default.rules:3

udev_rules_apply_to_event: GROUP 6 /lib/udev/rules.d/50-udev-default.rules:76

udev_device_read_db: device 0x7fa67ba6bae0 filled with db file data

udev_rules_apply_to_event: LINK 'disk/by-path/xen-vbd-832-part1' /lib/udev/rules.d/60-persistent-storage.rules:62

udev_rules_apply_to_event: IMPORT '/sbin/blkid -o udev -p /dev/xvdb1' /lib/udev/rules.d/60-persistent-storage.rules:78

util_run_program: '/sbin/blkid -o udev -p /dev/xvdb1' started

util_run_program: '/sbin/blkid -o udev -p /dev/xvdb1' returned with exitcode 2

udev_rules_apply_to_event: RUN '/bin/raw /dev/raw/raw1 %N' /etc/udev/rules.d/60-raw.rules:9

udev_rules_apply_to_event: IMPORT 'udisks-part-id /dev/xvdb1' /lib/udev/rules.d/80-udisks.rules:87

util_run_program: 'udisks-part-id /dev/xvdb1' started

util_run_program: '/lib/udev/udisks-part-id' (stderr) 'libudev: udev_device_new_from_syspath: '

util_run_program: '/lib/udev/udisks-part-id' (stderr) 'device 0x230b270 has devpath '/devices/vbd-832/block/xvdb/xvdb1''

util_run_program: '/lib/udev/udisks-part-id' (stderr) 'libudev: udev_device_new_from_syspath: '

util_run_program: '/lib/udev/udisks-part-id' (stderr) 'device 0x230cb70 has devpath '/devices/vbd-832/block/xvdb''

util_run_program: '/lib/udev/udisks-part-id' (stderr) 'libudev: udev_device_read_db: '

util_run_program: '/lib/udev/udisks-part-id' (stderr) 'device 0x230cb70 filled with db file data'

util_run_program: '/lib/udev/udisks-part-id' (stderr) 'using device_file=/dev/xvdb syspath=/sys/devices/vbd-832/block/xvdb, offset=32256 ao=0 and number=1 for /dev/xvdb1'

util_run_program: '/lib/udev/udisks-part-id' (stderr) 'Entering MS-DOS parser (offset=0, size=21474836480)'

util_run_program: '/lib/udev/udisks-part-id' (stderr) ''

util_run_program: '/lib/udev/udisks-part-id' (stderr) 'MSDOS_MAGIC found'

util_run_program: '/lib/udev/udisks-part-id' (stderr) ''

util_run_program: '/lib/udev/udisks-part-id' (stderr) 'looking at part 0 (offset 32256, size 21467948544, type 0x83)'

util_run_program: '/lib/udev/udisks-part-id' (stderr) ''

util_run_program: '/lib/udev/udisks-part-id' (stderr) 'new part entry'

util_run_program: '/lib/udev/udisks-part-id' (stderr) ''

util_run_program: '/lib/udev/udisks-part-id' (stderr) 'looking at part 1 (offset 0, size 0, type 0x00)'

util_run_program: '/lib/udev/udisks-part-id' (stderr) ''

util_run_program: '/lib/udev/udisks-part-id' (stderr) 'new part entry'

util_run_program: '/lib/udev/udisks-part-id' (stderr) ''

util_run_program: '/lib/udev/udisks-part-id' (stderr) 'looking at part 2 (offset 0, size 0, type 0x00)'

util_run_program: '/lib/udev/udisks-part-id' (stderr) ''

util_run_program: '/lib/udev/udisks-part-id' (stderr) 'new part entry'

util_run_program: '/lib/udev/udisks-part-id' (stderr) ''

util_run_program: '/lib/udev/udisks-part-id' (stderr) 'looking at part 3 (offset 0, size 0, type 0x00)'

util_run_program: '/lib/udev/udisks-part-id' (stderr) ''

util_run_program: '/lib/udev/udisks-part-id' (stderr) 'new part entry'

util_run_program: '/lib/udev/udisks-part-id' (stderr) ''

util_run_program: '/lib/udev/udisks-part-id' (stderr) 'Exiting MS-DOS parser'

util_run_program: '/lib/udev/udisks-part-id' (stderr) ''

util_run_program: '/lib/udev/udisks-part-id' (stderr) 'MSDOS partition table detected'

util_run_program: '/lib/udev/udisks-part-id' (stderr) ''

util_run_program: '/lib/udev/udisks-part-id' (stdout) 'UDISKS_PARTITION=1'

util_run_program: '/lib/udev/udisks-part-id' (stdout) 'UDISKS_PARTITION_SCHEME=mbr'

util_run_program: '/lib/udev/udisks-part-id' (stdout) 'UDISKS_PARTITION_NUMBER=1'

util_run_program: '/lib/udev/udisks-part-id' (stdout) 'UDISKS_PARTITION_TYPE=0x83'

util_run_program: '/lib/udev/udisks-part-id' (stdout) 'UDISKS_PARTITION_SIZE=21467948544'

util_run_program: '/lib/udev/udisks-part-id' (stdout) 'UDISKS_PARTITION_LABEL='

util_run_program: '/lib/udev/udisks-part-id' (stdout) 'UDISKS_PARTITION_UUID='

util_run_program: '/lib/udev/udisks-part-id' (stdout) 'UDISKS_PARTITION_FLAGS='

util_run_program: '/lib/udev/udisks-part-id' (stdout) 'UDISKS_PARTITION_SLAVE=/sys/devices/vbd-832/block/xvdb'

util_run_program: '/lib/udev/udisks-part-id' (stdout) 'UDISKS_PARTITION_OFFSET=32256'

util_run_program: '/lib/udev/udisks-part-id' (stdout) 'UDISKS_PARTITION_ALIGNMENT_OFFSET=0'

util_run_program: 'udisks-part-id /dev/xvdb1' returned with exitcode 0

udev_rules_apply_to_event: RUN 'socket:@/org/freedesktop/hal/udev_event' /etc/udev/rules.d/90-hal.rules:2

udev_event_execute_rules: no node name set, will use kernel supplied name 'xvdb1'

udev_device_update_db: created db file for '/devices/vbd-832/block/xvdb/xvdb1' in '/dev/.udev/db/block:xvdb1'

udev_node_add: creating device node '/dev/xvdb1', devnum=202:17, mode=0660, uid=0, gid=6

udev_node_mknod: preserve file '/dev/xvdb1', because it has correct dev_t

udev_node_mknod: preserve permissions /dev/xvdb1, 060660, uid=0, gid=6

node_symlink: preserve already existing symlink '/dev/block/202:17' to '../xvdb1'

link_find_prioritized: found '/sys/devices/vbd-832/block/xvdb/xvdb1' claiming '/dev/.udev/links/disk\x2fby-path\x2fxen-vbd-832-part1'

link_update: creating link '/dev/disk/by-path/xen-vbd-832-part1' to '/dev/xvdb1'

node_symlink: preserve already existing symlink '/dev/disk/by-path/xen-vbd-832-part1' to '../../xvdb1'

udevadm_test: UDEV_LOG=6

udevadm_test: DEVPATH=/devices/vbd-832/block/xvdb/xvdb1

udevadm_test: MAJOR=202

udevadm_test: MINOR=17

udevadm_test: DEVNAME=/dev/xvdb1

udevadm_test: DEVTYPE=partition

udevadm_test: ACTION=add

udevadm_test: SUBSYSTEM=block

udevadm_test: DEVLINKS=/dev/block/202:17 /dev/disk/by-path/xen-vbd-832-part1

udevadm_test: ID_PATH=xen-vbd-832

udevadm_test: ID_PART_TABLE_TYPE=dos

udevadm_test: LVM_SBIN_PATH=/sbin

udevadm_test: UDISKS_PRESENTATION_NOPOLICY=1

udevadm_test: UDISKS_PARTITION=1

udevadm_test: UDISKS_PARTITION_SCHEME=mbr

udevadm_test: UDISKS_PARTITION_NUMBER=1

udevadm_test: UDISKS_PARTITION_TYPE=0x83

udevadm_test: UDISKS_PARTITION_SIZE=21467948544

udevadm_test: UDISKS_PARTITION_SLAVE=/sys/devices/vbd-832/block/xvdb

udevadm_test: UDISKS_PARTITION_OFFSET=32256

udevadm_test: UDISKS_PARTITION_ALIGNMENT_OFFSET=0

udevadm_test: run: '/bin/raw /dev/raw/raw1 /dev/xvdb1'

udevadm_test: run: 'socket:@/org/freedesktop/hal/udev_event'

[root@raeydb rules.d]#

[root@raeydb rules.d]# udevadm test /block/xvdb/xvdb1 | grep mode

For RHEL 5.X
[root@raeydb rules.d]# udevtest /block/xvdb/xvdb1 | grep mode