Showing posts with label RAC 10g. Show all posts
Showing posts with label RAC 10g. Show all posts

Sunday, October 16, 2011

Setup ssh user equivalence for all RAC nodes


This Example Done on Two Node SSH Setup:
Log on as user ORACLE on nodes. Do not give pass phrase to any of the methods below.

ON NODE 1

Step 1

[root@rac1pub ~]# su - oracle

Step 2

[oracle@rac1pub ~]$ ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/home/oracle/.ssh/id_dsa):
Created directory '/home/oracle/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/oracle/.ssh/id_dsa.
Your public key has been saved in /home/oracle/.ssh/id_dsa.pub.
The key fingerprint is:
b1:24:2f:8f:5a:27:4b:ce:aa:09:ce:13:bd:d8:b1:3e
oracle@rac1pub.kasb.com

Step 3

[oracle@rac1pub ~]$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/oracle/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/oracle/.ssh/id_rsa.
Your public key has been saved in /home/oracle/.ssh/id_rsa.pub.
The key fingerprint is:
68:0f:79:13:76:a7:a1:2e:ce:6e:1f:a6:2b:1a:15:f3
oracle@rac1pub.kasb.com

Step 4

[oracle@rac1pub ~]$ cd /home/oracle/.ssh

Step 5

[oracle@rac1pub .ssh]$ ls -ltr
total 16
-rw-r--r--  1 oracle oinstall 613 Oct 16 23:06 id_dsa.pub
-rw-------  1 oracle oinstall 668 Oct 16 23:06 id_dsa
-rw-r--r--  1 oracle oinstall 233 Oct 16 23:06 id_rsa.pub
-rw-------  1 oracle oinstall 883 Oct 16 23:06 id_rsa


Step 6

Note: The file name should be (authorized_keys) their is no chance for spelling mistake.
otherwise your ssh will be failed.


[oracle@rac1pub .ssh]$ cat id_dsa.pub >> authorized_keys

[oracle@rac1pub .ssh]$ cat id_rsa.pub >> authorized_keys

Step 7
Now we are sending node1 ssh keys to node 2 in temporary file because to get this keys in authorized_keys file of Node 2
 
[oracle@rac1pub .ssh]$ scp authorized_keys  rac2pub:/tmp/rac1keys.tmp


The authenticity of host 'storage (192.168.1.192)' can't be established.
RSA key fingerprint is 97:b4:a8:13:a1:76:57:44:e2:0b:60:c1:b8:13:db:27.
Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added 'rac2pub,192.168.1.192' (RSA) to the list of known hosts.oracle@rac2pub's password:
authorized_keys                                                   100% 1692     1.7KB/s   00:00
 

ON NODE 2
Now you can verfiy on Rac node 2 in tmp directory file(rac1keys.tmp) should be their

[oracle@rac2pub ~]$ cd /tmp
[oracle@rac2pub tmp]$ ls -ltr
-rw-r--r--  1 oracle oinstall  1692 Oct 17 00:06 rac1keys.tmp

STEPS ON NODE 2==================================================
Note: Follow same Steps(1 to 6) done on RAC node 1


Step 7
Now we are sending node2 ssh keys to node 1 in temporary file because to get this keys in authorized_keys file of Node 1


[oracle@rac2pub .ssh]$ scp authorized_keys  rac1pub:/tmp/rac2keys.tmp
The authenticity of host 'storage (192.168.1.191)' can't be established.
RSA key fingerprint is 97:b4:a8:13:a1:76:57:44:e2:0b:60:c1:b8:13:db:27.
Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added 'rac1pub,192.168.1.191' (RSA) to the list of known hosts.oracle@rac1pub's password:
authorized_keys                                                   100% 1692     1.7KB/s   00:00



ON NODE 1
Now you can verfiy on Rac node 1 in tmp directory file (rac2keys.tmp) should be their
[oracle@rac1pub ~]$ cd /tmp
[oracle@rac1pub tmp]$ ls -ltr
-rw-r--r--  1 oracle oinstall  1692 Oct 17 00:15 rac2keys.tmp


After you have done all above now proceed further

ON NODE 1

[oracle@rac1pub ~]$ cd /home/oracle/.ssh/
[oracle@rac1pub .ssh]$ cat /tmp/rac2keys.tmp >> authorized_keys

ON NODE 2


[oracle@rac2pub ~]$ cd /home/oracle/.ssh/
[oracle@rac2pub .ssh]$ cat /tmp/rac1keys.tmp >> authorized_keys


Gather ssh fingerprints of all RAC nodes.
Now get fingerprints of all possible interfaces / nodes of this RAC setup using ssh.

NOTE: You need to exit after each successful logon to avoid confusion.

On Node 1:
ssh rac1pub.kasb.com
ssh rac1pvt.kasb.com
ssh rac2pub.kasb.com
ssh rac2pvt.kasb.com
ssh rac1pub
ssh rac1pvt
ssh rac2pub
ssh rac2pvt


On Node 2:
ssh rac1pub.kasb.com
ssh rac2pub.kasb.com
ssh rac1pvt.kasb.com
ssh rac2pvt.kasb.com
ssh rac1pub
ssh rac1pvt
ssh rac2pub
ssh rac2pvt


I put two practical example for your more uderstanding that what I am trying to say in Gather ssh fingerprints of all RAC nodes.
============================================================================
[oracle@rac1pub .ssh]$ ssh rac1pub.kasb.com

The authenticity of host 'rac1pub.kasb.com (192.168.1.191)' can't be established.
RSA key fingerprint is 97:b4:a8:13:a1:76:57:44:e2:0b:60:c1:b8:13:db:27.
Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added 'rac1pub.kasb.com,192.168.1.191' (RSA) to the list of known hosts.
[oracle@rac1pub ~]$ exit
logout
Connection to rac1pub.kasb.com closed.

[oracle@rac1pub .ssh]$ ssh rac1pvt.kasb.com
The authenticity of host 'rac1pvt.kasb.com (10.0.0.1)' can't be established.
RSA key fingerprint is 97:b4:a8:13:a1:76:57:44:e2:0b:60:c1:b8:13:db:27.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'rac1pvt.kasb.com,10.0.0.1' (RSA) to the list of known hosts.
Last login: Sun Oct 16 23:34:57 2011 from rac1pub.kasb.com

[oracle@rac1pub ~]$ exit
logout

=================================================================================

Verfication of Passwordless SSH connection
Here your passwordless connection established between both node

ON NODE 1You can see date of both nodes will be appeared without password prompt

[oracle@rac1pub ~]$ ssh rac1pub date
Sun Oct 16 23:41:29 PKT 2011
[oracle@rac1pub ~]$ ssh rac2pub date
Sun Oct 16 23:41:03 PKT 2011
[oracle@rac1pub ~]$ ssh rac2pvt date
Sun Oct 16 23:41:20 PKT 2011

ON Node 2
[oracle@rac2pub ~]$ ssh rac1pub date
Sun Oct 16 23:42:18 PKT 2011
[oracle@rac2pub ~]$ ssh rac1pvt date
Sun Oct 16 23:42:26 PKT 2011
[oracle@rac2pub ~]$ ssh rac2pvt date
Sun Oct 16 23:41:59 PKT 2011
[oracle@rac2pub ~]$ ssh rac2pub date
Sun Oct 16 23:42:06 PKT 2011
[oracle@rac2pub ~]$ ssh 192.168.1.191 date
Sun Oct 16 23:42:52 PKT 2011


Congratualtion your SSH have now Configured for Oracle Rac 10g.




Wednesday, July 20, 2011

Configuring Linux bonding driver for the Oracle 10g VIP or private interconnect

PURPOSE
-------
In order to avoid the public LAN from being a single point of failure, Oracle highly recommends
configuring a redundant set of public network interface cards (NIC's) on each cluster node.
Network redundancy can be achieved On Linux platforms using NIC Teaming (configuring multiple
interfaces to a team or using the Linux kernel bonding module).

This note will go over the two possible choices for achieving redundancy on Linux.

As inter-node IP address failover is achieved by using the Oracle managed VIP, 3rd party
clusterware based inter-node IP address failover technologies should not be configured on the
same set of NIC's that are used by the Oracle VIP. Only intra-node IP address failover
functionalities should be used in conjunction with the Oracle VIP.

SCOPE & APPLICATION
-------------------
This article is intended for experienced DBAs and Support Engineers.

1. NIC TEAMING BY CONFIGURING MULTIPLE INTERFACES TO A TEAM
-----------------------------------------------------------
Various hardware vendors provide network interface drivers and utilities to achieve NIC teaming.
Please consult your hardware vendor for details on how to configure your system for NIC teaming

2. NIC TEAMING USING THE LINUX KERNEL BONDING MODULE
----------------------------------------------------
The Linux kernel includes a bonding module that can be used to achieve software level NIC
teaming. The kernel bonding module can be used to team multiple physical interfaces to a single
logical interface, which is used to achieve fault tolerance and load balancing. The bonding
driver is available as part of the Linux kernel version 2.4.12 or newer versions. Since the
bonding module is delivered as part of the Linux kernel, it can be configured independently
from the interface driver vendor (different interfaces can constitute a single logical
interface).

The configuration steps are different among Linux distributions. This note will go over the
steps required to configure the bonding module in RedHat Enterprise Linux 3.0.

In the following example, two physical interfaces (eth0 and eth1) will be bonded together to a
single logical interface (bond0), and the VIP will run on top of the single logical interface.

A sample network configuration is as follows:

Default Gateway:
192.168.1.254

Netmask:
255.255.255.0

Interface configuration before bonding:
eth0: IP Address 192.168.1.1
eth1: IP Address 192.168.1.2

After configuring the bonding driver, a logical interface named bondX (where X is a number
higher than zero) representing the team of interfaces.

Interface configuration after bonding:
bond0: IP Address 192.168.1.10

Note: Before continuing, be sure the bonding module is mounted. The command "modprobe bonding" can be used to mount the module.

2-1 CONFIGURING THE BONDING DRIVER
----------------------------------
Since the bonding driver is delivered as a kernel module, the following lines need to be added
to /etc/modules.conf as root.

alias bond0 bonding
options bond0 miimon=100

For details on the "options" parameter, please refer to the documents referred to in section
2.8. In the above configuration, the MII link monitoring interval is set to 100ms. MII is used
to monitor the interface link status, and this is a typical configuration for mission critical
systems that require fast failure detection.

Note: MII is an abbreviation of "Media Independent Interface". Many popular fast ethernet
adapters use MII to autonegotiate the link speed and duplex mode.

By default, the bonding driver will transmit outgoing packets in a round-robin fashion using
each "slave" interface. The above example uses this default behavior. For details on changing
this behavior, please also refer to the documents referred to in section 2.8.

If you want to use muitlple bonding interfaces you should modify /etc/modules.conf like below example.

alias bond0 bonding
alias bond1 bonding
options bond0 miimon=100 max_bonds=2
options bond1 miimon=100 max_bonds=2

(In this example we have 2 bonding interfaces.)

The "max_bonds" parameter defines how many bonding interfaces we are
going to have.
For details on the "max_bonds" parameter, please refer to the documents
referred to in section 2.8.

2-2. CONFIGURING THE bond0 INTERFACE
------------------------------------
On RHEL 3.0, network interface parameters are configured in configuration files named
"ifcfg-<interface name>", found in the /etc/sysconfig/network-scripts directory. In order to
enable the bonding driver, a configuration file "ifcfg-bond0" needs to be created with
appropriate parameters. As root, create the file "/etc/sysconfig/network-scripts/ifcfg-bond0"
as shown below.

DEVICE=bond0
IPADDR=192.168.1.10
NETMASK=255.255.255.0
NETWORK=192.168.1.0
BROADCAST=192.168.1.255
ONBOOT=yes
BOOTPROTO=none
USERCTL=no
(Please change the IP address, Netmask, Broadcast to match your network configuration)

2-3. CHANGING THE CONFIGURATION FOR THE EXISTING INTERFACES
-----------------------------------------------------------
As root, please change the configuration file "/etc/sysconfig/network-scripts/ifcfg-eth0" as
shown below:

DEVICE=eth0
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
BOOTPROTO=none

Please also change the configuration file "/etc/sysconfig/network-scripts/ifcfg-eth1" as shown
below:

DEVICE=eth1
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
BOOTPROTO=none

These steps are necessary to associate the bond0 interface to its slave interfaces (eth0 and eth1).

2-4. RESTART THE NETWORK
------------------------
Execute the following commands as root to reflect the changes.

# service network stop
# service network start

If the configuration is correct, the above commands should both return [ OK ].

2-5. CONFIRMING THE NEW CONFIGURATION
-------------------------------------
The following messages should appear in your syslog (/var/log/messages).

Jan 28 16:00:09 rac01 kernel: bonding: MII link monitoring set to 100 ms
Jan 28 16:00:09 rac01 kernel: ip_tables: (C) 2000-2002 Netfilter core team
Jan 28 16:00:11 rac01 ifup: Enslaving eth0 to bond0
Jan 28 16:00:11 rac01 kernel: bonding: bond0: enslaving eth0 as a backup interface with a down link.
Jan 28 16:00:11 rac01 kernel: e1000: eth0: e1000_watchdog: NIC Link is Up 1000 Mbps Full Duplex
Jan 28 16:00:11 rac01 kernel: bonding: bond0: link status definitely up for interface eth0.
Jan 28 16:00:11 rac01 kernel: bonding: bond0: making interface eth0 the new active one.
Jan 28 16:00:11 rac01 ifup: ENslaving eth1 to bond0
Jan 28 16:00:11 rac01 kernel: bonding: bond0: enslaving eth1 as a backup interface with a down link.
Jan 28 16:00:11 rac01 kernel: e1000: eth1: e1000_watchdog: NIC Link is Up 1000 Mbps Full Duplex
Jan 28 16:00:11 rac01 kernel: bonding: bond0: link status definitely up for interface eth1.
Jan 28 16:00:11 rac01 network: Bringing up interface bond0: succeeded

The "ifconfig -a" command should return the following output.

bond0 Link encap:Ethernet HWaddr 00:0C:29:DC:83:E8
inet addr:192.168.1.10 Bcast:192.168.1.255 Mask:255.255.255.0
UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
RX packets:27 errors:0 dropped:0 overruns:0 frame:0
TX packets:1 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:3462 (3.3 Kb) TX bytes:42 (42.0 b)

eth0 Link encap:Ethernet HWaddr 00:0C:29:DC:83:E8
inet addr:192.168.1.10 Bcast:192.168.1.255 Mask:255.255.255.0
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:13 errors:0 dropped:0 overruns:0 frame:0
TX packets:1 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1701 (1.6 Kb) TX bytes:42 (42.0 b)
Interrupt:10 Base address:0x1424

eth1 Link encap:Ethernet HWaddr 00:0C:29:DC:83:E8
inet addr:192.168.1.10 Bcast:192.168.1.255 Mask:255.255.255.0
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:14 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1761 (1.7 Kb) TX bytes:0 (0.0 b)
Interrupt:11 Base address:0x14a4

(Note that other interfaces will also appear in a typical RAC installation)

2-6. CONSIDERATIONS FOR CRS INSTALLATION
-----------------------------------------
During CRS installation, choose "Public" for the "bond0" interface in the "Specify Network
Interface Usage" OUI page. If the "eth0" and "eth1" interfaces appear in OUI, then make sure
to choose "Do not use" for their types.

2-6a. IF YOU WANT TO CONFIGURE BONDING DEVICES AFTER INSTALLATION OF CRS.
-----------------------------------------
You can change your interconnect/public interface configuration using oifcfg command.
Please refer Note 283684.1

2-7. VIPCA CONFIGURATION
------------------------
The single interface name (i.e. "bond0") representing the redundant set of NIC's is the
interfaces that should be specified in the second screen in VIPCA (VIP Configuration Assistant,
1 of 2). Make sure not to select any of the underlying non-redundant NIC names in VIPCA, as
they should not be used by Oracle in a NIC teaming configuration.

2-8. OPTIONS FOR THE BONDING DRIVER
-----------------------------------
Various advanced interface, driver and switch configurations are available for achieving a
highly available network configuration. Please refer to the "Linux Ethernet Bonding Driver
mini-howto" for more details.

http://www.kernel.org/pub/linux/kernel/people/marcelo/linux-2.4/Documentation/networking/bonding.txt

RELATED DOCUMENTS
-----------------
Linux Ethernet Bonding Driver mini-howto:

http://www.kernel.org/pub/linux/kernel/people/marcelo/linux-2.4/Documentation/networking/bonding.txt

Red Hat Enterprise Linux 3: Reference Guide -> Appendix A. General Parameters and Modules -> A.3 Ethernet Parameters:

http://www.redhat.com/docs/manuals/enterprise/RHEL-3-Manual/ref-guide/s1-modules-ethernet.html

Note.283684.1 - How to Change Interconnect/Public Interface IP Subnet in a 10g Cluster
Note.291962.1 - Setting Up Bonding on SLES 9
Note.291958.1 - Setting Up Bonding in Suse SLES8