ORACLE 11203 RAC FOR REHL63(64BIT)安装文档.doc

ORACLE 11.2.0.3 RAC for REHL6.3(64bit)安装文档作者：魔心之泪1、安装环境22、安装前系统配置22.1存储规划22.2 ip地址规划32.3创建用户、组(所有节点)32.4配置各节点环境变量42.5关闭所有节点防火墙42.6配置NTP服务(所有节点)52.7系统参数设置52.8存储分区52.9 UDEV绑定裸设备63、安装GRID74、创建ASM磁盘组225、安装数据库246、建库291、安装环境 rootdb1 # uname -a Linux db1 2.6.32-279.el6.x86_64 #1 SMP Wed Jun 13 18:24:36 EDT 2012 x86_64 x86_64 x86_64 GNU/Linuxrootdb1 # lsb_releaseLSBVersion: :core-4.0-amd64:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-noarch2、安装前系统配置2.1存储规划名称映射裸设备大小OCR_VOTING/dev/sdb1/dev/raw/raw11G/dev/sdb2/dev/raw/raw21G/dev/sdb3/dev/raw/raw31GFlash_Recovery/dev/sdb5/dev/raw/raw45G/dev/sdb6/dev/raw/raw55G/dev/sdb7/dev/raw/raw65GDATA/dev/sdc1/dev/raw/raw9536.9G/dev/sdd1/dev/raw/raw10536.9G/dev/sde1/dev/raw/raw11536.9G/dev/sdf1/dev/raw/raw12536.9G/dev/sdg1/dev/raw/raw13536.9G/dev/sdh1/dev/raw/raw14536.9G/dev/sdi1/dev/raw/raw15536.9G/dev/sdj1/dev/raw/raw16536.9G2.2 ip地址规划主机名称db1db2eth0 IP(PUBLIC)10.238.78.3410.238.78.35eth0 IP(VIP)10.238.78.4010.238.78.41eth1(PRIV)192.168.0.1192.168.0.2SCAN IP10.238.78.42SIDpartner1partner2ORACLE 版本11.2.0.3修改所有节点hosts文件rootdb1 # cat /etc/hosts127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain410.238.73.135 websv110.238.78.34 db110.238.78.35 db210.238.78.40 db1-vip10.238.78.41 db2-vip192.168.0.1 db1-priv192.168.0.2 db2-priv10.238.78.42 rac-scan2.3创建用户、组(所有节点)groupadd -g 1000 oinstallgroupadd -g 1200 asmadmingroupadd -g 1201 asmdbagroupadd -g 1202 asmopergroupadd -g 1300 dbagroupadd -g 1301 operuseradd -m -u 1100 -g oinstall -G asmadmin,asmdba,asmoper griduseradd -m -u 1101 -g oinstall -G dba,oper,asmdba oraclemkdir -p /oracle/gridchown -R grid:oinstall /oracle/gridchown -R grid:oinstall /oracle/grid/11.2.0.3mkdir -p /oracle/dbchown -R oracle:oinstall /oracle/dbchmod -R 775 /oraclepasswd gridpasswd oracle2.4配置各节点环境变量修改grid用户.bash_profileexport ORACLE_SID=+ASM1#节点2为+ASM2export ORACLE_BASE=/oracle/oracle11gexport ORACLE_HOME=/oracle/grid/11.2.0.3export PATH=$ORACLE_HOME/bin:$PATH:/usr/local/bin/:.export TEMP=/tmpexport TMP=/tmpexport TMPDIR=/tmpumask 022修改oracle用户.bash_profileexport ORACLE_SID=partner1#节点2为partner2export ORACLE_UNQNAME=partnerexport ORACLE_BASE=/oracle/dbexport ORACLE_HOME=/oracle/db/11.2.0.3export PATH=$ORACLE_HOME/bin:$PATH:$ORA_CRS_HOME/binexport TEMP=/tmpexport TMPDIR=/tmpumask 0222.5关闭所有节点防火墙Service iptables status Service iptables stop chkconfig iptables off chkconfig iptables list设置/etc/selinux/config 文件，将 SELINUX 设置为 disabledrootdb1 # cat /etc/selinux/config# This file controls the state of SELinux on the system.# SELINUX= can take one of these three values:# enforcing - SELinux security policy is enforced.# permissive - SELinux prints warnings instead of enforcing.# disabled - No SELinux policy is loaded.SELINUX=disabled# SELINUXTYPE= can take one of these two values:# targeted - Targeted processes are protected,# mls - Multi Level Security protection.SELINUXTYPE=targeted2.6配置NTP服务(所有节点)LINUX客户端使用ntpdate 172.30.218.114 来向NTP服务器同步自己的时间其它LINUX如果仅作为只客户端的话,则不能启动ntpd服务!否则无法运行ntpdata 服务器地址 来同步时间之后可以使用cron或修改crontab文件定期向NTP服务器更新时间,并用# hwclock -systohc  将系统时间设置为硬件时间rootdb2 # crontab -l*/15 * * * * ntpdate 172.30.218.1142.7系统参数设置/etc/security/limits.conf文件添加如下内容：grid soft nproc 2047 grid hard nproc 16384 grid soft nofile 1024 grid hard nofile 65536 oracle soft nproc 2047 oracle hard nproc 16384 oracle soft nofile 1024 oracle hard nofile 65536/etc/pam.d/login添加：session required pam_limits.so2.8存储分区所有规划的设备进行分区，例如在节点1执行：rootdb1 # fdisk /dev/sdeWARNING: DOS-compatible mode is deprecated. It's strongly recommended to switch off the mode (command 'c') and change display units to sectors (command 'u').Command (m for help): pDisk /dev/sde: 536.9 GB, 536870912000 bytes255 heads, 63 sectors/track, 65270 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x000d60dc Device Boot Start End Blocks Id SystemCommand (m for help): nCommand action e extended p primary partition (1-4)pPartition number (1-4): 1First cylinder (1-65270, default 1): Using default value 1Last cylinder, +cylinders or +sizeK,M,G (1-65270, default 65270): Using default value 65270Command (m for help): wThe partition table has been altered!Calling ioctl() to re-read partition table.Syncing disks.所有节点同步分区partprobe2.9 UDEV绑定裸设备vi /etc/udev/rules.d/60-raw.rules# Enter raw device bindings here.# An example would be:# ACTION="add", KERNEL="sda", RUN+="/bin/raw /dev/raw/raw1 %N"# to bind /dev/raw/raw1 to /dev/sda, or# ACTION="add", ENVMAJOR="8", ENVMINOR="1", RUN+="/bin/raw /dev/raw/raw2 %M %m"# to bind /dev/raw/raw2 to the device with major 8, minor 1.#for ocr_votingACTION="add",KERNEL="sdb1",RUN+="/bin/raw /dev/raw/raw1 %N"ACTION="add",KERNEL="sdb2",RUN+="/bin/raw /dev/raw/raw2 %N"ACTION="add",KERNEL="sdb3",RUN+="/bin/raw /dev/raw/raw3 %N"#for Flash_RecoveryACTION="add",KERNEL="sdb5",RUN+="/bin/raw /dev/raw/raw4 %N"ACTION="add",KERNEL="sdb6",RUN+="/bin/raw /dev/raw/raw5 %N"ACTION="add",KERNEL="sdb7",RUN+="/bin/raw /dev/raw/raw6 %N"#for dataACTION="add",KERNEL="sdc1",RUN+="/bin/raw /dev/raw/raw7 %N"ACTION="add",KERNEL="sdd1",RUN+="/bin/raw /dev/raw/raw8 %N"ACTION="add",KERNEL="sde1",RUN+="/bin/raw /dev/raw/raw9 %N"ACTION="add",KERNEL="sdf1",RUN+="/bin/raw /dev/raw/raw10 %N"ACTION="add",KERNEL="sdg1",RUN+="/bin/raw /dev/raw/raw11 %N"ACTION="add",KERNEL="sdh1",RUN+="/bin/raw /dev/raw/raw12 %N"ACTION="add",KERNEL="sdi1",RUN+="/bin/raw /dev/raw/raw13 %N"ACTION="add",KERNEL="sdj1",RUN+="/bin/raw /dev/raw/raw14 %N"KERNEL="raw1-9*", OWNER="grid", GROUP="asmadmin", MODE="660"重启 UDEV： # /sbin/start_udev以上内容需要在所有节点执行验证：rootdb1 rules.d# ll /dev/rawtotal 0crw-rw- 1 grid asmadmin 162, 1 Jul 11 15:19 raw1crw-rw- 1 grid asmadmin 162, 10 Jul 11 15:19 raw10crw-rw- 1 grid asmadmin 162, 11 Jul 11 15:19 raw11crw-rw- 1 grid asmadmin 162, 12 Jul 11 15:19 raw12crw-rw- 1 grid asmadmin 162, 13 Jul 11 15:19 raw13crw-rw- 1 grid asmadmin 162, 14 Jul 11 15:19 raw14crw-rw- 1 grid asmadmin 162, 2 Jul 11 15:19 raw2crw-rw- 1 grid asmadmin 162, 3 Jul 11 15:19 raw3crw-rw- 1 grid asmadmin 162, 4 Jul 11 15:19 raw4crw-rw- 1 grid asmadmin 162, 5 Jul 11 15:19 raw5crw-rw- 1 grid asmadmin 162, 6 Jul 11 15:19 raw6crw-rw- 1 grid asmadmin 162, 7 Jul 11 15:19 raw7crw-rw- 1 grid asmadmin 162, 8 Jul 11 15:19 raw8crw-rw- 1 grid asmadmin 162, 9 Jul 11 15:19 raw9crw-rw- 1 root disk 162, 0 Jul 11 15:19 rawctl3、安装GRID节点1用GRID用执行：griddb1 grid$ export DISPLAY=10.238.73.144:0.0griddb1 grid$ export LANG=en_US.utf-8griddb1 grid$ ./runI*这里没有配置DNS,使用/etc/hosts文件来解析SCAN这里可以使用自动配置节点直接ssh互信，也可手动配置用grid、oracle用户在所有节点执行ssh-keygen -t dsassh-keygen -t rsacd .sshtouch authorized_keyscat id_rsa.pub >> authorized_keyscat id_dsa.pub >> authorized_keys#grid用户执行db1执行ssh db2 cat /home/grid/.ssh/id_rsa.pub >> authorized_keysssh db2 cat /home/grid/.ssh/id_dsa.pub >> authorized_keysdb2执行ssh db1 cat /home/grid/.ssh/id_rsa.pub >> authorized_keysssh db1 cat /home/grid/.ssh/id_dsa.pub >> authorized_keys#oracle执行db1执行ssh db2 cat /home/oracle/.ssh/id_rsa.pub >> authorized_keysssh db2 cat /home/oracle/.ssh/id_dsa.pub >> authorized_keysdb2执行ssh db1 cat /home/oracle/.ssh/id_rsa.pub >> authorized_keysssh db1 cat /home/oracle/.ssh/id_dsa.pub >> authorized_keysgrid、oracle用户验证ssh db1 datessh db2 datessh db1-priv datessh db2-priv date如果仍然需要输入密码，需检查相关目录权限chmod 644 authorized_keyschmod 755 /oracle/gridchmod 700 /oracle/grid/.ssh小提示：如果有的设备没有显示，则该设备可能被使用过，使用DD命令清除：dd if=/dev/zero of=/dev/raw/raw1 bs=10M count=10 grid安装包中找到cvuqdisk-1.0.9-1.rpm后运行(所有节点)：rootdb2 rpm# rpm -ivh cvuqdisk-1.0.9-1.rpm Preparing. # 100%Using default group oinstall to install package 1:cvuqdisk # 100%安装oracleasm,可在官网下载rootdb2 rpm# rpm -ivh oracleasm-support-2.1.8-1.el6.x86_64.rpmwarning: oracleasm-support-2.1.8-1.el6.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID ec551f03: NOKEYPreparing. # 100% package oracleasm-support-2.1.8-1.el6.x86_64 is already installed使用/etc/hosts解析scan可忽略此错误先在节点1和节点2执行：/oracle/oraInventory/orainstRoot.sh其次：/oracle/grid/11.2.0.3/root.sh执行结果：rootdb1 oracle# /oracle/grid/11.2.0.3/root.shPerforming root user operation for Oracle 11g The following environment variables are set as: ORACLE_OWNER= grid ORACLE_HOME= /oracle/grid/11.2.0.3Enter the full pathname of the local bin directory: /usr/local/bin: The contents of "dbhome" have not changed. No need to overwrite.The contents of "oraenv" have not changed. No need to overwrite.The contents of "coraenv" have not changed. No need to overwrite.Creating /etc/oratab file.Entries will be added to the /etc/oratab file as needed byDatabase Configuration Assistant when a database is createdFinished running generic part of root script.Now product-specific root actions will be performed.Using configuration parameter file: /oracle/grid/11.2.0.3/crs/install/crsconfig_paramsCreating trace directoryUser ignored Prerequisites during installationOLR initialization - successful root wallet root wallet cert root cert export peer wallet profile reader wallet pa wallet peer wallet keys pa wallet keys peer cert request pa cert request peer cert pa cert peer root cert TP profile reader root cert TP pa root cert TP peer pa cert TP pa peer cert TP profile reader pa cert TP profile reader peer cert TP peer user cert pa user certAdding Clusterware entries to upstartCRS-2672: Attempting to start 'ora.mdnsd' on 'db1'CRS-2676: Start of 'ora.mdnsd' on 'db1' succeededCRS-2672: Attempting to start 'ora.gpnpd' on 'db1'CRS-2676: Start of 'ora.gpnpd' on 'db1' succeededCRS-2672: Attempting to start 'ora.cssdmonitor' on 'db1'CRS-2672: Attempting to start 'ora.gipcd' on 'db1'CRS-2676: Start of 'ora.cssdmonitor' on 'db1' succeededCRS-2676: Start of 'ora.gipcd' on 'db1' succeededCRS-2672: Attempting to start 'ora.cssd' on 'db1'CRS-2672: Attempting to start 'ora.diskmon' on 'db1'CRS-2676: Start of 'ora.diskmon' on 'db1' succeededCRS-2676: Start of 'ora.cssd' on 'db1' succeededASM created and started successfully.Disk Group OCRVOTING created successfully.clscfg: -install mode specifiedSuccessfully accumulated necessary OCR keys.Creating OCR keys for user 'root', privgrp 'root'.Operation successful.CRS-4256: Updating the profileSuccessful addition of voting disk 14bd8fb4dff14f05bf40fca499a9a11f.Successful addition of voting disk 25c05e68002f4f6bbf722c09f8c7b068.Successful addition of voting disk 0231b788ad924fb3bf7eab84fd1de9ac.Successfully replaced voting disk group with +OCRVOTING.CRS-4256: Updating the profileCRS-4266: Voting file(s) successfully replaced# STATE File Universal Id File Name Disk group- - - - - 1. ONLINE 14bd8fb4dff14f05bf40fca499a9a11f (/dev/raw/raw1) OCRVOTING 2. ONLINE 25c05e68002f4f6bbf722c09f8c7b068 (/dev/raw/raw2) OCRVOTING 3. ONLINE 0231b788ad924fb3bf7eab84fd1de9ac (/dev/raw/raw3) OCRVOTINGLocated 3 voting disk(s).CRS-2672: Attempting to start 'ora.asm' on 'db1'CRS-2676: Start of 'ora.asm' on 'db1' succeededCRS-2672: Attempting to start 'ora.OCRVOTING.dg' on 'db1'CRS-2676: Start of 'ora.OCRVOTING.dg' on 'db1' succeededConfigure Oracle Grid Infrastructure for a Cluster . succeededrootdb2 rpm# /oracle/oraInventory/orainstRoot.shChanging permissions of /oracle/oraInventory.Adding read,write permissions for group.Removing read,write,execute permissions for world.Changing groupname of /oracle/oraInventory to oinstall.The execution of the script is complete.You have new mail in /var/spool/mail/rootrootdb2 rpm# /oracle/grid/11.2.0.3/root.shPerforming root user operation for Oracle 11g The following environment variables are set as: ORACLE_OWNER= grid ORACLE_HOME= /oracle/grid/11.2.0.3Enter the full pathname of the local bin directory: /usr/local/bin: The contents of "dbhome" have not changed. No need to overwrite.The contents of "oraenv" have not changed. No need to overwrite.The contents of "coraenv" have not changed. No need to overwrite.Creating /etc/oratab file.Entries will be added to the /etc/oratab file as needed byDatabase Configuration Assistant when a database is createdFinished running generic part of root script.Now product-specific root actions will be performed.Using configuration parameter file: /oracle/grid/11.2.0.3/crs/install/crsconfig_paramsCreating trace directoryUser ignored Prerequisites during installationOLR initialization - successfulAdding Clusterware entries to upstartCRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node db1, number 1, and is terminatingAn active cluster was found during exclusive startup, restarting to join the clusterConfigure Oracle Grid Infrastructure for a Cluster . succeededYou have new mail in /var/spool/mail/root验证：4、创建ASM磁盘组用gird用户登录$asmca5、安装数据库用/etc/hosts解析SCAN可忽略此错误安装完成后所有节点执行rootdb2 db2# /oracle/db/11.2.0.3/root.shPerforming root user operation for Oracle 11g The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME= /oracle/db/11.2.0.3Enter the full pathname of the local bin directory: /usr/local/bin: The contents of "dbhome" have not changed. No need to overwrite.The contents of "oraenv" have not changed. No need to overwrite.The contents of "coraenv" have not changed. No need to overwrite.Entries will be added to the /etc/oratab file as needed byDatabase Configuration Assistant when a database is createdFinished running generic part of root script.Now product-specific root actions will be performed.