CRS-1205 Auto-start failed for the CRS resource | Start of ora.<SID>.inst on member <node> failed.
| Topic ID: 3282 | |
| Created By: | 2008-JUL-16 15:53:28 [Vitaliy] |
| Updated By: | 2008-JUL-16 15:53:59 [Vitaliy] |
| Status: | Open |
| Severity: | Normal |
| Read Only: | No |
|
9346
2008-JUL-16 15:53:28
|
||||
|
Issue
--- alertqarac1.log shows --- 2008-07-15 19:08:16.303 [crsd(18464)]CRS-1205:Auto-start failed for the CRS resource . Details in qarac1. RAC Instance is not coming up after the node is rebooted.
RESEARCH
cd /u01/app/oracle/product/10.2.0/crs/log/qarac1 find . -ls | grep -v "4 drw" | grep -v client 4686403 588 -rw-r--r-- 1 root root 596559 Jul 15 19:13 ./crsd/crsd.log 4686401 4 -rw-r--r-- 1 oracle oinstall 794 Jul 15 19:07 ./evmd/evmdOUT.log 4686402 188 -rw-r--r-- 1 oracle oinstall 187483 Jul 15 19:08 ./evmd/evmd.log 4686404 4 -rw-r--r-- 1 oracle oinstall 193 Jul 15 19:07 ./cssd/cssdOUT.log 4686405 4 -rw-r--r-- 1 oracle oinstall 5 Jul 10 20:00 ./cssd/qarac1.pid 4686406 316 -rw-r--r-- 1 oracle oinstall 315799 Jul 15 19:08 ./cssd/ocssd.log 4917004 0 -rw-r--r-- 1 oracle oinstall 0 Jul 15 19:07 ./cssd/oclsomon/oclsomon.log 4917003 0 -rw-r--r-- 1 oracle oinstall 0 Jul 11 19:09 ./cssd/oclsomon/oclsomon.ba1 4915672 0 -rw-r--r-- 1 oracle oinstall 0 Jul 11 14:32 ./cssd/oclsomon/oclsomon.ba2 4686418 4 -rw-r--r-- 1 oracle oinstall 3356 Jul 15 19:08 ./racg/ora.qarac1.ons.log 4686396 12 -rw-rw-r-- 1 root oinstall 10289 Jul 15 19:08 ./alertqarac1.log qarac1.domain.com.ORASID1-> qarac1.domain.com.ORASID1-> grep -i xgrid ./crsd/crsd.log 2008-07-15 15:42:41.747: [ CRSRES][1495452000]0Resource Registered: ora.ORASID.db 2008-07-15 15:44:27.609: [ CRSRES][1495452000]0Resource Registered: ora.ORASID.ORASID1.inst 2008-07-15 15:44:33.143: [ CRSRES][1495452000]0Resource Registered: ora.ORASID.ORASID2.inst 2008-07-15 15:54:35.000: [ CRSRES][1497553248]0Attempting to start `ora.ORASID.ORASID1.inst` on member `qarac1` 2008-07-15 15:54:35.115: [ CRSRES][1510144352]0Attempting to start `ora.ORASID.ORASID2.inst` on member `qarac2` 2008-07-15 15:55:01.283: [ CRSRES][1497553248]0Start of `ora.ORASID.ORASID1.inst` on member `qarac1` succeeded. 2008-07-15 15:55:01.420: [ CRSRES][1497553248]0Attempting to start `ora.ORASID.db` on member `qarac2` 2008-07-15 15:55:01.542: [ CRSRES][1497553248]0Start of `ora.ORASID.db` on member `qarac2` succeeded. 2008-07-15 15:55:04.931: [ CRSRES][1510144352]0Start of `ora.ORASID.ORASID2.inst` on member `qarac2` succeeded. 2008-07-15 16:50:22.377: [ CRSRES][1524836704]0Attempting to start `ora.ORASID.db` on member `qarac1` 2008-07-15 16:50:22.566: [ CRSRES][1524836704]0Start of `ora.ORASID.db` on member `qarac1` succeeded. 2008-07-15 16:57:40.637: [ CRSRES][1524836704]0Attempting to start `ora.ORASID.ORASID2.inst` on member `qarac2` 2008-07-15 16:58:05.271: [ CRSRES][1524836704]0Start of `ora.ORASID.ORASID2.inst` on member `qarac2` succeeded. 2008-07-15 17:56:08.083: [ CRSRES][1522735456]0Attempting to start `ora.ORASID.ORASID2.inst` on member `qarac2` 2008-07-15 17:56:32.719: [ CRSRES][1522735456]0Start of `ora.ORASID.ORASID2.inst` on member `qarac2` succeeded. 2008-07-15 19:08:03.990: [ CRSRES][1484962144]0Attempting to start `ora.ORASID.ORASID1.inst` on member `qarac1` 2008-07-15 19:08:14.731: [ CRSAPP][1484962144]0StartResource error for ora.ORASID.ORASID1.inst error code = 1 2008-07-15 19:08:16.283: [ CRSRES][1484962144]0Start of `ora.ORASID.ORASID1.inst` on member `qarac1` failed. qarac1.domain.com.ORASID1-> vi ./crsd/crsd.log 2008-07-15 19:08:14.731: [ CRSAPP][1484962144]0StartResource error for ora.ORASID.ORASID1.inst error code = 1 2008-07-15 19:08:16.283: [ CRSRES][1484962144]0Start of `ora.ORASID.ORASID1.inst` on member `qarac1` failed. 2008-07-15 19:08:24.519: [ CRSRES][1482860896]0startRunnable: setting CLI values 2008-07-15 19:08:24.542: [ CRSRES][1482860896]0Attempting to start `ora.qarac1.ons` on member `qarac1` 2008-07-15 19:08:25.993: [ CRSRES][1482860896]0Start of `ora.qarac1.ons` on member `qarac1` succeeded. 2008-07-15 19:08:26.081: [ CRSRES][1487063392]0Start of `ora.qarac1.ASM1.asm` on member `qarac1` succeeded. 2008-07-15 19:08:26.082: [ CRSRES][1484962144]0Skip online resource: ora.qarac1.ons 2008-07-15 19:08:26.117: [ CRSRES][1487063392]0startRunnable: setting CLI values 2008-07-15 19:08:26.125: [ CRSRES][1487063392]0Attempting to start `ora.qarac1.gsd` on member `qarac1` 2008-07-15 19:08:26.459: [ CRSRES][1487063392]0Start of `ora.qarac1.gsd` on member `qarac1` succeeded. 2008-07-15 19:13:16.581: [ CRSRES][1487063392]0CRS-1002: Resource 'ora.qarac1.LISTENER_QARAC1.lsnr' is already running on member 'qarac1' ## BINGO -- CRS tried to bring up ora.ORASID.ORASID1.inst BEFORE ora.qarac1.ASM1.asm! ## So there's no dependency on ASM -- here's hot to verify this: export ORACLE_HOME=$CRS_HOME export PATH=$ORACLE_HOME/bin:$PATH crs_stat -p | grep REQ ## qarac1.domain.com.ORASID1-> crs_stat -p | grep REQ ## REQUIRED_RESOURCES= ## REQUIRED_RESOURCES= ## REQUIRED_RESOURCES= ## REQUIRED_RESOURCES= ## REQUIRED_RESOURCES=ora.qarac1.vip ## REQUIRED_RESOURCES= ## REQUIRED_RESOURCES= ## REQUIRED_RESOURCES= ## REQUIRED_RESOURCES= ## REQUIRED_RESOURCES=ora.qarac2.vip ## REQUIRED_RESOURCES= ## REQUIRED_RESOURCES= ## REQUIRED_RESOURCES= ## qarac1.domain.com.ORASID1-> ## NOTE the only REQUIRED_RESOURCES are vip's it should also include ASM ...
SOLUTION
## run this on the first node ## export ORACLE_HOME=$CRS_HOME export PATH=$ORACLE_HOME/bin:$PATH srvctl modify instance -d ORASID -i ORASID1 -s +ASM1 srvctl modify instance -d ORASID -i ORASID2 -s +ASM2 ## qarac1.domain.com.ORASID1-> crs_stat -p | grep REQ ## REQUIRED_RESOURCES=ora.qarac1.ASM1.asm ## REQUIRED_RESOURCES=ora.qarac2.ASM2.asm ## REQUIRED_RESOURCES= ## REQUIRED_RESOURCES= ## REQUIRED_RESOURCES=ora.qarac1.vip ## REQUIRED_RESOURCES= ## REQUIRED_RESOURCES= ## REQUIRED_RESOURCES= ## REQUIRED_RESOURCES= ## REQUIRED_RESOURCES=ora.qarac2.vip ## REQUIRED_RESOURCES= ## REQUIRED_RESOURCES= ## REQUIRED_RESOURCES= ## qarac1.domain.com.ORASID1-> ## REFERENCE: ## Note:276208.1 CRS is not starting first the ASM instance I used the following search terms to find it: "Start of" inst "on member" failed