Search: For:
Browsing Single Category
www.dbatoolz.com ORACLE DBA Forums Solutions › Topic Id: 2902 | Permalink

127.0.0.1 6101 missing form factor - thousands of TCP sockets on port 6101

Topic ID: 2902
Created By: 2008-JAN-10 16:03:30 [Vitaliy]
Updated By: 2008-JAN-10 16:06:15 [Vitaliy]
Status: Open
Severity: Normal
Read Only: No
8754
2008-JAN-10 16:03:30
Moderator
 
 
Registered On: Mar 2006
Total Posts: 267
Issue
Right after starting the Oracle TNS listener you notice thousands of TCP 
sockets that are being opened on PORT 6101:

hostname.10GR2-> netstat -a | grep 6101 | wc -l
   32022

hostname.10GR2-> netstat -a | grep 6101 | head
localhost.6101       localhost.54114      49152      0 49204      0 TIME_WAIT
localhost.6101       localhost.53091      49152      0 49204      0 TIME_WAIT
localhost.6101       localhost.52579      49152      0 49204      0 TIME_WAIT
localhost.6101       localhost.52064      49152      0 49204      0 TIME_WAIT
localhost.6101       localhost.51238      49152      0 49204      0 TIME_WAIT
localhost.6101       localhost.51750      49152      0 49204      0 TIME_WAIT
localhost.6101       localhost.50530      49152      0 49204      0 TIME_WAIT
localhost.6101       localhost.49461      49152      0 49204      0 TIME_WAIT
localhost.6101       localhost.49973      49152      0 49204      0 TIME_WAIT
localhost.6101       localhost.48993      49152      0 49204      0 TIME_WAIT
hostname.10GR2->

At the same time the opmn logs are being filled with the following messages:

hostname.10GR2-> pwd
/u01/app/oracle/product/oms10g/opmn/logs
hostname.10GR2->
hostname.10GR2-> ls -lta | head
total 421590616
-rw-rw----   1 oracle   dba      706178336 Jan 10 13:45 ons.log
drwx------   3 oracle   dba         5632 Jan 10 09:12 .
-rw-rw----   1 oracle   dba      1536000850 Jan 10 09:12 ons.log.08-01-10_09:12:13
-rw-rw----   1 oracle   dba      1536000850 Jan  9 23:18 ons.log.08-01-09_23:18:21
-rw-rw----   1 oracle   dba      1536000850 Jan  9 13:24 ons.log.08-01-09_13:24:21
-rw-rw----   1 oracle   dba      1536000850 Jan  9 03:30 ons.log.08-01-09_03:30:31
-rw-rw----   1 oracle   dba      1536000850 Jan  8 17:36 ons.log.08-01-08_17:36:25
-rw-rw----   1 oracle   dba      1536000850 Jan  8 07:42 ons.log.08-01-08_07:42:35
-rw-rw----   1 oracle   dba      1536000850 Jan  7 21:48 ons.log.08-01-07_21:48:26
hostname.10GR2->

hostname.10GR2-> tail ons.log
08/01/10 13:45:32 [4] Local connection 0,127.0.0.1,6101 missing form factor
<unknown>
08/01/10 13:45:32 [4] Local connection 0,127.0.0.1,6101 missing form factor
<unknown>
08/01/10 13:45:32 [4] Local connection 0,127.0.0.1,6101 missing form factor
<unknown>
08/01/10 13:45:32 [4] Local connection 0,127.0.0.1,6101 missing form factor
<unknown>
08/01/10 13:45:32 [4] Local connection 0,127.0.0.1,6101 missing form factor
<unknown>
hostname.10GR2->

If you let this go for a while you are running a chance of running out of 
space on the file system (in this case the "logs" directory grew by 200gb):

hostname.10GR2-> pwd
/u01/app/oracle/product/oms10g/opmn
hostname.10GR2->
hostname.10GR2->
hostname.10GR2-> ls -l
total 20
drwx------   2 oracle   dba          512 Nov 13 15:19 bin
drwx------   3 oracle   dba          512 Nov 13 16:17 conf
drwx------   2 oracle   dba         1024 Nov 13 15:17 lib
drwx------   3 oracle   dba         5632 Jan  9 13:24 logs
drwx------   2 oracle   dba          512 Nov 13 15:17 mesg
hostname.10GR2-> du -ks *
3315    bin
179     conf
667     lib
208113578       logs
26      mesg
hostname.10GR2->

Solution
Disable Oracle TNS Listener from subscribing to the Oracle Notification 
Server (ONS) by putting the following line in the listener.ora file and 
restarting the listener process:

   SUBSCRIBE_FOR_NODE_DOWN_EVENT_<listener_name>=OFF
Example
hostname.10GR2-> pwd
/u01/app/oracle/product/10.2.0/db_1/network/admin

hostname.10GR2-> cat listener.ora
SID_LIST_LISTENER =
  (SID_LIST =
    (SID_DESC =
      (GLOBAL_DBNAME = 10GR2.DOMAIN.COM)
      (ORACLE_HOME = /u01/app/oracle/product/10.2.0/db_1)
      (SID_NAME = 10GR2)
    )
  )

LISTENER =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = hostname.domain.com)(PORT = 1521))
  )
SUBSCRIBE_FOR_NODE_DOWN_EVENT_LISTENER=OFF
[edited by: Vitaliy at 16:06 (CST) on Jan. 10, 2008]