Forgot your password?    
+ Reply to Thread
Results 1 to 2 of 2

Thread: Troubleshooting concurrent manager startup problems

  1. #1
    Expert Oracle Administrator
    Join Date
    Oct 2011
    Delhi, India, India

    Troubleshooting concurrent manager startup problems



    1. Check the log files for any reported errors. The default name for the Internal Manager log file is 'CM_<hostname>.log' on NT or >ManagerName>.mgr on Unix. Also check any other manager log files in the $APPLCSF/$LOG directory for relevant messages. Do these errors point to any obvious places to check (e.g. database not running, TNS alias incorrect)? If these files do not exist at all, there may be permission problems that prevent processes from writing

    files to the proper directories.

    2) Ensure that you are logged in as the 'applmgr' (or equivalent) user on Unix,
    or if on NT that the service is being started by the 'applmgr' (or equivalent)

    3) If you are getting errors such as 'establish ICM unable to insert ICM record

    APP-1167 APP-1244 Oracle error 1631 in insert_icm_record' Check your database's tablespace for the FND tables to ensure you enough free space to create the next extent. Also check the FND tables to ensure no table has hit the maximum number of extents.

    4) Log in to SQLPLUS as database user "apps".

    Verify that view fnd_v$process exists: SELECT view_name FROM all_views WHERE view_name LIKE '%V$%';

    If the view is missing or invalid, rebuild it. From the operating system, as the applmgr user type:

    FNDLIBR FND FNDCPBWV applsys/fnd SYSADMIN 'System Administrator' SYSADMIN or as the applsys user run the following SQL statement

    CREATE VIEW fnd_v$process AS
    ( SELECT P.Addr, P.Pid, S.Process Spid,
    S.Osuser Username, P.Serial#,
    S.Terminal, S.Program, P.Background,
    P.Latchwait, S.Lockwait
    FROM V$process P, V$session S
    WHERE P.Addr = S.Paddr );

    5) Close down all Concurrent Manager processes and clear database tables/log files. This process will remove all history of Concurrent Jobs and remove all scheduled jobs. Please contact Oracle Support for advice when doing this to a Production installation.

    1. Check there are no FNDLIBR, FNDCRM or other Concurrent Manager processes running. This can be done via the Concurrent Manager-->Administer screen, or from the operating system - Note 68993.1 has a
    list of the process names to look for.
    2. In SQLPLUS run the following SQL TRUNCATE TABLE fnd_concurrent_processes;

    UPDATE fnd_concurrent_requests
    set phase_code = 'C', status_code = 'X'
    where status_code = 'T';

    UPDATE fnd_concurrent_queues SET running_processes = 0;

    SELECT concurrent_queue_name, control_code, running_processes,
    FROM fnd_concurrent_queues;

    The control_codes should be 'E' for deactivated managers, blank for all others except FNDICM which should be 'X'. You will need to update the table manually if you find any exceptions to these values.
    For example:-

    UPDATE fnd_concurrent_queues SET control_code = 'X'
    WHERE concurrent_queue_name = 'FNDICM';

    c) Delete all log and output files in FND_TOP/log, APPLCSF/log and APPLCSF/out if these files are no longer needed.

    6) On Unix check :-

    If any of these settings are incorrect, reset them to the correct value.

    7) Check the program versions

    On NT check versions of FND_TOP/bin/FNDLIBR.EXE and AU_TOP/bin/FNDCORE.DLL.

    This is achieved by highlighting the program file, then using the right mouse
    button to click-->Properties-->Version. Compare this information to that found
    in bug 881201 - this patch delivers v11.1.179 of FNDCORE.DLL (pre-req of this
    patch is to be on 11.0.3) Alternatively you may need to apply patch 872159

    which is FNDCORE.dll v11.0.215

    On Unix you can run the following command to determine the version numbers:

    'strings -a $FND_TOP\bin\FNDLIBR | grep Header > fndlibr_versions.txt'

    This will create a file called 'fndlibr_versions.txt' to send to Oracle Support.

    8) Set up SQLNET trace and Database SQL tracing.

    1. set TRACE_LEVEL_SERVER=16 in the SQLNET.ORA file on the database server.
    2. Set SQL_TRACE=true in the init.ora file

    NOTE - the database will need to be restarted for this change to take effect.It will also generate trace files for every database session, which induces a great deal of processing overhead as well as a lot of disk
    writing. Once the database has been restarted, try starting the Concurrent Manager, then stop the database and reset 'SQL_TRACE=false' in the init.ora file and restart the database again.

    9) Check the RDBMS version and ensure it is certified; certified combinations change periodically.

    1. Check dual tables: (assuming no-one is in the system and Concurrent Manager processes are not running)
    2. select count(*) from sys.dual; There must be one and only one row returned.
    3. select count(*) from apps.fnd_dual; There must be at least one row.
    4. Check there are no invalid objects in the database. Connect as database
    SELECT owner, object_name, object_type
    FROM dba_objects
    WHERE status != 'VALID';

    If this returns any rows, we may need to identify why the object is invalid and get it to compile before proceeding.

    1. Check/change Applications profile options. Check system level profile option 'Concurrent:Use ICM'; by default this should be set to 'No'. Try changing this value to 'Yes' and see if this changes the nature of the problem.
    2. Check process monitor (PMON) method. Connect as database user "APPS". SELECT profile_option_value FROM fnd_profile_option_values WHERE level_id = 10001 AND level_value = 0 AND application_id = 0 AND profile_option_id = (SELECT profile_option_id FROM fnd_profile_options WHERE profile_option_name = 'CONC_PMON_METHOD');

    This should return one row with a value of 'RDBMS' or 'LOCK'

    If the value is 'RDBMS' run the script FND_TOP/sql/AFIMPMON.SQL - this will set
    the PMON method to LOCK instead of RDBMS.

    1. Stop database and restart database server. If not already shutdown in previous step, stop and restart the database.
    1. If the above still fails, if running on NT delete the concurrent manager and recreate from scratch. This is described in the Applications on NT Installation manual.

    Once the above has been tried, please advise/send in the following information to Oracle Support:-

    1. On NT, versions of FND_TOP/bin/FNDLIBR.EXE and AU_TOP/bin/FNDCORE.DLL. On Unix run 'strings $FND_TOP/bin/fndlibr| grep Header > fndlibr_version.txt'.
    2. SQLNET trace files (svr_*.trc)
    3. Database trace files.
    4. Confirmation there are no invalid packages for the APPS user.
    5. Manager log files.
    6. Advise of how many and which Concurrent Manager processes are running.

  2. #2
    Oracle Administrator
    Join Date
    Apr 2012
    If any particular request is going to pending standby status and others are getting completed, it means that either it is waiting for the output of some other request or is conflicting with some other request. If the request is conflicting, check the queue of the conflict resolution manager for troubleshooting.


+ Reply to Thread

Similar Threads

  1. Concurrent Request troubleshooting HELP NEEDED
    By saioracleappsdba in forum Concurrent Processing Server / Workflow
    Replies: 1
    Last Post: 06-15-2012, 04:51 AM
  2. Inside the Oracle Concurrent Manager
    By mdmasood in forum Concurrent Processing Server / Workflow
    Replies: 0
    Last Post: 02-04-2012, 04:44 PM
  3. Profile options for concurrent manager(cm)
    By mdmasood in forum Concurrent Processing Server / Workflow
    Replies: 0
    Last Post: 02-03-2012, 04:37 PM
  4. Retrieve OS Process Id for a Concurrent Manager
    By Hemant in forum SCRIPTS : Concurrent Manager and Workflow
    Replies: 0
    Last Post: 10-31-2011, 08:54 PM
  5. Manager that ran a Concurrent Request
    By Hemant in forum SCRIPTS : Concurrent Requests & Concurrent Request Sessions
    Replies: 0
    Last Post: 10-31-2011, 08:27 PM

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

DBA Lounge (P) Ltd. deals in Oracle Technologies on Consulting, Resourcing, Corporate Training

Online and corporate training available on Oracle Database 11g, Oracle 11g Real Application Cluster (RAC), Oracle Applications 11i/R12, Oracle Fusion Middleware 11g, Oracle Identity Management-OIM, Oracle Internet Directory 11g-OID, Oracle Business Intelligence Enterprise Edition-OBIEE, Oracle Golden Gate, Oracle Access Management-OAM, Oracle Internet Directory-ODS, Oracle Identity Analytics Architecture-OIA Statistics