Showing posts with label oracle. Show all posts
Showing posts with label oracle. Show all posts

2019-02-22

Showing Total Memory Used Per Oracle DB Instance


The following command gets the output of "ps aux"...
... takes only the lines regarding Oracle DB ("ora_") ...
... and sums up the RSS (resident set size, the non-swapped physical memory that a task has used (in kiloBytes)) ...
... grouped by DB SID


ps aux --sort -rss | awk '/USER +PID | ora_/{ split($11,a,"_"); db=a[3]; if(db!="" && db!="RSS"){ m[db]+=$6; } } END{for (db in m){ printf("DB:%9s uses %7.1f MB RAM\n",db,m[db]/1024); tot+=m[db]} printf("%12sTotal %7.1f GB RAM\n"," ",tot/(1024*1024))}'

Output Example
(DB "Dfug" is a temporary DB that has been automatically created by RMAN in order to make a TSPITR)
DB:     Dfug uses   884.9 MB RAM
DB: C0201Z01 uses  4196.6 MB RAM
            Total     5.0 GB RAM

2019-01-16

Oracle Data Pump Errors: ORA-31626 and ORA-06512 and ORA-31637 and ORA-31632 and ORA-01422


Problem
While implementing a Data Pump Export script, I started getting the following error stack:
Connected to: Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production
ORA-31626: job does not exist
ORA-06512: at "SYS.DBMS_SYS_ERROR", line 79
ORA-06512: at "SYS.KUPV$FT", line 1183
ORA-31637: cannot create job CSEXPDP_01 for user SYS
ORA-06512: at "SYS.DBMS_SYS_ERROR", line 95
ORA-06512: at "SYS.KUPV$FT", line 1176
ORA-31632: master table "SYS.CSEXPDP_01" not found, invalid, or inaccessible
ORA-06512: at "SYS.DBMS_SYS_ERROR", line 95
ORA-06512: at "SYS.KUPV$FT", line 1167
ORA-01422: exact fetch returns more than requested number of rows
ORA-06512: at "SYS.KUPV$FT_INT", line 2566
ORA-06512: at "SYS.KUPV$FT_INT", line 2556
ORA-06512: at "SYS.KUPV$FT", line 1092

Cause & Solution
In my case the error was caused, because my script used the same name for the Database Directory and the Data Pump Job.
For example:
It created a DB-Directory like this . . .
CREATE OR REPLACE DIRECTORY tmp_auto_dp_01 AS '...';

. . . and specified a DataPump job name like this:
JOB_NAME=tmp_auto_dp_01
DIRECTORY=tmp_auto_dp_01
SCHEMAS=...
DUMPFILE=...
LOGFILE=...

I hope it helps someone

2018-11-20

ORA-65011 Pluggable database ... does not exist


PROBLEM: ORA-65011 Pluggable database ... does not exist

If trying to start the DB service ...
OS> srvctl start service -s MY_DB_SERVICE -d MY_CDB_UNIQUE_NAME -pdb MY_PDB_SID

... you get the following error ORA-65011 ...
PRCD-1084 : Failed to start service MY_DB_SERVICE
PRCR-1079 : Failed to start resource ora. MY_CDB_UNIQUE_NAME.MY_DB_SERVICE.svc
ORA-65011: Pluggable database MY_CDB_UNIQUE_NAME does not exist.
CRS-5017: The resource action "ora. MY_CDB_UNIQUE_NAME.MY_DB_SERVICE.svc start" encountered the following error:
ORA-65011: Pluggable database MY_PDB_SD does not exist.
. For details refer to "(:CLSN00107:)" in "/u00/app/oracle/diag/crs/my_host/crs/trace/ohasd_oraagent_oracle.trc".
CRS-2674: Start of 'ora.CDB_UNIQUE_NAME.MY_DB_SERVICE.svc' on 'my_host' failed

(Since I has just created the DB using RMAN dupplicate and skipping some tablespaces, I though that my PDB (pluggable DB) had problems because of missing tablespaces.)

... but, as the error says, there is no PDB with the given name:
OS> oerr ORA 65011
65011, 00000, "Pluggable database %s does not exist."
// *Cause:  User attempted to specify a pluggable database
//          that does not exist.
// *Action: Check DBA_PDBS to see if it exists.


SOLUTION
Check how the service has been configured ...
I remarked that the name of my PDB (MY_PDB_SID)
was misspelled (MY_PDB_SD <= missing "I")
OS> srvctl config service -s MY_DB_SERVICE -db MY_CDB_UNIQUE_NAME
Service name: MY_DB_SERVICE
Cardinality: SINGLETON
Service role: PRIMARY
. . .
Runtime Load Balancing Goal: NONE
TAF policy specification: NONE
Edition:
Pluggable database name: MY_PDB_SD
Maximum lag time: ANY
SQL Translation Profile:
. . .
GSM Flags: 0
Service is enabled

... and re-create the DB service, using the correct PDB name
(So I re-created my MY_DB_SERVICE)
OS> srvctl remove service -s MY_DB_SERVICE -db MY_CDB_UNIQUE_NAME
OS> srvctl add service -service MY_DB_SERVICE -db MY_CDB_UNIQUE_NAME -pdb MY_PDB_SID -l primary/u00/app/oracle

Then it worked
OS> srvctl start service -s MY_DB_SERVICE -db MY_CDB_UNIQUE_NAME
OS> srvctl status service -s MY_DB_SERVICE -db MY_CDB_UNIQUE_NAME
Service MY_DB_SERVICE is running


2018-11-16

Oracle Database: How To "Shrink" the UNDO Tablespace



-- Create "temporary" undo tablespace and make it the default undo tablespace
create undo tablespace UNDO_TMP datafile '+U01' size 2G AUTOEXTEND ON NEXT 10M MAXSIZE UNLIMITED;
alter system set undo_tablespace=UNDO_TMP;

-- Drop the original undo tablespace
drop tablespace UNDOTS including contents and datafiles;

-- Recreate the old undo tablespace (smaler) and make it the default undo tablespace again
create undo tablespace UNDOTS datafile '+U01' size 2G AUTOEXTEND ON NEXT 10M MAXSIZE UNLIMITED;
alter system set undo_tablespace=UNDOTS;

-- Drop the "temporary" undo tablespace
drop tablespace UNDO_TMP including contents and datafiles;

2018-11-13

Oracle Enterprise Manager / Cloud Control: Historical Data Not Showing Issue

Problem Description

Last week after one of Oracle Database hosts had to be rebooted because of memory problem. I had the issue with Oracle Enterprise Manager / Cloud Control that historical data of this DB (only for this DB) was not showing any more:

  • In "Performance" => "Performance Home" => "View Data: Historical"
    Error Message
    Data Not Available. Statspack data is not available for this database instance. Make sure that Statspack is installed on the target instance.
  • In "Performance" => "Top Activity"  => "View Data: Historical"
    Error Message
    (No Data Available)
  • In: "Performance"  => "ASH Analytics" in "Top Activity"
    Error
    No useful data was showing

Possible Solution

Completely logout of Enterprise Manager (not only from the DB) and logon back.

2018-08-15

ORA-16548: database not enabled


PROBLEM

The standby DB is/stays disabled (cannot be enabled)

DGMGRL> show configuration;

Configuration - MYDB

  Protection Mode: MaxAvailability
  Databases:
    MYDB_PRIMARY - Primary database
    MYDB_STANDBY - Physical standby database (disabled)

Fast-Start Failover: DISABLED
OR
DGMGRL> edit database MYDB_STANDBY . . .
Error: ORA-16548: database not enabled

and the . . .
DGMGRL> enable database MYDB_STANDBY;
Enabled.
. . . does not work. Despite showing "Enabled." the DB is not really enabled.

The DataGuard Log file drc<SID>.log(cd /u00/app/oracle/diag/rdbms/mydb_primary/mydb/trace
or - withTrivadis Baseenv:  cdd; cd trace
shows:
. . .
ENABLE DATABASE MYDB_STANDBY
Warning, database MYDB_STANDBY that was marked for re-creation
      will be re-enabled. There may be errors or warnings
      if the database was not properly re-created. See this
      log and the alert log for more details.
Metadata Resync failed. Status = ORA-16603
. . .


POSSIBLE CAUSE

I caused the problem myself, because I've tried to solve a Data Guard problem by re-creating the configuration using a configuration name in upper case letters instead of re-using the same name as before (with lower case letters).

POSSIBLE SOLUTION

On the STANDBY DB
DGMGRL> connect sys/<Pwd>@MYDB_PRIMARY
Connected.
DGMGRL> remove configuration;
Error: ORA-16627: operation disallowed since no standby databases would remain to support protection mode

Failed.
DGMGRL> disable configuration;
Disabled.
DGMGRL> remove configuration;
Removed configuration

On the PRIMARY DB
DGMGRL> connect sys/<Pwd>@MYDB_PRIMARY

DGMGRL> enable database MYDB_STANDBY;
Enabled.


2018-06-06

How To DB Restore an Oracle DB on Another Host


Precondition on the host where the DB will be restored to (called test host / TEST_DB):
  • this description is valid for Linux
  • an own installed test DB, so that the binaries and directory structure are ready
  • enough storage like the original DB to be restored

The original DB to be restored on a different host (test host) will be called:
SID           : ORIG_DB
DB_UNIQUE_NAME: ORIG_DB_SITE1

All steps are performed on the test host.
If running stop your test DB.

Go to the admin directory
cd /u00/app/oracle/admin

Perhaps delete not log files that are not needed in order to save place, if your /u00 is not big enough
cd TEST_DB
find . \( -name "*.log" -o -name "*.aud" \) -exec rm {} \;

Create Oracle admin/SID directory as copy of your TEST_DB with the name of the DB 
cd ..
cp -R TEST_DB ORIG_DB

Insert the following line in file /etc/oratab 
ORIG_DB:/u00/app/oracle/product/12.2.0.1:N

Re-Logon as OS oracle user.
For example:
exit
sudo -u oracle sudosh

Set your Linux environment.
For example (using Baseenv from Triivadis):  
ORIG_DB

Two variants how to prepare the DB init file:
A) copy the init file from the ORIG_DB to your test host
B) edit the init file of your TEST_DB (if you do not have access to the original DB init file any more)

For both variants, here is the target directory where the DB init files are located:
cd /u00/app/oracle/admin/${ORACLE_SID}/pfile
A) the copy variant should be clear

B) edit variant
Rename the init file of your TEST_DB in order to match your ORIG_DB
mv initTEST_DB.ora initORIG_DB.ora
vi initORIG_DB.ora

In the initORIG_DB.ora, replace all TEST_DB by ORIG_DB substrings.
The following parameters are concerned:
    • control_files
    • db_name
    • db_unique_name
    • audit_file_dest

Perhaps delete other non used init*.ora, spfile*.ora and pfile.ora from this directory in order to avoid confusion.
ATTENTION: do not delete your prepared initORIG_DB.ora file

Create a symbolic link to your new initORIG_DB.ora file
cd $ORACLE_HOME/dbs
ln -s /u00/app/oracle/admin/${ORACLE_SID}/pfile/init${ORACLE_SID}.ora init${ORACLE_SID}.ora
  
Result. For Example:
oracle@test_host [ORIG_DB] ll
. . .
lrwxrwxrwx  1 oracle dba        49 Jun  5 11:22 initORIG_DB.ora -> /u00/app/oracle/admin/ORIG_DB/pfile/initORIG_DB.ora
. . .

Now you are ready to perform your restore.

Here is an example of an RMAN command files.
You will need to adapt:
  • how you connect to your RMAN-Catalog DB
  • the number of RMAN channels
  • the RMAN channels type and parameters (PARMS) (look at the RMAN log files of the ORIG_DB)
  • adapt the DBID
  • adapt the SCN for your needs / or use a time instead of SCN

You should be able to find the DBID in the logs of the RMAN backups of the ORIG_DB.
For example:
. . .
RMAN>
connected to target database: ORIG_DB (DBID=767753736)
. . .

You can also query the real ORIG_DB (not the new temporary DB on the test host) for the DBID (if available).
For example:
SQL> SELECT dbid FROM v$database;

      DBID
----------
 767753736

Startup the DB in nomount mode
SQL> startup nomount

Copy the controlfile of the ORIG_DB to the test host ***  . . .
(Assuming you have this controlfile in ASM file sytem or backup of the file system)
. . . and "restore" this controlfile:
rman TARGET/ NOCATALOG
SET DBID=767753736
RESTORE CONTROLFILE FROM "<Path to *** from above>";

ALTER DATABASE MOUNT;

Or, if you need to use the RMAN catalog (which will cause RMAN problems with the "original DB"):
RMAN command file that restore the DB control and start the DB in mount mode.
connect target /
connect catalog /@<Your RMANCATALOG>

SET DBID 767753736;

run {
  ALLOCATE CHANNEL CH1 TYPE 'SBT_TAPE' PARMS="SBT_LIBRARY=/opt/hds/Base/libobk.so";
  SET UNTIL SCN=1815512098911;
  RESTORE CONTROLFILE;
  RELEASE CHANNEL CH1;
}
ALTER DATABASE MOUNT;

Or, if you need to use the RMAN catalog (which will cause RMAN problems with the ORIG_DB)

connect target /
connect catalog /@<Your RMAN catalog alias>

SET DBID 767753736;

run {
  ALLOCATE CHANNEL CH1 TYPE 'SBT_TAPE' PARMS="SBT_LIBRARY=/opt/hds/Base/libobk.so";
  SET UNTIL SCN=1815512098911;
  RESTORE CONTROLFILE;
  RELEASE CHANNEL CH1;
}
ALTER DATABASE MOUNT;

ATENTION: After having used the RMAN catalog from another DB with the same DBID (here the TEST_DB), you will bee to go to the ORIG_DB and perform an RMAN crosscheck:

OS> rman target / catalog /@<Your RMAN catalog alias>

RMAN> crosscheck archivelog all;


Now the DB can be restored. (Again adapt the yellow entries)
connect target /
connect nocatalog

#SET DBID 767753736;

RUN
{
  ALLOCATE CHANNEL CH1 TYPE 'SBT_TAPE' PARMS='SBT_LIBRARY=/opt/hds/Base/libobk.so';
  ALLOCATE CHANNEL CH2 TYPE 'SBT_TAPE' PARMS='SBT_LIBRARY=/opt/hds/Base/libobk.so';
  ALLOCATE CHANNEL CH3 TYPE 'SBT_TAPE' PARMS='SBT_LIBRARY=/opt/hds/Base/libobk.so';
  ALLOCATE CHANNEL CH4 TYPE 'SBT_TAPE' PARMS='SBT_LIBRARY=/opt/hds/Base/libobk.so';
  SET UNTIL SCN=1815512098911;
  RESTORE DATABASE;
  RECOVER DATABASE;
  #ALTER DATABASE OPEN RESETLOGS;
}






2018-05-02

Flashback Database After Failover: Correcting "Error ORA-38754: FLASHBACK DATABASE not started"


If the alert log file of the former primary DB contains the following error messages:
. . .
FLASHBACK DATABASE TO SCN 6693334
ORA-38754 signalled during: FLASHBACK DATABASE TO SCN 6693334...
. . .

Try to perform a flashback manually and you might get the following error:
SQL> FLASHBACK DATABASE TO SCN 6693334;
FLASHBACK DATABASE TO SCN 6693334
*
ERROR at line 1:
ORA-38754: FLASHBACK DATABASE not started; required redo log is not available
ORA-38762: redo logs needed for SCN 6691520 to SCN 6693334
ORA-38761: redo log sequence 124 in thread 1, incarnation 13 could not be accessed

In this case . . .

. . . on the new standby DB (former primary DB - before the failover),
use rman by connecting to the local DB and using the control file (NOCATALOG):
rman target=/ NOCATALOG

or (if baseenv from Triivadis is available - with history)

rmanh target=/ NOCATALOG

Since this command using the SCN did not return any archivelogs . . .
RMAN> LIST ARCHIVELOG SCN BETWEEN 6691520 AND 6693334;

. . . I searched for the backup of the archivelog with sequence 124 by guessing the time range . . .
LIST BACKUP OF ARCHIVELOG
      FROM TIME "TO_DATE('2018.05.01 20:00:00','YYYY.MM.DD HH24:MI:SS')"
      UNTIL TIME "TO_DATE('2018.05.02 12:00:00','YYYY.MM.DD HH24:MI:SS')";

. . . and was lucky and got the SCN of the desired archivelog (SCN 6691790) (but for restoring I choose the Low SCN of the previous archivelog i.e. 6689656 )
BS Key  Size       Device Type Elapsed Time Completion Time
------- ---------- ----------- ------------ -------------------
343     60.00M     SBT_TAPE    00:00:52     2018-05-02 10:57:31
        BP Key: 343   Status: AVAILABLE  Compressed: NO  Tag: TAG20180502T105639
        Handle: 6446830_C0200Z01_c9t1sib7_1_1   Media: V_3683633_19130778

  List of Archived Logs in backup set 343
  Thrd Seq     Low SCN    Low Time            Next SCN   Next Time
  ---- ------- ---------- ------------------- ---------- ---------
. . .
  1    122     6687465    2018-05-02 10:27:49 6689656    2018-05-02 10:42:47
  1    123     6689656    2018-05-02 10:42:47 6691790    2018-05-02 10:56:38
  1    124     6691790    2018-05-02 10:56:38 6691798    2018-05-02 10:56:38


HINT
It can happen, that the needed archivelog, belongs to another incarnation of the DB.
(If this is the case - on Oracle 12c - you can use the incarnation number with the rman restore statement.)
This statement list your incarnations.
RMAN> list incarnation;

List of Database Incarnations
DB Key  Inc Key DB Name  DB ID            STATUS  Reset SCN  Reset Time
------- ------- -------- ---------------- --- ---------- ----------
1       1       DB200Z01 2998214488       PARENT  1          2018-04-20 08:00:24
2       2       DB200Z01 2998214488       PARENT  2449245    2018-04-23 07:56:18
. . .
11      11      DB200Z01 2998214488       PARENT  5928815    2018-05-01 09:13:12
12      12      DB200Z01 2998214488       PARENT  6070931    2018-05-01 13:49:26
13      13      DB200Z01 2998214488       CURRENT 6179737    2018-05-01 14:47:30
==> Incarnation 13 is current (I do not need to use it with the restore statement)

Performing the restore of the archivelog (adapt the PARMS= for your needs):
RUN {
  ALLOCATE CHANNEL ch1 TYPE 'sbt_tape' PARMS="SBT_LIBRARY=/opt/hds/Base/libobk.so";
  # ALLOCATE CHANNEL ch1 TYPE disk;
  #
  RESTORE ARCHIVELOG FROM SCN 6689656;  # INCARNATION 13;
}

Output
allocated channel: ch1
channel ch1: SID=778 device type=SBT_TAPE
channel ch1: CommVault Systems for Oracle: Version 11.0.0(BUILD80)

Starting restore at 2018-05-02 13:15:52

archived log for thread 1 with sequence 125 is already on disk as file +U02/DB200Z01_HSB1/ARCHIVELOG/2018_05_02/thread_1_seq_125.759.975063515
. . .
archived log for thread 1 with sequence 133 is already on disk as file +U02/DB200Z01_HSB1/ARCHIVELOG/2018_05_02/thread_1_seq_133.741.975064009
channel ch1: starting archived log restore to default destination
channel ch1: restoring archived log
archived log thread=1 sequence=123
channel ch1: restoring archived log
archived log thread=1 sequence=124
channel ch1: reading from backup piece 6446830_DB200Z01_c9t1sib7_1_1

Now the standby DB should by able to automatically perform a flashback.

I hope it helps.

[ ]

2018-04-26

RMAN: Restoring ARCHIVELOG For Specific Database Incarnation


This description is only valid for Oracle DBs from version 11.2.x and above.

After an Oracle database has performed a failover, the former primary DB needs some archivelogs in order to become a synchronized standby DB. Sometimes test archive logs have been deleted from the DB host by the RMAN backup.

For cases, that it is not possible to simply restore the missing archivelogs for the actual DB incarnation, RMAN with Oracle 11.2.x and above offers the new syntax "RESTORE ARCHIVELOG ... INCARNATION ...".


Normally you can simply use the number of the previous incarnation. But for some "unclear" cases, this description shows how to find the correct incarnation for a needed specific archivelog.


In case the DB misses an archivelog, the DB alertlog should contain the following message after the restart of the standby DB:
. . .
2018-04-25T15:31:59.503251+02:00
Media Recovery start incarnation depth : 1, target inc# : 8, irscn : 3815571
Media Recovery Waiting for thread 1 sequence 32 branch(resetlogs_id) 974369770
. . .

Find the DB incarnation based in the resetlogs_id:
SQL>
set lines 400

select INCARNATION#, RESETLOGS_CHANGE#, RESETLOGS_TIME, STATUS, RESETLOGS_ID
  from V$DATABASE_INCARNATION
 where RESETLOGS_ID = 974369770;

INCARNATION# RESETLOGS_CHANGE# RESETLOGS_TIME      STATUS                       RESETLOGS_ID
------------ ----------------- ------------------- ---------------------------- ------------
           7           3674168 2018-04-25 10:16:10 PARENT                          974369770

Here the DB incarnations and their correspondent SCN:
RMAN>
list incarnation;

List of Database Incarnations
DB Key  Inc Key DB Name  DB ID            STATUS  Reset SCN  Reset Time
------- ------- -------- ---------------- --- ---------- ----------
1       1       X0300Z01 2998214488       PARENT  1          2018-04-20 08:00:24
2       2       X0300Z01 2998214488       PARENT  2449245    2018-04-23 07:56:18
3       3       X0300Z01 2998214488       PARENT  2450480    2018-04-23 08:03:30
4       4       X0300Z01 2998214488       PARENT  2451567    2018-04-23 08:07:43
5       5       X0300Z01 2998214488       PARENT  2798708    2018-04-24 08:31:27
6       6       X0300Z01 2998214488       PARENT  3360141    2018-04-25 09:05:27
7       7       X0300Z01 2998214488       PARENT  3674168    2018-04-25 10:16:10
8       8       X0300Z01 2998214488       CURRENT 3815572    2018-04-25 14:44:33

Here the confirmation, that the archivelog sequence number 32 shown in the alertlog is linked to the resetlogs_id 974369770 :
RMAN> LIST ARCHIVELOG SCN BETWEEN 3674168 AND 3815572;

List of Archived Log Copies for database with db_unique_name C0200Z01_HSB2
=====================================================================
Key     Thrd Seq     S Low Time
------- ---- ------- - -------------------
1217    1    32      A 2018-04-25 14:34:32
        Name: +U02/X0300Z01_SIT2/ARCHIVELOG/2018_04_26/thread_1_seq_32.783.974446349 (TERMINAL)

Examples how to restore the archive logs for DB incarnation 7
RUN {
  ALLOCATE CHANNEL ch1 TYPE 'sbt_tape' PARMS="XXXXXXXXXXXXXXXX";
  # ALLOCATE CHANNEL ch1 TYPE disk;
  #
  RESTORE ARCHIVELOG SEQUENCE 32 INCARNATION 7;
  # RESTORE ARCHIVELOG FROM SEQUENCE 32 INCARNATION ALL;
  # RESTORE ARCHIVELOG FROM SEQUENCE 32 INCARNATION 7;
  # RESTORE ARCHIVELOG SEQUENCE BETWEEN 32 AND 35 INCARNATION 7;
  # RESTORE ARCHIVELOG FROM SCN 3674168 INCARNATION 7;
  # RESTORE ARCHIVELOG SCN BETWEEN 3674168 AND 3815572 INCARNATION 7;
  #
  # Syntax OK, but does not restore from other DB incarnations (non current DB incarnation)
  # The syntax diagram shows, that it is not possible to restore by TAG from a
  # non current DB incarnation.
  #   RESTORE ARCHIVELOG ALL FROM TAG 'TAG20180425T145639';  
}






2018-01-26

Unblocking the Oracle VM Manager (OVM) Command Line Interface (CLI)

If OVM CLI queries block as follows . . .
OS>
/usr/local/bin/sshpass -p <OvmPassword> ssh -p <OvmPort> admin@<OvmHost> "show server name=<NameOfTheVmToQueyFor>"
Permission denied, please try again.

Logon to the OVM host (<OvmHost> from above) as OS root user and restart the OVM CLI:
# service ovmcli stop
Stopping Oracle VM Manager CLI   [  OK  ]

# service ovmcli start
Starting Oracle VM Manager CLI   [  OK  ]

# service ovmcli status
Oracle VM Manager CLI is running...

Now you can retry the query from top:
OVM> show server name=<NameOfTheVmToQueyFor>
Command: show server name=<NameOfTheVmToQueyFor>
Status: Success
Time: 2018-01-26 12:17:23,011 CET
Data:
  Status = Running
  Role 1 = Utility
  Role 2 = Vm
  Ip Address = 159.*********
  Maintenance Mode = Off
  Inbound Migration Locked = No
  Agent Login = oracle
  Statistic Interval = 40
  NTP Server 1 = b****************
  NTP Server 2 = m******************
  BIOS Vendor = HP
  BIOS Version = P70
. . .


I hope it helps.

2018-01-19

AWK Usage Example: Generating Archivelog Register Statements

Here examples of generating Oracle DB statements in order to register archived log files stored in ASM into the DB.

In order to keep the implementation simple, only one archived log directory (variable P) is processed by each call.

Meaning of the awk Variables below:
  • A <- used in order print the character '
  • p <- holds the archived log directory path specified by the environment variable P
  • $8 <- Holds the strings of the 8th column. That means, contains the names of the archive log files

OS>
P="U02/<DbUniqueName>/ARCHIVELOG/2018_01_19/"
echo "ls -l $P" |
  asmcmd |
  awk -v A="'" -v p="$P" '/^ARCH/{ print "ALTER DATABASE REGISTER LOGFILE "A p $8 A";" }' |
  more
 
Output
. . .
ALTER DATABASE REGISTER LOGFILE 'U02/<DbUniqueName>/ARCHIVELOG/2018_01_19/thread_1_seq_12145.643.965816317';
ALTER DATABASE REGISTER LOGFILE 'U02/<DbUniqueName>/ARCHIVELOG/2018_01_19/thread_1_seq_12146.644.965816317';
ALTER DATABASE REGISTER LOGFILE 'U02/<DbUniqueName>/ARCHIVELOG/2018_01_19/thread_1_seq_12147.266.965815979';
 
Instead of the statement more as last command in the pipe, the output could by written into a file.

Another example of generating statements for archivelogs with sequence numbers >= than a given number (here 12090)  (SeqNo="12090"):
SeqNo="12094"
P="U02/<DbUniqueName>/ARCHIVELOG/2018_01_19/"
echo "ls -l $P" |
  asmcmd |
  head |
  awk -v A="'" -v p="$P" -v n="$SeqNo" '/^ARCH/{
    if(n!="") { 
      match($0, /.+_([0-9]+)\./, sn)
    }
    if(n=="" || sn[1]>=n) {
      print "ALTER DATABASE REGISTER LOGFILE "A p $8 A";" 
    }
  }' |
  more

Output
ALTER DATABASE REGISTER LOGFILE 'U02/<DbUniqueName>/ARCHIVELOG/2018_01_19/thread_1_seq_12094.415.965815957';
ALTER DATABASE REGISTER LOGFILE 'U02/<DbUniqueName>/ARCHIVELOG/2018_01_19/thread_1_seq_12095.432.965815957';
ALTER DATABASE REGISTER LOGFILE 'U02/<DbUniqueName>/ARCHIVELOG/2018_01_19/thread_1_seq_12096.428.965815957';


If an "empty" SeqNo is given, this parameter will be ignored
SeqNo=""
P="U02/<DbUniqueName>/ARCHIVELOG/2018_01_19/"
echo "ls -l $P" |
  asmcmd |
  head |
  awk -v A="'" -v p="$P" -v n="$SeqNo" '/^ARCH/{
    if(n!="") { 
      match($0, /.+_([0-9]+)\./, sn)
    }
    if(n=="" || sn[1]>=n) {
      print "ALTER DATABASE REGISTER LOGFILE "A p $8 A";" 
    }
  }' |
  more

Output: all archivelogs are now listed
ALTER DATABASE REGISTER LOGFILE 'U02/<DbUniqueName>/ARCHIVELOG/2018_01_19/thread_1_seq_1.444.965820029';
ALTER DATABASE REGISTER LOGFILE 'U02/<DbUniqueName>/ARCHIVELOG/2018_01_19/thread_1_seq_10.440.965828087';
ALTER DATABASE REGISTER LOGFILE 'U02/<DbUniqueName>/ARCHIVELOG/2018_01_19/thread_1_seq_12090.404.965815957';
ALTER DATABASE REGISTER LOGFILE 'U02/<DbUniqueName>/ARCHIVELOG/2018_01_19/thread_1_seq_12091.399.965815959';
ALTER DATABASE REGISTER LOGFILE 'U02/<DbUniqueName>/ARCHIVELOG/2018_01_19/thread_1_seq_12092.398.965815959';
ALTER DATABASE REGISTER LOGFILE 'U02/<DbUniqueName>/ARCHIVELOG/2018_01_19/thread_1_seq_12093.397.965815959';
ALTER DATABASE REGISTER LOGFILE 'U02/<DbUniqueName>/ARCHIVELOG/2018_01_19/thread_1_seq_12094.415.965815957';
ALTER DATABASE REGISTER LOGFILE 'U02/<DbUniqueName>/ARCHIVELOG/2018_01_19/thread_1_seq_12095.432.965815957';
ALTER DATABASE REGISTER LOGFILE 'U02/<DbUniqueName>/ARCHIVELOG/2018_01_19/thread_1_seq_12096.428.965815957';


Correcting Oracle DB Errors: ORA-01153, ORA-16136, ORA-16766, ORA-16826, ORA-38706, ORA-38788

After having re-build the standby DB, I've got the following error trying to enable the flashback:
SQL> 
alter database flashback on;

alter database flashback on
*
ERROR at line 1:
ORA-38706: Cannot turn on FLASHBACK DATABASE logging.
ORA-38788: More standby database recovery is needed

In order to solve the problem, I tried to start the recovery . . .
SQL> 
ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT;
ORA-16136: Managed Standby Recovery not active

. . . the error from above, reminded my that I forgot to start the DataGuard Broker on the primary DB. So I did it:
SQL> alter system set dg_broker_start=true;

This was not enough.
On the Data Guard Manager console, I've remarked the following error for the standby DB
. . .
DGMGRL> show configuration verbose;
. . .
        Error: ORA-16766: Redo Apply is stopped

Since I remembered, that I once solved this error by making sure that the physical standby DB really starts on MOUNT mode, I've checked it an restarted the standby DB:
OS>
srvctl modify database -d <MyPhysicalStandbyDbUniqueName> -r PHYSICAL_STANDBY -s mount

srvctl stop database -d <MyPhysicalStandbyDbUniqueName>

srvctl start database -d <MyPhysicalStandbyDbUniqueName>

And checked its configuration
OS>
srvctl config database -d <MyPhysicalStandbyDbUniqueName>

But when re-trying to start the recovery I've got another error:
SQL>
ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT;

ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT
*
ERROR at line 1:
ORA-01153: an incompatible media recovery is active

This was caused by the running recovery process (MRP0) . . .
SQL> 
select PROCESS,CLIENT_PROCESS,THREAD#,SEQUENCE#,BLOCK# from v$managed_standby where process = 'MRP0' or client_process='LGWR';

PROCESS CLIENT_P THREAD# SEQUENCE# BLOCK#
--------- -------- ---------- ---------- ----------
RFS LGWR 1 1551 125
MRP0 N/A 1 1551 125

. . . that I've stopped:
SQL>
alter database recover managed standby database cancel;

Database altered.

Check:
SQL>
select PROCESS,CLIENT_PROCESS,THREAD#,SEQUENCE#,BLOCK# from v$managed_standby where process = 'MRP0' or client_process='LGWR';

PROCESS CLIENT_P THREAD# SEQUENCE# BLOCK#
--------- -------- ---------- ---------- ----------
RFS LGWR 1 1551 125

Not I've re-tryied to enable the flash back again => with success
SQL>
alter database flashback on;

Database altered.

But the Data Guard Manager console shows that the redo apply is still stopped
DGMGRL>
show configuration verbose;
. . .
      Error: ORA-16766: Redo Apply is stopped

So, I re-tried to start it again:
SQL>
alter database recover managed standby database disconnect from session;

Database altered.

But the DataGuard Manager console shows that 
DGMGRL>
show configuration verbose;
. . .
      Warning: ORA-16826: apply service state is inconsistent with the DelayMins property

Ops, I used the wrong option when re-trying to start the recovery (I did not start it on the "MANAGED REAL TIME APPLY" mode).
Verify this with this query on the PRIMARY SITE:
SQL>
SET LINES 200
column DEST_NAME format A20;
column RECOVERY_MODE format A30;
select DEST_ID, DEST_NAME, RECOVERY_MODE FROM V$ARCHIVE_DEST_STATUS;

Output example:
   DEST_ID DEST_NAME            RECOVERY_MODE
---------- -------------------- --------------------
         1 LOG_ARCHIVE_DEST_1   IDLE
         2 LOG_ARCHIVE_DEST_2   MANAGED
         3 LOG_ARCHIVE_DEST_3   IDLE
. . .
It is only MANAGER, not 

After stopping the recovery again and restarting it "with REAL TIME APPLY"
SQL>
ALTER DATABASE RECOVER MANAGED STANDBY DATABASE cancel;

Database altered.


SQL>
ALTER DATABASE RECOVER MANAGED STANDBY DATABASE USING CURRENT LOGFILE disconnect;

Database altered.

The recovery is now on "REAL TIME APPLY" mode:
SQL>
SET LINES 200
column DEST_NAME format A20;
column RECOVERY_MODE format A30;
select DEST_ID, DEST_NAME, RECOVERY_MODE FROM V$ARCHIVE_DEST_STATUS;

   DEST_ID DEST_NAME            RECOVERY_MODE
---------- -------------------- ------------------------------
         1 LOG_ARCHIVE_DEST_1   IDLE
         2 LOG_ARCHIVE_DEST_2   MANAGED REAL TIME APPLY
         3 LOG_ARCHIVE_DEST_3   IDLE
         4 LOG_ARCHIVE_DEST_4   IDLE
. . .

And the DataGuard Manager console shows that all is OK:
DGMGRL> show configuration verbose;

Configuration Status:
SUCCESS