The Case.. The Mess up and The Quick Fix or Re Creating APPS Synonyms without using ADADMIN
The Case.. The Mess up and The Quick Fix or Re Creating APPS Synonyms without using ADADMIN
My current post(or a story rather) deals about a real life mess up and the quck solution we did to fix it with the least possible down time.
The content in this post is scritly for informational purposes only. DO NOT try this method even on your TEST or Development environment.
The Case
We all know that the APPS schema in Oracle Applications does not really hold many objects but holds synonyms to all the application objects in various schemas.
Also it is a common practice for the development environment to have a schema similar to the apps schema in Oracle Applications with only read access.
Some of these synonyms had gone invalid after a cloning process and were giving a synonym translation invalid error. It was decided to drop and recreate all the synonyms for the READ ONLY schema.
A dynamic script was crated for dropping all the synonyms form this schema.
spool /tmp/dropsynonym.sql
select 'drop synonym '||SYNONYM_NAME ||';' from user_synonyms;
spool off
This was supposed to be executed in the READ ONLY schema.
After this the synonyms were to be recreated again using another script
spool /tmp/synonym.sql
select distinct 'create synonym "'||object_name||'" for '||owner||'."'||object_n
ame||'";' from dba_objects where owner = 'APPS' ;
spool off
The above process worked fine always and seemed like a fairly tale.. until one fine day..
The Mess up
It all looked OK that day until a poor soul did this.
The drop synonym script was executed from the APPS schema instead of the APPS read only schema. The result.. All the synonyms from the APPS user got knocked off. Everything in Oracle Applications came crashing down.
(This is the part where my pager starts beeping..)
After we realized what had happened the first thought to strike my mind was to run adadmin and re create the grants and synonyms. But as expected adadmin also refused to work in the absence of the apps synonyms. Classic!!
The Quick Fix
We did have multiple environments of oracle applications at this particular client so we just made up a script to create all the synonyms again from a working similar instance. We this script in the APPS schema of the working environment.
spool fixsynonym.sql
select 'create synonym "'||synonym_name||'" for '||table_owner||'."'||table_name||'";' from user_synonyms;
spool off
Next we ran the fixsynonym.sql from the apps schema of the environment which had all its synonyms deleted.
Once done we were able to get back almost all of our synonyms and atleast were able to get adadmin working back again.
Next we ran adadmin and selected to recreate the grants and sysnonyms for the APPS Schema.
We did a clean shutdown of the instance and then brought it up again.. and it was back to perfect.
My current post(or a story rather) deals about a real life mess up and the quck solution we did to fix it with the least possible down time.
The content in this post is scritly for informational purposes only. DO NOT try this method even on your TEST or Development environment.
The Case
We all know that the APPS schema in Oracle Applications does not really hold many objects but holds synonyms to all the application objects in various schemas.
Also it is a common practice for the development environment to have a schema similar to the apps schema in Oracle Applications with only read access.
Some of these synonyms had gone invalid after a cloning process and were giving a synonym translation invalid error. It was decided to drop and recreate all the synonyms for the READ ONLY schema.
A dynamic script was crated for dropping all the synonyms form this schema.
spool /tmp/dropsynonym.sql
select 'drop synonym '||SYNONYM_NAME ||';' from user_synonyms;
spool off
This was supposed to be executed in the READ ONLY schema.
After this the synonyms were to be recreated again using another script
spool /tmp/synonym.sql
select distinct 'create synonym "'||object_name||'" for '||owner||'."'||object_n
ame||'";' from dba_objects where owner = 'APPS' ;
spool off
The above process worked fine always and seemed like a fairly tale.. until one fine day..
The Mess up
It all looked OK that day until a poor soul did this.
The drop synonym script was executed from the APPS schema instead of the APPS read only schema. The result.. All the synonyms from the APPS user got knocked off. Everything in Oracle Applications came crashing down.
(This is the part where my pager starts beeping..)
After we realized what had happened the first thought to strike my mind was to run adadmin and re create the grants and synonyms. But as expected adadmin also refused to work in the absence of the apps synonyms. Classic!!
The Quick Fix
We did have multiple environments of oracle applications at this particular client so we just made up a script to create all the synonyms again from a working similar instance. We this script in the APPS schema of the working environment.
spool fixsynonym.sql
select 'create synonym "'||synonym_name||'" for '||table_owner||'."'||table_name||'";' from user_synonyms;
spool off
Next we ran the fixsynonym.sql from the apps schema of the environment which had all its synonyms deleted.
Once done we were able to get back almost all of our synonyms and atleast were able to get adadmin working back again.
Next we ran adadmin and selected to recreate the grants and sysnonyms for the APPS Schema.
We did a clean shutdown of the instance and then brought it up again.. and it was back to perfect.
Comments
Great post and nice hints, I like the way you wrote this post.
Fadi
Posted by: Fadi Hasweh | May 27, 2007 02:28 PM
Good Post.. interesting :)
Serge
Posted by: Serge | May 29, 2007 07:09 PM
Thanks Fadi and Serge
Posted by: Sam | May 30, 2007 03:14 AM
Sam,
I've been following your posts for sometime and i think you are doing a wonderful job. Keep it up!
Cheers,
Ramesh
Posted by: Ramesh | May 30, 2007 07:22 AM
Oh - the old, I am sure I was connected to development issue.
Scripts like this are dangerous and I suggest DBA's really think about the implications of having a 'blind' script executed in any environment if it is done routinely . There are other safer ways to achieve the same result that would not break production if they were executed in the wrong environment.
I would suggest using exp/imp to create your read only environment schema. Or even better build the R/O environment in production once and lock the account. This also helps if a developer needs to get into prod for an emergency to help with a critical problem.
Anyway posts like this make us better DBA's, lets learn by others mistakes. :-|
Brett
Posted by: Brett | May 30, 2007 07:57 AM
yes i do agree with your view on that, Brett.Personally i am against these read only schema in production.
Sam
Posted by: Sam | May 30, 2007 09:16 PM
Thanks Ramesh.
Posted by: Sam | May 30, 2007 09:17 PM
Nice job done :)
Posted by: Arunkumar | June 2, 2007 01:36 AM
Thanks Arun.
Posted by: Sam | June 9, 2007 01:51 AM
It's really a good site.
Thanks,
Kevin
Posted by: Kevin | June 12, 2007 02:16 AM
Thanks Kevin.
Posted by: Sam | June 12, 2007 07:22 PM
This web site is great and very much useful for apps dba.
It will be great if you put some notes on Discover serve. I mean
1 concept,
2 Installation and configuration of 10g discoverer with 11i and R12
3. about EUL and report generation.
I think expectation is too much :)
Thanks,
Himadri.
Posted by: himadri DAS | June 28, 2007 01:46 AM
Hi Sam,
Thanks for your valuable tips/technics among your busy schedule. I have a question:
Sometimes I realize increasing the number of workers will expedite the Big Oracle patch to finish soon.
Is there any possibility to increase the number of workers in ADPATCH when the patching is in progress?
Thanks
Siva
Posted by: Sivakumar Vaithiyam Mohan | August 9, 2007 07:33 AM