Oracle and Linux

so with the keynote today, Oracle announced that it is supporting Red Hat and backporting bug fixes and patches to previous and older versions of the operating system. The VP of Engineering at Yahoo used a phrase that brought back memories. The term “legacy systems” to me always meant Dell and IBM hardware. Today, that means Solaris Sparc machines. With this announcement I wonder how this will change the industry. Since Red Hat makes the majority of their revenue with service and Oracle is giving away the Linux kernel, where will Red Hat make money?

The argument that Larry brought forth that patches and bug fixes go into the new kernel for new and previous kernel versions. This is an interesting concept. The ability to change your patch system from Red Hat to Oracle is amazing. To make this happen you need to go to and download the up2date rpm. This rpm is configured to go to This is a great way of getting adoption and migration of existing Red Hat customers to switch to Oracle.

This should be very interesting.

Data Guard tips and tricks

If you don’t know about Data Guard, it is something that everyone should know about. A few tips and tricks from the session

  1. network bandwidth is important. There are a bunch of tuning metrics that can be done at the TCP layer level
  2. low latency, low problems. high latency, problems start to creep in.
  3. async redo transport – no async buffer to fill up. LNS can fall behind and will go into archive log if it is too slow.
  4. gap resolution – increase the number of arch processes from 2 to 4. Also use MAX_CONNECTIONS to 2. this reduces the overall time needed to resolve a gap by transferring an archive log using multiple processes.
  5. if the gap between primary and physical gets very large, use incremental backup to speed up gap resolution. Stop redo apply process, determine the current SCN of physical, take incremental backup of primary, catalog and recovery the incremental to physical, restart redo.
  6. automate archive log management. move backups to a physical standby. use RMAN to setup the archive log retention. requires flash recovery area on all databases
  7. this is slightly different for SQL apply. For 10.1, it required a manual purge applied to archive logs. For 10R2 they are automatic. This can be disabled on 10R2 but why?


  • multiple standby configurations is not that difficult and works well. It is recommended to define standby machines on the standby just incase it becomes the primary.This does require more redo being shipped between systems.
  • switchover stops all xactions and the redo log is drained. Once the log is cleared, the switchover will happen. You can make a physical standby in a logical switchover reinitiate itself as a physical standby but this is a manual process.
  • always try to failover to your physical standby. If you must switchover to a logical, use flashback to recovery primary and process redo logs to bring it back in sync.
  • switchover should not take more than a few minutes. you can speed this up by using real time apply.
  • in 10r2, DB_ROLE_CHANGE can fire a trigger when roles change

If I want to run a report on a standby I can suspend redo apply, flash back the database to a point in time, and run the report. This will allow you to write data to the database then flush the changed when you flash it back to a restore point and turn on the redo apply.

Like I said, there is a good reason to understand Data Guard. It is a very powerful tool that allows you to offload your production server and have a live standby backup of your system.

some things never change

Ok, I haven’t exactly kept up with blogging during the conference. I just figured out how to connect with my laptop from the conference hall. I am currently sitting in the keynote address from Sun. I miss Scott McNealy as a speaker. Johnathan Schwarts is interesting but not as good as Scott. It is interesting to note that he has updated his wardrobe and is starting to look like a CEO.

“Our [Sun’s] business model is to drive change and innovation into the data center”. Interesting concept. How do you run a $26B company with this concept? Not sure about that but I guess it is how you run a $13B corporation.

“A massive global buildout is underway”. Every business is growing. On average 4% growth in GDP for countries is expected. Some grow more, some grow less. Faster, better computers are not necessarily a good thing. It takes less computer and less capital to run the same investment. It talkes about $8M for the hardware to run infrastructure for a $13B company. Some companies are growing faster than Moore’s Law because they are responding to consumers that demand interaction. Some examples are EBay, Major League Baseball, ExxonMobil. These companies need very large data centers that require significantly more horsepower. This is where Sun is targeting because the traditional customer will eventually turn to service suppliers to get servers in their offices.

Sun is focusing on four verticals. Software, servers, storage, and services. – need to look at this…..GlassFish is the name for the open source development
building a Java EE 5 application server. It is based on the source code
for Sun Java System Application Server PE 9 donated by Sun Microsystems
and TopLink persistence code donated by Oracle. The question is how does it correlate to Tomcat or the Oracle Apps server

“Solaris crossed the 6M install base. 70% of these are on x64/x86”. This begs the question of why Sparc and why Sun continues to develop these chips. The Linux variant on ubuntu now runs on Sparc so this might add some lifetime to the chipset.

“There are times when our hardware will be the most innovative, there are times that our software will be the most innovative”. This is an interesting concept. It is also a great statement because the hardware group typically lags in delivery of product while someone else leapfrogs the technology. Using this strategy is interesting.

The $1k rebate from PG&E is an interesting sales concept. This successfully focuses the fact that other vendors consume too much power. If a customer has a competitors server running in California, PG&E will give the customer $1k rebate if they replace this with a Sun server. Interesting idea. I wonder how they structured this deal and how we could potentially replicate it with Entergy or Reliant.

The new concept…. 40 foot shipping container full of Sun equipment. Project blackbox. 250 systems/container. 2 Petabytes of tape. Up and running in five minutes.  Interesting packaging. To connect it requires three phase power, network connection, and hot and cold water connections. The racks are water cooled with chillers between the sideways mounted racks and has a center walkway for servicing. This is interesting and my guess is that everyone else will start delivering this as a solution within a few years.

more later….

book review – High Availability with RAC, Flashback and Data Guard

For this blog entry, I will be reviewing a book.
Oracle Database 10g High Availability with RAC, Flashback & Data Guard by Matthew Hart and Scott Jesse

The book is organized well with part 1 being logical availability, part 2 RAC, part 3 disaster planning and part 4 being the distributed database. The book starts out talking about high availability as an unreachable concept that typically requires a large budget. The author compares the computing grid to the utility grid. I sure hope that the grid that they are speaking of isn’t the power grid because it is down on a regular basis at my house. I have been looking at a UPS because ours is down for at least five minutes a month, every month. More during rainstorms.

I usually don’t mind a book that dives straight into the details but putting code on page 4 seems a bit much for me. The code isn’t explained very well but it is used as an example for the chapter. The code looks very simple, create tablespace, create a user, connect, create table and populate. The sample that they give stating that there is no window for a backup except on the weekends is a little hard to believe. The example on accounts receivable comsuming resources is a totally believable problem. The rest of the chapter just details more and more potential failures. Unfortunately, this chapter is not comprehensive and does not recommend solutions or processes to stop the problems. They also don’t detail the impact of the problems but implies that there are significant problems if the failure mode happens.

more later…..

Oracle World – day 1

Ok, it isn’t truly day 1 but it is my first day. Today is Sunday which is the second day of the tutorials. I’ve been to JavaOne but it wasn’t nearly this large. The conference is spread across Moscone West, South, and the Hilton hotel. It looks like I will be doing a substantial amount of walking between the conference center and the Hilton. I’ve already done a bit of walking today to find out what is where and what hands on labs that I can sneak into.

I’ve already gotten kicked out of one session. It was funny because I went in and did 99% of the lab and jumped out to answer a phone call. I got tagged going back into the room but they let me go back in to get my stuff and leave. The hands on tracks are very interesting. Very open format with lectures between labs. I like this idea.

I don’t think that I have seen this many laptops in one place in my life. The open access systems are all laptops. The hands on labs are all laptops. They all have 19 inch screens attached to them. I wonder what these machines will do once this event is finished. My guess is that they are leased and will go to the next big trade show. When I was with Sun we deployed a large server and a series of JavaStations. I wonder what the cost tradeoff now is for laptops and java consoles.

I have been trying to plan my schedule. It is challenging because they won’t let Oracle employees register. We have to wait until 10 minutes into the talk and take a seat if there is one. I understand but it does make planning and scheduling difficult. The biggest problem that I am having is deciding on what I want to be when I grow up. I’ve always adopted the philosophy of doing one or two things very well, making a name for myself, and moving on to the next topic. Oracle seems to have an information page setup for all of the products that is driven by product marketing. I’ve used this tactic before. I might need to write a series of tutorials or possibly a book to bring up my levels of expertise.

more later….

an atypical day

Today has been an unusual day. I didn’t have any meetings. I didn’t have any conference calls. I didn’t have anywhere that I was required to be. Very unlike the rest of the week. First, let’s recap the events leading upto today.

 – 6am work out. must start the week out properly
 – 7:45 in office for meeting with team
 – 8:30 meetings start to plan out how to accomplish tasks for the week
 – 11:30 skip out of office and head to customer site
 – 1:00 meeting at customer site, lasts until 2:30
 – 2:30 follow up to meeting, write follow up emails, finish some documentation, do some research
 – 4:30 drive home to make soccer practice for the kids
 – 5:30 soccer practice
 – 7pm dinner
 – 7:30 finish up project on the planet Mercury (3rd grade project)
 – 8:30 start hearding kids to bed
 – 9:30 finished with bedtime, get to read paper, check on email again, watch last quarter of Monday Night Football

 – 6:00 work out. wow, two days back to back
 – 8:30 doctors appointment
 – 10:30 did it really take two hours at the doctor? finished reading book
 – 11:30 customer meeting/event
 – 1:30 event rained out
 – 2:30 research and training, prep demo system
 – 6pm dinner, did we really have dinner two nights in a row? wow.
 – 7pm play games, watch really bad kids tv show
 – 8:30 start hearding kids to bed
 – 9:30 finish with bedtime, check on email again, watch tv show
 – 10:45 read boring technical manuals instead of sleeping pill

 – 7:30 breakfast with the family, unusual event. I’m usually gone by this time
 – 8:00 check email, follow up on correspondance
 – 9:00 customer meeting
 – 11:30 back in office, pick up salad at desk and read email while working
 – 1pm internal meetings to plan customer architecture presentation
 – 4:30 – drive home, tired from all day meetings
 – 6pm – dinner with kids, wife playing soccer
 – 7pm – cub scouts at our house
 – 9pm – finished cleaning up, everyone to bed
 – 10pm – check email, read until time to go to bed

 – breakfast with the family, two days in a row. I will get spoiled if this keeps up.
 – nothing planned. could go to golf tournament, looks like rain. tough to justify “skipping out on work” for tournament.
 – long range planning
 – research
 – prep for presentations next week
 – prep for presentations tomorrow
 – get up to date on expenses, plan training
 – training, playing with demo systems.

 – work out in morning
 – paperwork in office
 – customer meeting and presentation
 – conference call to set next weeks schedule.

total time at customer sites: 11 hours
number of customers visited in person: 7
number of customers contacted via phone and email: 17
number of phone calls with customers: 13
number of internal phone calls: 15
number of emails with customers: 27
number of internal emails: 8
time spent on blogging: 1 hour
time spent on fantasy football: 1 1/2 hours + 5 minutes trash talking

Days like today are rare. I wanted to record what I did in a week so that I could look back and see if this was typical, unusual, productive, or waste of time.


importance of context

My kids and I are constantly quoting movies. Some movies have more relevance than others and some are more quotable as well. We have been doing this for years. Every Christmas we watch It’s A Wonderful Life and every February 2nd we watch Groundhogs Day. I realize that they are corny movies but they help build context. Two of our favorite movies are It’s a Mad, Mad, Mad, Mad World and What’s Up Doc. These movies are movies that you can watch with the kids.

At work we joke about movies like Caddy Shack and The Godfather, neither of which are appropriate for kids. To me, the importance for these touchstones is a commonality. We try to build similar commonalities with cutomers. Some of them are contrived and some of them are really shared experiences. I’ve been talking to some of Oracle’s long time customers and they really like some of the events that we host. I’m not talking about the lunch and learn lectures or speaking events at hotels. I am talking about events like plays, volunteer events, or even golf tournaments. Years ago I got tickets to the Shell Open and sat on the 16th green with one of my friends/customers. We still talk about it. It was very interesting watching the players round the corner and see how each of them made their approach shots to the hole.

I guess that a common theme is starting to repeat itself, history. History is what makes us good at what we do. History is what makes us have a common ground to share more experiences. History is what lets people trust each other. If I was right the last ten times, there is a good chance that I will be right again. If I bent the truth the last two times and it cost you money, it will probably cost you money again.

My only hope is that I am building up enough history and karma so that I will be trustworthy. I know that part of the Boy Scout oath is a scout will be trustworthy. I guess this is what it means.

risk and reward

Does everyone think in terms of risk and reward. It seems like most people that I have talked to lately are greatly focused on risk and reward. I understand if a manager is looking at investing money in a product and needs to figure out if the cost of the software is worth the risk of not deploying it. I struggle with justification of software. Ok, I understand the need to get a word processor and presentation software package. Is it worth the $200 or so dollars to get Word and Powerpoint or is downloading StarOffice adequate? If I send a thank you note, can I send it via email or do I really need to get a card and send it through the US mail? At what point does being too cheap really get in the way of success? At what point does doing what is required cost you in lost opportunity or perception? Would going a little farther really make difference? I keep coming back to a story that I heard about Larry Bird the basketball player. To become truly great he would spend hours practicing one shot and one shot only. After weeks and weeks of making the same shot, he would move slightly to the left or right on the court and repeat the training.

Does the same apply to non-sports activities? Can I read documentation, play with software, give presentations, crate a prototype deployment and truly know enough about a product to be an expert? Do I need to do this multiple times to truly see it from different angles? What are my cost and risk for cutting corners? If I use StarOffice to create a document, what am I loosing? If I use a public domain database do I sacrifice functionality or features when compared to using a commercial version? (Ok, I know the answer to this one and can even explain it in gory detail).

I wish I knew how to analyze risk enough to make the informed decisions. One of my daughters friends is constantly analyzing risk and reward. She has been doing this since she was able to talk. “If I don’t make my bed, what will happen?” “If I don’t eat my peas, what are the consequences?” I’m constantly amazed by her rationalization that the peas are not worth the bowl full of ice cream. Not making her bed is not a fair trade off for an hour of television. My friends finally found the breaking point now that she wants a cell phone. It is amazing what she will compromise to get her phone back since it is her lifeline to her friends. I’ll stop here because I don’t want to get started on how old someone should be to have a phone with them.

Risk vs reward? I guess that this is what separates really good decision makers from really bad ones.



before we talk, what is your job function

I understand the concept of delivering a message to the audience. I also understand of not wanting to create enemies by saying the wrong thing to the wrong person. For example, if I say that a new business process or software package improves your life by making a daily task take 40% less time. If you are a DBA, your response is — great. I hate typing in these commands or explaining my scripts to the latest new guy in the office. If the DBA’s manager is in the room, it translates into —- great, his time just got free so I can assign this project that has been on hold. If an executive is in the room, it translates to — great, I can deny that new hire request or cut staff by one person. If all three are in the room, no one says anything because they see it from the other persons perspective and it gets translated into fear across the board because the DBA will loose his job, the manager will loose headcount, and the executive is now held accountable for reducing expenses to pay for this software. Net result is that nothing gets done and everything stays the same.

I go to lunch with a group of administrators that I worked with years ago. It is amazing how slowly things change. Projects that I worked on two years ago are still being rolled out. The manager that I locked horns with has run two more people out of the group. A replacement has not been hired to fill my position. And most surprisingly, my accounts still work on most of the machines that I managed. I haven’t tried going to superuser because for some reason that crosses a line for me. For some reason, me logging into the university web server is not a violation of morals and principals. I can update a web page that has my name on it (and my personal content) but running sudo crosses the line for me. Dosen’t it seem that I should not have a web page at this institution? I was a student at one university and graduated last December. It is now October and I still have account access on most machines. I was an employee at a different university two years ago and I still have an LDAP entry as well as home directory access on most IT machines. Is it me or is this typical? If I were the IT manager, LDAP and Kerberos would be the first thing that I implement and restrict access to only those that should have access. When you leave the group or organization, your account is disabled.

One thing that I am getting really good at of late is doing a demo of a subset of our products. I really like having a corporate demo site managed by corporate. When I was at Sun, we started to have a set of software demos so that we could show the portal and ldap servers in action. At Oracle, the product suite is much larger. The rack of demo machines consists of a few hundred machines that are all refreshed daily from a golden image. I like the ability to schedule a demo and go to a customer site so that we can drive the software from my laptop. I’ve done two of them in the past few weeks and have been to a couple of training classes that used the demo suite to explain the products. I guess I am a visual learner and do much better seeing the product and not bullets on a presentation.

I guess I should have asked first before I started rambling; who am I talking to? I sure hope I didn’t offend you with my ramblings. If you are a former co-worker and don’t like being made fun of; get over it. Bring it up at lunch next week. If you are reading this to find out about Oracle software, this probably isn’t the place to learn. I’ll throw out tidbits here and there but this is more of a high level moral discussion and relevancy discussion. If this is my manager, forget that you read any of this. If anyone else, why are you reading this blog? What are you hoping to get out of my ramblings?