Category: Oracle

Having an ssh term into an instance isn’t the easiest thing to work with. I was hoping to get an X11 window into the session and display it back to my desktop. My desktop is a WindowsXP so the only real option is to use TightVNC to get an X11 interface.

Fortunately the Oracle Unbreakable Linux that I am running as an instance has vncserver installed as a default module. I tried running vncserver on the instance and attaching to it from my desktop. Unfortunately, it did not work.

To connect to the vncserver, I first tried to use putty to tunnel the ports. VNC runs on ports 5900 and 5901. I tried to configure the proxy to have it tunnel through ssh and connect on the other end. Fortunately, this did not work. I have done this in the past and it is very slow and difficult to use.

My second attempt was to create a new port opening in the Elasticfox tool and connect directly with TightVNC. I defined the ports 5900 and 5901 as the default connection and mapped it to the security group used to start the instance. This worked and allowed me to connect to the VNC server running on the test instance.

Doing this, we can now launch firefox and view web sites. The ones that we want to look at are https://localhost:1158/em and http://localhost:8080/apex. With this configuration we can now start looking at installing, testing, and experimenting with instances. We can also use the Amazon storage to keep our tests persistent so that we don’t have to recreate the environment every time.

more later…..

Continuation of the Oracle Education class on EBS Foundations……

Today we continued the discussion on shared entities and integration. I think I understand this concept. Basically it is similar to Java Objects and inheritance. If I define something at a global level, it is shared across all entities. If I define something inside an object, it is valid for that object only. The example given was with a supplier. If I create a supplier, the supplier name is global. If I create a site for invoices or delivery, the site is specific to the role that I was in when I created the site. I can have this object shared across sites but the secondary role must define this and allow it to be the same or different. The example given was with a company that does business in the US and Canada. If I work for a company that has divisions (and different roles) in both locations, the invoice group in Canada has a site that they invoice that might be the same or different from the site in the US.

Chapter 5 of the class focuses on security of EBS. The basic model is what you would expect. End users have access to self service and approvals. A smaller class of users has access to registered applications. An even smaller class of system administrators have access to delegate application access and define access control. There is also a class of higher system administrators that define the data security model and functional security of EBS. By functional security, I mean individual menus of functions, forms, and html pages. What was new to me was that this is the same as Oracle Access Manager. Given that EBS already has this, there really isn’t a need to OAM to front end an EBS installation. The only reason you would use OAM is to time restrict or restrict by ip address a users access to the EBS installation. This does not come up very often so the need to use OAM is decreased.

It is interesting that EBS allows you to define effective dates but not times. It seems that effective date range should also allow you to restrict on a time basis and EBS might be able to do this but out of the box it does not. OAM gives you this option along with some other options and we have recommended this to some customers. They were more excited about the password reset features in OAM and less interested in the time access. EBS does not have the self service password reset that OAM does have.

The reporting mechanism to look at security inside of EBS is also something that is potentially very powerful but also dangerous. Many of the customers that I have talked to have a difficult time managing concurrent processing. I think I understand why now. Users can be given the right to define and launch reports at any time. If these reports are search intensive, it can log jam a system. Using something like partitioning can reduce the search by reducing the result set to a month instead of six or seven years. Incorporation of the EBS Management Pack into OEM does give you a finer grain administration of concurrent processes. OEM gives you a view into processes and allow you to raise or lower the priority of the process. EBS administration tools do not give you this fine a grain of control. OEM integrates better into the operating system thus has better controls to raise or lower process priorities through EBS or OS commands.

Chapter 6 is a discussion of flexifields. Flexifields is a configurable field that opens in a window from a regular EBS window. It allows you to structure identifiers required by EBS according to your own business definition. It also allows you to collect and display additional information that EBS does not require as fine a grain definition. You can create key flexfields to generate a shortcut name for an item. For example, if you encode components with descriptive information in a series of values, these values can be concatenated together. The 11g version of the database allows you to partition based on these entries. The two fit very nicely together and allows you to separate data based on a partial field and not have to waste storage to split information into different columns.

more tomorrow…..

I am taking an Oracle Edu class this week – D49493GC10 (or D52216) which is the intro to EBS R12 version. The class material looks very itneresting. The assumption is that you have EBS and are trying to use the features and functions of the class.

The course starts relatively basic with tips and tricks for navigating the forms interfaces. This is very helpful for someone who is doing this for the first time. It shows you different ways of doing things like searching, personalizing, and customizing the forms interface. Once you get through the basics of the user interface, the class dives into the different components of EBS and how it is installed and configured. The next chapter is discussing shared data and integration. This is a good introduction chapter. The next chapter is fundamentals of system admin. This is an abreviated chapter that summarizes the five day class. It is more of an introduction chapter and not a detailed chapter on administration.

Overall this has been a good class. It is an introductory class but it does give you a good feel for what it takes to navigate and manage some of the features in EBS. I can see where someone would want to take this class if they were getting into consulting or deploying EBS for the first time. It seems to me that the admin class or the installation class is more in line with someone starting. The workflow class is probably what someone wants to take if they have an existing installation, understand basic administration, and want to enhance an existing deployment.

more tomorrow on the admin issues…..

It appears that there is more than hype with the HP-Oracle Exadata Storage solution. I like the architecture and way that it splits out queries between boxes. A good source of more information can be found at Kevin Closson’s blog. Kevin is an Oracle Employee and is a performance architect. He goes into some good detail about the operations.

What impressed me the most about this device is that is currently is focused on data warehouse solutions. It looks for select statements and farms these requests to the storage node. Since the storage node understands the table structure and query formatting, it only returns the relevant data. Instead of returning a 1G table and doing the where select on the database machine, the select…where is done on the storage node and the relevant columns and rows are returned. This means that things like returning a current billing statement for a specific cell phone will return the current bill and not all of the historical data. Partitioning does this but it does not reduce the number of columns returned. If you have a lot of data associated with a user in different columns, you can reduce the amount transferred from the storage by returning only the relevant information from the tables and not the entire row.

Interesting stuff. If we only had a few telco companies where I work…..

The huge benefit that I see from this device is that the statements that are shipped across to the storage node is warehouse centric. I can foresee this working with spatial data and being applied to seismic reservoir systems. I can see this working with materialized views for manufacturing. The only thing that needs to change is the engine on the database that farms out the request to the storage component. This has huge long term potential where large data sets are used. Many of the complex models that are currently designed to reduce this complexity can be expanded and simplified. The storage node can be customized to understand this format and pre-process the requests. If we follow this to the extreme, you might be able to embed some of the stored procedures for things like velocity correction of seismic data and have the storage node process this data and present the corrected trace instead of the raw trace. You can also do thing like image recognition or enhancement in the storage component and not at the higher levels. More complex stuff that currently requires parallel processing can be forced down into the storage and not to parallel processes. Agreed, the product is not here yet but I can see it going in that direction. Since Oracle typically provides open APIs to most of its products, I can see new ventures springing up based on this technology and specific solutions being created for specific industries.

I spent some time playing with the Amazon Cloud and Oracle today. I wanted to see if I could allocate some persistent storage with an instance so that I didn’t have to create a new repository every time I started an instance. I wanted to create an instance, store data there, and stop the instance. I didn’t want to have to reload the data or the instance every time I started.

I was reading LewisC’s blog and he suggested that you could not use the S3 storage for a file system. I didn’t think that was true so I thought I would play. It turns out that you can create a file system and even create a database in that area. Here are the steps.

1) create an S3 persistent storage area. I did this using the Elastifox extension to Firefox. This is done in the Volumes and Snapshots tab. If you click on the green + button it allows you to define a new storage. I created a 400G area just to see if I could create a file system of interest. When I did it assigned it to a VOL ID but did not populate anything else.

2) start an AMI. I did a simple search for an oracle instance and started an 11g 32-bit instance. Once I did this, I was able to use putty and login to the system as root and oracle. Setting this up did require downloading putty and puttygen. The puttygen was needed to convert the keys that aws generates into the right format for putty. Once I was able to do this, I could create to the public DNS instance name with the keys that aws generated.

3) once I had the instance started, I went back to the Volumes and Snapshots and associated storage with the running instance. I had to associate the storage with a device so I associated it with /dev/sdb. I chose this because /dev/sda is the default storage from the AMI.

4) After associating the S3 storage with the instance, I was able to treat it as a local disk. I partitioned it with fdisk and created a Linux partition and write the data to the partition. Since this was a simple test, I created one instance and only one instance thus I see /dev/sdb1.

5) Once I had a partition defined, I formatted the partition with a filesystem using the mkfs /dev/sdb1 command. This partitions the file system using the ext3 filesystem by default. I did not test any other file systems but I assume that you could do something like reiser or newer filesystem.

6) Once I had a formatted partition, I was able to mount it. I first had to create /u03 with the mkdir /u03 command. I then mounted the file system with the mount /dev/sdb1 /u03 command. To make this persistent I had to edit the /etc/fstab file with the following line:
/dev/sdb1 /u03 ext3 defaults 1 3

7) I can verify that the file system exists with the df -k command. I should see
/dev/sdb1 412843916 3434932 388437736 1% /u03
as one of the lines.

8) now that I have the file system created and mounted I can run_dbca.sh as root and create a database on the file system.

At OpenWorld last week, Oracle and HP announced an HP Oracle database machine and Exadata storage server hardware. product page.

If you read the news article, which has a broken link from the Oracle home page, you get the basics that HP and Oracle have jointly announced a new rack solution that is pre-configured to run the Oracle 11g database in a real application cluster (RAC), using 64 processing cores, Oracle Enterprise Linux, and upto 168 Terabytes of storage. The servers are HP Proliant D180 servers with two Intel 2.66 Ghz quad-core processors, capacity for 12 three and a half inch disks, 8G of memory, and dual InfiniBand network connections. I recommend reading more in the data sheet. What is new from Oracle is the Exadata Storage Server Software to manage this storage. Along with this is a new plug-in for Enterprise Manager to help manage this storage device.

The difference between this and standard storage is that the storage can either be raw storage or can be used as an offload storage device to minimize queries. It is typically used for a data warehouse where queries return large sets of data. With the exadata storage software, the storage processors can pre-process the SQL request and return the results to the database engine. It is important to note that this software is designed to be used with the HP hardware and only the HP hardware. It is not designed to be used with other servers or storage solutions. This product is directly targeted at the Netezza and Teradata solutions. The intent is to bring some of these old solutions to 10g and beyond and help with performance tuning. Most of the Netezza and Teradata solutions are on 9i or older and not tuned very well. A high end comparison of the Netezza, Teradata, and HP-Oracle solution shows that they both hold the same amount of disk but the HP-Oracle solution has substantially more database cores (4 for Netezza, 32 for Teradata, and 64 for HP-Oracle). The HP-Oracle solution also has 3x the memory and 20x the network speeds. Many of the Netezza customers are running into scalability problems where they need multiple boxes to host the data and are running into one source of truth issues because some data is replicated on both boxes and there are no consistency tools to keep the data synchronized.

You can scale the solution from one box to 18 boxes inside of one physical hardware rack. This gives you 65 Terabyes of storage using SAS storage or 216 Terabytes using SATA storage. The peak performance inside the box is 18 GB/second for data transfer. You can also use Infiniband to connect to another hardware rack to scale across multiple servers since each system has 2 Infiniband switches. The database engine comes pre-installed but is not configured to use RAC, partitioning, advanced compression, security, or enterprise manager options like diagnostic and tuning. You can run this system in single instance without RAC and split the data appropriately for a data warehouse. The single instance is configured to have a database instance that is managed by ASM. ASM then knows how to talk to the Oracle exadata cell manager and offload the queries to the exadata cell. You can have multiple exadata cells behind the Infiniband switch.

The 11g database has been enhanced to take advantage of the exadata storage. A new component called iDB (Intelligent Database protocol) runs in the database kernel and maps database operations to the exadata processors or launches the query locally. This protocol allows the exadata processors to perform functions like table scans, compression, and OLAP functions. The iDB protocol is built on top of RDSv3 (Reliable Datagram Sockets) and runs over InfiniBand. All of these protocols are open standard and publicly defined.

Two important points about this architecture is that when a SELECT…WHERE is issued against a table, a traditional storage device will return the table and the database engine will sort the where clause and drop the results that are not relevant. The exadata storage will do the select and the sort at the disk and only return the results after the where clause is performed. The second way of reducing the data transferred is to return only the columns requested and not the whole table. In a traditional storage device the table is returned and the columns that are not requested are dropped. In the exadata storage, the storage processor will perform the sort and only return the requested information across the data channel.

Another new feature of the exadata storage node is the ability to manage I/O resources. In traditional database management, you have been able to allocate CPU and memory utilization based on jobs or processes. You can allocate 30% of your processing to finance and 70% of your processing to payroll. The problem with this model is that a finance job could swamp the I/O resources with a simple request even though it does not use more than 30% of the processor. With the exadata storage you can allocate resources to the I/O channel as well. With this model you get 30% of your I/O allocated to finance as well as 30% of your database engine.

One of the impacts that this might have on business is that a single user can be given 10% of a machine and 10% of the I/O bandwidth. This user can be given the ability to run ad-hoc queries into the data warehouse without consuming more than 10% of all resources. This isn’t true with most business intelligence engines. We typically see problems with Cognos or Business Objects systems in products. Database resources can be restricted to a user or group and not let them overload the storage system with open ended queries.

The sales model for this box is a little different. Oracle sells the product and HP delivers the hardware pre-configured to the customers. Oracle handles the first line of support and HP provides the 2nd tier along with break-fix and spares. This machine does not come with database licenses. This allows customers to use existing licenses and not have to purchase new licenses. The warranty for this machine is 3 years parts and labor.

For those of you that are graphical in nature, technical” overview, is a good pictorial review of what I have been talking about.

My gut feeling… this is a new technology. It is a game changer. The big question is will this become mainstream for many companies or will it just be an Amazon and Google solution? If I were a hosting company and wanted to host Oracle, this would be the solution. I would choose this over visualization any day. If I were a telco or vendor with a large number of customers that needed target marketing, it would be an easy choice. My biggest problem is that in Houston, this isn’t the case outside the oil and gas industry. It might be viable for health care. It might be viable for utility companies. It might be viable for service companies. Time will tell.

there is a percieved link between business data and firms performance

Restrictive access to data to exec level only

Lack of trust of data to lower level employees

Small minority of firms use business data

Too hard to use tools

33% of tech companies think they adequately use business data
Financial firms come in at 20%

Silos, lack of standards, and error prone data impede use. Used more in other countries than US

Dirty data a big problem in Europe

Defining a strategy and admitting that data is good or bad is the first step

Data governance is also an issue for moving forward

I recently read a book, http://www.amazon.com/Good-Great-Companies-Leap-Others/dp/0066620996/ref=pd_bbs_sr_1?ie=UTF8&s=books&qid=1219157911&sr=1-1 – Good to Great, and tried to apply it to myself, my immediate organization, and the company that I work for.

I need to be careful here because some of my managers read this blog so I won’t use real names to protect myself.

The three key things that I got from this book are
– different levels of leadership
– hedgehog concept (single focus)
– todo vs stop doing list

The different levels of leadership are:
1) highly capable and makes productive contributions through talent, knowledge, skills and good work habits.
2) contributing team member. conributed individual capabilities to the achievement of group objectives and works effectively with others in a group setting
3) competent manager. organizes people and resources toward the effective and efficient pursuit of a predetermined objective
4) effictive leader. catalyzes commitment to and vigorous pursuit of a clear and compelling vision. stimulating higher performance standards
5) executive. builds enduring greatness through a paradoxical blend of personal humility and professional will.

First my perspective. When I worked at Sun I did self destructive things like not wearing socks, wearing polo shirts, coming into the office in sandals because my actions should drive my success. I wanted to wow everyone with my brilliance and abilities. I had level 1 down and made it part of my personality. When I came to Oracle I realized that I could not do it on my own and needed to help others and rely upon others to get my job done. I have grown into a level 2 and think that I know how to do this. This year I am working on level 3 by helping the new sales reps and OD reps as they get hired on. The learning curve at Oracle is a little daunting and helping new people learn our product suite helps them communicate this to a customer. I think that I have levels one and two down and am building the foundation to level 3 this year.

I do fall short in level 4 because I don’t have a clear vision other than a revenue target. I don’t know what my company, my group, or I want to accomplish other than grow revenues. This isn’t a good thing. I think I know what the corporate goal is. I need to research to clarify it. I think it is to create an integrated end to end stack of applications and services with open program interfaces to manage the day to day operation of a corporation. As for my group, I think I know what the goal is but I don’t like what I am hearing. My group got split into two so the goal for this year is to reconnect with the remaining customers that we have ignored and rebuild. I think that this is the goal but I don’t like it as a goal. My personal goal is to expand my knowledge base and understand the applications more to align myself with the corporate vision. (I got lucky and was able to ask one of our corporate visionaries who works in acquisitions the question of vision and he clearly stated this while I was typing)

I also fall short of the level 5 leadership because it is not part of my job function. I can try to build this as part of training the sales reps, creating a wiki, and creating workshops/seminars that repeat year over year. I am trying to do this but have not accomplished this.

Here is the scary part. If I do this same evaluation for the people that I work with I would say that most of them are level 2 to level 3 (more level 2 than 3). There are some exceptions that rise up to level 4. The people that I consider to be level 5 are very good at what they do but are area or geography resources and I find it difficult to get their attention for the smaller customers. When I go beyond two levels above me I don’t have enough interaction with these managers to make an analysis. The stuff that I do see from the mid-level managers is technocrat stuff that do not add value to anything that I do. It is more adding value to their managers and not the group. The executives in our group are level 4 and a couple are level 5. I met with the US SC manager last year and he didn’t talk the first 10 minutes of our meeting other than to ask questions. His questions were direct and showed that he did some research on me before getting together. His direct report asked me about programs that he started and things that he did last year that I really didn’t know much about or things that helped me. One of the sales managers that we worked for in years past recently came down to our office to talk about changes in our organization. He did nothing but ask questions. He didn’t offer solutions to our problems. He didn’t recommend how to fix things. He asked what our issues were and asked for more input to help him solve the problems. Unfortunately, the managers below him are all about filling in the check box and being there just in case they are needed. It could be that I don’t know them enough but I am not motivated from my interactions with them.

The hedgehog concept is single focus on a single topic. The three things that correlate to this are passion, what are you best at, and what drives your financial engine.

The passion was the easiest. I can break complex ideas into simple concepts and bring it down to a productive discussion. This is probably my biggest asset and something that I can help my team the most.

What am I best at is probably the next toughest. I have been good at pre-sales. I think I showed this while at Sun. I thought I was good at research until I went back for a Ph.D. program and found out that I didn’t have the motivation or focus to make it happen. I also realized that I am not that good of a developer. What I am good at is breaking down issues and correlating facts from different areas.

My driving engine? This is the metric for measuring success. The book talks about how Walgreens wanted to maximize revenue per custom visit. How would I measure something like this? Sales per hour with a customer? Change in sales quarter over quarter? The number of face to face meetings per week? The number of research hours after a meeting? The number of interactions after a meeting? I need to work with my manager to define this metric. I also plan on going a level or two above my manager to interact positively with them.

The to-do list and the stop-doing list are easy and difficult. I have monthly to-do goals and weekly to-do lists. I even started using rememberthemilk.com to help create and manage these lists for different parts of my life. When I created a stop-doing list all of them were personal goals like stop watching tv, stop eating high sugar foods, stop going out for dinner to save money. I had trouble creating a stop-doing work list. This is something that is difficult to do and I need to focus more time on.

I also liked the comment on stopping the flywheel. I feel like my group just got run over by the flywheel and we are trying to get up and start it rolling faster again. I think I have three or four good flywheel spinning goals that I am moving forward with but only time will tell.

Overall, this was a good book. It made me think. It made me analyze how I am doing my job. It helped me focus on what needed to get done (as well as start blogging again).

It seems to me that ITIL best practices and consolidation conflict in goals. ITIL focuses on isolation of faults and features to make it easier to manage services. Consolidation aggregates services to reduce the need to manage. I have been working with customers on converting large iron database to commodity rac clusters. It seems like the biggest benefit is not consolidation but making everything the same. The same OS version. The same database release. The same stored procedures. The same app features. Splitting services tends to cause drift when it comes to patches, tuning, and customizations.

A year and a half ago I took the 10g Administration I class from Oracle Education. As part of my quarterly education class I signed up for the 11g Administration I class. I was hoping to learn new stuff about 11g and sharpen the saw as my manager puts it. Unfortunately, the material has not substantially changed between versions of the class.

The main differences are in chapters 13, and 18. In the 11g class, Chapter 13 –  Performance Management has been updated to include more about automatic memory management and less talk about SQL Tuning. The chapter on Flashback has been moved into the Administration II class and is not covered in this class. In Chapter 18 – Enhancing Database Capabilities, new features on alerting and incident reporting are detailed. This is all part of the Enterprise Manager Console that comes with 11g. The chapter goes on to detail the use of Metalink, logging service requests, and managing patches. This is important information that is relevant to any database rollout into production.

Unfortunately, I would not recommend attending this class if you have already attended the 10g Administration I class. 90% of the material is the same and the 10% that is different is covered in the Enterprise Manager class. If you have not taken the 10g Administration I class I strongly advise that you take the 11g class even if you are rolling out 10g into production. There really are not any substantial differences in the class. In talking with the instructor, it appears that all of the significant changes have been pushed into the Administration II class which details new features of the 11g database.