Storage in the Oracle Cloud

This week we are going to focus on storage. Storage is a slippery slope and difficult conversation to have. Are we talking about a file synchronization like,, or Are we talking about raw block storage or long term archive storage? There are many services available from many vendors. We are going to focus on block storage in the cloud that can be used for files if desired or for backups of databases and virtual machines. Some of the cloud vendors have specific focused storage like Azure tables that offer a noSQL type storage or Amazon S3 allowing you to run a website without a web server. Today we will look at the Oracle IaaS Storage set of products. This is different than the Oracle PaaS Documents option which is more of a Google Docs like solution. The IaaS Storage is a block of storage that you pay for either on a metered usage or non-metered usage basis.

Notice from the web page, we click on Infrastructure and follow the Storage route. We see that we get the raw block storage or the archive storage as options. We also have the option of an on-site cache front end that reduces latency and offers an NFS front end to the users providing more of a document management strategy rather than a raw block option.

Before we dive a little deeper into the options and differences between the storage appliance, spinning disk, and spinning tape in the cloud, we need to have a discussion about pricing and usage models. If you click on the Pricing tab at the top of the screen you see the screens below.

Metered pricing consists of three parts. 1) how much storage are you going to start with, 2) how much storage are you going to grow to, and 3) how much are you going to read back? Metering is difficult to guestimate and unfortunately it has a significant cost associated with being wrong. Many long term customers of AWS S3 understand this and have gotten sticker shock when the first bill comes in. The basic cost for outbound transfer is measured on a per GB basis. The more that you read across the internet, the more you pay. You can circumvent this by reading into a compute server in the Oracle cloud and not have to pay the outbound transfer. If, for example, you are backing up video surveillance data and uploading 24 hours of video at the end of they day, you can read the 24 hour bundle into a compute server and extract the 10-15 minutes that you are interested in and pay for the outbound charges on compute for the smaller video file.

Non-Metered pricing consists of one part. How much storage are you going to use over the year. Oracle does not charge for the amount of data transferred in-bound or out-bound with this storage. You can read and write as much as you want and there is no charge for data transfer across the internet. In the previous example you could read the 24 hours of video from the cloud storage, throw away 90% of it from a server in your data center, and not incur any charges for the volume of transfer.

Given that pricing is difficult to calculate, we created our own spreadsheet to estimate pricing as well as part numbers that should be ordered when consuming Oracle cloud resources. The images below show the cost of 120 TB of archive storage, metered block storage, and non-metered block storage.

Note that the data transfer price is non-trivial. Reading the data back from the cloud can get significantly more expensive than the cost of the storage itself. A good rule of thumb is the cost of spinning disk in the cloud should not exceed $30/TB/month or $400/TB/year. If you look at the cost of a NetApp or EMC storage system, you are looking at $3K-$4K/TB purchase price with 10% annual maintenance per year ($300-$400). If you are currently running out of storage and your NFS filer is filling up, you can purchase cloud resources for a few months and see if it works. It won’t cost you anything more than paying support and you can grow your cloud storage as needed rather than buying 3 years ahead as you would with a filer in your data center. The key issue with cloud storage is latency and access times. Access to a filer in your data center is typically 10ms where access time to cloud storage is typically 80+ms. All cloud storage vendors have on site appliance solutions that act as cache front ends to address this latency problem. Oracle has one that talks NFS. Amazon has one that talks iSCSI. Microsoft has one that talk SMB. There truly is no single vendor with a generic solution that addresses all problems.

Enough with the business side of storage. Unfortunately, storage is a commodity so the key conversation is economics, reliability, and security. We have already addressed economics. When it comes to reliability the three cloud vendors address data replication and availability in different ways. Oracle triple mirrors the data and provides public-private key encryption of all data uploaded to the cloud. Data can be mirrored to another data center in the same geography but can not be mirrored across an ocean. This selection is done post configuration and is tied to your account as a storage configuration.

Now to the ugly part of block storage. Traditionally, block storage has been addressed through an operating system as a logical unit or aggregation of blocks on a disk drive. Terms like tracks and sectors bleed into the conversation. With cloud storage, it is not part of the discussion. Storage in the cloud is storage. It is accessed through an interface called a REST api. The data can be created, read, updated, and deleted using html calls. All of this is documented in the Oracle Documents – Cloud Storage web site.

The first step is to authenticate to the cloud site with an instance name, username, and password. What is passed back is an authentication token. Fortunately, there are a ton of tools to help read and write HTML code and are specifically tuned to help create headers and JSON structured data packets for the REST api interfaces. The screen below shows the Postman interface available through Chrome. A similar one exists for Firefox called RESTClient API. Unfortunately, there is no extension for Internet Explorer.

The first step is to get an auth header by typing in the username and password into the Basic Authentication screen.

Once we are authorized, we connect to the service by going to domain) where identity domain is the cloud provider account that we have been assigned. In our example we are connecting to metcsgse00029 as our identity domain and logging in as the user cloud.admin. We can see what “containers” are available by sending a GET call or create a new container by sending a PUT call with the new container name at the end of our html string. I use the word container because the top level of storage consists of different areas. These areas are not directories. They are not file systems. The are containers that hold special properties. We can create a container that is standard storage which represents spinning disk in the cloud or we can create a container that is archive storage which represents a tape unit in the cloud. This is done by sending the X-Storage-Class header. If there is no header, the default is block storage and spinning disk. If the X-Storage-Class is assigned to Archive it is tape in the cloud. Some examples of creating a container are shown below. We can do this via Postman inside Chrome or a command line

From the command line this would look like

export OUID=cloud.admin
export OPASS=mypassword
export ODOMAIN=metcsgse00029
c url -is -X GET -H "X-Storage-User:Storage-$ODOMAIN:$OUID"
-H "X-Storage-Pass:$OPASS"

This should return an html header with HTTP 200 OK and an embedded header of X-Auth-Token: AUTH_tk578061b9ae7f864ae9cde3cfdd75d706. Note that the value after the X-Auth-Token is what we will use to pass into all other requests. This token will change with each request and is good for 30 minutes from first execution. Once we have the authentication finished we either change the request type from a GET to a PUT and append the container name to the end. The screen above shows how to do this with Postman. The results should look like the screen below. We can do this from the command line as show below as well.

c url -is -X PUT -H "X-Auth-Token: AUTH_tk578061b9ae7f864ae9cde3cfdd75d706"$ODOMAIN/new_area

In this example we create a new container from the command line called new_area. We can verify this by reviewing the cloud storage by changing the PUT to a GET.

c url -is -X GET -H "X-Auth-Token: AUTH_tk578061b9ae7f864ae9cde3cfdd75d706"$ODOMAIN

Both of these methods allow us to see the storage that we created. I personally do not like this interface. It is not intended to be human consumable. Uploading and downloading a file is difficult at best. A user interface that makes dragging and dropping files is desirable. This is where dropbox and google docs shine. They allow you to drag and drop as well as synchronize directories to cloud storage. The Oracle Storage Cloud is not intended to be this solution. It is designed so that you can drop a new library into your rman backup and backup straight from your database to the cloud. You can point your ComVault or Legato backup software to a cloud instance and replicate your data to the cloud. If you want a human readable interface you need to purchase something like the Cloudberry Explorer from Cloudberry. This give you a Windows Explorer like interface and allows your to drag and drop files, create containers and directories, and schedule archives or backups as desired.

Note that the way that you create a block storage container vs an archive container is a simple menu selection. Retrieving the archive storage is a little more complex because the tape unit must stage the file from the tape to disk and notify you that the restoration has been completed. This is a little more complex and we will defer this discussion to a later blog.

Copying files is little more than dragging and dropping a file between sections of a window in Cloudberry.

For completeness, I have included the command line screen shots so that you can see the request/response of a command line interaction.

It is important to remember our objective. We can use the cloud block storage as a repository for things like database and a holding point for our backups. When we configure a database in the cloud, we backup and restore from this storage. This is configured in the database provisioning screen. The Storage-metcsgse00029/backup is the location of RMAN backup and restores. The backup container is created through the REST api or Cloudberry interface. We can also attach to the cloud storage through the cloud storage appliance software which runs inside a virtual machine and listens for NFS requests and translates them into REST api calls. A small disk is attached to the virtual machine and it acts as a cache front end to the cloud storage. As files are written via NFS they are copied to the cloud storage. As the cache fills up, files contents are dropped from local storage and the metadata pointing to where the files are located are updated relocating the storage to the cloud rather than the cache disk. If a file is retrieved via NFS, the file is read from cache or retrieved from the cloud and inserted into the cache as it is written to the client that requested it.

In summary, we covered the economics behind why you would select cloud storage over on site storage. We talked about how to access the storage from a browser based interface, web based interface, or command line. We talked about improving latency and security. Overall, cloud based storage is something that everyone is familiar with. Products like Facebook, Picaso, or Instagram do nothing more than store photos in cloud storage for you to retrieve when you want. You pay for these services by advertisements injected into the web page. Corporations are turning more and more towards cloud storage as a cheaper way to consume long term storage at a much lower price. The Oracle Storage Cloud service is first of three that we will evaluate this week.

182 thoughts on “Storage in the Oracle Cloud”

  1. Thanks for your article. One other thing is always that individual states in the United states of america have their own laws that will affect home owners, which makes it very, very hard for the Congress to come up with a brand new set of rules concerning foreclosure on homeowners. The problem is that each state possesses own legislation which may interact in an unfavorable manner with regards to foreclosure procedures.

  2. My spouse and i ended up being really glad Raymond managed to deal with his studies by way of the precious recommendations he made through the site. It’s not at all simplistic just to possibly be freely giving key points that many other folks could have been selling. And we also recognize we need the writer to thank for this. All of the illustrations you made, the simple web site navigation, the friendships you can make it possible to engender – it is all awesome, and it’s making our son and us reason why that matter is thrilling, and that’s particularly serious. Many thanks for all!

  3. Thank you so much for giving everyone remarkably brilliant possiblity to discover important secrets from this website. It is always so lovely and jam-packed with a lot of fun for me personally and my office fellow workers to visit your site more than 3 times per week to read through the latest things you will have. And definitely, I am also actually impressed with your eye-popping secrets you serve. Certain 2 areas on this page are certainly the best we have had.

  4. I have to show some thanks to this writer just for rescuing me from this predicament. Because of researching through the internet and coming across concepts that were not helpful, I figured my entire life was gone. Being alive without the presence of answers to the problems you’ve solved by way of your good write-up is a serious case, as well as the kind that would have badly damaged my career if I had not encountered the blog. Your expertise and kindness in touching all the things was useful. I am not sure what I would’ve done if I hadn’t come across such a thing like this. I am able to at this time look ahead to my future. Thanks a lot so much for your specialized and sensible guide. I will not hesitate to endorse the sites to anybody who requires counselling about this situation.

  5. I and my guys were actually reviewing the nice recommendations found on your web page and so unexpectedly I had a terrible suspicion I had not thanked the web site owner for those tips. Most of the ladies appeared to be joyful to learn them and have actually been making the most of these things. Thanks for simply being well kind and for pick out variety of helpful subjects millions of individuals are really eager to be aware of. My honest regret for not expressing gratitude to you sooner.

  6. Needed to put you that little remark to help give many thanks as before for all the pretty methods you’ve shown above. It has been simply pretty generous with people like you to deliver unreservedly all some people would’ve made available for an electronic book to earn some money for their own end, especially considering that you could have done it in the event you wanted. Those pointers also acted to provide a fantastic way to recognize that many people have the identical dreams the same as mine to know good deal more when it comes to this issue. I am certain there are many more pleasurable moments ahead for people who look over your site.

  7. I and my buddies appeared to be reading the excellent information and facts on your web blog and all of a sudden I got a horrible suspicion I had not expressed respect to the blog owner for those strategies. Most of the women became absolutely happy to learn all of them and have now without a doubt been making the most of these things. Appreciation for simply being indeed accommodating and also for deciding upon varieties of ideal subjects most people are really desperate to understand about. Our own sincere apologies for not saying thanks to you sooner.

  8. Needed to draft you the very little remark just to thank you so much over again with the stunning guidelines you have provided at this time. This is simply unbelievably generous of you to present publicly just what most people could possibly have made available for an e-book in order to make some dough for themselves, most notably considering that you might well have tried it if you ever decided. These strategies as well worked to be the fantastic way to understand that the rest have a similar dreams just like my own to find out a great deal more on the subject of this problem. I’m sure there are millions of more fun opportunities in the future for people who scan through your site.

  9. I not to mention my guys were found to be examining the best tactics on the website and so quickly I had an awful suspicion I had not expressed respect to the web site owner for them. The young men were for that reason excited to learn all of them and now have sincerely been using these things. We appreciate you truly being very kind and for picking out certain smart information millions of individuals are really eager to discover. My personal sincere apologies for not expressing appreciation to you sooner.

  10. I would like to convey my respect for your generosity for people that need help on the theme. Your personal commitment to passing the message up and down came to be unbelievably helpful and has continuously empowered men and women much like me to reach their dreams. Your insightful tips and hints means much a person like me and far more to my mates. Regards; from each one of us.

  11. My wife and i were so thrilled that Ervin could carry out his investigations using the ideas he had in your web pages. It’s not at all simplistic just to choose to be handing out points which usually some others might have been selling. And we also already know we now have the website owner to appreciate for that. The specific illustrations you have made, the simple website menu, the friendships you will aid to create – it’s got everything fantastic, and it’s really aiding our son in addition to the family know that this subject is brilliant, which is incredibly mandatory. Thank you for all!

  12. I precisely needed to thank you so much all over again. I am not sure what I would’ve sorted out without these concepts shown by you regarding my topic. It previously was a real frustrating case for me, nevertheless taking note of a skilled manner you solved it forced me to jump over gladness. Extremely happier for the advice and as well , have high hopes you are aware of an amazing job your are providing educating most people thru your websites. Probably you’ve never met all of us.

  13. Thank you a lot for giving everyone a very marvellous opportunity to discover important secrets from this blog. It is always so terrific plus stuffed with amusement for me personally and my office mates to search your site nearly three times a week to see the new things you have got. And of course, we are certainly impressed for the eye-popping points served by you. Selected 3 points on this page are undoubtedly the most beneficial I’ve had.

  14. I must express my thanks to you for rescuing me from this instance. Right after scouting through the world wide web and getting basics that were not productive, I assumed my entire life was over. Existing devoid of the solutions to the issues you have resolved as a result of your article is a crucial case, as well as those that could have in a wrong way damaged my career if I had not discovered your blog post. The understanding and kindness in playing with all the things was valuable. I’m not sure what I would’ve done if I hadn’t discovered such a subject like this. I am able to at this moment look forward to my future. Thanks for your time very much for the high quality and effective guide. I will not be reluctant to refer the website to anybody who would need guide about this topic.

  15. I wanted to construct a brief comment to express gratitude to you for all of the wonderful tips you are placing at this website. My prolonged internet investigation has now been compensated with high-quality ideas to exchange with my contacts. I ‘d admit that most of us site visitors actually are undeniably lucky to be in a really good website with so many lovely individuals with interesting basics. I feel somewhat fortunate to have come across your webpage and look forward to tons of more pleasurable minutes reading here. Thanks a lot once again for everything.

  16. My husband and i have been very happy John could deal with his research while using the ideas he got when using the blog. It’s not at all simplistic just to possibly be giving for free guidance which usually some other people may have been selling. We really know we need the writer to give thanks to because of that. All of the illustrations you have made, the straightforward website navigation, the friendships your site help engender – it is most astounding, and it is aiding our son and our family reason why that subject is amusing, which is certainly unbelievably vital. Thanks for everything!

  17. I just wanted to compose a simple word to appreciate you for all the precious facts you are sharing at this website. My time consuming internet research has at the end of the day been honored with sensible insight to go over with my co-workers. I would repeat that many of us website visitors actually are undeniably fortunate to be in a fantastic community with many outstanding professionals with valuable secrets. I feel somewhat grateful to have seen your webpage and look forward to really more thrilling times reading here. Thanks once again for a lot of things.

  18. Thanks for all of the hard work on this web page. My aunt loves working on investigation and it’s really obvious why. A lot of people notice all concerning the compelling medium you deliver powerful suggestions by means of your website and even recommend contribution from the others on that matter and my girl is without question understanding so much. Enjoy the rest of the new year. You have been carrying out a great job.

  19. Thank you a lot for providing individuals with an exceptionally remarkable possiblity to check tips from this blog. It is always so pleasant plus packed with a great time for me and my office mates to visit your website minimum 3 times per week to find out the latest issues you have. And of course, I’m actually astounded with all the good creative concepts you give. Selected 3 areas in this posting are surely the most impressive I have ever had.

  20. I and my guys happened to be going through the nice tactics from your web site then quickly developed a horrible feeling I never expressed respect to the site owner for those tips. Those people were for this reason joyful to see them and now have in actuality been tapping into them. Appreciate your really being well accommodating and for utilizing certain ideal subjects millions of individuals are really wanting to be informed on. My sincere regret for not saying thanks to you earlier.

  21. Thanks for all of your labor on this web site. My mum takes pleasure in participating in investigations and it’s really simple to grasp why. All of us learn all concerning the dynamic way you give very useful solutions by means of your web blog and in addition boost participation from visitors on this area plus our child is in fact discovering a lot. Enjoy the remaining portion of the new year. You’re the one doing a stunning job.

  22. I enjoy you because of your whole work on this website. My mother really likes participating in investigations and it’s obvious why. Many of us know all of the lively form you render informative tips through this website and even attract response from other ones on that issue while our own princess is always learning a great deal. Take advantage of the rest of the new year. You’re the one carrying out a superb job.

  23. Needed to post you a very little observation to be able to give many thanks the moment again for the amazing pointers you’ve shared on this page. It is simply particularly generous of people like you to allow openly what exactly a number of people might have offered as an ebook to help with making some cash for themselves, specifically since you might well have tried it in case you considered necessary. These smart ideas as well acted to be a good way to fully grasp that the rest have the same zeal much like mine to see a little more in regard to this issue. I am certain there are numerous more pleasurable instances ahead for those who see your website.

  24. I intended to post you a little note in order to thank you so much the moment again for all the great ideas you’ve contributed in this case. It was really strangely generous of people like you to deliver unreservedly precisely what numerous people would’ve offered for sale as an e-book in making some bucks for their own end, mostly seeing that you could have done it in the event you considered necessary. The advice additionally acted to be the easy way to be sure that other individuals have the same dreams just like my personal own to learn more with regards to this problem. I know there are some more fun times ahead for people who browse through your website.

  25. I needed to draft you that little remark to help give thanks the moment again regarding the fantastic pointers you have discussed in this article. This is simply unbelievably open-handed with people like you to make easily all that a lot of folks could possibly have advertised as an e book to help with making some bucks on their own, and in particular considering the fact that you might have done it if you considered necessary. These advice also served to provide a easy way to recognize that someone else have similar keenness just as my own to see somewhat more with regard to this matter. Certainly there are lots of more fun periods in the future for those who browse through your website.

  26. I would like to point out my affection for your kindness in support of all those that have the need for help with this particular area of interest. Your special dedication to getting the message along became remarkably interesting and has in most cases empowered workers much like me to reach their dreams. Your new useful publication can mean a whole lot to me and substantially more to my office workers. Thanks a ton; from everyone of us.

  27. I definitely wanted to write down a brief comment to appreciate you for all of the fabulous tips and tricks you are placing at this website. My rather long internet search has at the end of the day been compensated with awesome suggestions to share with my guests. I would believe that we readers are rather blessed to be in a decent site with so many brilliant individuals with valuable opinions. I feel extremely fortunate to have encountered your website page and look forward to really more cool minutes reading here. Thanks a lot once more for all the details.

  28. I wish to show some thanks to you for bailing me out of this particular crisis. Just after looking out through the the net and getting recommendations which were not powerful, I figured my life was gone. Living without the presence of solutions to the difficulties you’ve fixed by way of this site is a serious case, as well as the ones which may have adversely affected my career if I hadn’t encountered your blog post. Your own personal expertise and kindness in controlling the whole thing was precious. I am not sure what I would’ve done if I had not come across such a point like this. I am able to now relish my future. Thanks for your time very much for the impressive and effective guide. I won’t hesitate to endorse your site to anyone who requires tips on this problem.

  29. My wife and i ended up being cheerful John could finish up his research with the precious recommendations he was given in your blog. It is now and again perplexing to simply happen to be handing out thoughts the others may have been selling. And we all keep in mind we’ve got the blog owner to give thanks to for that. The illustrations you have made, the easy web site menu, the friendships your site assist to engender – it is all overwhelming, and it’s making our son in addition to our family imagine that this subject is cool, which is highly essential. Thank you for everything!

Leave a Reply

Your email address will not be published. Required fields are marked *