Supporting multiple providers

One of the key uses of Terraform is to deploy development and production systems. Terraform can be used to manage what is deployed, manage resources, and restrict resources available to an instance. In our last blog entry we looked at the vSphere provider and looked at some of they key parameters that are needed to deploy solutions into this virtual environment.

In a perfect world we should be able to develop definitions to deploy development systems to a small or older system, deploy production to a more expensive and powerful vSphere cluster, and a disaster recovery copy to make sure that we can failover to an alternate datacenter in times of emergency. We should then be able to take the data for this provider and move it to Amazon AWS or Microsoft Azure or Google GCP by just changing the provider. Unfortunately, this is not a perfect world and there are a ton of reasons that this won’t work.

If we look at the documentation for the AWS provider we note that we don’t need a username and password or IP address but rather need a public and private key to connect to an AWS serviced and these parameters can be provided by command line environment variables. We can also define multiple providers and give an alias for the multiple providers and deploy services into different accounts, regions, and zones based on the terraform provider definition.

A typical aws provider main.tf file looks like…

provider “aws” {
version = “> 2”
profile = “default”
region = var.dev_location
alias = “dev”
}

provider “aws” {
version = “> 2”
profile = “default”
region = “us-west-1”
alias = “prod”
}

allowing you to deploy resources into “aws.dev” or “aws.prod” with a variable.tf file containing nothing or

variable “createdby” {
type = string
default = “TechEnablement”
}

variable “environment” {
type = string
default = “TechEnablement”
}

variable “dev_location” {
type = string
default = “us-east-1”
}

With this variable.tf definition you need to define environment variables to define the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY or define shared_credentials_file in a terraform configuration file to point to the location of a key file. On Windows this is typically “%USERPROFILE%\.aws\credentials”. The format of the credentials file looks like

[default]

aws_access_key_id=AWSSAMPLE7EXAMPLE

aws_secret_access_key=long/keywith/numbers4&letters

Unfortunately the vSphere provider does not allow for an alias tag and use of different account credentials and vsphere host address. Rather than defining multiple providers you need to define different directories and different variable.tf and main.tf files for each of the environments. In our earlier example we would have a dev, prod, and dr folder under our main folder. Each folder would have terraform configuration files to define what each environment would look like and resources available.

A typical multi-environment tree would look different from out initial single tree deployment with a dev, prod, and dr folder each containing the same main.tf files but different variable.tf definitions. Each folder would have their own terraform.tfstate file as well given that there are different environment variables and states on different servers.

If you try to define multiple vsphere providers in one file you get the error

Given the differences between the two provider types it begs the question of changing the aws provider to the same file format as the vsphere provider and have three different folders that deploy different environments to different servers. This would work but having everything in one file reduces complexity and potential errors by having multiple copies in multiple folders. Editing one does not guarantee changes to the other directories and there might be subtle differences between the different environments, like datastore names or locations as well as network definitions, that are unique to each environment.

In summary, there are multiple ways of solving the same problem. The ultimate solution is to write a generic provider that can deploy services into vSphere, Hyper-V, Nutanix, other on-premises virtual machine hosts, AWS, Azure, Google GCP, and other cloud virtual machine hosts. Given that there is no generic provider that works across all or even multiple environments you have to decide how to deploy multiple terraform configuration files to multiple target locations without doubling or tripling your work and code that needs support and maintenance. My recommendation is to go with different folders for different environments and have different variable.tf and main.tf files in each folder.

390 thoughts on “Supporting multiple providers”

  1. Thanks a lot for giving everyone such a special chance to discover important secrets from here. It is often so nice and full of fun for me and my office colleagues to search your web site no less than 3 times per week to study the new issues you have. Not to mention, we are at all times amazed with all the staggering knowledge you serve. Certain 3 tips in this post are in reality the finest I have had.

  2. I intended to send you the little bit of note to help give thanks again relating to the lovely methods you have featured on this page. It has been quite particularly generous of you to convey openly all that some people would have distributed for an electronic book to end up making some profit for their own end, and in particular given that you could have tried it if you wanted. Those suggestions likewise served as the easy way to understand that the rest have the same dream like my personal own to realize a good deal more in regard to this issue. I am sure there are lots of more enjoyable sessions up front for those who browse through your site.

  3. I and also my pals were reading through the good advice located on your web page while quickly I had an awful suspicion I never expressed respect to the web site owner for those strategies. My ladies happened to be for this reason stimulated to read all of them and already have quite simply been taking advantage of those things. I appreciate you for genuinely really kind as well as for selecting some useful tips most people are really wanting to be informed on. Our own sincere regret for not expressing appreciation to earlier.

  4. I and also my buddies were checking the good secrets located on your site while suddenly I got an awful feeling I had not thanked the site owner for those techniques. All the guys are actually stimulated to see all of them and have definitely been enjoying those things. Appreciation for genuinely really kind and then for picking this form of useful tips most people are really desperate to be aware of. My honest regret for not expressing gratitude to you earlier.

  5. I have to show thanks to this writer for bailing me out of such a trouble. After browsing through the search engines and seeing things that were not pleasant, I was thinking my entire life was done. Being alive without the answers to the issues you have resolved by means of your short article is a crucial case, as well as the kind that could have badly damaged my entire career if I had not come across your site. Your own knowledge and kindness in maneuvering all the things was priceless. I don’t know what I would have done if I had not discovered such a stuff like this. I am able to now look ahead to my future. Thanks a lot very much for the high quality and result oriented help. I won’t think twice to recommend your web sites to any individual who would need recommendations about this situation.

  6. Thanks a lot for providing individuals with such a pleasant chance to discover important secrets from this site. It is usually so brilliant plus jam-packed with a lot of fun for me personally and my office acquaintances to search your website at the least 3 times in 7 days to read the fresh things you have got. And definitely, we are at all times happy concerning the astounding tips and hints you serve. Certain 2 areas in this post are honestly the most effective I have had.

  7. My wife and i were quite happy Louis could finish up his reports through your ideas he got in your weblog. It is now and again perplexing to just happen to be giving for free helpful hints which some people might have been trying to sell. So we figure out we have got the writer to thank because of that. The explanations you made, the easy site menu, the friendships you will make it easier to engender – it is everything fantastic, and it’s letting our son in addition to our family feel that the content is awesome, which is unbelievably mandatory. Thank you for everything!

  8. I precisely desired to thank you very much yet again. I am not sure the things that I might have used without these points revealed by you relating to my subject. It previously was a very scary concern in my circumstances, but seeing a specialized fashion you processed that took me to jump with gladness. Now i am happier for your service and thus sincerely hope you recognize what an amazing job you were carrying out instructing people thru your blog post. I know that you’ve never met all of us.

  9. Thank you for your whole labor on this site. My mother delights in conducting investigation and it’s really easy to see why. Many of us hear all about the compelling manner you provide sensible suggestions on your blog and as well as foster contribution from people on the issue so our favorite simple princess is now studying a great deal. Take pleasure in the remaining portion of the new year. You’re the one performing a first class job.

  10. I must express some thanks to this writer for bailing me out of this particular problem. Right after browsing throughout the world wide web and finding tricks which were not productive, I figured my life was done. Living without the presence of strategies to the problems you have solved by means of your good website is a critical case, and ones which might have adversely damaged my entire career if I hadn’t noticed your web site. The capability and kindness in controlling all things was very helpful. I don’t know what I would’ve done if I had not come upon such a thing like this. I’m able to at this time look forward to my future. Thanks so much for this specialized and sensible help. I won’t think twice to propose your blog to any person who would like tips about this topic.

  11. Needed to put you a very little word to be able to thank you very much as before for these marvelous techniques you have shown in this article. It has been so particularly generous with people like you in giving freely all that numerous people might have offered for an e-book to make some dough for their own end, even more so given that you might well have done it if you decided. The tactics also acted as a great way to realize that someone else have the same dream similar to mine to learn a good deal more with regards to this issue. I am certain there are several more fun situations in the future for individuals who view your blog post.

  12. I wish to express my love for your kind-heartedness in support of men and women who actually need help with your issue. Your very own dedication to getting the solution all through ended up being exceptionally beneficial and has surely allowed workers just like me to achieve their dreams. Your own interesting publication denotes so much to me and even more to my mates. Thanks a ton; from each one of us.

  13. I not to mention my guys happened to be checking out the excellent things on the blog and so the sudden developed a horrible feeling I had not thanked the site owner for those techniques. Those women were so joyful to read all of them and already have very much been loving them. Many thanks for actually being really considerate as well as for using such important subject areas millions of individuals are really wanting to learn about. Our sincere apologies for not expressing gratitude to you sooner.

  14. I simply wanted to write down a quick message in order to express gratitude to you for these unique tips and hints you are placing here. My prolonged internet investigation has finally been rewarded with high-quality facts and techniques to share with my colleagues. I would admit that we website visitors actually are undoubtedly fortunate to live in a superb community with many lovely individuals with useful basics. I feel somewhat fortunate to have discovered your entire website page and look forward to many more amazing minutes reading here. Thanks a lot again for everything.

  15. Thanks a lot for giving everyone such a special opportunity to read in detail from this website. It is always so great plus stuffed with a good time for me personally and my office colleagues to search the blog at a minimum 3 times in 7 days to study the latest tips you have. Of course, I’m so actually happy for the special methods served by you. Selected 4 tips in this posting are completely the most suitable we’ve had.

  16. Pingback: 2drainage
  17. I must get across my gratitude for your generosity for those people that absolutely need help on this important subject. Your personal dedication to passing the solution throughout came to be wonderfully practical and have all the time empowered most people like me to get to their objectives. Your entire invaluable instruction entails a great deal to me and additionally to my office colleagues. Thanks a lot; from everyone of us.

  18. I wish to express my passion for your kind-heartedness supporting people who absolutely need assistance with this particular concept. Your real commitment to getting the message up and down had become surprisingly productive and have truly made guys just like me to arrive at their desired goals. Your amazing helpful facts indicates a whole lot a person like me and a whole lot more to my mates. Thanks a lot; from each one of us.

Leave a Reply

Your email address will not be published. Required fields are marked *