Terraform data vs resources

In previous blog posts we have talked about Installing Terraform on Windows, Installing Terraform on Ubuntu, Providers part 1 and Providers part 2. In this blog we will look at differentiating between data sources and resources. Note that we are looking at terraform constructs differently from what is presented in the HashiCorp Terraform Associate Certification and not following the suggested study guide format but diving into what is required to build something from scratch.

The assumptions that we are going to make in this blog are that first we have a terraform binary installed on Windows or Ubuntu and second that our target system is going to be vSphere initially either for development or in house production systems.

Given that we talked about the vSphere provider in an earlier blog posting we won’t go into a discussion on adding and configuring providers other than to say that vSphere is a tricky configuration when it comes to managing multiple servers. If we were managing multiple Amazon or Azure regions or zones we would define one provider for one zone and another provider with a different alias for the second zone. The vSphere provider does not support multiple datacenters or aliases for providers so managing multiple hosts with terraform must be done with one vSphere server controlling multiple hosts or multiple folders and directories managing different vSphere servers. Both of these strategies work well. For this example we will look at multiple servers under one vSphere and a single terraform configuration to manage all of these hosts.

Before we get started on data vs resource definitions it is important to note that the Terraform Associate certification focuses on Cloud Engineering and not necessarily on-premises servers. The concepts are generic enough but for this blog we will cover vSphere as our example target.

Scrolling down to the outline of the exam note that “Create and differentate resource and data configuration” is one of the topics in the Read, generate, and modify configuration section.

If we look at the documentation for the Terraform CLI for Data Sources it notes that data sources allow data to be fetched or computed for use elsewhere in Terraform configuration. Each provider may offer data sources alongside its set of resource types. What this means is that we can declare things that we know exist and should not change. For vSphere we know that there is a datacenter as well as datastore. The datacenter is the root of the vSphere environment. Under the datacenter is a host and a datastore. If we correlate this back to our vSphere console we can pull this data from the user interface.

From our lab environment we log into the vSphere Client and go to the Summary screen of the root vSphere instance.

Note that we have 4 hosts displayed on the left side, 4 hosts listed in the center as well as 23 virtual machines. We do have a number of templates and virtual machines listed under our server 10.0.0.92 with two of the virtual machines currently running.

For our Terraform vSphere provider we need to first get the name of the Datacenter associated with this installation. We can do this by clicking on the Datacenters tab on the right side of the screen.

For this example we see that “Home-lab” is the Datacenter defined for our instance. In Terraform we would define this with a data element rather than a resource.

data "vsphere_datacenter" "datacenter" {
  name = "Home-lab"
}

The key reason that we use data rather than resource is that the terraform command will try to create this datacenter when an apply is run and destroy the datacenter when a destroy is run. We don’t want to destroy our datacenter but want to retain the definition to use as needed to access hosts and virtual machines.

From the Resources definition in the Terraform CLI, a resource describes one or more infrastructure object and running the apply command creates, updates, or destroys the object to force them to match the configuration in the configuration files. If your configuration gets changed then the terraform configuration tries to delete and reconfigure your datacenter. This is not the behavior that we want. What we want is to define everything that is static and should not change with the data declaration and things that are dynamic with the resource declaration. The elements that typically don’t change for vSphere are:

  • Datacenter
  • Host
  • Network
  • Datastore

Things that typically change but are good to use data declarations for are

  • Folders
  • Templates
  • Tags

Things that typically change are need a resource definition are

  • Virtual Machines
  • VMFS Disks
  • Roles
  • Users

To reference a data element you start with the data.<value> and use the type definition followed by the alias name. In the following example we define a host that is associated with a compute cluster that is inside a datacenter. We refer to the datacenter as data.vsphere_datacenter.dc and the cluster as data.vsphere_compute_cluster.c1 where “dc” and “c1” are aliases to identify the data definitions. We use the id parameter in the data construct to uniquely identify the resource.

data "vsphere_datacenter" "dc" {
  name = "TfDatacenter"
}

data "vsphere_compute_cluster" "c1" {
  name          = "DC0_C0"
  datacenter_id = data.vsphere_datacenter.dc.id
}

resource "vsphere_host" "h1" {
  hostname = "10.10.10.1"
  username = "root"
  password = "password"
  license  = "00000-00000-00000-00000i-00000"
  cluster  = data.vsphere_compute_cluster.c1.id
}

In the above example we are talking about the host “10.10.10.1” located in the compute cluster “DC0_C0” that lives in the TfDatacenter. In our Home-lab datacenter we can define our host 10.0.0.92 as a member of the Home-lab datacenter and provision virtual machines onto this host.

We would also define our vsphere_datastore defined by “name” that lives in the data.vsphere_datacenter.dc. To view the datastores from the vSphere Client, click on the Datastores tab or the disk icon to get a list of potential datastores.

Note that some of the datastores are listed as inacessible. These servers are currently turned off and the accessible ones are connected to our primary host and vSphere server which is the only one powered on for demonstration purposes.

In summary, data declarations should be used for static and permanent structures. Resource declarations should be used for things that can be created or considered transient or malleable. For a production datacenter that I managed we defined folders, virtual machine templates, and datastores with the data declaration.

data “vsphere_datacenter” “dc” {
name = “QM Lab”
}

data “vsphere_resource_pool” “pool” {
name = “TintonFalls/Resources”
datacenter_id = data.vsphere_datacenter.dc.id
}

data “vsphere_host” “host” {
name = “172.19.21.54”
datacenter_id = data.vsphere_datacenter.dc.id
}

data “vsphere_datastore” “DS6” {
name = “DS6”
datacenter_id = data.vsphere_datacenter.dc.id
}

data “vsphere_datastore” “EQL1-Raid5” {
name = “EQL1-Raid5”
datacenter_id = data.vsphere_datacenter.dc.id
}

data “vsphere_datastore” “EQL2-Raid6” {
name = “EQL2-Raid6”
datacenter_id = data.vsphere_datacenter.dc.id
}

data “vsphere_datastore” “ADV-ESX4-maglib1” {
name = “ADV-ESX4 maglib1(7.2K)”
datacenter_id = data.vsphere_datacenter.dc.id
}

data “vsphere_datastore” “ADV-ESX4-maglib2” {
name = “ADV-ESX4 maglib2(7.2K)”
datacenter_id = data.vsphere_datacenter.dc.id
}

data “vsphere_datastore” “ADV-ESX4storage1” {
name = “ADV-ESX4 storage1(15K)”
datacenter_id = data.vsphere_datacenter.dc.id
}

data “vsphere_datastore” “ADV-ESX4storage2” {
name = “ADV-ESX4 storage2(15K)”
datacenter_id = data.vsphere_datacenter.dc.id
}

data “vsphere_datastore” “ADV-ESX4storage3” {
name = “ADV-ESX4 storage3(15K)”
datacenter_id = data.vsphere_datacenter.dc.id
}

data “vsphere_datastore” “ADV-ESX4storage4” {
name = “ADV-ESX4 storage4(15K)”
datacenter_id = data.vsphere_datacenter.dc.id
}

data “vsphere_datastore” “ADV-ESX4storage5” {
name = “ADV-ESX4 storage5(15K)”
datacenter_id = data.vsphere_datacenter.dc.id
}

data “vsphere_datastore” “corenfsshare” {
name = “corenfsshare”
datacenter_id = data.vsphere_datacenter.dc.id
}

data “vsphere_network” “network” {
name = “VM Network”
datacenter_id = data.vsphere_datacenter.dc.id
}

data “vsphere_virtual_machine” “W2k8-sp18-template” {
name = “W2k8-sp18-template”
datacenter_id = data.vsphere_datacenter.dc.id
}

data “vsphere_virtual_machine” “Windows2016-template” {
name = “2016_template”
datacenter_id = data.vsphere_datacenter.dc.id
}

data “vsphere_virtual_machine” “W2k16-Aug-2020-template” {
name = “W2k16-Aug-2020-template”
datacenter_id = data.vsphere_datacenter.dc.id
}

data “vsphere_virtual_machine” “centos-base” {
name = “centos-base”
datacenter_id = data.vsphere_datacenter.dc.id
}

data “vsphere_virtual_machine” “rhel-base” {
name = “rhel-base”
datacenter_id = data.vsphere_datacenter.dc.id
}

data “vsphere_virtual_machine” “suse-sap-base” {
name = “suse-sap-base”
datacenter_id = data.vsphere_datacenter.dc.id
}

data “vsphere_virtual_machine” “te-kubernetes-base” {
name = “te-kubernetes-base”
datacenter_id = data.vsphere_datacenter.dc.id
}

data “vsphere_virtual_machine” “ubuntu-base” {
name = “ubuntu-base”
datacenter_id = data.vsphere_datacenter.dc.id
}

data “vsphere_virtual_machine” “ubuntu-docker-desktop” {
name = “ubuntu-docker-desktop”
datacenter_id = data.vsphere_datacenter.dc.id
}

data “vsphere_virtual_machine” “Windows-10-template” {
name = “Windows-10-template”
datacenter_id = data.vsphere_datacenter.dc.id
}

data “vsphere_folder” “parent” {

path = “${data.vsphere_datacenter.dc.name}/vm/Technical Enablement”
}

data “vsphere_folder” “Apps” {
path = “${data.vsphere_folder.parent.path}/Apps”
}

data “vsphere_folder” “testing” {
path = “${data.vsphere_folder.Apps.path}/testing”
}

data “vsphere_folder” “sp22” {
path = “${data.vsphere_folder.Apps.path}/sp22”
}

data “vsphere_folder” “sp21” {
path = “${data.vsphere_folder.Apps.path}/sp21”
}

data “vsphere_folder” “sp20” {
path = “${data.vsphere_folder.Apps.path}/sp20”
}

389 thoughts on “Terraform data vs resources”

  1. Hi! I could have sworn I’ve been to this site before but after checking through some of the post I realized it’s new to me. Anyways, I’m definitely glad I found it and I’ll be book-marking and checking back frequently!

  2. I have been absent for a while, but now I remember why I used to love this site. Thanks , I抣l try and check back more frequently. How frequently you update your website?

  3. Great items from you, man. I’ve take note your stuff prior to and you are simply too fantastic. I really like what you have bought right here, certainly like what you are saying and the way in which in which you assert it. You make it entertaining and you continue to care for to keep it sensible. I can not wait to learn far more from you. That is actually a wonderful website.

  4. Excellent blog right here! Also your web site rather a lot up fast! What host are you the usage of? Can I am getting your affiliate hyperlink to your host? I wish my website loaded up as quickly as yours lol

  5. Pretty nice post. I just stumbled upon your weblog and wanted to say that I’ve truly enjoyed browsing your weblog posts. After all I will be subscribing for your feed and I’m hoping you write once more very soon!

  6. I must express appreciation to the writer just for bailing me out of this particular difficulty. Right after searching through the the net and obtaining tricks which are not helpful, I believed my life was well over. Living without the answers to the difficulties you have sorted out by way of this report is a crucial case, and those which might have negatively affected my entire career if I hadn’t come across the blog. Your actual ability and kindness in dealing with every aspect was important. I am not sure what I would have done if I had not discovered such a point like this. It’s possible to now look forward to my future. Thanks for your time very much for this impressive and results-oriented help. I will not hesitate to suggest your web site to any individual who would like guidance on this issue.

  7. Hello, Neat post. There is an issue along with your web site in web explorer, might check this?IE still is the market leader and a good component of other folks will leave out your wonderful writing due to this problem.

  8. A lot of thanks for your whole effort on this blog. Debby really likes setting aside time for research and it is easy to see why. My partner and i notice all concerning the powerful form you deliver very helpful tactics through your web site and therefore attract participation from the others on this subject so our princess is undoubtedly starting to learn a lot. Take pleasure in the remaining portion of the new year. You’re carrying out a glorious job.

  9. I wish to express my gratitude for your kind-heartedness in support of men and women that actually need assistance with in this theme. Your personal commitment to passing the solution all through came to be really good and has all the time empowered girls much like me to get to their aims. Your important tips and hints implies a lot to me and still more to my colleagues. With thanks; from everyone of us.

  10. A lot of thanks for all your effort on this web site. My mum delights in conducting investigations and it’s easy to see why. I know all relating to the dynamic tactic you provide rewarding steps on this website and as well as foster contribution from people on that area of interest plus our favorite princess is becoming educated a whole lot. Take advantage of the remaining portion of the new year. You are always performing a first class job.

  11. I precisely had to say thanks once more. I am not sure the things I could possibly have sorted out without the actual aspects provided by you directly on such a subject matter. It was before a real frustrating problem in my view, but viewing your specialised mode you processed that took me to weep with fulfillment. I will be grateful for this service as well as hope that you know what a powerful job your are putting in instructing many others via your webpage. Most likely you’ve never got to know any of us.

  12. I wish to express some thanks to you just for bailing me out of this difficulty. As a result of checking through the world wide web and getting basics which are not pleasant, I was thinking my life was well over. Existing minus the approaches to the difficulties you’ve fixed by way of your good site is a serious case, and the kind which may have in a wrong way damaged my career if I had not discovered your blog post. That training and kindness in dealing with all the things was helpful. I’m not sure what I would have done if I hadn’t discovered such a point like this. I can also now look ahead to my future. Thanks very much for the professional and effective help. I won’t be reluctant to propose your web site to anybody who needs and wants support on this area.

  13. I wanted to post you the tiny note to help thank you so much as before for these great techniques you have featured in this article. It was quite wonderfully generous with you to make freely just what a lot of folks would’ve distributed as an e book to help make some profit for themselves, certainly since you could have done it in the event you considered necessary. The secrets likewise acted like a easy way to realize that some people have the identical desire just like my own to realize whole lot more on the topic of this condition. I’m sure there are numerous more enjoyable times up front for those who view your site.

  14. Needed to post you this little remark to finally thank you very much the moment again with the lovely tricks you have contributed above. It’s simply pretty generous of you to present freely all that many people might have sold for an e-book to generate some profit for their own end, especially since you might well have tried it if you desired. The basics additionally served to provide a good way to be certain that other people have a similar keenness much like my personal own to learn more and more concerning this issue. I know there are a lot more fun periods ahead for individuals who take a look at your blog.

  15. A lot of thanks for your whole effort on this website. My niece take interest in making time for investigation and it’s really easy to see why. Most people hear all concerning the powerful medium you deliver useful secrets through this web blog and in addition attract response from some others on that theme while our favorite child is actually learning a whole lot. Enjoy the remaining portion of the new year. You are carrying out a remarkable job.

  16. I precisely desired to say thanks all over again. I am not sure the things I would’ve accomplished without the type of information shared by you about my field. Certainly was a real troublesome scenario for me, but taking note of your skilled form you processed that took me to cry for delight. I’m just grateful for this support and in addition wish you are aware of an amazing job you happen to be providing instructing the mediocre ones with the aid of a site. Most likely you’ve never come across all of us.

  17. Thanks for each of your effort on this web page. Debby really likes getting into investigation and it’s really obvious why. All of us know all of the lively tactic you deliver valuable tips and hints on this web blog and even foster participation from people on the content while our simple princess has always been understanding a lot of things. Have fun with the remaining portion of the year. You have been carrying out a glorious job.

  18. I want to show thanks to this writer for rescuing me from such a crisis. Just after surfing around throughout the world wide web and finding things that were not productive, I was thinking my life was over. Being alive without the strategies to the difficulties you have fixed by way of the site is a crucial case, as well as those which might have in a negative way damaged my entire career if I hadn’t discovered your website. Your good ability and kindness in controlling all the things was very helpful. I don’t know what I would’ve done if I had not come across such a thing like this. It’s possible to at this point look ahead to my future. Thank you very much for the specialized and effective help. I will not be reluctant to refer the blog to any individual who should receive direction on this problem.

Leave a Reply

Your email address will not be published. Required fields are marked *