Page MenuHomePhabricator

Extend Tofu provider to allow configuration via environment variables
Open, LowPublic

Description

Background

With the main OpenStack provider, it's possible to define it something like this:

terraform {
  required_providers {
    openstack = {
      source  = "terraform-provider-openstack/openstack"
      version = "~> 1.48.0"
    }
  }

  required_version = ">= 1.3.0"
}

provider "openstack" {
  tenant_name                   = var.project
}

The remaining settings can be imported via environment variables, simply by downloading the OpenRC file for the application credentials from Horizon, and sourcing it into your environment prior to running Terraform. That OpenRC file looks something like this:

#!/usr/bin/env bash

export OS_AUTH_TYPE=v3applicationcredential
export OS_AUTH_URL=https://openstack.eqiad1.wikimediacloud.org:25000/v3
export OS_IDENTITY_API_VERSION=3
export OS_REGION_NAME="eqiad1-r"
export OS_INTERFACE=public
export OS_APPLICATION_CREDENTIAL_ID=[snip]
export OS_APPLICATION_CREDENTIAL_SECRET=[snip]

Since Terraform variables which are marked as sensitive=true only protects them from being displayed accidentally. While I'd hope that variables defining provider credentials aren't re-used elsewhere in a Terraform configuration and thus won't find their way into state, it's not guaranteed.

By forcing the credentials to be loaded dynamically, it avoids the issue of either forcing the user to specify the credentials at runtime (via -var, -var-file, or being prompted), or storing them in a *.tfvars file (or worse, in the configuration itself) which might get accidentally committed to source control. Using the OpenRC file also makes it much easier to set short-lived credentials which can be easily rotated by simply downloading and sourcing a new file into their shell.

The ask

Please can the Wikimedia CloudVPS provider *also* read from the OS_* environment variables if the values are not defined directly in Terraform configuration, in the same way that the existing OpenStack provider can?

Event Timeline

nskaggs renamed this task from Extend TF provider to allow configuration via environment variables to Extend TF (terraform) provider to allow configuration via environment variables.Oct 28 2022, 3:16 PM
taavi removed taavi as the assignee of this task.Oct 28 2022, 5:29 PM
taavi triaged this task as Low priority.

This is a reasonable feature request, although the new Terraform provider SDK (terraform-plugin-framework) that terraform-cloudvps uses doesn't seem to have an easy way to do it yet.

taavi renamed this task from Extend TF (terraform) provider to allow configuration via environment variables to Extend Tofu provider to allow configuration via environment variables.Sep 28 2024, 1:33 PM

I've got an implementation of this working.

Using the following OpenTofu code:

main.tf
terraform {
  required_providers {
    cloudvps = {
        source = "terraform.wmcloud.org/registry/cloudvps"
    }
  }
}

provider "cloudvps" {}

resource "cloudvps_web_proxy" "debug" {
    hostname = "accounts-provider-debug"
    domain = "wmcloud.org"
    backends = ["http://172.16.6.17:80"]
}

And OpenTofu configured to use my local system as the provider source (this is the cause of the warnings below):

~/.terraformrc
provider_installation {
  dev_overrides {
    "terraform.wmcloud.org/registry/cloudvps" = "/home/stwalkerster/go/bin"
  }

  direct {}
}

OpenTofu produces this output:

bash $ tofu plan
╷
│ Warning: Provider development overrides are in effect
│ 
│ The following provider development overrides are set in the CLI configuration:
│  - terraform.wmcloud.org/registry/cloudvps in /home/stwalkerster/go/bin
│ 
│ The behavior may therefore not match any released version of the provider and applying changes may cause the state to become incompatible with published releases.
╵

Planning failed. OpenTofu encountered an error while generating this plan.

╷
│ Error: OpenStack Auth URL not set
│ 
│   with provider["terraform.wmcloud.org/registry/cloudvps"],
│   on main.tf line 9, in provider "cloudvps":
│    9: provider "cloudvps" {}
│ 
│ The provider is unable to determine the OpenStack Auth URL. Please configure this setting in the provider block or by using the OS_AUTH_URL environment variable.
╵
╷
│ Error: OpenStack Project ID not set
│ 
│   with provider["terraform.wmcloud.org/registry/cloudvps"],
│   on main.tf line 9, in provider "cloudvps":
│    9: provider "cloudvps" {}
│ 
│ The provider is unable to determine the OpenStack Project ID. Please configure this setting in the provider block or by using the OS_PROJECT_ID environment variable.
╵
╷
│ Error: OpenStack Application Credential ID not set
│ 
│   with provider["terraform.wmcloud.org/registry/cloudvps"],
│   on main.tf line 9, in provider "cloudvps":
│    9: provider "cloudvps" {}
│ 
│ The provider is unable to determine the OpenStack Application Credential ID. Please configure this setting in the provider block or by using the OS_APPLICATION_CREDENTIAL_ID environment variable.
╵
╷
│ Error: OpenStack Application Credential Secret not set
│ 
│   with provider["terraform.wmcloud.org/registry/cloudvps"],
│   on main.tf line 9, in provider "cloudvps":
│    9: provider "cloudvps" {}
│ 
│ The provider is unable to determine the OpenStack Application Credential Secret. Please configure this setting in the provider block or by using the OS_APPLICATION_CREDENTIAL_SECRET environment variable.
╵

However, running the following set of commands now allows the plan to execute successfully:

bash $ export OS_APPLICATION_CREDENTIAL_ID="e62...[snip]...30f"
bash $ export OS_APPLICATION_CREDENTIAL_SECRET="zB1...[snip]...NBA"
bash $ export OS_AUTH_URL="https://openstack.eqiad1.wikimediacloud.org:25000/v3"
bash $ export OS_PROJECT_ID=account-creation-assistance
bash $ tofu plan
╷
│ Warning: Provider development overrides are in effect
│ 
│ The following provider development overrides are set in the CLI configuration:
│  - terraform.wmcloud.org/registry/cloudvps in /home/stwalkerster/go/bin
│ 
│ The behavior may therefore not match any released version of the provider and applying changes
│ may cause the state to become incompatible with published releases.
╵
cloudvps_web_proxy.debug: Refreshing state...

No changes. Your infrastructure matches the configuration.

OpenTofu has compared your real infrastructure against your configuration and found no
differences, so no changes are needed.

I don't have permissions to even push to a new branch of the GitLab repo in order to submit a merge request for this. I've uploaded the patch here instead:

Never mind, I'm being silly. https://gitlab.wikimedia.org/repos/cloud/cloud-vps/terraform-cloudvps/-/merge_requests/2