×

使用配置了默认集群选项的Terraform集群模板快速创建AWS上的Red Hat OpenShift服务(ROSA)集群。

下面描述的集群创建过程使用Terraform配置来准备具有以下资源的ROSA with HCP集群

  • 具有托管oidc-config配置的OIDC提供程序

  • 具有关联的AWS托管ROSA策略的必备IAM操作员角色

  • 具有关联的AWS托管ROSA策略的IAM账户角色

  • 创建ROSA with STS集群所需的所有其他AWS资源

Terraform概述

Terraform是一个基础设施即代码工具,它提供了一种方法来配置一次资源,并根据需要复制这些资源。Terraform通过使用声明性语言来完成创建任务。您声明您希望基础设施资源的最终状态是什么,Terraform将根据您的规格创建这些资源。

先决条件

要在您的Terraform配置中使用Red Hat云服务提供商,您必须满足以下先决条件

  • 您已安装AWS上的Red Hat OpenShift服务(ROSA)命令行界面(CLI)工具。

  • 您拥有您的离线Red Hat OpenShift集群管理器令牌

  • 您已安装Terraform版本1.4.6或更高版本。

  • 您已创建您的AWS帐户范围的IAM角色。

    特定的帐户范围IAM角色和策略提供ROSA支持、安装、控制平面和计算功能所需的STS权限。这包括帐户范围的操作员策略。有关AWS账户角色的更多信息,请参见其他资源。

  • 您拥有一个AWS账户关联的凭证,允许您创建资源。凭证已配置用于AWS提供商。请参见AWS Terraform提供商文档中的身份验证和配置部分。

  • 在运行Terraform的AWS IAM角色策略中,您至少具有以下权限。在AWS控制台中检查这些权限。

    Terraform的最低AWS权限
    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Sid": "VisualEditor0",
          "Effect": "Allow",
          "Action": [
            "iam:GetPolicyVersion",
            "iam:DeletePolicyVersion",
            "iam:CreatePolicyVersion",
            "iam:UpdateAssumeRolePolicy",
            "secretsmanager:DescribeSecret",
            "iam:ListRoleTags",
            "secretsmanager:PutSecretValue",
            "secretsmanager:CreateSecret",
            "iam:TagRole",
            "secretsmanager:DeleteSecret",
            "iam:UpdateOpenIDConnectProviderThumbprint",
            "iam:DeletePolicy",
            "iam:CreateRole",
            "iam:AttachRolePolicy",
            "iam:ListInstanceProfilesForRole",
            "secretsmanager:GetSecretValue",
            "iam:DetachRolePolicy",
            "iam:ListAttachedRolePolicies",
            "iam:ListPolicyTags",
            "iam:ListRolePolicies",
            "iam:DeleteOpenIDConnectProvider",
            "iam:DeleteInstanceProfile",
            "iam:GetRole",
            "iam:GetPolicy",
            "iam:ListEntitiesForPolicy",
            "iam:DeleteRole",
            "iam:TagPolicy",
            "iam:CreateOpenIDConnectProvider",
            "iam:CreatePolicy",
            "secretsmanager:GetResourcePolicy",
            "iam:ListPolicyVersions",
            "iam:UpdateRole",
            "iam:GetOpenIDConnectProvider",
            "iam:TagOpenIDConnectProvider",
            "secretsmanager:TagResource",
            "sts:AssumeRoleWithWebIdentity",
            "iam:ListRoles"
          ],
          "Resource": [
            "arn:aws:secretsmanager:*:<ACCOUNT_ID>:secret:*",
            "arn:aws:iam::<ACCOUNT_ID>:instance-profile/*",
            "arn:aws:iam::<ACCOUNT_ID>:role/*",
            "arn:aws:iam::<ACCOUNT_ID>:oidc-provider/*",
            "arn:aws:iam::<ACCOUNT_ID>:policy/*"
          ]
        },
        {
          "Sid": "VisualEditor1",
          "Effect": "Allow",
          "Action": [
            "s3:*"
            ],
          "Resource": "*"
        }
      ]
    }

使用Terraform时的注意事项

一般来说,使用Terraform管理云资源应以预期任何更改都应使用Terraform方法来进行。在使用AWS控制台或Red Hat控制台等Terraform外部的工具修改由Terraform创建的云资源时,请谨慎操作。使用Terraform外部的工具来管理已由Terraform管理的云资源会导致与您声明的Terraform配置产生配置漂移。

例如,如果您使用Red Hat混合云控制台升级您的Terraform创建的集群,则需要在应用任何即将到来的配置更改之前协调您的Terraform状态。有关更多信息,请参见HashiCorp开发者文档中的管理Terraform状态中的资源

默认集群规范概述

表1. 默认ROSA with STS集群规范
组件 默认规范

帐户和角色

  • 默认IAM角色前缀:rosa-<6位字母数字字符串>

  • 未创建集群管理员角色

集群设置

  • 默认集群版本:4.14

  • 集群名称:rosa-<6位字母数字字符串>

  • 使用Red Hat OpenShift集群管理器混合云控制台进行安装的默认AWS区域:us-east-2(美国东部,俄亥俄州)

  • 可用性:数据平面的多区域

  • 启用了EC2实例元数据服务(IMDS),并允许使用IMDSv1或IMDSv2(令牌可选)

  • 用户定义项目的监控:已启用

加密

  • 云存储在静止状态下加密

  • 未启用其他etcd加密

  • 默认AWS密钥管理服务(KMS)密钥用作持久性数据的加密密钥

  • 默认情况下未启用AWS密钥管理服务(KMS)密钥加密

控制平面节点配置

  • 控制平面节点实例类型:m5.2xlarge(8 vCPU,32 GiB RAM)

  • 控制平面节点数量:3

基础设施节点配置

  • 基础设施节点实例类型:r5.xlarge(4 vCPU,32 GiB RAM)

  • 基础设施节点数量:2

计算节点机器池

  • 计算节点实例类型:m5.xlarge(4 vCPU 16,GiB RAM)

  • 计算节点数量:3

  • 自动缩放:未启用

  • 无其他节点标签

网络配置

  • 集群隐私:公有或私有

  • 您可以在Terraform集群创建过程中选择创建一个新的VPC。

  • 您必须已配置您自己的虚拟私有云(VPC)

  • 未配置集群范围的代理

无类别域间路由(CIDR)范围

  • 机器CIDR:10.0.0.0/16

  • 服务CIDR:172.30.0.0/16

  • Pod CIDR:10.128.0.0/14

  • 主机前缀:/23

集群角色和策略

  • 用于创建操作员角色和OpenID Connect (OIDC)提供程序的模式:auto

    对于使用混合云控制台上的OpenShift集群管理器的安装,auto模式需要一个具有管理员权限的OpenShift集群管理器角色。

  • 默认操作员角色前缀:rosa-<6位字母数字字符串>

集群更新策略

  • 个别更新

  • 1小时节点排水宽限期

使用Terraform创建默认ROSA集群

下面概述的集群创建过程展示了如何使用Terraform创建您的帐户范围IAM角色和具有托管OIDC配置的ROSA集群。

准备您的Terraform环境

在您可以使用Terraform创建AWS上的Red Hat OpenShift服务集群之前,您需要导出您的离线Red Hat OpenShift集群管理器令牌

步骤
  1. 可选:由于此过程中 Terraform 文件会创建在您的当前目录中,您可以创建一个新目录来存储这些文件,并通过运行以下命令导航到该目录

    $ mkdir terraform-cluster && cd terraform-cluster
  2. 使用离线 Red Hat OpenShift 集群管理器令牌授予您的帐户权限。

  3. 复制您的离线令牌,并通过运行以下命令将其设置为环境变量

    $ export RHCS_TOKEN=<your_offline_token>

    此环境变量在每次会话结束时都会重置,例如重新启动计算机或关闭终端。

验证
  • 导出令牌后,通过运行以下命令验证其值

    $ echo $RHCS_TOKEN

在本地创建 Terraform 文件

设置离线 Red Hat OpenShift 集群管理器令牌后,您需要在本地创建 Terraform 文件来构建您的集群。您可以使用以下代码模板创建这些文件。

步骤
  1. 通过运行以下命令创建main.tf文件

    $ cat<<-EOF>main.tf
    #
    # Copyright (c) 2023 Red Hat, Inc.
    #
    # Licensed under the Apache License, Version 2.0 (the "License");
    # you may not use this file except in compliance with the License.
    # You may obtain a copy of the License at
    #
    #   http://apache.ac.cn/licenses/LICENSE-2.0
    #
    # Unless required by applicable law or agreed to in writing, software
    # distributed under the License is distributed on an "AS IS" BASIS,
    # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    # See the License for the specific language governing permissions and
    # limitations under the License.
    #
    terraform {
      required_providers {
        aws = {
          source  = "hashicorp/aws"
          version = ">= 4.20.0"
        }
        rhcs = {
          version = ">= 1.6.3"
          source  = "terraform-redhat/rhcs"
        }
      }
    }
    
    # Export token using the RHCS_TOKEN environment variable
    provider "rhcs" {}
    
    provider "aws" {
      region = var.aws_region
      ignore_tags {
        key_prefixes = ["kubernetes.io/"]
      }
      default_tags {
        tags = var.default_aws_tags
      }
    }
    
    data "aws_availability_zones" "available" {}
    
    locals {
      # Extract availability zone names for the specified region, limit it to 3 if multi az or 1 if single
      region_azs = var.multi_az ? slice([for zone in data.aws_availability_zones.available.names : format("%s", zone)], 0, 3) : slice([for zone in data.aws_availability_zones.available.names : format("%s", zone)], 0, 1)
    }
    
    resource "random_string" "random_name" {
      length  = 6
      special = false
      upper   = false
    }
    
    locals {
      worker_node_replicas = var.multi_az ? 3 : 2
      # If cluster_name is not null, use that, otherwise generate a random cluster name
      cluster_name = coalesce(var.cluster_name, "rosa-\${random_string.random_name.result}")
    }
    
    # The network validator requires an additional 60 seconds to validate Terraform clusters.
    resource "time_sleep" "wait_60_seconds" {
      count = var.create_vpc ? 1 : 0
      depends_on = [module.vpc]
      create_duration = "60s"
    }
    
    module "rosa-hcp" {
      source                 = "terraform-redhat/rosa-hcp/rhcs"
      version                = "1.6.3"
      cluster_name           = local.cluster_name
      openshift_version      = var.openshift_version
      account_role_prefix    = local.cluster_name
      operator_role_prefix   = local.cluster_name
      replicas               = local.worker_node_replicas
      aws_availability_zones = local.region_azs
      create_oidc            = true
      private                = var.private_cluster
      aws_subnet_ids         = var.create_vpc ? var.private_cluster ? module.vpc[0].private_subnets : concat(module.vpc[0].public_subnets, module.vpc[0].private_subnets) : var.aws_subnet_ids
      create_account_roles   = true
      create_operator_roles  = true
    # Optional: Configure a cluster administrator user (1)
    #
    # Option 1: Default cluster-admin user
    # Create an administrator user (cluster-admin) and automatically
    # generate a password by uncommenting the following parameter:
    #  create_admin_user = true
    # Generated administrator credentials are displayed in terminal output.
    #
    # Option 2: Specify administrator username and password
    # Create an administrator user and define your own password
    # by uncommenting and editing the values of the following parameters:
    #  admin_credentials_username = <username>
    #  admin_credentials_password = <password>
    
      depends_on = [time_sleep.wait_60_seconds]
    }
    EOF
    1 可选:在集群创建过程中创建管理员用户,方法是取消注释相应的参数并在需要时编辑其值。
  2. 通过运行以下命令创建variables.tf文件

    在运行构建集群的命令之前复制并编辑此文件。

    $ cat<<-EOF>variables.tf
    #
    # Copyright (c) 2023 Red Hat, Inc.
    #
    # Licensed under the Apache License, Version 2.0 (the "License");
    # you may not use this file except in compliance with the License.
    # You may obtain a copy of the License at
    #
    #   http://apache.ac.cn/licenses/LICENSE-2.0
    #
    # Unless required by applicable law or agreed to in writing, software
    # distributed under the License is distributed on an "AS IS" BASIS,
    # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    # See the License for the specific language governing permissions and
    # limitations under the License.
    #
    variable "openshift_version" {
      type        = string
      default     = "4.14.20"
      description = "Desired version of OpenShift for the cluster, for example '4.14.20'. If version is greater than the currently running version, an upgrade will be scheduled."
    }
    
    variable "create_vpc" {
      type        = bool
      description = "If you would like to create a new VPC, set this value to 'true'. If you do not want to create a new VPC, set this value to 'false'."
    }
    
    # ROSA Cluster info
    variable "cluster_name" {
      default     = null
      type        = string
      description = "The name of the ROSA cluster to create"
    }
    
    variable "additional_tags" {
      default = {
        Terraform   = "true"
        Environment = "dev"
      }
      description = "Additional AWS resource tags"
      type        = map(string)
    }
    
    variable "multi_az" {
      type        = bool
      description = "Multi AZ Cluster for High Availability"
      default     = true
    }
    
    variable "worker_node_replicas" {
      default     = 3
      description = "Number of worker nodes to provision. Single zone clusters need at least 2 nodes, multizone clusters need at least 3 nodes"
      type        = number
    }
    
    variable "aws_subnet_ids" {
      type        = list(any)
      description = "A list of either the public or public + private subnet IDs to use for the cluster blocks to use for the cluster"
      default     = ["subnet-01234567890abcdef", "subnet-01234567890abcdef", "subnet-01234567890abcdef"]
    }
    
    variable "private_cluster" {
      type        = bool
      description = "If you want to create a private cluster, set this value to 'true'. If you want a publicly available cluster, set this value to 'false'."
    }
    
    #VPC Info
    variable "vpc_name" {
      type        = string
      description = "VPC Name"
      default     = "tf-qs-vpc"
    }
    
    variable "vpc_cidr_block" {
      type        = string
      description = "value of the CIDR block to use for the VPC"
      default     = "10.0.0.0/16"
    }
    
    variable "private_subnet_cidrs" {
      type        = list(any)
      description = "The CIDR blocks to use for the private subnets"
      default     = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]
    }
    
    variable "public_subnet_cidrs" {
      type        = list(any)
      description = "The CIDR blocks to use for the public subnets"
      default     = ["10.0.101.0/24", "10.0.102.0/24", "10.0.103.0/24"]
    }
    
    variable "single_nat_gateway" {
      type        = bool
      description = "Single NAT or per NAT for subnet"
      default     = false
    }
    
    #AWS Info
    variable "aws_region" {
      type    = string
      default = "us-east-2"
    }
    
    variable "default_aws_tags" {
      type        = map(string)
      description = "Default tags for AWS"
      default     = {}
    }
    EOF
  3. 通过运行以下命令创建vpc.tf文件

    $ cat<<-EOF>vpc.tf
    #
    # Copyright (c) 2023 Red Hat, Inc.
    #
    # Licensed under the Apache License, Version 2.0 (the "License");
    # you may not use this file except in compliance with the License.
    # You may obtain a copy of the License at
    #
    #   http://apache.ac.cn/licenses/LICENSE-2.0
    #
    # Unless required by applicable law or agreed to in writing, software
    # distributed under the License is distributed on an "AS IS" BASIS,
    # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    # See the License for the specific language governing permissions and
    # limitations under the License.
    #
    module "vpc" {
      source  = "terraform-aws-modules/vpc/aws"
      version = "5.1.2"
    
      count = var.create_vpc ? 1 : 0
      name  = var.vpc_name
      cidr  = var.vpc_cidr_block
    
      azs             = local.region_azs
      private_subnets = var.multi_az ? var.private_subnet_cidrs : [var.private_subnet_cidrs[0]]
      public_subnets  = var.multi_az ? var.public_subnet_cidrs : [var.public_subnet_cidrs[0]]
    
      enable_nat_gateway   = true
      single_nat_gateway   = var.single_nat_gateway
      enable_dns_hostnames = true
      enable_dns_support   = true
    
      tags = var.additional_tags
    }
    EOF

    您可以开始使用 Terraform 了。

使用 Terraform 创建您的 ROSA 集群

创建 Terraform 文件后,必须启动 Terraform 以提供所有必需的依赖项。然后应用 Terraform 计划。

请勿修改 Terraform 状态文件。有关更多信息,请参阅使用 Terraform 时的注意事项

步骤
  1. 设置 Terraform 以根据您的 Terraform 文件创建资源,运行以下命令

    $ terraform init
  2. 可选:通过运行以下命令验证您复制的 Terraform 是否正确

    $ terraform validate
    示例输出
    Success! The configuration is valid.
  3. 通过运行以下命令使用 Terraform 创建您的集群

    $ terraform apply

    Terraform 接口会提出两个问题来创建您的集群,类似于以下内容

    示例输出
    var.create_vpc
      If you would like to create a new VPC, set this value to 'true'. If you do not want to create a new VPC, set this value to 'false'.
    
      Enter a value:
    
    var.private_cluster
      If you want to create a private cluster, set this value to 'true'. If you want a publicly available cluster, set this value to 'false'.
    
      Enter a value:
  4. 当 Terraform 接口列出要创建或更改的资源并提示确认时,输入yes继续或输入no取消。

    示例输出
    Plan: 63 to add, 0 to change, 0 to destroy.
    
    Do you want to perform these actions?
      Terraform will perform the actions described above.
      Only 'yes' will be accepted to approve.

    如果您输入yes,您的 Terraform 计划将开始,创建您的 AWS 帐户角色、Operator 角色和您的 ROSA Classic 集群。

验证
  1. 通过运行以下命令验证您的集群是否已创建

    $ rosa list clusters
    显示集群 ID、名称和状态的示例输出
    ID                                NAME          STATE  TOPOLOGY
    27c3snjsupa9obua74ba8se5kcj11269  rosa-tf-demo  ready  Classic (STS)
  2. 通过运行以下命令验证您的帐户角色是否已创建

    $ rosa list account-roles
    示例输出
    I: Fetching account roles
    ROLE NAME                                   ROLE TYPE      ROLE ARN                                                           OPENSHIFT VERSION  AWS Managed
    ROSA-demo-Installer-Role                    Installer      arn:aws:iam::<ID>:role/ROSA-demo-Installer-Role                    4.14               No
    ROSA-demo-Support-Role                      Support        arn:aws:iam::<ID>:role/ROSA-demo-Support-Role                      4.14               No
    ROSA-demo-Worker-Role                       Worker         arn:aws:iam::<ID>:role/ROSA-demo-Worker-Role                       4.14               No
  3. 通过运行以下命令验证您的 Operator 角色是否已创建

    $ rosa list operator-roles
    显示 Terraform 创建的 Operator 角色的示例输出
    I: Fetching operator roles
    ROLE PREFIX    AMOUNT IN BUNDLE
    rosa-demo      8

使用 Terraform 删除您的 ROSA 集群

使用terraform destroy命令删除使用terraform apply命令创建的所有资源。

在销毁资源之前,请勿修改您的 Terraform .tf 文件。这些变量与要删除的资源匹配。

步骤
  1. 在您运行terraform apply命令创建集群的目录中,运行以下命令以删除集群

    $ terraform destroy

    Terraform 接口将提示您输入两个变量。这些变量应与您在创建集群时提供的答案匹配。

    var.create_vpc
      If you would like to create a new VPC, set this value to 'true.' If you do not want to create a new VPC, set this value to 'false.'
    
      Enter a value:
    
    var.private_cluster
      If you want to create a private cluster, set this value to 'true.' If you want a publicly available cluster, set this value to 'false.'
    
      Enter a value:
  2. 输入yes开始角色和集群删除。

    示例输出
    Plan: 0 to add, 0 to change, 63 to destroy.
    
    Do you really want to destroy all resources?
      Terraform will destroy all your managed infrastructure, as shown above.
      There is no undo. Only 'yes' will be accepted to confirm.
    
      Enter a value: yes
验证
  1. 通过运行以下命令验证您的集群是否已销毁

    $ rosa list clusters
    显示没有集群的示例输出
    I: No clusters available
  2. 通过运行以下命令验证帐户角色是否已销毁

    $ rosa list account-roles
    显示没有 Terraform 创建的帐户角色的示例输出
    I: Fetching account roles
    I: No account roles available
  3. 通过运行以下命令验证 Operator 角色是否已销毁

    $ rosa list operator-roles
    显示没有 Terraform 创建的 Operator 角色的示例输出
    I: Fetching operator roles
    I: No operator roles available