×

使用配置有默认集群选项的 Terraform 集群模板快速创建 Red Hat OpenShift Service on AWS (ROSA)(经典架构)集群。

下面描述的集群创建过程使用 Terraform 配置,该配置准备了一个具有以下资源的 ROSA(经典架构)AWS 安全令牌服务 (STS) 集群

  • 具有托管oidc-config配置的 OIDC 提供程序

  • 具有关联的 AWS 托管 ROSA 策略的先决条件 IAM 运算符角色

  • 具有关联的 AWS 托管 ROSA 策略的 IAM 账户角色

  • 创建具有 STS 集群的 ROSA 所需的所有其他 AWS 资源

Terraform 概述

Terraform 是一种基础设施即代码工具,它提供了一种方法来配置一次资源,并根据需要复制这些资源。Terraform 通过使用声明性语言来完成创建任务。您声明所需的基础设施资源的最终状态,Terraform 将根据您的规范创建这些资源。

先决条件

要在您的 Terraform 配置中使用Red Hat Cloud Services 提供程序,您必须满足以下先决条件

  • 您已安装 Red Hat OpenShift Service on AWS (ROSA) 命令行界面 (CLI) 工具。

  • 您拥有您的离线Red Hat OpenShift 集群管理器令牌

  • 您已安装Terraform 版本 1.4.6 或更高版本。

  • 您已创建您的 AWS 帐户范围的 IAM 角色。

    特定的帐户范围 IAM 角色和策略提供 ROSA 支持、安装、控制平面和计算功能所需的 STS 权限。这包括帐户范围的运算符策略。有关 AWS 帐户角色的更多信息,请参阅其他资源。

  • 您拥有一个AWS 帐户关联的凭据,允许您创建资源。凭据已配置为 AWS 提供程序。请参阅 AWS Terraform 提供程序文档中的身份验证和配置部分。

  • 在运行 Terraform 的 AWS IAM 角色策略中,您至少具有以下权限。在 AWS 控制台中检查这些权限。

    Terraform 的最低 AWS 权限
    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Sid": "VisualEditor0",
          "Effect": "Allow",
          "Action": [
            "iam:GetPolicyVersion",
            "iam:DeletePolicyVersion",
            "iam:CreatePolicyVersion",
            "iam:UpdateAssumeRolePolicy",
            "secretsmanager:DescribeSecret",
            "iam:ListRoleTags",
            "secretsmanager:PutSecretValue",
            "secretsmanager:CreateSecret",
            "iam:TagRole",
            "secretsmanager:DeleteSecret",
            "iam:UpdateOpenIDConnectProviderThumbprint",
            "iam:DeletePolicy",
            "iam:CreateRole",
            "iam:AttachRolePolicy",
            "iam:ListInstanceProfilesForRole",
            "secretsmanager:GetSecretValue",
            "iam:DetachRolePolicy",
            "iam:ListAttachedRolePolicies",
            "iam:ListPolicyTags",
            "iam:ListRolePolicies",
            "iam:DeleteOpenIDConnectProvider",
            "iam:DeleteInstanceProfile",
            "iam:GetRole",
            "iam:GetPolicy",
            "iam:ListEntitiesForPolicy",
            "iam:DeleteRole",
            "iam:TagPolicy",
            "iam:CreateOpenIDConnectProvider",
            "iam:CreatePolicy",
            "secretsmanager:GetResourcePolicy",
            "iam:ListPolicyVersions",
            "iam:UpdateRole",
            "iam:GetOpenIDConnectProvider",
            "iam:TagOpenIDConnectProvider",
            "secretsmanager:TagResource",
            "sts:AssumeRoleWithWebIdentity",
            "iam:ListRoles"
          ],
          "Resource": [
            "arn:aws:secretsmanager:*:<ACCOUNT_ID>:secret:*",
            "arn:aws:iam::<ACCOUNT_ID>:instance-profile/*",
            "arn:aws:iam::<ACCOUNT_ID>:role/*",
            "arn:aws:iam::<ACCOUNT_ID>:oidc-provider/*",
            "arn:aws:iam::<ACCOUNT_ID>:policy/*"
          ]
        },
        {
          "Sid": "VisualEditor1",
          "Effect": "Allow",
          "Action": [
            "s3:*"
            ],
          "Resource": "*"
        }
      ]
    }

使用 Terraform 时的注意事项

通常,使用 Terraform 管理云资源时,应预期任何更改都应使用 Terraform 方法进行。使用 AWS 控制台或 Red Hat 控制台等 Terraform 之外的工具修改 Terraform 创建的云资源时,请谨慎操作。使用 Terraform 之外的工具来管理已由 Terraform 管理的云资源会导致与您声明的 Terraform 配置出现配置漂移。

例如,如果您使用Red Hat Hybrid Cloud Console升级您的 Terraform 创建的集群,则需要在应用任何即将进行的配置更改之前协调您的 Terraform 状态。有关更多信息,请参阅 HashiCorp 开发人员文档中的在 Terraform 状态中管理资源

默认集群规范概述

表 1. 使用 STS 集群的默认 ROSA 规范
组件 默认规范

帐户和角色

  • 默认 IAM 角色前缀:rosa-<6 位字母数字字符串>

  • 未创建集群管理员角色

集群设置

  • 默认集群版本:4.14

  • 集群名称:rosa-<6 位字母数字字符串>

  • 使用 Red Hat OpenShift 集群管理器混合云控制台进行安装的默认 AWS 区域:us-east-2(美国东部,俄亥俄州)

  • 可用性:数据平面的多区域

  • 已启用 EC2 实例元数据服务 (IMDS),并允许使用 IMDSv1 或 IMDSv2(令牌可选)

  • 已启用 EC2 实例元数据服务 (IMDS),并允许使用 IMDSv1 或 IMDSv2(令牌可选)

  • 用户定义项目的监控:已启用

加密

  • 云存储在静止状态下加密

  • 未启用额外的 etcd 加密

  • 默认 AWS 密钥管理服务 (KMS) 密钥用作持久数据的加密密钥

控制平面节点配置

  • 控制平面节点实例类型:m5.2xlarge(8 个 vCPU,32 GiB RAM)

  • 控制平面节点数量:3

基础架构节点配置

  • 基础架构节点实例类型:r5.xlarge(4 个 vCPU,32 GiB RAM)

  • 基础架构节点数量:2

计算节点机器池

  • 计算节点实例类型:m5.xlarge(4 个 vCPU 16,GiB RAM)

  • 计算节点数量:3

  • 自动缩放:未启用

  • 无其他节点标签

网络配置

  • 集群隐私:公共或私有

  • 您可以在 Terraform 集群创建过程中选择创建一个新的 VPC。

  • 您必须已配置您自己的虚拟私有云 (VPC)

  • 未配置集群范围的代理

无类别域间路由 (CIDR) 范围

  • 机器 CIDR:10.0.0.0/16

  • 服务 CIDR:172.30.0.0/16

  • Pod CIDR:10.128.0.0/14

  • 主机前缀:/23

集群角色和策略

  • 用于创建 Operator 角色和 OpenID Connect (OIDC) 提供程序的模式:auto

    对于在混合云控制台上使用 OpenShift 集群管理器进行安装的安装,auto 模式需要一个具有管理员权限的 OpenShift 集群管理器角色。

  • 默认 Operator 角色前缀:rosa-<6位字母数字字符串>

集群更新策略

  • 单个更新

  • 节点驱逐的宽限期为 1 小时

使用 Terraform 创建默认 ROSA(经典架构)集群

下面概述的集群创建过程展示了如何使用 Terraform 创建您的帐户范围的 IAM 角色和具有托管 OIDC 配置的 ROSA(经典架构)集群。

为 Terraform 准备您的环境

在您可以使用 Terraform 创建 AWS 上的 Red Hat OpenShift Service 集群之前,您需要导出您的离线 Red Hat OpenShift 集群管理器令牌

步骤
  1. 可选:因为在此过程中 Terraform 文件会在您的当前目录中创建,您可以创建一个新目录来存储这些文件,并通过运行以下命令导航到该目录

    $ mkdir terraform-cluster && cd terraform-cluster
  2. 使用离线 Red Hat OpenShift 集群管理器令牌授予您的帐户权限。

  3. 复制您的离线令牌,并通过运行以下命令将令牌设置为环境变量

    $ export RHCS_TOKEN=<your_offline_token>

    此环境变量在每个会话结束时都会重置,例如重新启动您的机器或关闭终端。

验证
  • 导出令牌后,通过运行以下命令验证该值

    $ echo $RHCS_TOKEN

在本地创建您的 Terraform 文件

设置您的离线 Red Hat OpenShift 集群管理器令牌后,您需要在本地创建 Terraform 文件以构建您的集群。您可以使用以下代码模板创建这些文件。

步骤
  1. 通过运行以下命令创建main.tf文件

    $ cat<<-EOF>main.tf
    #
    # Copyright (c) 2023 Red Hat, Inc.
    #
    # Licensed under the Apache License, Version 2.0 (the "License");
    # you may not use this file except in compliance with the License.
    # You may obtain a copy of the License at
    #
    #   http://apache.ac.cn/licenses/LICENSE-2.0
    #
    # Unless required by applicable law or agreed to in writing, software
    # distributed under the License is distributed on an "AS IS" BASIS,
    # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    # See the License for the specific language governing permissions and
    # limitations under the License.
    #
    
    terraform {
      required_providers {
        aws = {
          source  = "hashicorp/aws"
          version = ">= 4.20.0"
        }
        rhcs = {
          version = ">= 1.6.2"
          source  = "terraform-redhat/rhcs"
        }
      }
    }
    
    # Export token using the RHCS_TOKEN environment variable
    provider "rhcs" {}
    
    provider "aws" {
      region = var.aws_region
      ignore_tags {
        key_prefixes = ["kubernetes.io/"]
      }
      default_tags {
        tags = var.default_aws_tags
      }
    }
    
    data "aws_availability_zones" "available" {}
    
    locals {
      # The default setting creates 3 availability zones. Set to "false" to create a single availability zones.
      region_azs = var.multi_az ? slice([for zone in data.aws_availability_zones.available.names : format("%s", zone)], 0, 3) : slice([for zone in data.aws_availability_zones.available.names : format("%s", zone)], 0, 1)
    }
    
    resource "random_string" "random_name" {
      length  = 6
      special = false
      upper   = false
    }
    
    locals {
      path                 = coalesce(var.path, "/")
      worker_node_replicas = try(var.worker_node_replicas, var.multi_az ? 3 : 2)
      # If cluster_name is not null, use that, otherwise generate a random cluster name
      cluster_name = coalesce(var.cluster_name, "rosa-\${random_string.random_name.result}")
    }
    
    # The network validator requires an additional 60 seconds to validate Terraform clusters.
    resource "time_sleep" "wait_60_seconds" {
      count = var.create_vpc ? 1 : 0
      depends_on = [module.vpc]
      create_duration = "60s"
    }
    
    module "rosa-classic" {
      source                 = "terraform-redhat/rosa-classic/rhcs"
      version                = "1.5.0"
      cluster_name           = local.cluster_name
      openshift_version      = var.openshift_version
      account_role_prefix    = local.cluster_name
      operator_role_prefix   = local.cluster_name
      replicas               = local.worker_node_replicas
      aws_availability_zones = local.region_azs
      create_oidc            = true
      private                = var.private_cluster
      aws_private_link       = var.private_cluster
      aws_subnet_ids         = var.create_vpc ? var.private_cluster ? module.vpc[0].private_subnets : concat(module.vpc[0].public_subnets, module.vpc[0].private_subnets) : var.aws_subnet_ids
      multi_az               = var.multi_az
      create_account_roles   = true
      create_operator_roles  = true
    # Optional: Configure a cluster administrator user (1)
    #
    # Option 1: Default cluster-admin user
    # Create an administrator user (cluster-admin) and automatically
    # generate a password by uncommenting the following parameter:
    #  create_admin_user = true
    # Generated administrator credentials are displayed in terminal output.
    #
    # Option 2: Specify administrator username and password
    # Create an administrator user and define your own password
    # by uncommenting and editing the values of the following parameters:
    #  admin_credentials_username = <username>
    #  admin_credentials_password = <password>
    
      depends_on = [time_sleep.wait_60_seconds]
    }
    EOF
    1 可选:通过取消注释相应的参数并编辑其值,在集群创建期间创建管理员用户。
  2. 通过运行以下命令创建variables.tf文件

    在运行构建集群的命令之前复制并编辑此文件。

    $ cat<<-EOF>variables.tf
    #
    # Copyright (c) 2023 Red Hat, Inc.
    #
    # Licensed under the Apache License, Version 2.0 (the "License");
    # you may not use this file except in compliance with the License.
    # You may obtain a copy of the License at
    #
    #   http://apache.ac.cn/licenses/LICENSE-2.0
    #
    # Unless required by applicable law or agreed to in writing, software
    # distributed under the License is distributed on an "AS IS" BASIS,
    # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    # See the License for the specific language governing permissions and
    # limitations under the License.
    #
    variable "openshift_version" {
      type        = string
      default     = "4.14.20"
      description = "Desired version of OpenShift for the cluster, for example '4.14.20'. If version is greater than the currently running version, an upgrade will be scheduled."
    }
    
    variable "create_vpc" {
      type        = bool
      description = "If you would like to create a new VPC, set this value to 'true'. If you do not want to create a new VPC, set this value to 'false'."
    }
    
    # ROSA Cluster info
    variable "cluster_name" {
      default     = null
      type        = string
      description = "The name of the ROSA cluster to create"
    }
    
    variable "additional_tags" {
      default = {
        Terraform   = "true"
        Environment = "dev"
      }
      description = "Additional AWS resource tags"
      type        = map(string)
    }
    
    variable "path" {
      description = "(Optional) The arn path for the account/operator roles as well as their policies."
      type        = string
      default     = null
    }
    
    variable "multi_az" {
      type        = bool
      description = "Multi AZ Cluster for High Availability"
      default     = true
    }
    
    variable "worker_node_replicas" {
      default     = 3
      description = "Number of worker nodes to provision. Single zone clusters need at least 2 nodes, multizone clusters need at least 3 nodes"
      type        = number
    }
    
    variable "aws_subnet_ids" {
      type        = list(any)
      description = "A list of either the public or public + private subnet IDs to use for the cluster blocks to use for the cluster"
      default     = ["subnet-01234567890abcdef", "subnet-01234567890abcdef", "subnet-01234567890abcdef"]
    }
    
    variable "private_cluster" {
      type        = bool
      description = "If you want to create a private cluster, set this value to 'true'. If you want a publicly available cluster, set this value to 'false'."
    }
    
    #VPC Info
    variable "vpc_name" {
      type        = string
      description = "VPC Name"
      default     = "tf-qs-vpc"
    }
    
    variable "vpc_cidr_block" {
      type        = string
      description = "value of the CIDR block to use for the VPC"
      default     = "10.0.0.0/16"
    }
    
    variable "private_subnet_cidrs" {
      type        = list(any)
      description = "The CIDR blocks to use for the private subnets"
      default     = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]
    }
    
    variable "public_subnet_cidrs" {
      type        = list(any)
      description = "The CIDR blocks to use for the public subnets"
      default     = ["10.0.101.0/24", "10.0.102.0/24", "10.0.103.0/24"]
    }
    
    variable "single_nat_gateway" {
      type        = bool
      description = "Single NAT or per NAT for subnet"
      default     = false
    }
    
    #AWS Info
    variable "aws_region" {
      type    = string
      default = "us-east-2"
    }
    
    variable "default_aws_tags" {
      type        = map(string)
      description = "Default tags for AWS"
      default     = {}
    }
    EOF
  3. 通过运行以下命令创建vpc.tf文件

    $ cat<<-EOF>vpc.tf
    #
    # Copyright (c) 2023 Red Hat, Inc.
    #
    # Licensed under the Apache License, Version 2.0 (the "License");
    # you may not use this file except in compliance with the License.
    # You may obtain a copy of the License at
    #
    #   http://apache.ac.cn/licenses/LICENSE-2.0
    #
    # Unless required by applicable law or agreed to in writing, software
    # distributed under the License is distributed on an "AS IS" BASIS,
    # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    # See the License for the specific language governing permissions and
    # limitations under the License.
    #
    module "vpc" {
      source  = "terraform-aws-modules/vpc/aws"
      version = "5.1.2"
    
      count = var.create_vpc ? 1 : 0
      name  = var.vpc_name
      cidr  = var.vpc_cidr_block
    
      azs             = local.region_azs
      private_subnets = var.private_subnet_cidrs
      public_subnets  = var.public_subnet_cidrs
    
      enable_nat_gateway   = true
      single_nat_gateway   = var.single_nat_gateway
      enable_dns_hostnames = true
      enable_dns_support   = true
    
      tags = var.additional_tags
    }
    EOF

    您已准备好启动 Terraform。

使用 Terraform 创建您的 ROSA 集群

创建 Terraform 文件后,必须启动 Terraform 以提供所有必需的依赖项。然后应用 Terraform 计划。

不要修改 Terraform 状态文件。有关更多信息,请参阅使用 Terraform 时的注意事项

步骤
  1. 设置 Terraform 以根据您的 Terraform 文件创建您的资源,运行以下命令

    $ terraform init
  2. 可选:通过运行以下命令验证您复制的 Terraform 是否正确

    $ terraform validate
    示例输出
    Success! The configuration is valid.
  3. 通过运行以下命令使用 Terraform 创建您的集群

    $ terraform apply

    Terraform 接口会提出两个问题来创建您的集群,类似于以下内容

    示例输出
    var.create_vpc
      If you would like to create a new VPC, set this value to 'true'. If you do not want to create a new VPC, set this value to 'false'.
    
      Enter a value:
    
    var.private_cluster
      If you want to create a private cluster, set this value to 'true'. If you want a publicly available cluster, set this value to 'false'.
    
      Enter a value:
  4. 当 Terraform 接口列出要创建或更改的资源并提示确认时,输入yes继续或输入no取消

    示例输出
    Plan: 74 to add, 0 to change, 0 to destroy.
    
    Do you want to perform these actions?
      Terraform will perform the actions described above.
      Only 'yes' will be accepted to approve.
    
      Enter a value: yes

    如果您输入yes,您的 Terraform 计划将开始,创建您的 AWS 帐户角色、Operator 角色和您的 ROSA Classic 集群。

验证
  1. 通过运行以下命令验证您的集群是否已创建

    $ rosa list clusters
    显示集群 ID、名称和状态的示例输出
    ID                                NAME          STATE  TOPOLOGY
    27c3snjsupa9obua74ba8se5kcj11269  rosa-tf-demo  ready  Classic (STS)
  2. 通过运行以下命令验证您的帐户角色是否已创建

    $ rosa list account-roles
    示例输出
    I: Fetching account roles
    ROLE NAME                                   ROLE TYPE      ROLE ARN                                                           OPENSHIFT VERSION  AWS Managed
    ROSA-demo-ControlPlane-Role                 Control plane  arn:aws:iam::<ID>:role/ROSA-demo-ControlPlane-Role                 4.14               No
    ROSA-demo-Installer-Role                    Installer      arn:aws:iam::<ID>:role/ROSA-demo-Installer-Role                    4.14               No
    ROSA-demo-Support-Role                      Support        arn:aws:iam::<ID>:role/ROSA-demo-Support-Role                      4.14               No
    ROSA-demo-Worker-Role                       Worker         arn:aws:iam::<ID>:role/ROSA-demo-Worker-Role                       4.14               No
  3. 通过运行以下命令验证您的 Operator 角色是否已创建

    $ rosa list operator-roles
    显示 Terraform 创建的 Operator 角色的示例输出
    I: Fetching operator roles
    ROLE PREFIX    AMOUNT IN BUNDLE
    rosa-demo      6

使用 Terraform 删除您的 ROSA 集群

使用terraform destroy命令删除使用terraform apply命令创建的所有资源。

在销毁资源之前,不要修改您的 Terraform .tf 文件。这些变量与要删除的资源匹配。

步骤
  1. 在您运行terraform apply命令创建集群的目录中,运行以下命令以删除集群

    $ terraform destroy

    Terraform 接口将提示您输入两个变量。这些变量应与您在创建集群时提供的答案匹配

    var.create_vpc
      If you would like to create a new VPC, set this value to 'true.' If you do not want to create a new VPC, set this value to 'false.'
    
      Enter a value:
    
    var.private_cluster
      If you want to create a private cluster, set this value to 'true.' If you want a publicly available cluster, set this value to 'false.'
    
      Enter a value:
  2. 输入yes开始角色和集群删除

    示例输出
    Plan: 0 to add, 0 to change, 74 to destroy.
    
    Do you really want to destroy all resources?
      Terraform will destroy all your managed infrastructure, as shown above.
      There is no undo. Only 'yes' will be accepted to confirm.
    
      Enter a value: yes
验证
  1. 通过运行以下命令验证您的集群是否已销毁

    $ rosa list clusters
    显示没有集群的示例输出
    I: No clusters available
  2. 通过运行以下命令验证帐户角色是否已销毁

    $ rosa list account-roles
    显示没有 Terraform 创建的帐户角色的示例输出
    I: Fetching account roles
    I: No account roles available
  3. 通过运行以下命令验证 Operator 角色是否已销毁

    $ rosa list operator-roles
    显示没有 Terraform 创建的 Operator 角色的示例输出
    I: Fetching operator roles
    I: No operator roles available