×

OpenShift Container Platform 审计提供了一个安全相关的、按时间顺序排列的记录集,记录了影响系统活动的序列,这些活动是由单个用户、管理员或系统的其他组件执行的。

关于 API 审计日志

审计在 API 服务器级别运行,记录所有传入服务器的请求。每个审计日志包含以下信息:

表 1. 审计日志字段
字段 描述

级别 (level)

生成事件的审计级别。

审计 ID (auditID)

每个请求生成的唯一审计 ID。

阶段 (stage)

生成此事件实例的请求处理阶段。

请求 URI (requestURI)

客户端发送到服务器的请求 URI。

动词 (verb)

与请求关联的 Kubernetes 动词。对于非资源请求,这是小写的 HTTP 方法。

用户 (user)

已认证的用户的信息。

模拟用户 (impersonatedUser)

可选。如果请求模拟另一个用户,则为模拟用户的用户信息。

源 IP (sourceIPs)

可选。请求发起的源 IP,以及任何中间代理。

用户代理 (userAgent)

可选。客户端报告的用户代理字符串。请注意,用户代理由客户端提供,不应被信任。

对象引用 (objectRef)

可选。此请求的目标对象引用。这并不适用于 `List` 类型请求或非资源请求。

响应状态 (responseStatus)

可选。响应状态,即使 `ResponseObject` 不是 `Status` 类型也会填充。对于成功的响应,这将只包含代码。对于非状态类型错误响应,这将自动填充错误消息。

请求对象 (requestObject)

可选。来自请求的 API 对象,以 JSON 格式。`RequestObject` 按原样记录在请求中(可能重新编码为 JSON),在版本转换、默认值、准入或合并之前。它是一个外部版本化的对象类型,本身可能不是一个有效的对象。对于非资源请求,此项被忽略,并且仅在请求级别及更高级别记录。

响应对象 (responseObject)

可选。响应中返回的 API 对象,以 JSON 格式。`ResponseObject` 在转换为外部类型后记录,并序列化为 JSON。对于非资源请求,此项被忽略,并且仅在响应级别记录。

请求接收时间戳 (requestReceivedTimestamp)

请求到达 API 服务器的时间。

阶段时间戳 (stageTimestamp)

请求到达当前审计阶段的时间。

注释 (annotations)

可选。存储在审计事件中的非结构化键值映射,可能由请求服务链中调用的插件设置,包括身份验证、授权和准入插件。请注意,这些注释用于审计事件,并不对应于提交对象的 `metadata.annotations`。键应唯一标识告知组件以避免名称冲突,例如 `podsecuritypolicy.admission.k8s.io/policy`。值应简短。注释包含在元数据级别。

Kubernetes API 服务器的示例输出

{"kind":"Event","apiVersion":"audit.k8s.io/v1","level":"Metadata","auditID":"ad209ce1-fec7-4130-8192-c4cc63f1d8cd","stage":"ResponseComplete","requestURI":"/api/v1/namespaces/openshift-kube-controller-manager/configmaps/cert-recovery-controller-lock?timeout=35s","verb":"update","user":{"username":"system:serviceaccount:openshift-kube-controller-manager:localhost-recovery-client","uid":"dd4997e3-d565-4e37-80f8-7fc122ccd785","groups":["system:serviceaccounts","system:serviceaccounts:openshift-kube-controller-manager","system:authenticated"]},"sourceIPs":["::1"],"userAgent":"cluster-kube-controller-manager-operator/v0.0.0 (linux/amd64) kubernetes/$Format","objectRef":{"resource":"configmaps","namespace":"openshift-kube-controller-manager","name":"cert-recovery-controller-lock","uid":"5c57190b-6993-425d-8101-8337e48c7548","apiVersion":"v1","resourceVersion":"574307"},"responseStatus":{"metadata":{},"code":200},"requestReceivedTimestamp":"2020-04-02T08:27:20.200962Z","stageTimestamp":"2020-04-02T08:27:20.206710Z","annotations":{"authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":"RBAC: allowed by ClusterRoleBinding \"system:openshift:operator:kube-controller-manager-recovery\" of ClusterRole \"cluster-admin\" to ServiceAccount \"localhost-recovery-client/openshift-kube-controller-manager\""}}

查看审计日志

您可以查看 OpenShift API 服务器、Kubernetes API 服务器、OpenShift OAuth API 服务器和 OpenShift OAuth 服务器的日志,这些日志适用于每个控制平面节点。

步骤

要查看审计日志:

  • 查看 OpenShift API 服务器审计日志

    1. 列出每个控制平面节点上可用的 OpenShift API 服务器审计日志

      $ oc adm node-logs --role=master --path=openshift-apiserver/
      示例输出
      ci-ln-m0wpfjb-f76d1-vnb5x-master-0 audit-2021-03-09T00-12-19.834.log
      ci-ln-m0wpfjb-f76d1-vnb5x-master-0 audit.log
      ci-ln-m0wpfjb-f76d1-vnb5x-master-1 audit-2021-03-09T00-11-49.835.log
      ci-ln-m0wpfjb-f76d1-vnb5x-master-1 audit.log
      ci-ln-m0wpfjb-f76d1-vnb5x-master-2 audit-2021-03-09T00-13-00.128.log
      ci-ln-m0wpfjb-f76d1-vnb5x-master-2 audit.log
    2. 通过提供节点名称和日志名称来查看特定的 OpenShift API 服务器审计日志

      $ oc adm node-logs <node_name> --path=openshift-apiserver/<log_name>

      例如

      $ oc adm node-logs ci-ln-m0wpfjb-f76d1-vnb5x-master-0 --path=openshift-apiserver/audit-2021-03-09T00-12-19.834.log
      示例输出
      {"kind":"Event","apiVersion":"audit.k8s.io/v1","level":"Metadata","auditID":"381acf6d-5f30-4c7d-8175-c9c317ae5893","stage":"ResponseComplete","requestURI":"/metrics","verb":"get","user":{"username":"system:serviceaccount:openshift-monitoring:prometheus-k8s","uid":"825b60a0-3976-4861-a342-3b2b561e8f82","groups":["system:serviceaccounts","system:serviceaccounts:openshift-monitoring","system:authenticated"]},"sourceIPs":["10.129.2.6"],"userAgent":"Prometheus/2.23.0","responseStatus":{"metadata":{},"code":200},"requestReceivedTimestamp":"2021-03-08T18:02:04.086545Z","stageTimestamp":"2021-03-08T18:02:04.107102Z","annotations":{"authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":"RBAC: allowed by ClusterRoleBinding \"prometheus-k8s\" of ClusterRole \"prometheus-k8s\" to ServiceAccount \"prometheus-k8s/openshift-monitoring\""}}
  • 查看 Kubernetes API 服务器审计日志

    1. 列出每个控制平面节点上可用的 Kubernetes API 服务器审计日志

      $ oc adm node-logs --role=master --path=kube-apiserver/
      示例输出
      ci-ln-m0wpfjb-f76d1-vnb5x-master-0 audit-2021-03-09T14-07-27.129.log
      ci-ln-m0wpfjb-f76d1-vnb5x-master-0 audit.log
      ci-ln-m0wpfjb-f76d1-vnb5x-master-1 audit-2021-03-09T19-24-22.620.log
      ci-ln-m0wpfjb-f76d1-vnb5x-master-1 audit.log
      ci-ln-m0wpfjb-f76d1-vnb5x-master-2 audit-2021-03-09T18-37-07.511.log
      ci-ln-m0wpfjb-f76d1-vnb5x-master-2 audit.log
    2. 通过提供节点名称和日志名称来查看特定的 Kubernetes API 服务器审计日志

      $ oc adm node-logs <node_name> --path=kube-apiserver/<log_name>

      例如

      $ oc adm node-logs ci-ln-m0wpfjb-f76d1-vnb5x-master-0 --path=kube-apiserver/audit-2021-03-09T14-07-27.129.log
      示例输出
      {"kind":"Event","apiVersion":"audit.k8s.io/v1","level":"Metadata","auditID":"cfce8a0b-b5f5-4365-8c9f-79c1227d10f9","stage":"ResponseComplete","requestURI":"/api/v1/namespaces/openshift-kube-scheduler/serviceaccounts/openshift-kube-scheduler-sa","verb":"get","user":{"username":"system:serviceaccount:openshift-kube-scheduler-operator:openshift-kube-scheduler-operator","uid":"2574b041-f3c8-44e6-a057-baef7aa81516","groups":["system:serviceaccounts","system:serviceaccounts:openshift-kube-scheduler-operator","system:authenticated"]},"sourceIPs":["10.128.0.8"],"userAgent":"cluster-kube-scheduler-operator/v0.0.0 (linux/amd64) kubernetes/$Format","objectRef":{"resource":"serviceaccounts","namespace":"openshift-kube-scheduler","name":"openshift-kube-scheduler-sa","apiVersion":"v1"},"responseStatus":{"metadata":{},"code":200},"requestReceivedTimestamp":"2021-03-08T18:06:42.512619Z","stageTimestamp":"2021-03-08T18:06:42.516145Z","annotations":{"authentication.k8s.io/legacy-token":"system:serviceaccount:openshift-kube-scheduler-operator:openshift-kube-scheduler-operator","authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":"RBAC: allowed by ClusterRoleBinding \"system:openshift:operator:cluster-kube-scheduler-operator\" of ClusterRole \"cluster-admin\" to ServiceAccount \"openshift-kube-scheduler-operator/openshift-kube-scheduler-operator\""}}
  • 查看 OpenShift OAuth API 服务器审计日志

    1. 列出每个控制平面节点上可用的 OpenShift OAuth API 服务器审计日志

      $ oc adm node-logs --role=master --path=oauth-apiserver/
      示例输出
      ci-ln-m0wpfjb-f76d1-vnb5x-master-0 audit-2021-03-09T13-06-26.128.log
      ci-ln-m0wpfjb-f76d1-vnb5x-master-0 audit.log
      ci-ln-m0wpfjb-f76d1-vnb5x-master-1 audit-2021-03-09T18-23-21.619.log
      ci-ln-m0wpfjb-f76d1-vnb5x-master-1 audit.log
      ci-ln-m0wpfjb-f76d1-vnb5x-master-2 audit-2021-03-09T17-36-06.510.log
      ci-ln-m0wpfjb-f76d1-vnb5x-master-2 audit.log
    2. 通过提供节点名称和日志名称来查看特定的 OpenShift OAuth API 服务器审计日志

      $ oc adm node-logs <node_name> --path=oauth-apiserver/<log_name>

      例如

      $ oc adm node-logs ci-ln-m0wpfjb-f76d1-vnb5x-master-0 --path=oauth-apiserver/audit-2021-03-09T13-06-26.128.log
      示例输出
      {"kind":"Event","apiVersion":"audit.k8s.io/v1","level":"Metadata","auditID":"dd4c44e2-3ea1-4830-9ab7-c91a5f1388d6","stage":"ResponseComplete","requestURI":"/apis/user.openshift.io/v1/users/~","verb":"get","user":{"username":"system:serviceaccount:openshift-monitoring:prometheus-k8s","groups":["system:serviceaccounts","system:serviceaccounts:openshift-monitoring","system:authenticated"]},"sourceIPs":["10.0.32.4","10.128.0.1"],"userAgent":"dockerregistry/v0.0.0 (linux/amd64) kubernetes/$Format","objectRef":{"resource":"users","name":"~","apiGroup":"user.openshift.io","apiVersion":"v1"},"responseStatus":{"metadata":{},"code":200},"requestReceivedTimestamp":"2021-03-08T17:47:43.653187Z","stageTimestamp":"2021-03-08T17:47:43.660187Z","annotations":{"authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":"RBAC: allowed by ClusterRoleBinding \"basic-users\" of ClusterRole \"basic-user\" to Group \"system:authenticated\""}}
  • 查看 OpenShift OAuth 服务器审计日志

    1. 列出每个控制平面节点上可用的 OpenShift OAuth 服务器审计日志

      $ oc adm node-logs --role=master --path=oauth-server/
      示例输出
      ci-ln-m0wpfjb-f76d1-vnb5x-master-0 audit-2022-05-11T18-57-32.395.log
      ci-ln-m0wpfjb-f76d1-vnb5x-master-0 audit.log
      ci-ln-m0wpfjb-f76d1-vnb5x-master-1 audit-2022-05-11T19-07-07.021.log
      ci-ln-m0wpfjb-f76d1-vnb5x-master-1 audit.log
      ci-ln-m0wpfjb-f76d1-vnb5x-master-2 audit-2022-05-11T19-06-51.844.log
      ci-ln-m0wpfjb-f76d1-vnb5x-master-2 audit.log
    2. 通过提供节点名称和日志名称来查看特定的 OpenShift OAuth 服务器审计日志

      $ oc adm node-logs <node_name> --path=oauth-server/<log_name>

      例如

      $ oc adm node-logs ci-ln-m0wpfjb-f76d1-vnb5x-master-0 --path=oauth-server/audit-2022-05-11T18-57-32.395.log
      示例输出
      {"kind":"Event","apiVersion":"audit.k8s.io/v1","level":"Metadata","auditID":"13c20345-f33b-4b7d-b3b6-e7793f805621","stage":"ResponseComplete","requestURI":"/login","verb":"post","user":{"username":"system:anonymous","groups":["system:unauthenticated"]},"sourceIPs":["10.128.2.6"],"userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0","responseStatus":{"metadata":{},"code":302},"requestReceivedTimestamp":"2022-05-11T17:31:16.280155Z","stageTimestamp":"2022-05-11T17:31:16.297083Z","annotations":{"authentication.openshift.io/decision":"error","authentication.openshift.io/username":"kubeadmin","authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":""}}

      `authentication.openshift.io/decision` 注释的可能值为 `allow`、`deny` 或 `error`。

过滤审计日志

您可以使用 `jq` 或其他 JSON 解析工具来过滤 API 服务器审计日志。

记录到 API 服务器审计日志的信息量由设置的审计日志策略控制。

以下步骤提供使用 `jq` 过滤控制平面节点 `node-1.example.com` 上的审计日志的示例。有关使用 `jq` 的详细信息,请参阅 jq 手册

先决条件
  • 您可以作为具有 `cluster-admin` 角色的用户访问集群。

  • 您已安装 `jq`。

步骤
  • 按用户过滤 OpenShift API 服务器审计日志

    $ oc adm node-logs node-1.example.com  \
      --path=openshift-apiserver/audit.log \
      | jq 'select(.user.username == "myusername")'
  • 按用户代理过滤 OpenShift API 服务器审计日志

    $ oc adm node-logs node-1.example.com  \
      --path=openshift-apiserver/audit.log \
      | jq 'select(.userAgent == "cluster-version-operator/v0.0.0 (linux/amd64) kubernetes/$Format")'
  • 按特定 API 版本过滤 Kubernetes API 服务器审计日志,仅输出用户代理

    $ oc adm node-logs node-1.example.com  \
      --path=kube-apiserver/audit.log \
      | jq 'select(.requestURI | startswith("/apis/apiextensions.k8s.io/v1beta1")) | .userAgent'
  • 通过排除动词来过滤 OpenShift OAuth API 服务器审计日志

    $ oc adm node-logs node-1.example.com  \
      --path=oauth-apiserver/audit.log \
      | jq 'select(.verb != "get")'
  • 过滤 OpenShift OAuth 服务器审计日志,筛选出识别用户名并发生错误的事件

    $ oc adm node-logs node-1.example.com  \
      --path=oauth-server/audit.log \
      | jq 'select(.annotations["authentication.openshift.io/username"] != null and .annotations["authentication.openshift.io/decision"] == "error")'

收集审计日志

您可以使用 must-gather 工具收集审计日志以调试您的集群,您可以查看这些日志或将其发送给 Red Hat 支持。

步骤
  1. 运行 `oc adm must-gather` 命令,并使用 `-- /usr/bin/gather_audit_logs`

    $ oc adm must-gather -- /usr/bin/gather_audit_logs
  2. 从刚在您的工作目录中创建的 `must-gather` 目录创建压缩文件。例如,在使用 Linux 操作系统的计算机上,运行以下命令:

    $ tar cvaf must-gather.tar.gz must-gather.local.472290403699006248 (1)
    1 将 `must-gather-local.472290403699006248` 替换为实际的目录名称。
  3. 将压缩文件附加到 Red Hat 客户门户网站的 **客户支持** 页面 上的支持案例中。