Kubernetes
Table of Contents
1. Kubernetes 简介
Kubernetes is an open-source container-orchestration system for automating application deployment, scaling, and management.
1.1. Kubernetes 架构
Kubernetes 架构如图 1 所示,图片来源:https://www.aquasec.com/wiki/display/containers/Kubernetes+Architecture+101
Figure 1: High level Kubernetes architecture showing a cluster with a master and two worker nodes
Kubernetes 主要由以下几个核心组件组成:
1、etcd 保存了整个集群的状态;
2、API Server 提供了资源操作的唯一入口,并提供认证、授权、访问控制、API 注册和发现等机制;
3、Controller Manager 负责维护集群的状态,比如故障检测、自动扩展、滚动更新等;
4、Scheduler 负责资源的调度,按照预定的调度策略将 Pod 调度到相应的机器上;
5、kubelet 负责维护容器的生命周期,同时也负责 Volume(CVI)和网络(CNI)的管理;
6、kube-proxy 负责为 Service 提供 Cluster 内部的服务发现和负载均衡。
看另一个图 2 ,它清晰表明了 Kubernetes 的架构设计以及组件之间的通信协议,图片摘自 ppt:kubeadm Cluster Creation Internals: From Self-Hosting to Upgradability and HA
Figure 2: Kubernetes’ high-level component architecture
1.2. Kubernetes 开放接口(CRI、CNI、CSI)
为了便于系统的扩展,Kubernetes 中开放的以下接口:
1、CRI(Container Runtime Interface):容器运行时接口;
2、CNI(Container Network Interface):容器网络接口;
3、CSI(Container Storage Interface):容器存储接口。
1.2.1. CRI(Container Runtime Interface)
CRI 中定义了“容器”和“镜像”的服务接口。
Kubelet communicates with the container runtime (or a CRI shim for the runtime) over Unix sockets using the gRPC framework, where kubelet acts as a client and the CRI shim as the server. 如图 3 所示,图片摘自:https://kubernetes.io/blog/2016/12/container-runtime-interface-cri-in-kubernetes/
Figure 3: k8s CRI 架构
CRI 两类服务(“容器”和“镜像”)的 grpc 定义如下,完整文件可参考:https://github.com/containerd/cri/blob/master/vendor/k8s.io/kubernetes/pkg/kubelet/apis/cri/runtime/v1alpha2/api.proto
// Runtime service defines the public APIs for remote container runtimes service RuntimeService { // Version returns the runtime name, runtime version, and runtime API version. rpc Version(VersionRequest) returns (VersionResponse) {} // RunPodSandbox creates and starts a pod-level sandbox. Runtimes must ensure // the sandbox is in the ready state on success. rpc RunPodSandbox(RunPodSandboxRequest) returns (RunPodSandboxResponse) {} // StopPodSandbox stops any running process that is part of the sandbox and // reclaims network resources (e.g., IP addresses) allocated to the sandbox. // If there are any running containers in the sandbox, they must be forcibly // terminated. // This call is idempotent, and must not return an error if all relevant // resources have already been reclaimed. kubelet will call StopPodSandbox // at least once before calling RemovePodSandbox. It will also attempt to // reclaim resources eagerly, as soon as a sandbox is not needed. Hence, // multiple StopPodSandbox calls are expected. rpc StopPodSandbox(StopPodSandboxRequest) returns (StopPodSandboxResponse) {} // RemovePodSandbox removes the sandbox. If there are any running containers // in the sandbox, they must be forcibly terminated and removed. // This call is idempotent, and must not return an error if the sandbox has // already been removed. rpc RemovePodSandbox(RemovePodSandboxRequest) returns (RemovePodSandboxResponse) {} // PodSandboxStatus returns the status of the PodSandbox. If the PodSandbox is not // present, returns an error. rpc PodSandboxStatus(PodSandboxStatusRequest) returns (PodSandboxStatusResponse) {} // ListPodSandbox returns a list of PodSandboxes. rpc ListPodSandbox(ListPodSandboxRequest) returns (ListPodSandboxResponse) {} // CreateContainer creates a new container in specified PodSandbox rpc CreateContainer(CreateContainerRequest) returns (CreateContainerResponse) {} // StartContainer starts the container. rpc StartContainer(StartContainerRequest) returns (StartContainerResponse) {} // StopContainer stops a running container with a grace period (i.e., timeout). // This call is idempotent, and must not return an error if the container has // already been stopped. // TODO: what must the runtime do after the grace period is reached? rpc StopContainer(StopContainerRequest) returns (StopContainerResponse) {} // RemoveContainer removes the container. If the container is running, the // container must be forcibly removed. // This call is idempotent, and must not return an error if the container has // already been removed. rpc RemoveContainer(RemoveContainerRequest) returns (RemoveContainerResponse) {} // ListContainers lists all containers by filters. rpc ListContainers(ListContainersRequest) returns (ListContainersResponse) {} // ContainerStatus returns status of the container. If the container is not // present, returns an error. rpc ContainerStatus(ContainerStatusRequest) returns (ContainerStatusResponse) {} // UpdateContainerResources updates ContainerConfig of the container. rpc UpdateContainerResources(UpdateContainerResourcesRequest) returns (UpdateContainerResourcesResponse) {} // ExecSync runs a command in a container synchronously. rpc ExecSync(ExecSyncRequest) returns (ExecSyncResponse) {} // Exec prepares a streaming endpoint to execute a command in the container. rpc Exec(ExecRequest) returns (ExecResponse) {} // Attach prepares a streaming endpoint to attach to a running container. rpc Attach(AttachRequest) returns (AttachResponse) {} // PortForward prepares a streaming endpoint to forward ports from a PodSandbox. rpc PortForward(PortForwardRequest) returns (PortForwardResponse) {} // ContainerStats returns stats of the container. If the container does not // exist, the call returns an error. rpc ContainerStats(ContainerStatsRequest) returns (ContainerStatsResponse) {} // ListContainerStats returns stats of all running containers. rpc ListContainerStats(ListContainerStatsRequest) returns (ListContainerStatsResponse) {} // UpdateRuntimeConfig updates the runtime configuration based on the given request. rpc UpdateRuntimeConfig(UpdateRuntimeConfigRequest) returns (UpdateRuntimeConfigResponse) {} // Status returns the status of the runtime. rpc Status(StatusRequest) returns (StatusResponse) {} } // ImageService defines the public APIs for managing images. service ImageService { // ListImages lists existing images. rpc ListImages(ListImagesRequest) returns (ListImagesResponse) {} // ImageStatus returns the status of the image. If the image is not // present, returns a response with ImageStatusResponse.Image set to // nil. rpc ImageStatus(ImageStatusRequest) returns (ImageStatusResponse) {} // PullImage pulls an image with authentication config. rpc PullImage(PullImageRequest) returns (PullImageResponse) {} // RemoveImage removes the image. // This call is idempotent, and must not return an error if the image has // already been removed. rpc RemoveImage(RemoveImageRequest) returns (RemoveImageResponse) {} // ImageFSInfo returns information of the filesystem that is used to store images. rpc ImageFsInfo(ImageFsInfoRequest) returns (ImageFsInfoResponse) {} }
目前,CRI 的后端主要有:
1、Docker:kuberentes 最初就开始支持的容器运行时,不过,docker 公司在推广 OCI 标准。
2、cri-containerd:基于 Containerd 的 Kubernetes CRI 实现。
3、rkt:由 CoreOS 主推的用来跟 docker 抗衡的容器运行时。
1.2.2. CNI(Container Network Interface)
Kubernetes 本身并不提供网络功能,只是把网络接口开放出来,通过插件的形式实现。
既然 Kubernetes 中将容器的联网通过插件的方式来实现,那么该如何解决容器的联网问题呢?
如果您在本地单台机器上运行 docker 容器的话会注意到所有容器都会处在 docker0 网桥自动分配的一个网络 IP 段内(默认为 172.17.0.1/16),该值可以通过 docker 启动参数--bip 来设置。这样所有本地的所有的容器都拥有了一个 IP 地址,而且还是在一个网段内彼此就可以互相通信了。
但是 Kubernetes 管理的是集群,Kubernetes 中的网络要解决的核心问题就是每台主机的 IP 地址网段划分,以及单个容器的 IP 地址分配。概括为:
1、保证每个 Pod 拥有一个集群内唯一的 IP 地址;
2、保证跨 Node 的 Pod 可以互相通信等等。
为了解决该问题,Kubernetes 提出了 CNI(容器网络接口),下面这些方案都实现了 CNI 接口:
1、Flannel
2、Calico
3、Contiv
4、Weave net
5、Kube-router
6、Cilium
7、Canal
还有很多就不一一列举了,只要实现 Kubernetes 官方的设计的 CNI 就可以自己写一个网络插件,其标准可参考:https://github.com/containernetworking/cni/blob/master/SPEC.md
注 1:不同的 CNI 实现,其性能的对比情况可参考:https://itnext.io/benchmark-results-of-kubernetes-network-plugins-cni-over-10gbit-s-network-updated-april-2019-4a9886efe9c4
注 2:Docker 推出了 CNM(Docker LibnetworkContainer Network Model),和 CNI 形成竞争。
1.2.3. CSI(Container Storage Interface)
CSI 试图建立一个行业标准接口的规范,使存储供应商能够开发一个插件,就能使存储系统在多个容器编排器系统中工作。其标准可参考:https://github.com/container-storage-interface/spec/blob/master/spec.md
1.3. 安装 Kubernetes
安装 Kubernetes 前,你需要先安装 etcd、容器运行时环境(如 docker)、CNI 实现(如 Flannel,它为跨机器的 Pod 之间通信提供支持),它们需要在 Cluster 所有节点中安装并启动它们。
下面介绍如何安装 Kubernetes 软件包:
kubeadm
: the command to bootstrap the cluster.
kubelet
: the component that runs on all of the machines in your cluster and does things like starting pods and containers.
kubectl
: the command line util to talk to your cluster.
在 Redhat 系统中,用 root 用户执行下面命令即可:
cat <<EOF > /etc/yum.repos.d/kubernetes.repo [kubernetes] name=Kubernetes baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64 enabled=1 gpgcheck=1 repo_gpgcheck=1 gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg exclude=kube* EOF # Set SELinux in permissive mode (effectively disabling it) setenforce 0 sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes systemctl enable --now kubelet
安装前有一些注意事项,如需要关闭 swap,一些特定端口不能被防火墙禁止等等,详情可参考:https://kubernetes.io/docs/setup/independent/install-kubeadm/
1.4. 基本概念
1.4.1. pod
在 Kubernetes 里,集群调度的最小单元就是一个 pod,一个 pod 可以是一个容器,也可以是多个容器,例如你运行一个程序,其中使用了 nginx,使用 php 了,那么可以将这两个窗口放在同一个 pod 中,对他们提供统一的调配能力, 一个 pod 只能运行在一个 Node 上,而一个 Node 上可以有多个 pod。
2. kubeadm
使用 kubeadm
可以更方便地创建一个 Cluster。
参考:https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/
2.1. 创建 Master 节点(kubeadm init)
使用 kubeadm init
可以创建 Master 节点(不过当使用 Flannel 作为 CNI 实现时,需要指定参数 --pod-network-cidr=10.244.0.0/16
),如:
$ kubeadm init --pod-network-cidr=10.244.0.0/16 ...... [addons] Applied essential addon: CoreDNS [addons] Applied essential addon: kube-proxy Your Kubernetes control-plane has initialized successfully! To start using your cluster, you need to run the following as a regular user: mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config You should now deploy a pod network to the cluster. Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/addons/ Then you can join any number of worker nodes by running the following on each as root: kubeadm join 172.31.27.248:6443 --token 77ozzy.ykywnuaaa1l83049 \ --discovery-token-ca-cert-hash sha256:2954f21a6a30ee3992b5cc45b2a3d63d098552a1f38716ce7395e6851694831c
按照 kubeadm init
执行完成后的提示,还需要执行下面命令:
$ mkdir -p $HOME/.kube $ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config $ sudo chown $(id -u):$(id -g) $HOME/.kube/config
这样当前用户就可以使用 kubectl
了,这是因为 kubectl
默认会读取 $HOME/.kube/config
配置文件了;如果不这样做,使用 kubectl
前你需要指定 --kubeconfig
参数或者导出下面的环境变量:
export KUBECONFIG=/etc/kubernetes/admin.conf
按照 kubeadm init
执行完成后的提示,还需要部署 pod network 到 Cluster。当使用 Flannel 作为 CNI 时,其命令如下:
$ kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/62e44c867a2846fefb68bd5f178daf4da3095ccb/Documentation/kube-flannel.yml
使用 kubectl get nodes
检查节点运行情况:
$ kubectl get nodes NAME STATUS ROLES AGE VERSION ip-172-31-27-248.ap-southeast-1.compute.internal Ready master 4m41s v1.14.1
可见,目前只有一个 master 节点,其状态为“Ready”。
注:如果状态显示“NotReady”,则可以通过下面命令来查找失败的原因:
$ kubectl describe nodes
在另外一个节点上,执行下面命令可以加入 Cluster:
$ kubeadm join 172.31.27.248:6443 --token 77ozzy.ykywnuaaa1l83049 \ --discovery-token-ca-cert-hash sha256:2954f21a6a30ee3992b5cc45b2a3d63d098552a1f38716ce7395e6851694831c
默认,master 节点不会参与工作负载,如果想要 master 节点参与工作负载,则可以执行下面命令:
$ kubectl taint nodes --all node-role.kubernetes.io/master- node/ip-172-31-27-248.ap-southeast-1.compute.internal untainted
2.1.1. kubeadm init 做了哪些事情
Running kubeadm init
bootstraps a Kubernetes cluster. This consists of the following steps:
- kubeadm runs a series of pre-flight checks to validate the system state before making changes. Some checks only trigger warnings, others are considered errors and will exit kubeadm until the problem is corrected or the user specifies --skip-preflight-checks.
- kubeadm generates a token that additional nodes can use to register themselves with the master in future. Optionally, the user can provide a token.
- kubeadm generates a self-signed CA to provision identities for each component (including nodes) in the cluster. It also generates client certificates to be used by various components. If the user has provided their own CA by dropping it in the cert directory (configured via --cert-dir, by default /etc/kubernetes/pki), this step is skipped.
- Outputting a kubeconfig file for the kubelet to use to connect to the API server, as well as an additional kubeconfig file for administration.
- kubeadm generates Kubernetes static Pod manifests for the API server, controller manager and scheduler. It places them in /etc/kubernetes/manifests. The kubelet watches this directory for Pods to create on startup. These are the core components of Kubernetes. Once they are up and running kubeadm can set up and manage any additional components.
- kubeadm “taints” the master node so that only control plane components will run there. It also sets up the RBAC authorization system and writes a special ConfigMap that is used to bootstrap trust with the kubelets.
- kubeadm installs add-on components via the API server. Right now this is the internal DNS server and the kube-proxy DaemonSet.
2.2. 添加 Worker 节点到 Cluster(kubeadm join)
kubeadm init
执行完成后有提示,在其它节点中执行下面命令可以加入 Cluster:
$ kubeadm join 172.31.27.248:6443 --token 77ozzy.ykywnuaaa1l83049 \ --discovery-token-ca-cert-hash sha256:2954f21a6a30ee3992b5cc45b2a3d63d098552a1f38716ce7395e6851694831c
如果不成功,可以在执行 kubeadm join
时指定参数 -v 3
以查看更多的日志。
成功添加当前节点到 Cluster 后,回到 master 节点,再次检测节点列表:
$ kubectl get nodes NAME STATUS ROLES AGE VERSION ip-172-31-22-197.ap-southeast-1.compute.internal Ready <none> 59s v1.14.1 ip-172-31-27-248.ap-southeast-1.compute.internal Ready master 20m v1.14.1
可见,现在已经多了一个节点。
2.2.1. kubeadm join 做了哪些事情
Running kubeadm join
on each node in the cluster consists of the following steps:
- kubeadm downloads root CA information from the API server. It uses the token to verify the authenticity of that data.
- kubeadm creates a local key pair. It prepares a certificate signing request (CSR) and sends that off to the API server for signing. The bootstrap token is used to authenticate. The control plane will sign this CSR requested automatically.
- kubeadm configures the local kubelet to connect to the API server
2.3. 重置 Cluster(kubeadm reset)
如果 kubeadm init
出现失败,可以使用 kubeadm reset
来撤销 kubeadm init
对系统产生的影响。
3. kubectl
kubectl
是操作 Kubernetes clusters 的命令行工具。
3.1. kubectl 基本语法
kubectl
的语法格式如下:
kubectl [operation] [TYPE] [NAME] [flags]
上面语法中:
1、operation 是所支持的操作,如 create
, get
, describe
, delete
等等;
2、TYPE 表示资源类型,注意资源类型是大小定不敏感的,可以使用单数或者复数形式,或者缩写形式,比如下面 3 个命令是等价的:
kubectl get pod pod1 kubectl get pods pod1 kubectl get po pod1
3、NAME 用于指定资源的名字(名字是大小写敏感的)。如果省略名字,则所有的资源都会显示出来,比如:
kubectl get pods # 省略了资源名字,会显示所有资源
4、flags 用于指定其它选项,比如 -s
or --server
用于指定 Kubernetes API server 的地址。注意,命令行中的选项比环境变量中相应的设置其优先级更高。
注意:如果要同时指定多个资源,可以采用下面的形式:
1、如果资源有相同的类型,则可以这样:
kubectl get pod example-pod1 example-pod2
2、如果多个资源的类型不同,则可以这样:
kubectl get pod/example-pod1 replicationcontroller/example-rc1
3、通过 -f
参数可以从一个或多个文件中指定资源,如:
kubectl get pod -f ./pod1.yaml -f ./pod2.yaml
参考:
https://kubernetes.io/docs/reference/kubectl/overview/
https://blog.csdn.net/xingwangc2014/article/details/51204224
3.2. Operations
kubectl
支持的操作如表 1 所示。
Operation | Syntax | Description |
---|---|---|
annotate | kubectl annotate (-f FILENAME \| TYPE NAME \| TYPE/NAME) KEY_1=VAL_1 ... KEY_N=VAL_N [--overwrite] [--all] [--resource-version=version] [flags] | Add or update the annotations of one or more resources. |
api-versions | kubectl api-versions [flags] | List the API versions that are available. |
apply | kubectl apply -f FILENAME [flags] | Apply a configuration change to a resource from a file or stdin. |
attach | kubectl attach POD -c CONTAINER [-i] [-t] [flags] | Attach to a running container either to view the output stream or interact with the container (stdin). |
autoscale | kubectl autoscale (-f FILENAME \| TYPE NAME \| TYPE/NAME) [--min=MINPODS] --max=MAXPODS [--cpu-percent=CPU] [flags] | Automatically scale the set of pods that are managed by a replication controller. |
cluster-info | kubectl cluster-info [flags] | Display endpoint information about the master and services in the cluster. |
config | kubectl config SUBCOMMAND [flags] | Modifies kubeconfig files. See the individual subcommands for details. |
create | kubectl create -f FILENAME [flags] | Create one or more resources from a file or stdin. |
delete | kubectl delete (-f FILENAME \| TYPE [NAME \| /NAME \| -l label \| --all]) [flags] | Delete resources either from a file, stdin, or specifying label selectors, names, resource selectors, or resources. |
describe | kubectl describe (-f FILENAME \| TYPE [NAME_PREFIX \| /NAME \| -l label]) [flags] | Display the detailed state of one or more resources. |
diff | kubectl diff -f FILENAME [flags] | Diff file or stdin against live configuration (BETA) |
edit | kubectl edit (-f FILENAME \| TYPE NAME \| TYPE/NAME) [flags] | Edit and update the definition of one or more resources on the server by using the default editor. |
exec | kubectl exec POD [-c CONTAINER] [-i] [-t] [flags] [-- COMMAND [args...]] | Execute a command against a container in a pod. |
explain | kubectl explain [--recursive=false] [flags] | Get documentation of various resources. For instance pods, nodes, services, etc. |
expose | kubectl expose (-f FILENAME \| TYPE NAME \| TYPE/NAME) [--port=port] [--protocol=TCP\vertUDP] [--target-port=number-or-name] [--name=name] [--external-ip=external-ip-of-service] [--type=type] [flags] | Expose a replication controller, service, or pod as a new Kubernetes service. |
get | kubectl get (-f FILENAME \| TYPE [NAME \| /NAME \| -l label]) [--watch] [--sort-by=FIELD] [[-o \| --output]=OUTPUT_FORMAT] [flags] | List one or more resources. |
label | kubectl label (-f FILENAME \| TYPE NAME \| TYPE/NAME) KEY_1=VAL_1 ... KEY_N=VAL_N [--overwrite] [--all] [--resource-version=version] [flags] | Add or update the labels of one or more resources. |
logs | kubectl logs POD [-c CONTAINER] [--follow] [flags] | Print the logs for a container in a pod. |
patch | kubectl patch (-f FILENAME \| TYPE NAME \| TYPE/NAME) --patch PATCH [flags] | Update one or more fields of a resource by using the strategic merge patch process. |
port-forward | kubectl port-forward POD [LOCAL_PORT:]REMOTE_PORT [...[LOCAL_PORT_N:]REMOTE_PORT_N] [flags] | Forward one or more local ports to a pod. |
proxy | kubectl proxy [--port=PORT] [--www=static-dir] [--www-prefix=prefix] [--api-prefix=prefix] [flags] | Run a proxy to the Kubernetes API server. |
replace | kubectl replace -f FILENAME | Replace a resource from a file or stdin. |
rolling-update | kubectl rolling-update OLD_CONTROLLER_NAME ([NEW_CONTROLLER_NAME] --image=NEW_CONTAINER_IMAGE \| -f NEW_CONTROLLER_SPEC) [flags] | Perform a rolling update by gradually replacing the specified replication controller and its pods. |
run | kubectl run NAME --image=image [--env="key=value"] [--port=port] [--replicas=replicas] [--dry-run=bool] [--overrides=inline-json] [flags] | Run a specified image on the cluster. |
scale | kubectl scale (-f FILENAME \| TYPE NAME \| TYPE/NAME) --replicas=COUNT [--resource-version=version] [--current-replicas=count] [flags] | Update the size of the specified replication controller. |
stop | kubectl stop | Deprecated: Instead, see kubectl delete. |
version | kubectl version [--client] [flags] | Display the Kubernetes version running on the client and server. |
3.3. Resource types
执行命令 kubectl api-resources
可以列出所有支持的 Resource types,如:
$ kubectl api-resources NAME SHORTNAMES APIGROUP NAMESPACED KIND bindings true Binding componentstatuses cs false ComponentStatus configmaps cm true ConfigMap endpoints ep true Endpoints events ev true Event limitranges limits true LimitRange namespaces ns false Namespace nodes no false Node persistentvolumeclaims pvc true PersistentVolumeClaim persistentvolumes pv false PersistentVolume pods po true Pod podtemplates true PodTemplate replicationcontrollers rc true ReplicationController resourcequotas quota true ResourceQuota secrets true Secret serviceaccounts sa true ServiceAccount services svc true Service mutatingwebhookconfigurations admissionregistration.k8s.io false MutatingWebhookConfiguration validatingwebhookconfigurations admissionregistration.k8s.io false ValidatingWebhookConfiguration customresourcedefinitions crd,crds apiextensions.k8s.io false CustomResourceDefinition apiservices apiregistration.k8s.io false APIService controllerrevisions apps true ControllerRevision daemonsets ds apps true DaemonSet deployments deploy apps true Deployment replicasets rs apps true ReplicaSet statefulsets sts apps true StatefulSet tokenreviews authentication.k8s.io false TokenReview localsubjectaccessreviews authorization.k8s.io true LocalSubjectAccessReview selfsubjectaccessreviews authorization.k8s.io false SelfSubjectAccessReview selfsubjectrulesreviews authorization.k8s.io false SelfSubjectRulesReview subjectaccessreviews authorization.k8s.io false SubjectAccessReview horizontalpodautoscalers hpa autoscaling true HorizontalPodAutoscaler cronjobs cj batch true CronJob jobs batch true Job certificatesigningrequests csr certificates.k8s.io false CertificateSigningRequest leases coordination.k8s.io true Lease events ev events.k8s.io true Event daemonsets ds extensions true DaemonSet deployments deploy extensions true Deployment ingresses ing extensions true Ingress networkpolicies netpol extensions true NetworkPolicy podsecuritypolicies psp extensions false PodSecurityPolicy replicasets rs extensions true ReplicaSet ingresses ing networking.k8s.io true Ingress networkpolicies netpol networking.k8s.io true NetworkPolicy runtimeclasses node.k8s.io false RuntimeClass poddisruptionbudgets pdb policy true PodDisruptionBudget podsecuritypolicies psp policy false PodSecurityPolicy clusterrolebindings rbac.authorization.k8s.io false ClusterRoleBinding clusterroles rbac.authorization.k8s.io false ClusterRole rolebindings rbac.authorization.k8s.io true RoleBinding roles rbac.authorization.k8s.io true Role priorityclasses pc scheduling.k8s.io false PriorityClass csidrivers storage.k8s.io false CSIDriver csinodes storage.k8s.io false CSINode storageclasses sc storage.k8s.io false StorageClass volumeattachments storage.k8s.io false VolumeAttachment
4. 参考
Kubernetes Handbook——Kubernetes 中文指南/云原生应用架构实践手册:https://jimmysong.io/kubernetes-handbook/
Kubernetes 之 kubectl 常用命令:https://blog.csdn.net/xingwangc2014/article/details/51204224
K8s 基本概念入门:https://blog.csdn.net/TM6zNf87MDG7Bo/article/details/79621510