Kubernetes

Table of Contents

1. Kubernetes 简介

Kubernetes is an open-source container-orchestration system for automating application deployment, scaling, and management.

参考:https://kubernetes.io/

1.1. Kubernetes 架构

Kubernetes 架构如图 1 所示,图片来源:https://www.aquasec.com/wiki/display/containers/Kubernetes+Architecture+101

k8s_architecture.png

Figure 1: High level Kubernetes architecture showing a cluster with a master and two worker nodes

Kubernetes 主要由以下几个核心组件组成:
1、etcd 保存了整个集群的状态;
2、API Server 提供了资源操作的唯一入口,并提供认证、授权、访问控制、API 注册和发现等机制;
3、Controller Manager 负责维护集群的状态,比如故障检测、自动扩展、滚动更新等;
4、Scheduler 负责资源的调度,按照预定的调度策略将 Pod 调度到相应的机器上;
5、kubelet 负责维护容器的生命周期,同时也负责 Volume(CVI)和网络(CNI)的管理;
6、kube-proxy 负责为 Service 提供 Cluster 内部的服务发现和负载均衡。

看另一个图 2 ,它清晰表明了 Kubernetes 的架构设计以及组件之间的通信协议,图片摘自 ppt:kubeadm Cluster Creation Internals: From Self-Hosting to Upgradability and HA

k8s_architecture2.gif

Figure 2: Kubernetes’ high-level component architecture

1.2. Kubernetes 开放接口(CRI、CNI、CSI)

为了便于系统的扩展,Kubernetes 中开放的以下接口:
1、CRI(Container Runtime Interface):容器运行时接口;
2、CNI(Container Network Interface):容器网络接口;
3、CSI(Container Storage Interface):容器存储接口。

1.2.1. CRI(Container Runtime Interface)

CRI 中定义了“容器”和“镜像”的服务接口。

Kubelet communicates with the container runtime (or a CRI shim for the runtime) over Unix sockets using the gRPC framework, where kubelet acts as a client and the CRI shim as the server. 如图 3 所示,图片摘自:https://kubernetes.io/blog/2016/12/container-runtime-interface-cri-in-kubernetes/

k8s_cri.png

Figure 3: k8s CRI 架构

CRI 两类服务(“容器”和“镜像”)的 grpc 定义如下,完整文件可参考:https://github.com/containerd/cri/blob/master/vendor/k8s.io/kubernetes/pkg/kubelet/apis/cri/runtime/v1alpha2/api.proto

// Runtime service defines the public APIs for remote container runtimes
service RuntimeService {
    // Version returns the runtime name, runtime version, and runtime API version.
    rpc Version(VersionRequest) returns (VersionResponse) {}

    // RunPodSandbox creates and starts a pod-level sandbox. Runtimes must ensure
    // the sandbox is in the ready state on success.
    rpc RunPodSandbox(RunPodSandboxRequest) returns (RunPodSandboxResponse) {}
    // StopPodSandbox stops any running process that is part of the sandbox and
    // reclaims network resources (e.g., IP addresses) allocated to the sandbox.
    // If there are any running containers in the sandbox, they must be forcibly
    // terminated.
    // This call is idempotent, and must not return an error if all relevant
    // resources have already been reclaimed. kubelet will call StopPodSandbox
    // at least once before calling RemovePodSandbox. It will also attempt to
    // reclaim resources eagerly, as soon as a sandbox is not needed. Hence,
    // multiple StopPodSandbox calls are expected.
    rpc StopPodSandbox(StopPodSandboxRequest) returns (StopPodSandboxResponse) {}
    // RemovePodSandbox removes the sandbox. If there are any running containers
    // in the sandbox, they must be forcibly terminated and removed.
    // This call is idempotent, and must not return an error if the sandbox has
    // already been removed.
    rpc RemovePodSandbox(RemovePodSandboxRequest) returns (RemovePodSandboxResponse) {}
    // PodSandboxStatus returns the status of the PodSandbox. If the PodSandbox is not
    // present, returns an error.
    rpc PodSandboxStatus(PodSandboxStatusRequest) returns (PodSandboxStatusResponse) {}
    // ListPodSandbox returns a list of PodSandboxes.
    rpc ListPodSandbox(ListPodSandboxRequest) returns (ListPodSandboxResponse) {}

    // CreateContainer creates a new container in specified PodSandbox
    rpc CreateContainer(CreateContainerRequest) returns (CreateContainerResponse) {}
    // StartContainer starts the container.
    rpc StartContainer(StartContainerRequest) returns (StartContainerResponse) {}
    // StopContainer stops a running container with a grace period (i.e., timeout).
    // This call is idempotent, and must not return an error if the container has
    // already been stopped.
    // TODO: what must the runtime do after the grace period is reached?
    rpc StopContainer(StopContainerRequest) returns (StopContainerResponse) {}
    // RemoveContainer removes the container. If the container is running, the
    // container must be forcibly removed.
    // This call is idempotent, and must not return an error if the container has
    // already been removed.
    rpc RemoveContainer(RemoveContainerRequest) returns (RemoveContainerResponse) {}
    // ListContainers lists all containers by filters.
    rpc ListContainers(ListContainersRequest) returns (ListContainersResponse) {}
    // ContainerStatus returns status of the container. If the container is not
    // present, returns an error.
    rpc ContainerStatus(ContainerStatusRequest) returns (ContainerStatusResponse) {}
    // UpdateContainerResources updates ContainerConfig of the container.
    rpc UpdateContainerResources(UpdateContainerResourcesRequest) returns (UpdateContainerResourcesResponse) {}

    // ExecSync runs a command in a container synchronously.
    rpc ExecSync(ExecSyncRequest) returns (ExecSyncResponse) {}
    // Exec prepares a streaming endpoint to execute a command in the container.
    rpc Exec(ExecRequest) returns (ExecResponse) {}
    // Attach prepares a streaming endpoint to attach to a running container.
    rpc Attach(AttachRequest) returns (AttachResponse) {}
    // PortForward prepares a streaming endpoint to forward ports from a PodSandbox.
    rpc PortForward(PortForwardRequest) returns (PortForwardResponse) {}

    // ContainerStats returns stats of the container. If the container does not
    // exist, the call returns an error.
    rpc ContainerStats(ContainerStatsRequest) returns (ContainerStatsResponse) {}
    // ListContainerStats returns stats of all running containers.
    rpc ListContainerStats(ListContainerStatsRequest) returns (ListContainerStatsResponse) {}

    // UpdateRuntimeConfig updates the runtime configuration based on the given request.
    rpc UpdateRuntimeConfig(UpdateRuntimeConfigRequest) returns (UpdateRuntimeConfigResponse) {}

    // Status returns the status of the runtime.
    rpc Status(StatusRequest) returns (StatusResponse) {}
}

// ImageService defines the public APIs for managing images.
service ImageService {
    // ListImages lists existing images.
    rpc ListImages(ListImagesRequest) returns (ListImagesResponse) {}
    // ImageStatus returns the status of the image. If the image is not
    // present, returns a response with ImageStatusResponse.Image set to
    // nil.
    rpc ImageStatus(ImageStatusRequest) returns (ImageStatusResponse) {}
    // PullImage pulls an image with authentication config.
    rpc PullImage(PullImageRequest) returns (PullImageResponse) {}
    // RemoveImage removes the image.
    // This call is idempotent, and must not return an error if the image has
    // already been removed.
    rpc RemoveImage(RemoveImageRequest) returns (RemoveImageResponse) {}
    // ImageFSInfo returns information of the filesystem that is used to store images.
    rpc ImageFsInfo(ImageFsInfoRequest) returns (ImageFsInfoResponse) {}
}

目前,CRI 的后端主要有:
1、Docker:kuberentes 最初就开始支持的容器运行时,不过,docker 公司在推广 OCI 标准。
2、cri-containerd:基于 Containerd 的 Kubernetes CRI 实现。
3、rkt:由 CoreOS 主推的用来跟 docker 抗衡的容器运行时。

1.2.2. CNI(Container Network Interface)

Kubernetes 本身并不提供网络功能,只是把网络接口开放出来,通过插件的形式实现。
既然 Kubernetes 中将容器的联网通过插件的方式来实现,那么该如何解决容器的联网问题呢?

如果您在本地单台机器上运行 docker 容器的话会注意到所有容器都会处在 docker0 网桥自动分配的一个网络 IP 段内(默认为 172.17.0.1/16),该值可以通过 docker 启动参数--bip 来设置。这样所有本地的所有的容器都拥有了一个 IP 地址,而且还是在一个网段内彼此就可以互相通信了。

但是 Kubernetes 管理的是集群,Kubernetes 中的网络要解决的核心问题就是每台主机的 IP 地址网段划分,以及单个容器的 IP 地址分配。概括为:
1、保证每个 Pod 拥有一个集群内唯一的 IP 地址;
2、保证跨 Node 的 Pod 可以互相通信等等。

为了解决该问题,Kubernetes 提出了 CNI(容器网络接口),下面这些方案都实现了 CNI 接口:
1、Flannel
2、Calico
3、Contiv
4、Weave net
5、Kube-router
6、Cilium
7、Canal

还有很多就不一一列举了,只要实现 Kubernetes 官方的设计的 CNI 就可以自己写一个网络插件,其标准可参考:https://github.com/containernetworking/cni/blob/master/SPEC.md

注 1:不同的 CNI 实现,其性能的对比情况可参考:https://itnext.io/benchmark-results-of-kubernetes-network-plugins-cni-over-10gbit-s-network-updated-april-2019-4a9886efe9c4
注 2:Docker 推出了 CNM(Docker LibnetworkContainer Network Model),和 CNI 形成竞争。

1.2.3. CSI(Container Storage Interface)

CSI 试图建立一个行业标准接口的规范,使存储供应商能够开发一个插件,就能使存储系统在多个容器编排器系统中工作。其标准可参考:https://github.com/container-storage-interface/spec/blob/master/spec.md

1.3. 安装 Kubernetes

安装 Kubernetes 前,你需要先安装 etcd、容器运行时环境(如 docker)、CNI 实现(如 Flannel,它为跨机器的 Pod 之间通信提供支持),它们需要在 Cluster 所有节点中安装并启动它们。

下面介绍如何安装 Kubernetes 软件包:
kubeadm: the command to bootstrap the cluster.
kubelet: the component that runs on all of the machines in your cluster and does things like starting pods and containers.
kubectl: the command line util to talk to your cluster.

在 Redhat 系统中,用 root 用户执行下面命令即可:

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
exclude=kube*
EOF

# Set SELinux in permissive mode (effectively disabling it)
setenforce 0
sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config

yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes

systemctl enable --now kubelet

安装前有一些注意事项,如需要关闭 swap,一些特定端口不能被防火墙禁止等等,详情可参考:https://kubernetes.io/docs/setup/independent/install-kubeadm/

1.4. 基本概念

1.4.1. pod

在 Kubernetes 里,集群调度的最小单元就是一个 pod,一个 pod 可以是一个容器,也可以是多个容器,例如你运行一个程序,其中使用了 nginx,使用 php 了,那么可以将这两个窗口放在同一个 pod 中,对他们提供统一的调配能力, 一个 pod 只能运行在一个 Node 上,而一个 Node 上可以有多个 pod。

2. kubeadm

使用 kubeadm 可以更方便地创建一个 Cluster。

参考:https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/

2.1. 创建 Master 节点(kubeadm init)

使用 kubeadm init 可以创建 Master 节点(不过当使用 Flannel 作为 CNI 实现时,需要指定参数 --pod-network-cidr=10.244.0.0/16 ),如:

$ kubeadm init --pod-network-cidr=10.244.0.0/16
......
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 172.31.27.248:6443 --token 77ozzy.ykywnuaaa1l83049 \
    --discovery-token-ca-cert-hash sha256:2954f21a6a30ee3992b5cc45b2a3d63d098552a1f38716ce7395e6851694831c

按照 kubeadm init 执行完成后的提示,还需要执行下面命令:

$ mkdir -p $HOME/.kube
$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
$ sudo chown $(id -u):$(id -g) $HOME/.kube/config

这样当前用户就可以使用 kubectl 了,这是因为 kubectl 默认会读取 $HOME/.kube/config 配置文件了;如果不这样做,使用 kubectl 前你需要指定 --kubeconfig 参数或者导出下面的环境变量:

export KUBECONFIG=/etc/kubernetes/admin.conf

按照 kubeadm init 执行完成后的提示,还需要部署 pod network 到 Cluster。当使用 Flannel 作为 CNI 时,其命令如下:

$ kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/62e44c867a2846fefb68bd5f178daf4da3095ccb/Documentation/kube-flannel.yml

使用 kubectl get nodes 检查节点运行情况:

$ kubectl get nodes
NAME                                               STATUS   ROLES    AGE     VERSION
ip-172-31-27-248.ap-southeast-1.compute.internal   Ready    master   4m41s   v1.14.1

可见,目前只有一个 master 节点,其状态为“Ready”。

注:如果状态显示“NotReady”,则可以通过下面命令来查找失败的原因:

$ kubectl describe nodes

在另外一个节点上,执行下面命令可以加入 Cluster:

$ kubeadm join 172.31.27.248:6443 --token 77ozzy.ykywnuaaa1l83049 \
    --discovery-token-ca-cert-hash sha256:2954f21a6a30ee3992b5cc45b2a3d63d098552a1f38716ce7395e6851694831c

默认,master 节点不会参与工作负载,如果想要 master 节点参与工作负载,则可以执行下面命令:

$ kubectl taint nodes --all node-role.kubernetes.io/master-
node/ip-172-31-27-248.ap-southeast-1.compute.internal untainted

2.1.1. kubeadm init 做了哪些事情

Running kubeadm init bootstraps a Kubernetes cluster. This consists of the following steps:

  1. kubeadm runs a series of pre-flight checks to validate the system state before making changes. Some checks only trigger warnings, others are considered errors and will exit kubeadm until the problem is corrected or the user specifies --skip-preflight-checks.
  2. kubeadm generates a token that additional nodes can use to register themselves with the master in future. Optionally, the user can provide a token.
  3. kubeadm generates a self-signed CA to provision identities for each component (including nodes) in the cluster. It also generates client certificates to be used by various components. If the user has provided their own CA by dropping it in the cert directory (configured via --cert-dir, by default /etc/kubernetes/pki), this step is skipped.
  4. Outputting a kubeconfig file for the kubelet to use to connect to the API server, as well as an additional kubeconfig file for administration.
  5. kubeadm generates Kubernetes static Pod manifests for the API server, controller manager and scheduler. It places them in /etc/kubernetes/manifests. The kubelet watches this directory for Pods to create on startup. These are the core components of Kubernetes. Once they are up and running kubeadm can set up and manage any additional components.
  6. kubeadm “taints” the master node so that only control plane components will run there. It also sets up the RBAC authorization system and writes a special ConfigMap that is used to bootstrap trust with the kubelets.
  7. kubeadm installs add-on components via the API server. Right now this is the internal DNS server and the kube-proxy DaemonSet.

2.2. 添加 Worker 节点到 Cluster(kubeadm join)

kubeadm init 执行完成后有提示,在其它节点中执行下面命令可以加入 Cluster:

$ kubeadm join 172.31.27.248:6443 --token 77ozzy.ykywnuaaa1l83049 \
    --discovery-token-ca-cert-hash sha256:2954f21a6a30ee3992b5cc45b2a3d63d098552a1f38716ce7395e6851694831c

如果不成功,可以在执行 kubeadm join 时指定参数 -v 3 以查看更多的日志。

成功添加当前节点到 Cluster 后,回到 master 节点,再次检测节点列表:

$ kubectl get nodes
NAME                                               STATUS   ROLES    AGE   VERSION
ip-172-31-22-197.ap-southeast-1.compute.internal   Ready    <none>   59s   v1.14.1
ip-172-31-27-248.ap-southeast-1.compute.internal   Ready    master   20m   v1.14.1

可见,现在已经多了一个节点。

2.2.1. kubeadm join 做了哪些事情

Running kubeadm join on each node in the cluster consists of the following steps:

  1. kubeadm downloads root CA information from the API server. It uses the token to verify the authenticity of that data.
  2. kubeadm creates a local key pair. It prepares a certificate signing request (CSR) and sends that off to the API server for signing. The bootstrap token is used to authenticate. The control plane will sign this CSR requested automatically.
  3. kubeadm configures the local kubelet to connect to the API server

2.3. 重置 Cluster(kubeadm reset)

如果 kubeadm init 出现失败,可以使用 kubeadm reset 来撤销 kubeadm init 对系统产生的影响。

3. kubectl

kubectl 是操作 Kubernetes clusters 的命令行工具。

3.1. kubectl 基本语法

kubectl 的语法格式如下:

kubectl [operation] [TYPE] [NAME] [flags]

上面语法中:
1、operation 是所支持的操作,如 create, get, describe, delete 等等;
2、TYPE 表示资源类型,注意资源类型是大小定不敏感的,可以使用单数或者复数形式,或者缩写形式,比如下面 3 个命令是等价的:

kubectl get pod pod1
kubectl get pods pod1
kubectl get po pod1

3、NAME 用于指定资源的名字(名字是大小写敏感的)。如果省略名字,则所有的资源都会显示出来,比如:

kubectl get pods         # 省略了资源名字,会显示所有资源

4、flags 用于指定其它选项,比如 -s or --server 用于指定 Kubernetes API server 的地址。注意,命令行中的选项比环境变量中相应的设置其优先级更高。

注意:如果要同时指定多个资源,可以采用下面的形式:
1、如果资源有相同的类型,则可以这样:

kubectl get pod example-pod1 example-pod2

2、如果多个资源的类型不同,则可以这样:

kubectl get pod/example-pod1 replicationcontroller/example-rc1

3、通过 -f 参数可以从一个或多个文件中指定资源,如:

kubectl get pod -f ./pod1.yaml -f ./pod2.yaml

参考:
https://kubernetes.io/docs/reference/kubectl/overview/
https://blog.csdn.net/xingwangc2014/article/details/51204224

3.2. Operations

kubectl 支持的操作如表 1 所示。

Table 1: kubectl operations
Operation Syntax Description
annotate kubectl annotate (-f FILENAME \| TYPE NAME \| TYPE/NAME) KEY_1=VAL_1 ... KEY_N=VAL_N [--overwrite] [--all] [--resource-version=version] [flags] Add or update the annotations of one or more resources.
api-versions kubectl api-versions [flags] List the API versions that are available.
apply kubectl apply -f FILENAME [flags] Apply a configuration change to a resource from a file or stdin.
attach kubectl attach POD -c CONTAINER [-i] [-t] [flags] Attach to a running container either to view the output stream or interact with the container (stdin).
autoscale kubectl autoscale (-f FILENAME \| TYPE NAME \| TYPE/NAME) [--min=MINPODS] --max=MAXPODS [--cpu-percent=CPU] [flags] Automatically scale the set of pods that are managed by a replication controller.
cluster-info kubectl cluster-info [flags] Display endpoint information about the master and services in the cluster.
config kubectl config SUBCOMMAND [flags] Modifies kubeconfig files. See the individual subcommands for details.
create kubectl create -f FILENAME [flags] Create one or more resources from a file or stdin.
delete kubectl delete (-f FILENAME \| TYPE [NAME \| /NAME \| -l label \| --all]) [flags] Delete resources either from a file, stdin, or specifying label selectors, names, resource selectors, or resources.
describe kubectl describe (-f FILENAME \| TYPE [NAME_PREFIX \| /NAME \| -l label]) [flags] Display the detailed state of one or more resources.
diff kubectl diff -f FILENAME [flags] Diff file or stdin against live configuration (BETA)
edit kubectl edit (-f FILENAME \| TYPE NAME \| TYPE/NAME) [flags] Edit and update the definition of one or more resources on the server by using the default editor.
exec kubectl exec POD [-c CONTAINER] [-i] [-t] [flags] [-- COMMAND [args...]] Execute a command against a container in a pod.
explain kubectl explain [--recursive=false] [flags] Get documentation of various resources. For instance pods, nodes, services, etc.
expose kubectl expose (-f FILENAME \| TYPE NAME \| TYPE/NAME) [--port=port] [--protocol=TCP\vertUDP] [--target-port=number-or-name] [--name=name] [--external-ip=external-ip-of-service] [--type=type] [flags] Expose a replication controller, service, or pod as a new Kubernetes service.
get kubectl get (-f FILENAME \| TYPE [NAME \| /NAME \| -l label]) [--watch] [--sort-by=FIELD] [[-o \| --output]=OUTPUT_FORMAT] [flags] List one or more resources.
label kubectl label (-f FILENAME \| TYPE NAME \| TYPE/NAME) KEY_1=VAL_1 ... KEY_N=VAL_N [--overwrite] [--all] [--resource-version=version] [flags] Add or update the labels of one or more resources.
logs kubectl logs POD [-c CONTAINER] [--follow] [flags] Print the logs for a container in a pod.
patch kubectl patch (-f FILENAME \| TYPE NAME \| TYPE/NAME) --patch PATCH [flags] Update one or more fields of a resource by using the strategic merge patch process.
port-forward kubectl port-forward POD [LOCAL_PORT:]REMOTE_PORT [...[LOCAL_PORT_N:]REMOTE_PORT_N] [flags] Forward one or more local ports to a pod.
proxy kubectl proxy [--port=PORT] [--www=static-dir] [--www-prefix=prefix] [--api-prefix=prefix] [flags] Run a proxy to the Kubernetes API server.
replace kubectl replace -f FILENAME Replace a resource from a file or stdin.
rolling-update kubectl rolling-update OLD_CONTROLLER_NAME ([NEW_CONTROLLER_NAME] --image=NEW_CONTAINER_IMAGE \| -f NEW_CONTROLLER_SPEC) [flags] Perform a rolling update by gradually replacing the specified replication controller and its pods.
run kubectl run NAME --image=image [--env="key=value"] [--port=port] [--replicas=replicas] [--dry-run=bool] [--overrides=inline-json] [flags] Run a specified image on the cluster.
scale kubectl scale (-f FILENAME \| TYPE NAME \| TYPE/NAME) --replicas=COUNT [--resource-version=version] [--current-replicas=count] [flags] Update the size of the specified replication controller.
stop kubectl stop Deprecated: Instead, see kubectl delete.
version kubectl version [--client] [flags] Display the Kubernetes version running on the client and server.

3.3. Resource types

执行命令 kubectl api-resources 可以列出所有支持的 Resource types,如:

$ kubectl api-resources
NAME                              SHORTNAMES   APIGROUP                       NAMESPACED   KIND
bindings                                                                      true         Binding
componentstatuses                 cs                                          false        ComponentStatus
configmaps                        cm                                          true         ConfigMap
endpoints                         ep                                          true         Endpoints
events                            ev                                          true         Event
limitranges                       limits                                      true         LimitRange
namespaces                        ns                                          false        Namespace
nodes                             no                                          false        Node
persistentvolumeclaims            pvc                                         true         PersistentVolumeClaim
persistentvolumes                 pv                                          false        PersistentVolume
pods                              po                                          true         Pod
podtemplates                                                                  true         PodTemplate
replicationcontrollers            rc                                          true         ReplicationController
resourcequotas                    quota                                       true         ResourceQuota
secrets                                                                       true         Secret
serviceaccounts                   sa                                          true         ServiceAccount
services                          svc                                         true         Service
mutatingwebhookconfigurations                  admissionregistration.k8s.io   false        MutatingWebhookConfiguration
validatingwebhookconfigurations                admissionregistration.k8s.io   false        ValidatingWebhookConfiguration
customresourcedefinitions         crd,crds     apiextensions.k8s.io           false        CustomResourceDefinition
apiservices                                    apiregistration.k8s.io         false        APIService
controllerrevisions                            apps                           true         ControllerRevision
daemonsets                        ds           apps                           true         DaemonSet
deployments                       deploy       apps                           true         Deployment
replicasets                       rs           apps                           true         ReplicaSet
statefulsets                      sts          apps                           true         StatefulSet
tokenreviews                                   authentication.k8s.io          false        TokenReview
localsubjectaccessreviews                      authorization.k8s.io           true         LocalSubjectAccessReview
selfsubjectaccessreviews                       authorization.k8s.io           false        SelfSubjectAccessReview
selfsubjectrulesreviews                        authorization.k8s.io           false        SelfSubjectRulesReview
subjectaccessreviews                           authorization.k8s.io           false        SubjectAccessReview
horizontalpodautoscalers          hpa          autoscaling                    true         HorizontalPodAutoscaler
cronjobs                          cj           batch                          true         CronJob
jobs                                           batch                          true         Job
certificatesigningrequests        csr          certificates.k8s.io            false        CertificateSigningRequest
leases                                         coordination.k8s.io            true         Lease
events                            ev           events.k8s.io                  true         Event
daemonsets                        ds           extensions                     true         DaemonSet
deployments                       deploy       extensions                     true         Deployment
ingresses                         ing          extensions                     true         Ingress
networkpolicies                   netpol       extensions                     true         NetworkPolicy
podsecuritypolicies               psp          extensions                     false        PodSecurityPolicy
replicasets                       rs           extensions                     true         ReplicaSet
ingresses                         ing          networking.k8s.io              true         Ingress
networkpolicies                   netpol       networking.k8s.io              true         NetworkPolicy
runtimeclasses                                 node.k8s.io                    false        RuntimeClass
poddisruptionbudgets              pdb          policy                         true         PodDisruptionBudget
podsecuritypolicies               psp          policy                         false        PodSecurityPolicy
clusterrolebindings                            rbac.authorization.k8s.io      false        ClusterRoleBinding
clusterroles                                   rbac.authorization.k8s.io      false        ClusterRole
rolebindings                                   rbac.authorization.k8s.io      true         RoleBinding
roles                                          rbac.authorization.k8s.io      true         Role
priorityclasses                   pc           scheduling.k8s.io              false        PriorityClass
csidrivers                                     storage.k8s.io                 false        CSIDriver
csinodes                                       storage.k8s.io                 false        CSINode
storageclasses                    sc           storage.k8s.io                 false        StorageClass
volumeattachments                              storage.k8s.io                 false        VolumeAttachment

4. 参考

Kubernetes Handbook——Kubernetes 中文指南/云原生应用架构实践手册:https://jimmysong.io/kubernetes-handbook/
Kubernetes 之 kubectl 常用命令:https://blog.csdn.net/xingwangc2014/article/details/51204224
K8s 基本概念入门:https://blog.csdn.net/TM6zNf87MDG7Bo/article/details/79621510

Author: cig01

Created: <2019-03-29 Fri>

Last updated: <2019-06-09 Sun>

Creator: Emacs 27.1 (Org mode 9.4)