ansible k8s inventory使用说明及示例

使用如下命令可以查看插件的使用方法。

# ansible-doc -t k8s

示例如下, 文档中说：“File must be named k8s.yaml l”。插件的配置名，必须是k8s.yaml或。但是实际好像并不是这样。

# cat k8s.yaml 
plugin: k8s
connections:- kubeconfig: /data/apps/admin.kubeconfig

然后使用ansible-inventory命令可以获取到集群内的信息。

# ansible-inventory -i k8s.yaml --list > /tmp/k8s_inventory.out

inventory的组织层次

• 10-6-56-10_7443

  • 包含了所有以namespace命名的子组,如namespace_default, namespace_devops

• _meta

  • 主要是hostvars

• all

  • 包含10-6-56-10_7443这个组
  • 包含以lable_开头的各个组
    • 如label_app.kubernetes.io/name_www
    • label_app_contour
    • 再如label_pod-template-hash_54899cb8f9
  • 不包含以namespace_开头的各个组

• 以namespace开头命名的组名

  • 如namespace_default(包含下面两个)
    • namespace_default_pods
      • 所有的pod名（实际上是${pod}_${container}这样的名称）
    • namespace_default_services
      • 就是其名称空间的svc名称

• 以label开头命名的组名, 把各个label都实现了分组

  • 如label_pod-template-hash_7cf4cfb78f
    • 此组内，只有pod名称
  • label_app.kubernetes.io/name_app1

kubectl connection连接插件

由于在_meta中，已经为各主机配置了"ansible_connection": &#kubectl"变量。

# cat /tmp/k8s_inventory.out  |jq '."_meta"."hostvars"."app1-6bc54678fd-btsfp_app1"'

{"annotations": {"kubectl.kubernetes.io/restartedAt": "2021-08-12T16:27:37+08:00","prometheus_io_path": "/metrics","prometheus_io_port": "18080","prometheus_io_scrape": "true"},"ansible_connection": &#kubectl","ansible_kubectl_container": "app1","ansible_kubectl_namespace": "test1","ansible_kubectl_pod": "app1-6bc54678fd-btsfp","ansible_remote_tmp": "/tmp/","cluster_name": null,"container_image": "10.6.63.12:5000/apps/app1:dev_20210423k8s-leo_2021042312094701","container_name": "10.6.63.12:5000/apps/app1:dev_20210423k8s-leo_2021042312094701","container_ready": true,"container_state": "Running","docker_cgroupdriver": "systemd","docker_insecure_registry": [],"labels": {"app.kubernetes.io/instance": "app1","app.kubernetes.io/name": "app1","pod-template-hash": "6bc54678fd"},"object_type": "pod","pod_host_ip": "10.6.56.163","pod_ip": "10.20.65.69","pod_name": null,"pod_node_name": "k8s-node-56-163","pod_phase": "Running","pod_resource_version": "77030206","pod_self_link": null,"pod_uid": "eb95536c-04ce-4d40-8abe-242cecabc686"
}

所以可以直接以inventory中的组名或${pod}_${container}名（不能直接使用pod的名称）来进行连接，而不需要再为其指定使用kubectl这个connection的插件。

# k get pod
NAME                                 READY   STATUS    RESTARTS   AGE
app1                                 1/1     Running   1          207d
mynginxdeployment-6c9b57b5b8-q447q   1/1     Running   0          65d
mynginxdeployment-6c9b57b5b8-v9plj   1/1     Running   0          65d
mynginxdeployment-6c9b57b5b8-x6hwn   1/1     Running   0          65d
mynginxdeployment-74f48d646b-jxph7   1/1     Running   0          65d
nginx-deploy-2-7b7b5f64cf-qq5sl      1/1     Running   2030       173d
nginx-deploy-2-7b7b5f64cf-zj5fx      1/1     Running   2030       173d
nginx-deploy-7b7b5f64cf-h4dt4        1/1     Running   2444       207d
nginx-deploy-7b7b5f64cf-j42vw        1/1     Running   2444       207d

连接单个pod

# ansible -i k8s.yaml nginx-deploy-7b7b5f64cf-j42vw_nginx -m shell -a "pwd; date"
[WARNING]: Invalid characters were found in group names but not replaced, use -vvvv to see details
nginx-deploy-7b7b5f64cf-j42vw_nginx | CHANGED | rc=0 >>
/
Mon Feb 28 06:23:09 UTC 2022

连接deploy

使用app.kubernetes.io/name的label

使用label来操作。这个会因为其中包含了svc的名称，而报一个错误，不过其他的存在的pod还是会有结果的。

# ansible  -i k8s.yaml label_app.kubernetes.io/name_app1 -m shell -a "date; hostname"

使用名为app的label

这种app: app1的label是仅包含pod名称列表的组名。

# ansible  -i k8s.yaml label_app_app1 -m shell -a "date; hostname"

问题

service名称的用处

以label开头的组内，总是包含一个以svc名称命名的主机，如label_app.kubernetes.io/instance_app1的hosts中有一个app1, 这个app1不是一个真正的pod, 所以在操作时，会报错。

# ansible -i k8s.yaml label_app.kubernetes.io/name_app1 -m shell -a "hostname"
[WARNING]: Invalid characters were found in group names but not replaced, use -vvvv to see details
app1 | UNREACHABLE! => {"changed": false,"msg": "Failed to connect to the host via ssh: ssh: Could not resolve hostname app1: Name or service not known","unreachable": true
}
app1-558cdfc558-72flq_app1 | CHANGED | rc=0 >>
app1-558cdfc558-72flq

后发现，这个app1，可能指代的是namespace_ox_services组中hosts中指代的app1。

但是这个app1的host，到底是有什么作用呢？

alermanager的组中没有额外的host:

"label_app.kubernetes.io/instance_main": {"hosts": ["alertmanager-main-0_alertmanager","alertmanager-main-0_config-reloader","alertmanager-main-1_alertmanager","alertmanager-main-1_config-reloader","alertmanager-main-2_alertmanager","alertmanager-main-2_config-reloader"]}

其namespace_monitoring_services中确实没有main这个svc。

"namespace_monitoring_services": {"hosts": ["alertmanager-main","alertmanager-operated","blackbox-exporter","grafana","kube-state-metrics","node-exporter","outer-node-exporter","prometheus-adapter","prometheus-k8s","prometheus-operated","prometheus-operator"]},

有可能是app.kubernetes.io/name和app.kubernetes.io/instance这两个label比较特殊的原因。

无法操作没有python环境的pod

# ansible -i k8s.yaml grafana-5d5658d6d9-2sp2z_grafana -m shell -a "date"
[WARNING]: Invalid characters were found in group names but not replaced, use -vvvv to see details
[WARNING]: No python interpreters found for host grafana-5d5658d6d9-2sp2z_grafana (tried ['python3.10', 'python3.9', 'python3.8', 'python3.7', 'python3.6', 'python3.5',
'/usr/bin/python3', '/usr/libexec/platform-python', 'python2.7', '/usr/bin/python', 'python'])
grafana-5d5658d6d9-2sp2z_grafana | FAILED! => {"ansible_facts": {"discovered_interpreter_python": "/usr/bin/python"},"changed": false,"module_stderr": "/bin/sh: /usr/bin/python: not foundncommand terminated with exit code 127n","module_stdout": "","msg": "The module failed to execute correctly, you probably need to set the interpreter.nSee stdout/stderr for the exact error","rc": 127
}

# ansible   localhost -k8s_exec -a "kubeconfig={{lookup('env','KUBECONFIG')}} namespace=monitoring pod=grafana-7c4bdc5894-pfqt5 command=pwd"
[DEPRECATION WARNING]: The 'return_code' return key is deprecated. Please use 'rc' instead. This feature will be removed  in version 4.0.0. Deprecationwarnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
localhost | CHANGED => {"changed": true,"rc": 0,"return_code": 0,"stderr": "","stderr_lines": [],"stdout": "/usr/share/grafanan","stdout_lines": ["/usr/share/grafana"]
}