DataKit 使用

DataKit 使用

目录

• 1. DataKit 使用
• • 1.1. 网站
  • 1.2. 常用脚本
  • • 1.2.1. 常用命令
    • 1.2.2. 卸载
  • 1.3. 命令
  • • 1.3.1. 卸载
    • 1.3.2. 输出默认配置
  • 1.4. 采集器
  • • 1.4.1. 采集器在 console(工作空间) 中的位置
    • • 1.4.1.1. Object
      • 1.4.1.2. CustomObject
      • 1.4.1.3. RUM
      • 1.4.1.4. Security
      • 1.4.1.5. Network
      • 1.4.1.6. KeyEvent
      • 1.4.1.7. Profile
  • 1.5. DQL
  • 1.6. 行协议
  • • 1.6.1. 示例
    • • 1.6.1.1. datakit self
      • 1.6.1.2. net
      • 1.6.1.3. ebpf
      • 1.6.1.4. profile
      • 1.6.1.5. scheck 安全巡检
      • 1.6.1.6. RUM
      • 1.6.1.7. KeyEvent
      • 1.6.1.8. Custom Object
      • 1.6.1.9. Logging
  • 1.7. 一些约束
  • 1.8. Troubleshooting
  • • 1.8.1. 打开本地监视
  • 1.9. 上传镜像至 harbor
  • 1.10. 集成测试
  • 1.11. 采集器脚本
  • • 1.11.1. zipkin 上报 span

1. DataKit 使用

DataKit is collection agent for Guance Cloud.

1.1. 网站

• Console(工作空间): /
• 源码:

1.2. 常用脚本

1.2.1. 常用命令

cat /var/log/datakit/log | grep 'xxx'

1.2.2. 卸载

clean.sh:

#!/bin/shdatakit service -Urm -rf /usr/local/datakit
rm -rf /var/log/datakit

1.3. 命令

1.3.1. 卸载

# Linux/Mac shell
datakit service -I # re-install
datakit service -U # uninstall 此处卸载 DataKit 并不会删除 DataKit 相关文件rm -rf /usr/local/datakit
rm -rf /var/log/datakit

1.3.2. 输出默认配置

sudo datakit tool --default-main-conf

1.4. 采集器

1.4.1. 采集器在 console(工作空间) 中的位置

1.4.1.1. Object

这个在 console 中看不到, 只能通过 DQL 查询, 语句如下:

O::`snmp_object`

1.4.1.2. CustomObject

基础设施 - 自定义。

=ObjectadminOther&router_type=table

1.4.1.3. RUM

用户访问监测。需要新增监控, 比方说阈值。

1.4.1.4. Security

安全巡检。

1.4.1.5. Network

基础设施 - 网络。

=HostNetwork&time=15m&is_refresh=true

1.4.1.6. KeyEvent

监控 - 监控器。

KeyEvent 就是告警。在这里可以新建一个告警, 比方说 “阈值”。

1.4.1.7. Profile

应用性能监测 - Profile

=15m&is_refresh=true

1.5. DQL

调起命令: datakit dql。使用 tab 键选择小分类。

• 查看网络: show_network_source()。
• 查看事件: show_event_source()。

查询所有工作空间下的事件(仅限一条):

E::abc LIMIT 1

其中 abc 是指标集名称。

除了指标 Metrics, 即 M, 不支持模糊查询外, 其它的 category 都支持:

L::`*` LIMIT 1 # 或 L::re(`.*`) LIMIT 1

因为后端存储引擎 TDEngine 不支持, 之前的 influxdb 可能支持。

其它:

O::host_processes { process_name=&#'} limit 1
O::host_processes limit 10
M::cpu limit 10
M::`snmp_metric`:(`ifBandwidthInUsageRate`)

查询 object 数据时, 如果只是指定了名字而没有指定额外条件, 则只会显示最近 5 分钟的数据, 如:

O::test5 limit 1; # 显示最近 5 分钟的数据, 超过了就不会显示了, 显示的内容是 no data.

如果想查 5 分钟之外的, 则必须指定额外条件:

O::`*` { class='test5'} limit 1;

1.6. 行协议

curl -d "param1=value1&param2=value2" -X POST localhost:9529/write

curl -d "netflow,direction=outgoing,dst_domain&#-a4b1,dst_ip=47.110.144.10,dst_ip_type=other,dst_port=442,family=IPv4,host=debian-gnu-linux-10,pid=16868,service=ebpf,src_ip=10.211.55.3,src_ip_type=private,src_port=57470,status=info,transport=tcp bytes_read=5244i,bytes_written=4834i,retransmits=0i,rtt=508i,rtt_var=151i,tcp_closed=0i,tcp_established=0i 1656404560535993599" -X POST localhost:9529/v1/write/network[io]...[io.filters]network = [ # 针对 Network 过滤"{ source = 'netflow' and dst_port IN [ '442' ] }"]

1.6.1. 示例

1.6.1.1. datakit self

datakit,arch=arm64,host=debian-gnu-linux-10,os=linux,os_version_detail=Debian GNU/Linux 10 (buster),uuid=debian-gnu-linux-10,version=1.4.0-156-gaa269ef4fe,vserion=1.4.0-156-gaa269ef4fe cpu_usage=26.02808425276026,dropped_points=0i,dropped_points_total=0i,elected=0i,heap_alloc=16410384i,heap_objects=81345i,heap_sys=24576000i,incumbency=0i,max_heap_alloc=16410384i,max_heap_objects=81345i,max_heap_sys=24576000i,max_num_goroutines=19i,min_heap_alloc=12573912i,min_heap_objects=77403i,min_heap_sys=24576000i,min_num_goroutines=19i,num_goroutines=19i,open_files=10i,pid=31584i,uptime=56i 1656402016576400450

1.6.1.2. net

net,host=debian-gnu-linux-10,interface=eth0 bytes_recv=24748227i,bytes_recv/sec=545i,bytes_sent=1440653i,bytes_sent/sec=236i,drop_in=0i,drop_out=0i,err_in=0i,err_out=0i,packets_recv=19756i,packets_recv/sec=1i,packets_sent=14521i,packets_sent/sec=1i 1656402032873381726
net,host=debian-gnu-linux-10,interface=all tcp_activeopens=6443i,tcp_attemptfails=35i,tcp_currestab=2i,tcp_estabresets=9770i,tcp_incsumerrors=0i,tcp_inerrs=0i,tcp_insegs=48089i,tcp_insegs/sec=1i,tcp_maxconn=-1i,tcp_outrsts=6864i,tcp_outsegs=43643i,tcp_outsegs/sec=1i,tcp_passiveopens=6229i,tcp_retranssegs=0i,tcp_rtoalgorithm=1i,tcp_rtomax=120000i,tcp_rtomin=200i,udp_ignoredmulti=27i,udp_incsumerrors=0i,udp_indatagrams=1340i,udp_indatagrams/sec=0i,udp_inerrors=0i,udp_noports=1i,udp_outdatagrams=1032i,udp_outdatagrams/sec=0i,udp_rcvbuferrors=0i,udp_sndbuferrors=0i 1656402032873381726

1.6.1.3. ebpf

netflow,direction=outgoing,dst_ip=35.161.136.21,dst_ip_type=other,dst_port=443,family=IPv4,host=debian-gnu-linux-10,pid=14120,service=ebpf,src_ip=10.211.55.3,src_ip_type=private,src_port=46852,status=info,transport=tcp bytes_read=31i,bytes_written=35i,retransmits=0i,rtt=110397i,rtt_var=116086i,tcp_closed=0i,tcp_established=1i 1656403960536936427
netflow,direction=outgoing,dst_domain&#-a4b1,dst_ip=118.31.126.76,dst_ip_type=other,dst_port=443,family=IPv4,host=debian-gnu-linux-10,pid=16868,service=ebpf,src_ip=10.211.55.3,src_ip_type=private,src_port=49954,status=info,transport=tcp bytes_read=0i,bytes_written=266i,retransmits=0i,rtt=17690i,rtt_var=8845i,tcp_closed=1i,tcp_established=1i 1656403960536936427
netflow,direction=outgoing,dst_domain&#-a4b1,dst_ip=47.114.74.166,dst_ip_type=other,dst_port=443,family=IPv4,host=debian-gnu-linux-10,pid=16868,service=ebpf,src_ip=10.211.55.3,src_ip_type=private,src_port=58222,status=info,transport=tcp bytes_read=4783i,bytes_written=4149i,retransmits=0i,rtt=1965i,rtt_var=2716i,tcp_closed=0i,tcp_established=1i 1656403960536936427
netflow,direction=outgoing,dst_ip=10.211.55.1,dst_ip_type=private,dst_port=53,family=IPv4,host=debian-gnu-linux-10,pid=16868,service=ebpf,src_ip=10.211.55.3,src_ip_type=private,src_port=*,status=info,transport=udp bytes_read=542i,bytes_written=144i 1656403960536936427
dnsflow,dst_ip=10.211.55.1,dst_port=53,family=IPv4,host=debian-gnu-linux-10,service=ebpf,src_ip=10.211.55.3,src_port=48974,transport=udp rcode=0i,resp_time=13719949i,timeout=false 1656403958807396745
dnsflow,dst_ip=10.211.55.1,dst_port=53,family=IPv4,host=debian-gnu-linux-10,service=ebpf,src_ip=10.211.55.3,src_port=42622,transport=udp rcode=0i,resp_time=15890057i,timeout=false 1656403958807343664
dnsflow,dst_ip=10.211.55.1,dst_port=53,family=IPv4,host=debian-gnu-linux-10,service=ebpf,src_ip=10.211.55.3,src_port=45533,transport=udp rcode=0i,resp_time=12643041i,timeout=false 1656403958837755466
dnsflow,dst_ip=10.211.55.1,dst_port=53,family=IPv4,host=debian-gnu-linux-10,service=ebpf,src_ip=10.211.55.3,src_port=54366,transport=udp rcode=0i,resp_time=16814431i,timeout=false 1656403958837375567
netflow,direction=outgoing,dst_domain&#-a4b1,dst_ip=47.110.144.10,dst_ip_type=other,dst_port=443,family=IPv4,host=debian-gnu-linux-10,pid=16868,service=ebpf,src_ip=10.211.55.3,src_ip_type=private,src_port=57470,status=info,transport=tcp bytes_read=5244i,bytes_written=4834i,retransmits=0i,rtt=508i,rtt_var=151i,tcp_closed=0i,tcp_established=0i 1656404560535993599
netflow,direction=outgoing,dst_ip=10.211.55.1,dst_ip_type=private,dst_port=53,family=IPv4,host=debian-gnu-linux-10,pid=16868,service=ebpf,src_ip=10.211.55.3,src_ip_type=private,src_port=*,status=info,transport=udp bytes_read=813i,bytes_written=216i 1656404560535993599

1.6.1.4. profile

profile,endpoint=/v1/upload/profiling,env=testing,host=MacBook-Air-2.local,language=python,runtime=CPython,service=python-profiling-manual,version=7.8.9 datakit_ver="1.4.0-156-gaa269ef4fe",duration=60000000000i,end=1656412909000000000i,format="",library_ver="1.2.1",pid="",profile_id="2dc55b7e-cbf5-4b1e-9bf2-81a12a37e7a5",profiler_version="1.2.1",runtime_id="4185e78ee69b47b9bf8675e392f5f748",runtime_version="3.9.13",start=1656412849000000000i 1656412849000000000
profile,endpoint=/v1/upload/profiling,env=testing,host=MacBook-Air-2.local,language=python,runtime=CPython,service=python-profiling-manual,version=7.8.9 datakit_ver="1.4.0-156-gaa269ef4fe",duration=60000000000i,end=1656412969000000000i,format="",library_ver="1.2.1",pid="",profile_id="96e1e529-0fe6-4ea8-bb1e-c4365939aa88",profiler_version="1.2.1",runtime_id="4185e78ee69b47b9bf8675e392f5f748",runtime_version="3.9.13",start=1656412909000000000i 1656412909000000000

1.6.1.5. scheck 安全巡检

0000-scheck-start,category=system,host=debian-gnu-linux-10,level=info,title=scheck start,version=1.0.8 message="scheck started, 137 rules ready at 2022-06-28 19:32:53" 1656415973381892236
0069-sudo-logfile,category=system,host=debian-gnu-linux-10,level=warn,title=sudo 日志未配置或被删除, version=1.0.8 message="sudo.log 未配置或被删除" 1656415973420972117

1.6.1.6. RUM

resource,sdk_name=df_web_rum_sdk,sdk_version=2.0.24,app_id=appid_JtcMjz7Kzg5n8eifTjyU6w,env=production,version=1.0.0,userid=f8e5d15f-0bfd-4ad9-b275-d7fc6761a5c1,session_id=02e7a8e7-243f-4ad9-bdf1-7866b9ac21c0,session_type=user,is_signin=F,os=Mac OS,os_version=10,os_version_major=10,browser=Firefox,browser_version=102.0,browser_version_major=102,screen_size=1440*900,view_id=90c69be1-a2b1-49e1-8ddd-b10d048dce5f,view_url=:8080/,view_host=:8080,view_path=/,view_path_group=/,view_url_query={},resource_url=.js,resource_url_host=static.guance,resource_url_path=/browser-sdk/v2/dataflux-rum.js,resource_url_path_group=/browser-sdk/?/dataflux-rum.js,resource_url_query={},resource_type=js,resource_status=200,resource_status_group=2xx,resource_method=GET duration=7000000,resource_ttfb=5000000,resource_trans=2000000,resource_first_byte=5000000 1656485594137
resource,sdk_name=df_web_rum_sdk,sdk_version=2.0.24,app_id=appid_JtcMjz7Kzg5n8eifTjyU6w,env=production,version=1.0.0,userid=f8e5d15f-0bfd-4ad9-b275-d7fc6761a5c1,session_id=02e7a8e7-243f-4ad9-bdf1-7866b9ac21c0,session_type=user,is_signin=F,os=Mac OS,os_version=10,os_version_major=10,browser=Firefox,browser_version=102.0,browser_version_major=102,screen_size=1440*900,view_id=90c69be1-a2b1-49e1-8ddd-b10d048dce5f,view_url=:8080/,view_host=:8080,view_path=/,view_path_group=/,view_url_query={},resource_url=:8080/,resource_url_host=,resource_url_path=/,resource_url_path_group=/,resource_url_query={},resource_type=document,resource_status=304,resource_status_group=3xx,resource_method=GET duration=33000000,resource_ttfb=2000000,resource_trans=0,resource_first_byte=25000000 1656485594092
resource,sdk_name=df_web_rum_sdk,sdk_version=2.0.24,app_id=appid_JtcMjz7Kzg5n8eifTjyU6w,env=production,version=1.0.0,userid=f8e5d15f-0bfd-4ad9-b275-d7fc6761a5c1,session_id=02e7a8e7-243f-4ad9-bdf1-7866b9ac21c0,session_type=user,is_signin=F,os=Mac OS,os_version=10,os_version_major=10,browser=Firefox,browser_version=102.0,browser_version_major=102,screen_size=1440*900,view_id=90c69be1-a2b1-49e1-8ddd-b10d048dce5f,view_url=:8080/,view_host=:8080,view_path=/,view_path_group=/,view_url_query={},resource_url=:8080/favicon.ico,resource_url_host=,resource_url_path=/favicon.ico,resource_url_path_group=/favicon.ico,resource_url_query={},resource_type=image,resource_status=200,resource_status_group=2xx,resource_method=GET duration=17000000,resource_ttfb=17000000,resource_trans=0,resource_first_byte=17000000 1656485594139
view,sdk_name=df_web_rum_sdk,sdk_version=2.0.24,app_id=appid_JtcMjz7Kzg5n8eifTjyU6w,env=production,version=1.0.0,userid=f8e5d15f-0bfd-4ad9-b275-d7fc6761a5c1,session_id=02e7a8e7-243f-4ad9-bdf1-7866b9ac21c0,session_type=user,is_signin=F,os=Mac OS,os_version=10,os_version_major=10,browser=Firefox,browser_version=102.0,browser_version_major=102,screen_size=1440*900,view_id=90c69be1-a2b1-49e1-8ddd-b10d048dce5f,view_url=:8080/,view_host=:8080,view_path=/,view_path_group=/,view_url_query={},view_loading_type=initial_load,view_apdex_level=0,is_active=true view_error_count=0,view_resource_count=3,view_long_task_count=0,view_action_count=0,first_contentful_paint=72000000,loading_time=67000000,dom_interactive=63000000,dom_content_loaded=66000000,dom_complete=67000000,first_paint_time=25000000,resource_load_time=1000000,time_to_interactive=55000000,dom=4000000,dom_ready=58000000,time_spent=3065000000 1656485594092

1.6.1.7. KeyEvent

user create_time=1656383652424,df_date_range="10",df_event_id="event-21946fc19eaf4c5cb1a698f659bf74cd",df_message="【lwc】(lwc@qq) 进入了工作空间",df_status="info",df_title="【lwc】(lwc@qq) 进入了工作空间",df_user_id="acnt_a5d6130c19524a6b9fe91d421eaf8603",user_email="lwc@qq",user_name="lwc" 1658040035652416000

其中 user 是 event source。

curl --location --request POST 'localhost:9529/v1/write/keyevent' 
--header 'Content-Type: text/plain' 
--data-raw 'user create_time=1656383652424,df_date_range="10",df_event_id="event-21946fc19eaf4c5cb1a698f659bf74cd",df_message="【lwc】(lwc@qq) 进入了工作空间",df_status="info",df_title="【lwc】(lwc@qq) 进入了工作空间",df_user_id="acnt_a5d6130c19524a6b9fe91d421eaf8603",user_email="lwc@qq",user_name="lwc" 1658040035652416000'

1.6.1.8. Custom Object

aliyun_ecs,name="ecs_name",host="ecs_host" instanceid="ecs_instanceid",os="ecs_os",status="ecs_status",creat_time="ecs_creat_time",publicip="1.1.1.1",regionid="cn-qinghai",privateip="192.168.1.12",cpu="ecs_cpu",memory=204800000000

curl --location --request POST 'localhost:9529/v1/write/custom_object' 
--header 'Content-Type: text/plain' 
--data-raw 'aliyun_ecs,name="ecs_name",host="ecs_host" instanceid="ecs_instanceid",os="ecs_os",status="ecs_status",creat_time="ecs_creat_time",publicip="1.1.1.1",regionid="cn-qingdao",privateip="192.168.1.12",cpu="ecs_cpu",memory=204800000000'

1.6.1.9. Logging

nginx,filename=access.log,filepath=/opt/homebrew/var/log/nginx/access.log,host=MacBook-Air-2.local,service=nginx agent="Go-http-client/1.1",browser="Go-http-client",browserVer="1.1",bytes=97i,client_ip="127.0.0.1",engine="",engineVer="",http_method="GET",http_url="/server_status",http_version="1.1",isBot=false,isMobile=false,log_read_lines=2i,log_read_offset=891i,message="127.0.0.1 - - [19/Aug/2022:19:45:36 +0800] "GET /server_status HTTP/1.1" 200 97 "-" "Go-http-client/1.1"",os="",referrer="-",status="OK",status_code=200i,ua="" 1660909536000000000nginx,filename=access.log,filepath=/opt/homebrew/var/log/nginx/access.log,host=MacBook-Air-2.local,service=nginx agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36",browser="Chrome",browserVer="104.0.0.0",bytes=0i,client_ip="127.0.0.1",engine="AppleWebKit",engineVer="537.36",http_method="GET",http_url="/",http_version="1.1",isBot=false,isMobile=false,log_read_lines=7i,log_read_offset=1501i,message="127.0.0.1 - - [19/Aug/2022:19:46:17 +0800] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36"",os="Intel Mac OS X 10_15_7",referrer="-",status="notice",status_code=304i,ua="Macintosh" 1660909577000000000nginx,filename=error.log,filepath=/opt/homebrew/var/log/nginx/error.log,host=MacBook-Air-2.local,service=nginx client_ip="127.0.0.1",http_method="GET",http_url="/hello",http_version="1.1",ip_or_host="localhost:8080",log_read_lines=1i,log_read_offset=888i,message="2022/08/19 19:47:01 [error] 95221#0: *4 open() "/opt/homebrew/Cellar/nginx/1.23.1/html/hello" failed (2: No such file or directory), client: 127.0.0.1, server: localhost, request: "GET /hello HTTP/1.1", host: "localhost:8080"",msg="95221#0: *4 open() "/opt/homebrew/Cellar/nginx/1.23.1/html/hello" failed (2: No such file or directory), client: 127.0.0.1, server: localhost, request: "GET /hello HTTP/1.1", host: "localhost:8080"",server="localhost",status="error" 1660909621000000000

1.7. 一些约束

1. Metric 数据, fields 字段的 value 值不能为 string;
2. Tags 的 value 只能为 string;

1.8. Troubleshooting

1.8.1. 打开本地监视

sudo datakit monitor

1.9. 上传镜像至 harbor

docker tag nginx:vts-1.20.2 pubrepo.jxxgouxxx/image-repo-for-testing/nginx/nginx:vts-1.20.2 
docker push pubrepo.jxxgouxxx/image-repo-for-testing/nginx/nginx:vts-1.20.2 

1.10. 集成测试

安装 docker 命令:

yum remove docker docker-client docker-client-latest docker-common docker-latest docker-latest-logrotate docker-logrotate docker-engineyum install -y yum-utils
yum-config-manager --add-repo .repo# yum list docker-ce --showduplicates | sort -ryum install docker-ce-20.10.18-3.el7 docker-ce-cli-20.10.18-3.el7 containerd.io docker-compose-plugincat <<EOF > /etc/docker/daemon.json
{"registry-mirrors": ["","",""]
}
EOFsystemctl start docker
systemctl enable docker
docker run hello-world
rm -rf /pos.po# systemctl daemon-reload 
# systemctl restart docker

不需要安装 docker-buildx-plugin, 会提示 conflicts, 因为 buildx 已经安装好了, 可以通过命令 docker buildx version 查看。

参考自: Install Docker Engine on CentOS

配置:

wget -O /pos.po .repo#非阿里云ECS用户会出现 Couldn't resolve host 'mirrors.cloud.aliyuncs' 信息，不影响使用。用户也可自行修改相关配置: eg:
sed -i -e '/mirrors.cloud.aliyuncs/d' -e '/mirrors.aliyuncs/d' /pos.poyum makecache

1.11. 采集器脚本

1.11.1. zipkin 上报 span

curl -X POST localhost:9411 -H 'Content-Type: application/json' -d '[{"id": "1234","traceId": "0123456789abcdef","timestamp": 1608239395286533,"duration": 100000,"name": "span from bash!","tags": {&#hod": "GET","http.path": "/api"},"localEndpoint": {"serviceName": "shell script"}
}]'