zombie 僵尸进程
巡检
Raspi 4B,发现存在30多个zombie进程.
1top
result
1top - 16:11:50 up 154 days, 17:24, 4 users, load average: 1.27, 1.46, 1.06
2Tasks: 437 total, 1 running, 436 sleeping, 0 stopped, 30 zombie
3%Cpu(s): 8.2 us, 15.1 sy, 0.0 ni, 76.7 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
4MiB Mem : 7821.1 total, 170.6 free, 5070.4 used, 2638.0 buff/cache
5MiB Swap: 11731.7 total, 7251.2 free, 4480.5 used. 2750.8 avail Mem
排查
1ps -A -ostat,ppid,pid,cmd |grep -e '^[Zz]'
1root 268331 3348371 0 Apr28 ? 00:00:00 [sh] <defunct>
2root 268341 3348371 0 Apr28 ? 00:00:00 [sh] <defunct>
3root 268381 3348371 0 Apr28 ? 00:00:00 [sh] <defunct>
4root 268724 3348371 0 Apr28 ? 00:00:00 [sh] <defunct>
5root 269149 3348371 0 Apr28 ? 00:00:00 [sh] <defunct>
6root 307388 3348371 0 Apr28 ? 00:00:00 [sh] <defunct>
7root 307399 3348371 0 Apr28 ? 00:00:00 [sh] <defunct>
8root 307435 3348371 0 Apr28 ? 00:00:00 [sh] <defunct>
9root 307546 3348371 0 Apr28 ? 00:00:00 [sh] <defunct>
10root 307588 3348371 0 Apr28 ? 00:00:00 [sh] <defunct>
11root 2224514 3348371 0 Apr23 ? 00:00:00 [sh] <defunct>
12root 2224524 3348371 0 Apr23 ? 00:00:00 [sh] <defunct>
13root 2224580 3348371 0 Apr23 ? 00:00:00 [sh] <defunct>
14root 2224919 3348371 0 Apr23 ? 00:00:00 [sh] <defunct>
15root 2225402 3348371 0 Apr23 ? 00:00:00 [sh] <defunct>
16root 2301659 3348371 0 Apr23 ? 00:00:00 [sh] <defunct>
17root 2301671 3348371 0 Apr23 ? 00:00:00 [sh] <defunct>
18root 2301708 3348371 0 Apr23 ? 00:00:00 [sh] <defunct>
19root 2302066 3348371 0 Apr23 ? 00:00:00 [sh] <defunct>
20root 2302530 3348371 0 Apr23 ? 00:00:00 [sh] <defunct>
21root 3237040 3348371 0 Apr25 ? 00:00:00 [sh] <defunct>
22root 3237050 3348371 0 Apr25 ? 00:00:00 [sh] <defunct>
23root 3237091 3348371 0 Apr25 ? 00:00:00 [sh] <defunct>
24root 3237424 3348371 0 Apr25 ? 00:00:00 [sh] <defunct>
25root 3237864 3348371 0 Apr25 ? 00:00:00 [sh] <defunct>
26root 3246934 3348371 0 Apr25 ? 00:00:00 [sh] <defunct>
27root 3246944 3348371 0 Apr25 ? 00:00:00 [sh] <defunct>
28root 3246983 3348371 0 Apr25 ? 00:00:00 [sh] <defunct>
29root 3247353 3348371 0 Apr25 ? 00:00:00 [sh] <defunct>
30root 3247793 3348371 0 Apr25 ? 00:00:00 [sh] <defunct>
发现父pid为3348371, 查看pid
1lsof -p 3348371
输出结果
1COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
2java 3348371 root cwd DIR 0,309 4096 4765082 /workdir
3java 3348371 root rtd DIR 0,309 4096 4796434 /
4java 3348371 root txt REG 0,309 10144 4401767 /usr/lib/jvm/java-17-openjdk-arm64/bin/java
5java 3348371 root mem REG 0,309 13758464 4402121 /usr/lib/jvm/java-17-openjdk-arm64/lib/server/classes.jsa
6java 3348371 root mem REG 0,309 10144 4401767 /usr/lib/jvm/java-17-openjdk-arm64/bin/java
7java 3348371 root mem REG 0,309 34176 4402100 /usr/lib/jvm/java-17-openjdk-arm64/lib/libmanagement_ext.so
8java 3348371 root mem REG 0,309 26368 4402098 /usr/lib/jvm/java-17-openjdk-arm64/lib/libmanagement.so
9java 3348371 root mem REG 0,309 133232 12340585 /usr/lib/aarch64-linux-gnu/libbrotlicommon.so.1.0.9
10java 3348371 root mem REG 0,309 415632 12212825 /usr/lib/aarch64-linux-gnu/libpcre.so.3.13.3
11java 3348371 root mem REG 0,309 43088 12340587 /usr/lib/aarch64-linux-gnu/libbrotlidec.so.1.0.9
12java 3348371 root mem REG 0,309 214888 4401694 /usr/lib/aarch64-linux-gnu/libpng16.so.16.37.0
13java 3348371 root mem REG 0,309 1285824 4401643 /usr/lib/aarch64-linux-gnu/libglib-2.0.so.0.7200.4
14java 3348371 root mem REG 0,309 751704 4401625 /usr/lib/aarch64-linux-gnu/libfreetype.so.6.18.1
15java 3348371 root mem REG 0,309 807424 4401658 /usr/lib/aarch64-linux-gnu/libharfbuzz.so.0.20704.0
16java 3348371 root mem REG 0,309 782504 4402078 /usr/lib/jvm/java-17-openjdk-arm64/lib/libawt.so
17java 3348371 root mem REG 0,309 116808 4401650 /usr/lib/aarch64-linux-gnu/libgraphite2.so.3.2.1
18java 3348371 root mem REG 0,309 65008 4402083 /usr/lib/jvm/java-17-openjdk-arm64/lib/libfontmanager.so
19java 3348371 root mem REG 0,309 39824 4402079 /usr/lib/jvm/java-17-openjdk-arm64/lib/libawt_headless.so
20java 3348371 root mem REG 0,309 60976 4402110 /usr/lib/jvm/java-17-openjdk-arm64/lib/libverify.so
21java 3348371 root mem REG 0,309 13248 4402082 /usr/lib/jvm/java-17-openjdk-arm64/lib/libextnet.so
22java 3348371 root mem REG 0,309 42144 4402111 /usr/lib/jvm/java-17-openjdk-arm64/lib/libzip.so
23java 3348371 root mem REG 0,309 111304 4402102 /usr/lib/jvm/java-17-openjdk-arm64/lib/libnet.so
24java 3348371 root mem REG 0,309 103152 4402103 /usr/lib/jvm/java-17-openjdk-arm64/lib/libnio.so
25java 3348371 root mem REG 0,309 128269364 4402112 /usr/lib/jvm/java-17-openjdk-arm64/lib/modules
26java 3348371 root mem REG 0,309 196112 4402089 /usr/lib/jvm/java-17-openjdk-arm64/lib/libjava.so
27java 3348371 root mem REG 0,309 33240 4402093 /usr/lib/jvm/java-17-openjdk-arm64/lib/libjimage.so
28java 3348371 root mem REG 0,309 84296 4769872 /usr/lib/aarch64-linux-gnu/libgcc_s.so.1
29java 3348371 root mem REG 0,309 551064 4769907 /usr/lib/aarch64-linux-gnu/libm.so.6
30java 3348371 root mem REG 0,309 32768 4761779 /tmp/hsperfdata_root/1
31java 3348371 root mem REG 0,309 2190752 4769958 /usr/lib/aarch64-linux-gnu/libstdc++.so.6.0.30
32java 3348371 root mem REG 0,309 24001728 4402124 /usr/lib/jvm/java-17-openjdk-arm64/lib/server/libjvm.so
33java 3348371 root mem REG 0,309 1641496 4769836 /usr/lib/aarch64-linux-gnu/libc.so.6
34java 3348371 root mem REG 0,309 70896 4402094 /usr/lib/jvm/java-17-openjdk-arm64/lib/libjli.so
35java 3348371 root mem REG 0,309 104608 4770006 /usr/lib/aarch64-linux-gnu/libz.so.1.2.11
36java 3348371 root mem REG 0,309 187776 4769553 /usr/lib/aarch64-linux-gnu/ld-linux-aarch64.so.1
37java 3348371 root mem REG 8,2 64 387090 /data/jenkins/secret.key
38java 3348371 root 0u CHR 1,3 0t0 61 /dev/null
39java 3348371 root 1w FIFO 0,12 0t0 396200203 pipe
40java 3348371 root 2w FIFO 0,12 0t0 396200204 pipe
41java 3348371 root 3r REG 0,309 128269364 4402112 /usr/lib/jvm/java-17-openjdk-arm64/lib/modules
42java 3348371 root 4r REG 0,309 89542047 4794629 /opt/jenkins.war
43java 3348371 root 5r CHR 1,9 0t0 162 /dev/urandom
44java 3348371 root 6r CHR 1,8 0t0 80 /dev/random
45java 3348371 root 7r REG 0,309 3398506 4761780 /tmp/winstone13428853651398353641.jar
46java 3348371 root 8u sock 0,8 0t0 396199873 protocol: TCP
47java 3348371 root 9r REG 8,2 11991 3628302 /data/jenkins/war/WEB-INF/lib/access-modifier-annotation-1.32.jar
48java 3348371 root 10u sock 0,8 0t0 396199899 protocol: UNIX-STREAM
49java 3348371 root 11r REG 8,2 17619 3628265 /data/jenkins/war/WEB-INF/lib/annotation-indexer-1.17.jar
50java 3348371 root 12r REG 8,2 2301284 3628295 /data/jenkins/war/WEB-INF/lib/ant-1.10.13.jar
51java 3348371 root 13r REG 8,2 20600 3628273 /data/jenkins/war/WEB-INF/lib/ant-launcher-1.10.13.jar
52java 3348371 root 14r REG 8,2 326307 3628262 /data/jenkins/war/WEB-INF/lib/antlr4-runtime-4.13.0.jar
53java 3348371 root 15r REG 8,2 155379 3628278 /data/jenkins/war/WEB-INF/lib/args4j-2.33.jar
54java 3348371 root 16r REG 8,2 121863 3628316 /data/jenkins/war/WEB-INF/lib/asm-9.5.jar
55java 3348371 root 17r REG 8,2 33978 3628339 /data/jenkins/war/WEB-INF/lib/asm-analysis-9.5.jar
56java 3348371 root 18r REG 8,2 72209 3628301 /data/jenkins/war/WEB-INF/lib/asm-commons-9.5.jar
57java 3348371 root 19r REG 8,2 51944 3628300 /data/jenkins/war/WEB-INF/lib/asm-tree-9.5.jar
58java 3348371 root 20r REG 8,2 91076 3628296 /data/jenkins/war/WEB-INF/lib/asm-util-9.5.jar
解决
通过lsof查看后是docker jenkins
需要启用**–init**，
1号进程设置docker-init, 出现僵尸进程是因为1号进程为 jenkins-agent， 开启init即可
原因：不会将操作系统的信号转发到子进程上，也不会回收孤儿进程
1version: "3"
2services:
3 web:
4 image: ...:latest
5 init: true # 启用
评论