awk常用操作

awk

1,输出指定列.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
$ cat netstat.txt
Proto Recv-Q Send-Q Local-Address Foreign-Address State
tcp 0 0 0.0.0.0:3306 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:9000 0.0.0.0:* LISTEN
tcp 0 0 coolshell.cn:80 124.205.5.146:18245 TIME_WAIT
tcp 0 0 coolshell.cn:80 61.140.101.185:37538 FIN_WAIT2
tcp 0 0 coolshell.cn:80 110.194.134.189:1032 ESTABLISHED
tcp 0 0 coolshell.cn:80 123.169.124.111:49809 ESTABLISHED
tcp 0 0 coolshell.cn:80 116.234.127.77:11502 FIN_WAIT2
tcp 0 0 coolshell.cn:80 123.169.124.111:49829 ESTABLISHED
tcp 0 0 coolshell.cn:80 183.60.215.36:36970 TIME_WAIT
tcp 0 4166 coolshell.cn:80 61.148.242.38:30901 ESTABLISHED
tcp 0 1 coolshell.cn:80 124.152.181.209:26825 FIN_WAIT1
tcp 0 0 coolshell.cn:80 110.194.134.189:4796 ESTABLISHED
tcp 0 0 coolshell.cn:80 183.60.212.163:51082 TIME_WAIT
tcp 0 1 coolshell.cn:80 208.115.113.92:50601 LAST_ACK
tcp 0 0 coolshell.cn:80 123.169.124.111:49840 ESTABLISHED
tcp 0 0 coolshell.cn:80 117.136.20.85:50025 FIN_WAIT2
tcp 0 0 :::22 :::* LISTEN

输出第一列和第四列.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
$ awk '{print $1, $4}' netstat.txt
Proto Local-Address
tcp 0.0.0.0:3306
tcp 0.0.0.0:80
tcp 127.0.0.1:9000
tcp coolshell.cn:80
tcp coolshell.cn:80
tcp coolshell.cn:80
tcp coolshell.cn:80
tcp coolshell.cn:80
tcp coolshell.cn:80
tcp coolshell.cn:80
tcp coolshell.cn:80
tcp coolshell.cn:80
tcp coolshell.cn:80
tcp coolshell.cn:80
tcp coolshell.cn:80
tcp coolshell.cn:80
tcp coolshell.cn:80
tcp :::22

2,过滤记录。显示第三列是0并且第6列是LISTEN的纪录。

1
2
3
4
5
$ awk '$3==0 && $6=="LISTEN"' netstat.txt
tcp 0 0 0.0.0.0:3306 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:9000 0.0.0.0:* LISTEN
tcp 0 0 :::22 :::* LISTEN

输出第三列大于0的全部列。

1
2
3
4
5
$ awk '$3>0 {print $0}' netstat.txt
Proto Recv-Q Send-Q Local-Address Foreign-Address State
tcp 0 4166 coolshell.cn:80 61.148.242.38:30901 ESTABLISHED
tcp 0 1 coolshell.cn:80 124.152.181.209:26825 FIN_WAIT1
tcp 0 1 coolshell.cn:80 208.115.113.92:50601 LAST_ACK

3,输出表头。

1
2
3
4
5
6
$ awk '$3==0 && $6=="LISTEN" || NR==1' netstat.txt
Proto Recv-Q Send-Q Local-Address Foreign-Address State
tcp 0 0 0.0.0.0:3306 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:9000 0.0.0.0:* LISTEN
tcp 0 0 :::22 :::* LISTEN

4,格式化输出。(打印输出:print,格式化打印输出:printf)

1
2
3
4
5
6
$ awk '$3==0 && $6=="LISTEN" || NR==1 {printf "%-20s %-20 %s\n", $4, $5, $6}' netstat.txt
Local-Address Foreign-Address State
0.0.0.0:3306 0.0.0.0:* LISTEN
0.0.0.0:80 0.0.0.0:* LISTEN
127.0.0.1:9000 0.0.0.0:* LISTEN
:::22 :::* LISTEN

5,字符串匹配。

1
2
3
4
5
6
$ awk '$6 ~ /FIN/ || NR==1 {print NR,$4,$5,$6}' OFS="\t" netstat.txt
1 Local-Address Foreign-Address State
6 coolshell.cn:80 61.140.101.185:37538 FIN_WAIT2
9 coolshell.cn:80 116.234.127.77:11502 FIN_WAIT2
13 coolshell.cn:80 124.152.181.209:26825 FIN_WAIT1
18 coolshell.cn:80 117.136.20.85:50025 FIN_WAIT2
1
2
3
4
5
$ awk '/LISTEN/' netstat.txt
tcp 0 0 0.0.0.0:3306 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:9000 0.0.0.0:* LISTEN
tcp 0 0 :::22 :::* LISTEN

可以使用”/FIN|TIME/“匹配FIN或TIME

1
2
3
4
5
6
7
8
$ awk '/FIN|TIME/' netstat.txt
tcp 0 0 coolshell.cn:80 124.205.5.146:18245 TIME_WAIT
tcp 0 0 coolshell.cn:80 61.140.101.185:37538 FIN_WAIT2
tcp 0 0 coolshell.cn:80 116.234.127.77:11502 FIN_WAIT2
tcp 0 0 coolshell.cn:80 183.60.215.36:36970 TIME_WAIT
tcp 0 1 coolshell.cn:80 124.152.181.209:26825 FIN_WAIT1
tcp 0 0 coolshell.cn:80 183.60.212.163:51082 TIME_WAIT
tcp 0 0 coolshell.cn:80 117.136.20.85:50025 FIN_WAIT2

模式取反。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
$ awk '$6 !~ /LISTEN/ || NR==1 {print NR,$4,$5,$6}' OFS="\t" netstat.txt
1 Local-Address Foreign-Address State
5 coolshell.cn:80 124.205.5.146:18245 TIME_WAIT
6 coolshell.cn:80 61.140.101.185:37538 FIN_WAIT2
7 coolshell.cn:80 110.194.134.189:1032 ESTABLISHED
8 coolshell.cn:80 123.169.124.111:49809 ESTABLISHED
9 coolshell.cn:80 116.234.127.77:11502 FIN_WAIT2
10 coolshell.cn:80 123.169.124.111:49829 ESTABLISHED
11 coolshell.cn:80 183.60.215.36:36970 TIME_WAIT
12 coolshell.cn:80 61.148.242.38:30901 ESTABLISHED
13 coolshell.cn:80 124.152.181.209:26825 FIN_WAIT1
14 coolshell.cn:80 110.194.134.189:4796 ESTABLISHED
15 coolshell.cn:80 183.60.212.163:51082 TIME_WAIT
16 coolshell.cn:80 208.115.113.92:50601 LAST_ACK
17 coolshell.cn:80 123.169.124.111:49840 ESTABLISHED
18 coolshell.cn:80 117.136.20.85:50025 FIN_WAIT2

or

1
$ awk '!/LISTEN/' netstat.txt

6,拆分文件。(按照第6列拆分),7个文件。

1
2
3
$ awk 'NR!=1{print > $6}' netstat.txt
$ ls
ESTABLISHED FIN_WAIT1 FIN_WAIT2 LAST_ACK LISTEN netstat.txt TIME_WAIT

按照第六列拆分,只输出第4,5列。

1
$ awk 'NR!=1{print $4,$5 > $6}' netstat.txt

7,使用if-else语句。

1
2
3
$ awk 'NR!=1{if($6 ~ /TIME|ESTABLISHED/) print > "1.txt";
else if($6 ~ /LISTEN/) print > "2.txt";
else print > "3.txt"}' netstat.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
$ cat 1.txt
tcp 0 0 coolshell.cn:80 124.205.5.146:18245 TIME_WAIT
tcp 0 0 coolshell.cn:80 110.194.134.189:1032 ESTABLISHED
tcp 0 0 coolshell.cn:80 123.169.124.111:49809 ESTABLISHED
tcp 0 0 coolshell.cn:80 123.169.124.111:49829 ESTABLISHED
tcp 0 0 coolshell.cn:80 183.60.215.36:36970 TIME_WAIT
tcp 0 4166 coolshell.cn:80 61.148.242.38:30901 ESTABLISHED
tcp 0 0 coolshell.cn:80 110.194.134.189:4796 ESTABLISHED
tcp 0 0 coolshell.cn:80 183.60.212.163:51082 TIME_WAIT
tcp 0 0 coolshell.cn:80 123.169.124.111:49840 ESTABLISHED

$ cat 2.txt
tcp 0 0 0.0.0.0:3306 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:9000 0.0.0.0:* LISTEN
tcp 0 0 :::22 :::* LISTEN

$ cat 3.txt
tcp 0 0 coolshell.cn:80 61.140.101.185:37538 FIN_WAIT2
tcp 0 0 coolshell.cn:80 116.234.127.77:11502 FIN_WAIT2
tcp 0 1 coolshell.cn:80 124.152.181.209:26825 FIN_WAIT1
tcp 0 1 coolshell.cn:80 208.115.113.92:50601 LAST_ACK
tcp 0 0 coolshell.cn:80 117.136.20.85:50025 FIN_WAIT2

8,统计。

计算文件数。

1
$ ls -l *.php *.go | awk '{$sum+=$5} END {print $sum}'

统计每个用户占用内存情况。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
$ ps aux | awk 'NR!=1{a[$1]+=$6;} END {for (i in a) print i ", " a[i]"KB";}'
nobody, 184KB
root, 390536KB
_usbmuxd, 792KB
_nsurlstoraged, 948KB
_applepay, 1520KB
_assetcache, 2576KB
_locationd, 16828KB
_gamecontrollerd, 1428KB
_mdnsresponder, 3244KB
_cmiodalassistants, 1320KB
_hidd, 3808KB
_timed, 3000KB
_spotlight, 12200KB
_appleevents, 2584KB
_displaypolicyd, 916KB
_mysql, 1432KB
_fpsd, 4116KB
_analyticsd, 6632KB
_iconservices, 856KB
_coreaudiod, 4724KB
_captiveagent, 684KB
_windowserver, 69096KB
_networkd, 4856KB
_distnote, 1712KB
tangchunlin, 3197076KB
_netbios, 7508KB
_atsserver, 1156KB
_datadetectors, 4KB
_ctkd, 4KB
_nsurlsessiond, 43068KB
_softwareupdate, 7272KB

9,awk脚本

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
$ cat score.txt
Jack 2001 78 89 90
Jerry 2002 87 99 87
Lily 2003 88 97 91
$ cat score.awk
#!/bin/awk -f
BEGIN {
math = 0
english = 0
computer = 0

printf "NAME NO. MATH ENGLIST COMPUTER TOTAL\n"
printf "-------------------------------------------------\n"
}

{
math+=$3
english+=$4
computer+=$5
printf "%-6s %-6s %4d %8d %8d %8d\n", $1, $2, $3, $4, $5, $3+$4+$5
}

END {
printf "-------------------------------------------------\n"
printf " TOTAL:%10d %8d %8d \n", math, english, computer
printf " AVERAGE:%10.2f %8.2f %8.2f\n", math/NR, english/NR, computer/NR
}

10,环境变量。

1
awk 'length>80' file // 找出file文件中长度大于80的行