uniq (識別并有選擇地過濾掉文件中重復的行)

嚯嚯發表于 2020-07-18 03:14

瀏覽次數：

命令行 Unix&Linux命令 uniq

在手機上閱讀

釋放雙眼，帶上耳機，聽聽看~！

00:00

在類似Unix的操作系統上，uniq命令報告或過濾掉文件中的重復行。本文檔介紹uniq的GNU / Linux版本。

查看英文版

1 uniq 運行系統環境

2 uniq 描述

3 uniq 語法

4 uniq 示例

uniq 運行系統環境

Linux

uniq 描述

uniq 從輸入文件INPUT中過濾出相鄰的匹配行，并將過濾后的數據寫入輸出文件OUTPUT中。

如果未指定INPUT，則uniq從標準輸入讀取。

如果未指定OUTPUT，則uniq寫入標準輸出。

如果未指定選項，則匹配的行將合并到第一個匹配項。

查看英文版

uniq 語法

uniq [OPTION]... [INPUT [OUTPUT]]

Options

-c, --count

帶數字的前綴行代表它們發生了多少次。

-d, --repeated

僅打印重復的行。

-D, --all-repeated[=delimit-method]

打印所有重復的行。delimit-method可能是以下之一：

none	根本不要分隔重復的行。這是默認值。
prepend	在每組重復的行之前插入空白行。
separate	在每組重復的行之間插入空白行。

該 -D 選項是等同于指定 --all-repeated=none.

-f N, --skip-fields=N

在確定唯一性之前，請避免比較一行的前N個字段。字段是一組字符，由空格分隔。例如，如果文檔的行已編號，并且您想比較該行中除行號以外的所有內容，此選項很有用。

如果指定了選項-f 1，則相鄰的行將被視為相同。如果未指定-f選項，則將其視為唯一。

1 This is a line. 2 This is a line.

-i, --ignore-case

通常，比較是區分大小寫的。該選項將執行不區分大小寫的比較。

-s N, --skip-chars=N

確定唯一性時，避免比較每行的前N個字符。就像-f選項一樣，但是它會跳過單個字符而不是字段。

-u, --unique

唯一僅打印唯一的行。

-z, --zero-terminated

以0字節（NULL）而不是換行符結束的行。

-w, --check-chars=N

每行比較不超過N個字符。

--help

顯示幫助消息并退出。

--version

輸出版本信息并退出。

Notes

uniq 不會檢測重復的行，除非它們相鄰。您可能要先對輸入進行排序，或者使用sort -u而不是uniq。

uniq [OPTION]... [INPUT [OUTPUT]]

Options

-c, --count

Prefix lines with a number representing how many times they occurred.

-d, --repeated

Only print duplicated lines.

-D, --all-repeated[=delimit-method]

Print all duplicate lines. delimit-method may be one of the following:

none	Do not delimit duplicate lines at all. This is the default.
prepend	Insert a blank line before each set of duplicated lines.
separate	Insert a blank line between each set of duplicated lines.

The -D option is the same as specifying --all-repeated=none.

-f N, --skip-fields=N

Avoid comparing the first N fields of a line before determining uniqueness. A field is a group of characters, delimited by whitespace.

This option is useful, for instance, if your document's lines are numbered, and you want to compare everything in the line except the line number. If the option -f 1 were specified, the adjacent lines

1 This is a line. 2 This is a line.

would be considered identical. If no -f option were specified, they would be considered unique.

-i, --ignore-case

Normally, comparisons are case-sensitive. This option performs case-insensitive comparisons instead.

-s N, --skip-chars=N

Avoid comparing the first N characters of each line when determining uniqueness. This is like the -f option, but it skips individual characters rather than fields.

-u, --unique

Only print unique lines.

-z, --zero-terminated

End lines with 0 byte (NULL), instead of a newline.

-w, --check-chars=N

Compare no more than N characters in lines.

--help

Display a help message and exit.

--version

Output version information and exit.

Notes

uniq does not detect repeated lines unless they are adjacent. You may want to sort the input first, or use sort -u instead of uniq.

查看英文版

uniq 示例

假設我們有一個八行文本文件myfile.txt，其中包含以下文本：

This is a line.
This is a line.
This is a line.
This is also a line.
This is also a line.
This is also also a line.

...以下是在此文件上運行uniq的幾種方法及其創建的輸出：

uniq myfile.txt

This is a line.
This is also a line.
This is also also a line.

uniq -c myfile.txt

3 This is a line.
1  
2 This is also a line.
1  
1 This is also also a line.

uniq -d myfile.txt

This is a line. 
This is also a line.

uniq -u myfile.txt

This is also also a line.

Let's say we have an eight-line text file, myfile.txt, which contains the following text:

This is a line.
This is a line.
This is a line.
This is also a line.
This is also a line.
This is also also a line.

...Here are several ways to run uniq on this file, and the output it creates:

uniq myfile.txt

This is a line.
This is also a line.
This is also also a line.

uniq -c myfile.txt

3 This is a line.
1  
2 This is also a line.
1  
1 This is also also a line.

uniq -d myfile.txt

This is a line. 
This is also a line.

uniq -u myfile.txt

This is also also a line.

查看英文版

其他命令行

如此好文，分享給朋友