View source

{{DISPLAYTITLE:uniq}}
uniq is a Unix utility which, when fed a text file, outputs the file with adjacent identical lines collapsed to one. It is a kind of filter program. Typically it is used after sort. It can also output only the duplicate lines (with the -d option), or add the number of occurrences of each line (with the -c option).

An example: To see the list of lines in a file, sorted by the number of times each occurs:

    sort file | uniq -c | sort -n

Using uniq like this is common when building pipelines in shell scripts. Be careful, uniq expects its input to be already sorted; sometimes sort -u is what you need.
*-u   Print only lines which are not repeated in the original file
*-d   Print one copy only of each repeated line in the input file.
*-c   Generate an output report in default style except that each line is preceded by a count of the number of times it occurred.  If this option is specified, the -u and -d options are ignored if either or both are also present.
*-i   Ignore case differences when comparing lines
*-s   Skips a number of characters in a line
*-w   Specifies the number of characters to compare in lines, after any characters and fields have been skipped
*--help    Displays a help message
*--version Displays version number on stdout and exits.
 [alibaba@ohnonono junk]$ cat shakes.txt
 to
 be
 or
 not
 to
 be
 that
 is
 the
 question
 [alibaba@ohnonono junk]$ sort shakes.txt | uniq
 be
 is
 not
 or
 question
 that
 the
 to

[[Category:Bash]]
[[Category:Linux Commands]]