Stupid SysAdmin Tricks: Uniq

SHAC: i just remembered one of the silliest sysadmin hacks ever

SHAC: if you have a file that sometimes has duplicate lines

SHAC: and sometimes doesnt

SHAC: and you want to process each line once

SHAC: if file is called foo

cat foo foo | sort | uniq -d > bar

SHAC: so you cat foo twice into sort so EVERY line has at least 2 instances… then you sort them all out… then you use uniq and say “tell me what lines are duplicated”

SHAC: of course every line is duplicated, but uniq only tells you that once

BRAIN: that is brilliant

SHAC: “boss” taught me it like 4 years ago

Leave a Reply