uniq command can be used the consecutive recurring lines in a file.
$ uniq filename > targetfile
There are multiple options available with uniq. They are not limited to and include the following:
uniq -c filename
Prefix lines by the number of occurrences
uniq -d fileanme
Only print duplicate lines
uniq -u fileanme
Only print unique lines
Please note that the uniq command expects the input in sorted format. If the input is not sorted, it will only remove the consecutive recurring rows. To remove recurring rows spread across the file (not necessarily consecutive), one must first sort the input in the below way.
$ sort filename | uniq > targetfile
Happy Scripting!!