See also, 'nauniq' (non-adjacent uniq), an implementation of this text-processing task as a full utility, with some convenient options for reducing memory usage:

https://metacpan.org/pod/distribution/App-nauniq/script/naun...