You need "gawk -M" for this for bignum support, so visited[$0]++ doesn't wrap back to zero, otherwise it is not correct for huge files with huge numbers of duplicates.
The portable one-liner that doesn't suffer from integer wraparound is actually
awk '!($0 in seen) { seen[$0]; print }'
which can be golfed a bit:
awk '!($0 in s); s[$0]'
$0 in s tests whether the line exists in the s[] assoc array. We negate that, so we print if it doesn't exist.
Then we unconditionally execute s[$0]. This has an undefined value that behaves like Boolean false. In awk if we mention an array location, it materializes, so this has the effect that "$0 in s" is now true, though s[$0] continues to have an undefined value.
At least on the stock MacOS awk, you can get up to 2^53 before arithmetic breaks (doesn't wrap, just doesn't go up any more which means the one-liner still works.)
The portable one-liner that doesn't suffer from integer wraparound is actually
which can be golfed a bit: $0 in s tests whether the line exists in the s[] assoc array. We negate that, so we print if it doesn't exist.Then we unconditionally execute s[$0]. This has an undefined value that behaves like Boolean false. In awk if we mention an array location, it materializes, so this has the effect that "$0 in s" is now true, though s[$0] continues to have an undefined value.