I agree that in the majority of cases it will cause bugs. But what I'm saying is that there are cases where it can legitimately not cause a bug.
The problem I have is this: this testing framework explicitly states the following:
> The quality of your tests can be gauged from the percentage of mutations killed
So, you write your tests to test that there is a consistent ordering, it changes the "<" to a ">", finds that all tests still pass (because, as you said, you're only testing if there is a consistent ordering, because that's the only thing that matters in this case), and considers your tests lower-quality because of that.
This is a flaw most mutation testing - and, for that matter, test driven design - suffers.
For another flaw: there are many things that end up along the lines of a bunch of shortcut "fast paths" at the start, with a slow path at the end that does slow but exhaustive checking.
For example, if you're checking that two objects are equal you may insert a check at the start to see if they are the same pointer, and if so return `true`. If you're doing division in software you may check if you're dividing by 1 instead of doing the computation. That sort of thing.
Any mutation that introduces false negatives into the shortcuts (but no false positives) won't be flagged by mutation testing.
The problem I have is this: this testing framework explicitly states the following:
> The quality of your tests can be gauged from the percentage of mutations killed
So, you write your tests to test that there is a consistent ordering, it changes the "<" to a ">", finds that all tests still pass (because, as you said, you're only testing if there is a consistent ordering, because that's the only thing that matters in this case), and considers your tests lower-quality because of that.
This is a flaw most mutation testing - and, for that matter, test driven design - suffers.
For another flaw: there are many things that end up along the lines of a bunch of shortcut "fast paths" at the start, with a slow path at the end that does slow but exhaustive checking.
For example, if you're checking that two objects are equal you may insert a check at the start to see if they are the same pointer, and if so return `true`. If you're doing division in software you may check if you're dividing by 1 instead of doing the computation. That sort of thing.
Any mutation that introduces false negatives into the shortcuts (but no false positives) won't be flagged by mutation testing.