I once nearly lost a contest because of a faulty SQL query on their side. If I didn't get to see the query, I wouldn't have been able to defend my entry and would've lost. Losing this contest would've been trivial, but if I applied this to a trial, it would be horrible.
The error was that a 'group by' was used to find the number of unique entries, even though they had leading spaces, that were part of their uniqueness. Group by doesn't take leading spaces into account, leading them to get a different result than me. I think that this could've happened to a lot of people, even forensic IT engineers.
Actually it isn't specific to GROUP BY, the problem comes from the fact that some DMBS (MySQL for example) which ignore leading and trailing whitespaces (and other stuff like capitalization) while comparing strings, leading them to count less unique strings that a stricter comparision would count.
Okay so this article is bumping up against the hysteria that I'd categorize as "semi-technology literate" yet makes some good points. Almost like talking about how dangerous it is to walk through a minefield and then stepping on one. There's a valid point in there somewhere.
Copyright reform is one of my favorite subjects, and for a multitude of reasons. Should the prosecution be able to dump a case straight up without recourse because the "Stingray" gathering tool is too lovely to submit to review? Nope. Should breathalyzer code be held from review just because it's a product made by somebody? Nope. Should FOIA be stonewalled or pay-walled and inhibit the Constitutional freedom of the press? Nope!
Innocent until proven guilty is a very, very important premise for the US legal system. It's backed up by both the Fourth and First amendments to the Constitution. Any justification to put them aside for "War on ____" might seem reasonable on the surface, until taking a closer look at multiple murder evidence that comes from within the borders more often than on a laptop of a Citizen who just so happens to be coming back from a foreign country and gets worked for passwords under duress or has to forfeit hardware without recourse.
I dunno, maybe I sound like some kind of off-the-rocker dude by thinking about such things, but I love my country, I'm willing to sit down and think about this kind of stuff. It doesn't have to be extreme. Taking the small steps of talking with one another about what we really value is important in my opinion.
So... if i build and sell a "breathalyzer" to your local pd that simply randomly selects a number between .08 and .16, you'd be ok with that? Pretty much every other element of the system is biased toward conviction. Police have quotas. Prosecutors need convictions. Private prisons get more money for more convicts. It would be easy to dodge questions for a long, long time. Why would police administer the test to a sober person? In this one weird example a sober person registered as drunk. oh well, its still good for 99.9% of other cases.
The right to confront your accuser is there for a reason. More and more, software is the accuser. We all, at HN, struggle to make our code correct. Step back, think a second, how do you QA a breathalyzer? How do you deal with variations in sensor packages? Yes, they probably do more good than harm, but are they "accurate"? how do you know that?
Breathalyzers are known to be wildly inaccurate, to such an extent that in some jurisdictions DUI is just defined in terms of a breathalyzer result because they know it doesn't bear a reliable relationship to the actual BAC.
But is it accurate enough to say "you're drunk driving"? IIRC at least on this side of the pond, if you're caught you either get fined, or you're taken to the station for a more accurate breath test or a blood test.
In the US, you have the right to request a blood test, but the blood test is not required and the breathalyzer is considered sufficient evidence (again, since the offense is defined in terms of the breathalyzer).
The presumption of innocence comes from English common law, not the Constitution; it in fact appears nowhere in the Constitution, let alone in the 1st or 4th Amendment.
Okay, so I've conflated the direct inspiration for the framing of the Constitution with what it actually says, and tossed in a couple Amendments that are relevant practically speaking, and yet you still understood the jist of what I was saying. Potatoe.
Not really. We're probably closer than we are far apart, but I think the distinction is important; for instance, we usually discuss the presumption of innocence in terms of "innocent until proven guilty" and then base our opinions off the modern (indeed, message-board) interpretation of the word "proof". Mathematical certainty was not an objective of the framers, or of their predecessors.
Which is relevant here, of course, because of the demand for access to source code to increase certainty.
But: you'd be right if you assumed the "1st and 4th Amendment" thing was what actually motivated me to comment. I'm a message board nerd too!
The constitution defines those rights granted to citizens from natural law that shall not be reduced by the govt. The constitution limits the government... citizens have #alltherights
There are similar issues with this in Structural and Mechanical engineering. Engineers are expected to rely more and more on software to execute and document complex calculations to verify designs, but how can you be sure those underlying calculations/theorems/models are correctly implemented? Some packages are constantly patching particular edge cases that get sent to them from their users. Many issue announcements to warn of bugs that could cause an incorrect result.
If a result from software led to a critical failure in a design, the onus is most likely still on the Engineer.
I have seen cases where software is formally reviewed by independent verification bodies, much in the same way your ISO 9001 compliance is. I can't see why this wouldn't apply here. Have an independent party, who has signed an appropriate NDA, asses and certify that your product does what it says on the tin and audit it at regular periods.
> Have an independent party, who has signed an appropriate NDA, asses and certify that your product does what it says on the tin and audit it at regular periods.
This may work for safety standards or financial liability, but it doesn't work for the legal system, because "independent" parties are influenced by who pays them.
Yes, that's always a factor. Larger IVBs will tend to err on the side of their reputation in the marketplace (which may also be a factor).
One could envisage a (horrible) situation where both parties engage an IVB (a practice common in some areas of engineering) and then they both jointly engage another body to effectively vet their processes and pass that joint 50/50 cost into their fees back to you.
Or the court appoints one and you agree to share the costs (probably wouldn't fly in the US).
I wonder if it's possible/practical to model the mechanical properties using multiple different tools, and check for agreement, along the lines of N-version programming[1] for software.
It might get tricky where you're doing extremely specialised stuff where only a single implementation exists, but I suppose you'd have to assign some time to validating your tools personally for your use.
I think things will become even worse now that criminal "scouting" and even vetting is being done via learning models. So you may not even find hard filters or conditionals...instead the errors (or stereotypes?) would be embedded deep inside some neural net. I'm not even sure how one would explain that one to a jury.
Machine learning doesn't make a difference between correlation and causation. No doubt such systems already help reproduce the inequalities present in the data they've been trained on.
Yeah, that's what I'm thinking. GIGO basically: the US records it would be using would suggest poor, black, mentally-ill etc. people are most likely to be criminals. The software will pick up on that.
This defense attorney was creative for asking to examine the source code, but that isn't the only way to cast doubt on the accuracy of the software that DNA matched his client to the crime scene. He could simply obtain a copy of it and have an expert run tests to determine a false positive rate and also what types of scenarios cause the software to deliver false positives, then call that expert as a witness.
...which could be much more expensive and sill not find the error you'd need to defend the client. You'd essentially be black box testing without even knowing what kind of errors you want to find. Or am I missing something?
This is what happens when you have companies profiting off the misery of others.
The biggest reason for companies wanting to protect their source code in this case is that they already know their software is broken, like pretty much every other software, and they don't want to fix it. The arguments against losing money and such are total bullshit as courts have plenty of procedures for disclosing materials only to the relevant parties present, not to the public as a whole. These companies simply don't want to spend the money auditing and making sure their code runs correctly because the only consequence of that is wrongfully convicting someone they don't give a fuck about.
I'd say, let them see the code and let the highest paid expert witness win. That is, after all, the American way.
The essential problem is that in such environments the process of doing a task must be open to inspection, but software exists as a loophole that circumvents making process public.
This use to be a problem in Florida with drunk driving arrests. The company that makes the code for the breathalyzers wouldn't allow their code to be reviewed by defendants. There was finally a precedent set that defendants couldn't mount a viable defense without reviewing the code.
So for a while people accused of a DUI could wind up getting off, under the right circumstances, by requesting the source code then getting refused by the company.
The company finally allowed pieces of the code to be reviewed by the courts.
I'm not sure I agree with that view. Independant testing by another lab should remove any doubt on the validity of a forensic, rather than forcing companies to open source their technology. And of course some form of certification/random tests that ensures that the company providing the forensic isn't a bunch of conmen.
This has nothing to do with open source. Competing companies can not suddenly use the copyrighted work, implement patented technologies, or for that matter redistribute the work.
I can go down to the government archive and as a citizen request to view blueprints of buildings. I may not copy the design to build my own building, nor use patented aspects, but I am allowed to inspect or ask a independent expert to review the design and find flaws. Unsurprisingly this has not killed the architect profession, nor forced construction companies to open source their technologies.
But how can you verify that the two independent companies aren't both using the same faulty code? You could ask someone under oath, but how would they know whether an employee bought or stole code from the competitor, or they both made similar errors in implementing a published algorithm, etc? As mentioned regarding Volkswagen, black box testing doesn't necessarily cut it (especially if the flaws are statistical in nature or triggered in unusual circumstances, and more so if they are deliberate).
How can you currently verify that 2 labs aren't using the same faulty forensic process when it comes to other types of evidence?
Most forensic evidence isn't 100% conclusive things like DNA and fingerprint identification aren't a 1 to 1 unique match they are usually in the range of 1 out of 100-400,000 match.
This is sufficient for most cases because the likelihood of that evidence being "wrong" when you combine it with other factors like motive, eye-witness accounts and other non-physical evidence is very slim.
I'm not sure but would hope that traditional forensic labs and processes are also open to scrutiny. E.g. if the defense suggests that some important mark on the body could have been introduced while it was being handled, it should be possible to show why that couldn't happen with the procedures that were followed, and that the procedures were followed.
They need to meet the specific regulatory requirements in their jurisdictions but it doesn't necessarily means that they are easy to inspect.
On the body part it's a bit more complicated it's more a CSI effect people think that all cases have tons of physical evidence and that everything is cool and flashy and high-techy in reality most cases have very little physical evidence and labs might not be used at all most bodies are inspected at by the county coroner's office which might be quite inadequate of collecting that sort of evidence we come to expect by watching crime procedurals.
> On the body part it's a bit more complicated it's more a CSI effect people think that all cases have tons of physical evidence and that everything is cool and flashy and high-techy
I sat on a jury a couple years back and the prosecutor's opening statement addressed this issue in the first seconds of the case. He worked hard to make a distinction between reality and television. This was a smart move, as some of the people on that jury turned out to be dumb as rocks when we went into deliberation. I'm sure this is true of most juries.
As a logical middle ground, companies can (and do) release source code for review under NDA without becoming "open source" (Usually to government / military clients where access to the source code is a requirement for the contract. See https://en.wikipedia.org/wiki/Shared_source#Microsoft_Govern... for an example). This would hopefully be acceptable for all parties concerned where courts are involved.
The error was that a 'group by' was used to find the number of unique entries, even though they had leading spaces, that were part of their uniqueness. Group by doesn't take leading spaces into account, leading them to get a different result than me. I think that this could've happened to a lot of people, even forensic IT engineers.