I read the paper, they had 11 patients with known cancer and trained the NN to detect volatile organic compounds in the breath from raw GC-MS (gas chromatography mass spectrometry) data. From what I understand, the interesting and novel application is in the NN learning to parse the data for the detection of the aldehydes they were interested in. Nothing about this paper involves testing in patients where you aren't sure if they have cancer or not from what I read. Lots of false positives as well. Furthermore, whether or not this type of breath testing is useful in diagnosis is an open research question.
In my opinion, from a medicine standpoint this paper is pretty far from being applicable to patients, and may never be, although that is par for the course for a lot of medical research. I'm not smart enough to comment on the ML details.
I was thinking how things like this could revolutionize healthcare by reducing the need for doctors, who we never have enough of. But you know the saying, AI is always the thing we can't do yet? Meaning the goalposts shift, and stuff that was previously considered a goal of AI research is now just boring automation, and so AI is still a distant dream. Well it occurs to me that there is something similar with healthcare: healthcare is whatever is still expensive to provide. Regardless of how many life-saving treatments and inventions are adopted, there will still be some things that only hospital/clinic staff can help you with. And so we will always have the same bottleneck and people will always complain about access to healthcare, using this bottleneck as their barometer.
The problem here is that due to the base rate fallacy, accuracy percentages for medical tests are widely misinterpreted, so a "90% accurate" disease-screening toilet is a recipe for lots of unnecessary procedures and emotional trauma. Even at 99%, a positive hit for most diseases is still probably a false positive.
That entirely depends what the outcome of a positive screening result is. Screening doesn't really care about the base rate —and yes, if you were going to start chopping bits off immediately, it would matter— it's just there to weed out the negatives.
Doctors currently manually screen for many cancers, over entire demographics. It takes up an extraordinary amount of time, takes decades of training, and still misses cancer.
A breathalyser or toilet can sample you twice a day. That can feed into a risk analysis that pushes you onto blood screening (again, largely automated chem detection, only nursing support required). If you're still setting off alarms at that point, you go to the specialist.
And that's the point. Whereas they'd have been seeing 100% of their demographic, these machines are doing layers of screening for them. They end up only seeing a tiny fraction, where their job shifts to final confirmation and treatment.
What is really important is these machines cannot miss things. A low false positive rate is acceptable. False negatives over zero cannot be allowed because they'll delay or prevent further testing.
> A breathalyser or toilet can sample you twice a day. That can feed into a risk analysis that pushes you onto blood screening (again, largely automated chem detection, only nursing support required). If you're still setting off alarms at that point, you go to the specialist.
The long term, personalised health view behind this is what's interesting to me.
Medicine has a lot of wide-windows of what's normal for "people", but often the first signs of an underlying issue is that something that was normal for you at one end of a window begins to move, or change, in ways it hasn't before.
One sample of a blood test might tell you "that level is a little lower than average, but still not low enough we'd worry". But the historical knowledge that "I've actually always tended to be on the higher side of average for that measurement" changes the picture drastically.
Like you say, when it comes to looking at false positives of new tools like this, sure, a tool may be useless at giving you a one time 100% accurate reading, but if you can watch the trend of it over time that may be valuable.
It might even be a good thing. With enough regular feedback, we might emotionally adjust to the realization that "maybe you should get checked out" is not OMG I'm going to die.
There is a huge misconception about the value of screening in the general population. It doesn't do what most people think it does. Even with emotional adjustment it is entirely unclear that screening for things like cancer is ever good for the patient. Look at work by the oncologist Vinay Prasad: https://www.bmj.com/content/352/bmj.h6080
a) It's just not that easy to "everybody needs to adjust" -- people are different wrt. what causes anxiety, etc. It seems a priory quite plausbile that there's a significant genetic component to this.
b) "Get checked" may itself cause problems, for example: Scans for breast cancer involve radiation which may actually cause cancer. Biopsies can also be quite invasive. Obviously the risk is low, but depending on the exact numbers and it might actually be better to not get screened until one is in a known high-risk group (where the base rate fallacy doesn't skew the results so much).
And if it was massively affordable, portable, etc.
(On a lighter note I just can't help but think of a few scenes from a particular movie when we approach new automation fields... "This one goes in your mouth, this one goes in your ear, this one goes in your butt. No wait...")
>and stuff that was previously considered a goal of AI research is now just boring automation, and so AI is still a distant dream.
I think that's exactly right, which is what makes it particularly amusing to look back at Hubert Dreyfuses' apocalyptic declarations that computers can't do things that resemble human-like judgment. One of those things was supposedly chess, which required "insight", which computers can't have. But once computers got better at it than humans, everyone kind of forgot about it and pointed at new examples.
Similar things have happened. Two independent neural nets "spontaneously" developed their own communication protocol in an experiment. I'll see if I can find a link.
Humans, oddly enough, have a pretty insatiable appetite for not dying.
You can build someone a 10 bedroom mansion and then offer to give them a 20 bedroom mansion instead, and they might say, "Thanks, but 10 bedrooms is quite enough." But people generally won't say no if you can offer them a longer lifespan and/or an increase in quality of life.
Incidentally, if retirement age is fixed, increases in longevity lead to a higher proportion of society that is only consuming but not producing. Unless you want a decrease in standard of living, you have to compensate somehow, be it increases in efficiency, rapid population growth, or raising the retirement age. If you raise the retirement age, then medicine needs to help make sure most people are not only alive but can also actually be fit to work at age 70-80.
I don't think we're ever going to end up replacing doctors, but in the same way that tech accelerates other industries, AI should accelerate healthcare. Tying "expert machine" to "machine learning" could massively improve the pace at which Emergency Departments can triage patients, for instance, or provide doctors another useful input tool when diagnosing tricky conditions.
We could make an AI to do the hellish coding and insurer reimbursement interactions, to free them up to do medicine. Most "low-end GPs" are still extremely impressive intellects.
And that is a key tenant of capitalism. Cost of labor is what drives the true value of money. It is also why computers will never take all our jobs and a universal basic income doesn’t make any sense. This isn’t just healthcare, it’s everything.
They have numbers in the paper[0] linked in TFA. They trained on 29 samples and tested on 12 samples. On those samples, all techniques (NN, SVM, CNN) all had 100% true positive rates. The CNNs did best on false positives, giving on average 2-3 per sample.
Also, it is very important to note: they are comparing a CNN analysing a GC-MS[1] sample to a human expert analysing a GC-MS sample, which is basically a 1D plot of intensities of chemicals. This is not a comparison to a human smelling someone's breath.
Also, as far as I can see, they never compare to a GC-MS run with selective ion monitoring (SIM), which would be a lot faster for the human to analyse.
[1] A gas-chromatograph mass spectrometer is a device which samples a gas and detects what chemical species are present. It's what they do at airports when they swipe that piece of fabric on your luggage and put it in a machine (GC-MS) to look for explosives. It's a nice tool, used extensively in labs worldwide, but expensive (usually a couple hundred thousand USD, depending on specifications).
Ah sorry, I was relying on the parent comment's statement that they hadn't published numbers. Hadn't actually checked myself. Glad that they did though.
That being said, those data numbers seem extremely small. In reading the paper, I see that they attempted to augment it a bit. But it still seems near meaningless given the tiny sample set.
Maybe obtaining and analyzing these samples is a more complicated process than we think? For example, acquiring and analyzing a malware sample is not equivalent to labeling or classifying an image.
Does anyone know if TSA gate agents can see tumors, and if they've ever told anyone? Like, say, a golf-ball sized tumor in the neck? Would they have to be medically trained? Do they avoid or encourage hiring former radiologists?
There's no human operators for the body scanners anymore, due to privacy concerns. The images are analyzed by computer, and any anomalies are highlighted on a cartoon diagram for extra attention during the pat down.
Also, if someone had a golf-ball sized growth on the outside of their body, I'm sure the person is already aware of it. (The body scanners can't see inside the body.)
> There was an interesting story of a doctor who noticed cancer while watching TV and passed the word on.
It's not hugely frequent, I don't think, but it's not unheard of for this kind of thing to happen.
Most of the family are in the industry, and just watching TV or whatever you'll occasionally hear a comment about noticing a tremor or slight droop in someone's face that could be indicative of whatever.
Nope, millimeter wave scanners barely penetrate the upper skin layers, and I don't think they have the contrast required to differentiate even a skin tumor from normal skin.
From what I can tell, dogs worked well in the laboratory, but it did not work well in the field. In the lab they usually had 1/5 or so positive samples, but in the field, it was a much lower positive rate depending upon cancer and the dog handler would not be able to give positive reinforcement immediately in the field, also different dogs have different success rates.
https://www.livescience.com/61234-how-dogs-smell-cancer.html
Worth pointing out that humans can also smell disease in human breath - broad odours point to disease of particular organs (liver, kidney) and also to some specific diseases (diabetes).
Having had a family member who was epileptic, I can attest he often had a unique odor on his breath shortly before a seizure occurred. Now, there are people doing research into training dogs to detect that and alert that a seizure is imminent.
I don't think we know 100% how much specific diseases contribute, but there is at least ongoing research about that, and it seems there is general changes in body odour composition: https://en.wikipedia.org/wiki/Old_person_smell
Are there any estimates about releasing this technology for general use? Or does the (US) FDA insist on slowing down this type of diagnostic technology a few decades, or can we expect to see these as kiosks in every clinic over the next few months?
You don't seem to understand the actual problems with screening. Try learning about the field before implying the FDA isn't doing their job: https://www.bmj.com/content/352/bmj.h6080
at the moment they are essentially saying it's a promising tech that helps experts and cuts down on their errors. It helps with one part of the analysis. Looks like it is a long way from anything general use, though it's on the path towards that.
I read a few anecdotes that said that nurses who were familiar with cancer patients would subconsciously identify an odor that was associated with them.
Then they would smell it in the wild and be struck with a sudden an anxiety about whether or not they should tell the person to get checked, given the odd nature of the clue.
It does from the perspective of the blood giver. Many people are dreadfully afraid of needles to the point their unwillingness to do preventative tests. Also reduces the knowledge needed to administrate the test and possibly the eventual cost of executing the test. No phlebotomists needed to exhale.
Like dismissing a brand new product prototype as being unlikely to be a big success, this is a very low-value comment, in that it can't be tested in the short term, and the penalty for being wrong in the long term is zero.
So you can comfortably make this sneer about every new medical innovation you hear about, losing nothing for being wrong but crediting yourself with intuitive powers if you're right, which you likely will be some day, the more so the more you repeat the move.
That said, for every scandal like Theranos, the likelihood of another one just like it diminishes (if regulators and investors are acting effectively and rationally), so it might be a while before your prediction comes up.
I have suffered with severe nasal allergies as a kid. And had a hard time smelling anything because my nose was stuffed. I realized that I developed a sense of "smell" in my hand. I could actually feel the air. And tell what it smells like.
For those who can't believe it, smells have a pH, humidity, heat capacity, temperature, density, and pressure to them. So if you can train your skin to be sensitive and concentrate on those things you can feel smells.
It was not long till I realized that people give off a certain odor. And those odors are often related to the germs on their body. And the food they ate. Sometimes I could even tell if the person was sick.
I never developed a method of detecting illnesses simply by waving my hand on the person unless I was acutely familiar with the sickness, but if we can develop machines to detect "smell" (a fact I did not know) then we can definitely detect machines to scan people as they walk through a door and tell what germs they carry.
Of course my dream powers are to tell what antigerm ointment each person should use.
In my opinion, from a medicine standpoint this paper is pretty far from being applicable to patients, and may never be, although that is par for the course for a lot of medical research. I'm not smart enough to comment on the ML details.