These types of anecdotes make the actual practice of both ML and AI seem rather, well, less than scientific. There is supposed to be Ph.D. level math behind all of this, yet an amateur with admittedly no ML background is part of the team.
In Star Wars, it takes Luke Skywalker years to learn to use a light saber skillfully. Then in The Force Awakens, some ex-Stormtrooper with no training picks up the light saber and within 5 minutes is a pro. Kinda ruined the mystique of Star Wars, just like people jumping into ML with no training ruins the mystique of ML.
I am very against the idea that somebody with a PhD is the only one that can do a certain kind of work. But I am ofcourse biased given that you call me out.
Creative and critical thinking is not exclusive to people with a PhD. The ability to understand ones strengths and weaknesses is not exclusive to a PhD.
I would never attempt to write or publish a paper without help of somebody with stronger mathematical or statistical knowledge. On the other hand they should not write source code for a paper without consulting somebody with a strong background in sw engineering. You complement each other. Power is in recognizing that.
You would be surprised how many software bugs I have found that invalidated entire (draft) papers. A PhD in ML doesn't save you from that.
Is mystique worth having? In fiction, sure, but I think not in R&D. I think a lot of ML and AI isn't especially scientific, but I also think there's a lot of low-hanging fruit in applying it to new areas, and both of those make it easier for an amateur to contribute.
Let me expound on this, as there's a lot of PhD hate in the comments parallel to mine.
What is unique to a PhD is that you took a very long time to master a small slice of the knowledge pie.
The emphasis here is on long time: Most people simply aren't willing to go for years on a low salary and tedious job.
It doesn't mean PhD's are more
(or less) creative, top coders and whatnot; it means we took the time to read all the papers, to know all previous solution attempts and who all the big players in the field are ("all" w.r.t. our niche).
It also mean we can read papers much faster than other people because that is basically what we do all day.
There you have it! Now don't send me ML job offers, cause I gotta read this next obscure paper to figure out if they are legit. :D Just kidding.
I think your metaphor kinda plays against your point.
Years and years ago in the movie universe, Luke had to painfully learn to use his lightsaber from a mentor who passed down techniques and philosophy to the student.
In the current day of the movies, lightsabers are understood to be powerful, yet temperamental and exotic, weapons. Mildly trained individuals can use them, even if it's to a limited extent (i.e. flick switch; shiny side is the business end; heat bad, ouch, no touch).
To belabor the metaphor, you've also had a tradition of people, from all walks of life, using vibroblades (basic to advanced standard statistical analysis and regression) in order to achieve some level of parity against users of lightsabers.
There is PhD level math involved. And yet, ML (deep learning in particular) is much more of an empirical endeavor than many would like to admit. A deep understanding of the underlying mathematics does not necessarily give you a better model. Modern models are so complicated that no one can reason through them. Parameter spaces are non-convex and fully of ugly pathologies that make neat and tidy analysis methods useless.
From one perspective, it is disheartening that a deep understanding of the underlying methods doesn't necessarily win the day. From another, it is quite remarkable that having good implementation skills and a methodical mindset can get you quite far.
Much 'applied' ML is based on tweaked existing models, getting them production ready and integrating them into a product. That's inherently a bit of non-trivial engineering work, but as you pointed out, not scientific per-se.
You're disillusioned if you think a PhD is what makes the difference.
Smart people will be able to contribute even if they don't have a PhD.
Some PhD are useless and everyone is wondering how the hell they go through that.
In Star Wars, it takes Luke Skywalker years to learn to use a light saber skillfully. Then in The Force Awakens, some ex-Stormtrooper with no training picks up the light saber and within 5 minutes is a pro. Kinda ruined the mystique of Star Wars, just like people jumping into ML with no training ruins the mystique of ML.