> The Jeopardy playing Watson (DeepQA pipeline) was a landmark success at Information Retrieval, its architecture is built largely on Apache UIMA and Lucene with proprietary code for scaling out (performance) and filtering & ranking. I'm not an expert on IR so I won't comment further. This is very different from Neural Nets that are all the rage in ML today.
The question I would like to ask is that if the architecture is built on UIMA and Lucence, what exactly did "Watson" do?
Apache UIMA came out of the efforts from Ferrucci and others from TJ Watson Research Lab.
"Watson's" secret sauce was the filtering and ranking, you get multiple results from the models, which ones do you pick?
There was significant engineering effort put into reducing the latency of getting answers out of the pipeline as well, you had to optimize given the constraints of latency, accuracy and confidence from the pipeline.
The main trick behind Watson is to take the various systems (parsers, search, et al.) and hacks (constraints imposed by the rules of jeopardy) needed by a jeopardy playing bot and put them all together.
So, in some sense, you could say UIMA is what Watson did -- because it allowed a lot of flexibility for researchers to combine their efforts. Ranking and filtering becomes of ultimate importance in a system like that because at some point you have to make a decision. However, it is terribly reliant on the other modules at least getting somewhere in the ballpark -- and the ranking is also not, by itself, anything impressive.
So, it's an interesting case of how far you can get by just setting a single goal and slamming everything together -- but as it turns out, for every new domain you wish to apply something like that to, that magic ballpark is hard to reach without a significant amount of engineering & research effort to come up with new systems/hacks combined with a lot of relevant data. In other words, just like any other adhoc AIish system with a particular goal. Change the goal, change the system.
So, of course Watson was oversold, it was a PR and Sales effort from the beginning. Sort of like AlphaGo or DeepBlue -- you might be able to find one or two interesting ideas in the bowels of such a system -- but the system itself is not a generic one.
If you can find them, read the papers on Question Analysis and Answer Coercion. I am still impressed by how powerful these systems were, and how effective. UIMA is a framework for tying algorithms together, and experimenting quickly with different combinations.
The question I would like to ask is that if the architecture is built on UIMA and Lucence, what exactly did "Watson" do?