I implemented neural network feature creation for a backgammon agent ("Automated feature selection to maximize learning in artificial intelligence").
Nowadays, I mainly do parallel machine learning on machines with higher network latency. I haven't used MPI since.
With the sort of work I do nowadays (large-scale ML and NLP), I generally need very little synchronization, i.e. my tasks are usually embarrassingly parallel. I typically save final results in a centralized store (DB or NFS) and look at it there.
I also use Hadoop where appropriate.
I implemented neural network feature creation for a backgammon agent ("Automated feature selection to maximize learning in artificial intelligence").
Nowadays, I mainly do parallel machine learning on machines with higher network latency. I haven't used MPI since.