I'm enthusiastic that Intel is working on optimizing Torch's Tensor and nn libraries, and optimizing Neural Nets in general.
This repo is most definitely not ready for primetime.
Two comments:
1. I haven't seen an Intel team that does non-trivial DNN optimizations, almost all the (multiple) teams that have worked on optimizing neural networks have worked on trivial stuff like adding OpenMP optimizations to existing code, and integrating MKL into existing frameworks. Usually adding OpenMP optimizations isn't as relevant, because one uses CPUs for neuralnet inference, on a server where multiple processes are running. Per-core efficiency is much more important than cute threadpool optimizations like OpenMP which look great in micro-benchmarks.
The meaty and practically relevant optimizations have come from folks in the community like Andrew Lavin and Marat Dukhan ( https://github.com/Maratyszcza/NNPACK ) which are focused more on optimizing both single-core efficiency as well as multi-core.
2. The way this fork integrates MKLDNN into Torch is not great, where they add MKLDNN fields into the core Tensor / Storage struct, which is not great from a broader perspective. I will try to reach out to the authors and make this more proper.
p.s.: torch installs in a self-contained folder already as pointed by @gcr
I agree and would go further: Intel should find ways to get its optimizations merged into Torch's mainline branch (instead of in a fork), like CUDA support.
I think this approach is wrong. While it may make their library more accessible, Torch installation can be conditioned to use different libraries at least during the compile time. There is no need to package it like this.
I think proper approach would be making their library accessible separately and explain how it should be used with Torch.
To be fair, my OpenCL distro follows the same approach http://github.com/hughperkins/distro-cl Mainstream torch is kind of like 'sid' debian: it's cutting edge, but not always as stable as one would like, if one wants to tweak it a bit, eg add OpenCL.
As far as the title of this news thread, note that the 'self contained torch folder' title of the intel distro comes from the original distro that Soumith made https://github.com/torch/distro It's not something that Intel wrote/added/assert.
Thanks for this clarification. As such it should be then possibly removed from the title here, as it will add confusion.
I think one of the reasons of Torch installation not beeing stable is lack of versioning in the LuaRocks package system.
I have experienced this and wasted fair amount of time because of this - you update one innocent looking package and it will introduce weird bugs that go away when the whole system is updated.
This is the reason why I am very wary to update anything when the deadline approaches. For example I updated image package because I wanted some of its new features and it did not compile because somebody added an unnecessary optimization to load only one function from another package that apparently was not supported yet by the rest of the installation.
About your OpenCL distro, have you tried it on ARM platform?
This repo is most definitely not ready for primetime.
Two comments:
1. I haven't seen an Intel team that does non-trivial DNN optimizations, almost all the (multiple) teams that have worked on optimizing neural networks have worked on trivial stuff like adding OpenMP optimizations to existing code, and integrating MKL into existing frameworks. Usually adding OpenMP optimizations isn't as relevant, because one uses CPUs for neuralnet inference, on a server where multiple processes are running. Per-core efficiency is much more important than cute threadpool optimizations like OpenMP which look great in micro-benchmarks.
The meaty and practically relevant optimizations have come from folks in the community like Andrew Lavin and Marat Dukhan ( https://github.com/Maratyszcza/NNPACK ) which are focused more on optimizing both single-core efficiency as well as multi-core.
2. The way this fork integrates MKLDNN into Torch is not great, where they add MKLDNN fields into the core Tensor / Storage struct, which is not great from a broader perspective. I will try to reach out to the authors and make this more proper.
p.s.: torch installs in a self-contained folder already as pointed by @gcr