Perturbing units and observing change is so much like the early days of neurology and neuroscience. Lesion studies in post-surgery patients or those with stroke or brain damage told us a lot, removing part of the circuit and observing what changed.
yup. i remember reading about something called ‘optimal brain damage’ which aims at pruning non-essential weights from the network for faster training etc.
We also think there should be more Ablation Studies in deep learning research. To make it easier, we are working on automating the process of disabling components of models with a framework called Maggy. Pass in your Keras Model and specify which layes/components to mask out and we will generate all models and train them in parallel.
https://databricks.com/session_eu20/parallel-ablation-studie...https://github.com/logicalclocks/maggy
This is all super cool, but for those of you who tl:dr;ed, check out their application - creating a "concept painting" app. You can literally click "door" and spray on the picture, and it'll put a door there for you.
