Much much more to how neural circuits work than just connectivity. Good example to keep in mind is C elegans (https://en.wikipedia.org/wiki/Caenorhabditis_elegans). Despite knowing the connectivity of its 302 neurons, and the whole organism consists of just a few times that many cells, AFAIK we still don't fully understand how its nervous system works, e.g., how it makes decisions etc.

(There is a lot of work on this organism, and I haven't kept up with the literature. So maybe the comment is a bit out of date... someone more knowledgeble should chime in.)

Well, I wouldn't say we understand C. elegans neurobiology and behavior "fully", but great advances have been made at the systems level:

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2904967/#__ffn_... [overview of behavioral assays at systems level]

http://conferences.union.wisc.edu/ceneuro/program-info/organ... [next major conference]

And, of course, my favorite, WormWeb:

http://wormweb.org/neuralnet#c=ADA&m=1

The big difference is that we understand chips: We know what kind of elements does an integrated circuit need (gates, registers) and we understand how each of the low- and high-level and elements work. Reverse engineering in this case is puzzle reconstruction with very strong priors. On the other hand, we are truly clueless for brains: we do not know how "algorithms" are run in brains, if there are any simple computing units, how memory is stored ... All what we have are some guesses and statistics.

Or perhaps the brain works nothing like a CPU.

(The very idea that it does is actually laughable, akin to medieval fantasists imagining that flying machines must have huge white wings with fleshy feathers.)

Yes, a brain does not work like a CPU. As noted by another user in this thread, understanding brain is a considerably harder problem. "CPUs are deliberately engineered so that their different functions are nicely decoupled and easy to reason about but brains are the result of a very messy evolutionary process" -SilasX

I think it's partly the regular, simple, easy to understand structure that makes the approach outlined in this paper fail so badly. Because CPUs are made up of a number of simple, regular pieces connected in a way that's generic and easy to reason about, each of which is general enough to be used across many kinds of programs, knocking out individual transistors and observing the effects on the output of complex programs doesn't tell you much. Which parts of the CPU are used for a particular "behaviour" depends on the whims of the programmer and the compiler's optimisation passes as much as anything else.

Engineers are also the result of a very messy evolutionary process ;)

Being the result of a messy evolutionary process doesn't give you any special insight into other messy evolutionary processes.

I don't think that "maybe we could understand the brain if we had a very detailed picture" is the same as "the brain works like a CPU".

Nobody is claiming it does.

So maybe we should synthesize a (probably huge) photograph of a chip, whose network would correspond to one brain ... and then hand it to hackers, as a reverse engineering challenge :-)

The problem then transfers to synthesizing such a network on a chip :)

We could start with something very small, like a worm.

So? We know how chips work, we make them! Going from a picture is much easier, you know exactly what you are looking at. Not just the individual structures, but we also already know what kind of higher level structure to expect. There is nothing unknown about what we see on a chip, by definition (we could not have made it in the first place).

Semiconductor manufacturing is very planar - it's made up of many layers on top of each other, with some interconnecting features or vias between layers. It's a best-case scenario for reverse engineering with photography.