Core Bibliography

Representative papers from the literature that inspired me to investigate whether the science of AI has proven to be the science of building synthetic brain cortexes.

This book project is not an academic paper. It is a work of investigative journalism and popular science, which has not yet been completed. The existing proof-of-concept materials are based on oral interviews with 30+ scientists and a close reading of the literature. However, I cannot hope to cite all the work that is relevant, especially not at this early point in the project. Suffice it to say, there are many, many, many researchers who have inspired this project with their research, and I apologize I can’t name them all yet. Nonetheless, the following is a chronological listing of some papers that may be considered representative of the broader literature:

2003: The neuroscientist James DiCarlo begins to develop a mastery of the macaque monkey vision experiments. DiCarlo, James J., and John H. R. Maunsell. “Anterior Inferotemporal Neurons of Monkeys Engaged in Object Recognition Can Be Highly Sensitive to Object Retinal Position.” Journal of Neurophysiology, vol. 89, no. 6, June 2003, pp. 3264–78. DOI.org (Crossref), https://doi.org/10.1152/jn.00358.2002.

2005: DiCarlo and colleagues find the first predictive but non-explanatory model of the macaque monkey IT cortex. Hung, Chou P., et al. “Fast Readout of Object Identity from Macaque Inferior Temporal Cortex.” Science, vol. 310, no. 5749, Nov. 2005, pp. 863–66. DOI.org (Crossref), https://doi.org/10.1126/science.1117593.

2008: A student in DiCarlo’s group leads a project that changes the course of the laboratory’s research towards neural networks. Pinto, Nicolas, et al. “Why Is Real-World Visual Object Recognition Hard?” PLOS Computational Biology, vol. 4, no. 1, Jan. 2008, p. e27. PLoS Journals, https://doi.org/10.1371/journal.pcbi.0040027.

2012: Early members of the OpenWorm project release a model of the whole C. elegans organism, which falls far short of realism. Palyanov, Andrey, et al. “Towards a Virtual C. Elegans: A Framework for Simulation and Visualization of the Neuromuscular System in a 3D Physical Environment.” In Silico Biology, vol. 11, no. 3,4, 2012, pp. 137–47. DOI.org (Crossref), https://doi.org/10.3233/ISB-2012-0445.

2012: AI scientists publish the first program to match the object recognition performance of the human visual cortex. Krizhevsky, Alex, et al. “ImageNet Classification with Deep Convolutional Neural Networks.” Advances in Neural Information Processing Systems, vol. 25, Curran Associates, Inc., 2012. Neural Information Processing Systems, https://proceedings.neurips.cc/paper/2012/hash/c399862d3b9d6b76c8436e924a68c45b-Abstract.html.

2014: Daniel Yamins, a postdoc in DiCarlo’s group, leads the discovery of the first predictive, explanatory models of IT cortex signals; they are neural networks. Yamins, Daniel L. K., et al. “Performance-Optimized Hierarchical Models Predict Neural Responses in Higher Visual Cortex.” Proceedings of the National Academy of Sciences, vol. 111, no. 23, June 2014, pp. 8619–24. DOI.org (Crossref), https://doi.org/10.1073/pnas.1403112111.

2014: Working independently, using different statistical methods, a separate team of neuroscientists co-discover the same class of explanatory models. Khaligh-Razavi, Seyed-Mahdi, and Nikolaus Kriegeskorte. “Deep Supervised, but Not Unsupervised, Models May Explain IT Cortical Representation.” PLOS Computational Biology, vol. 10, no. 11, Nov. 2014, p. e1003915. PLoS Journals, https://doi.org/10.1371/journal.pcbi.1003915.

2014: AI scientists discover that the first AI models of object recognition can also be adapted into general models of visual cortex function. Sharif Razavian, Ali, et al. CNN Features Off-the-Shelf: An Astounding Baseline for Recognition. 2014, pp. 806–13. www.cv-foundation.org, https://www.cv-foundation.org/openaccess/content_cvpr_workshops_2014/W15/html/Razavian_CNN_Features_Off-the-Shelf_2014_CVPR_paper.html.

2016: Neuroscientists discover that the first explanatory, deep learning models of the IT cortex also serve as predictive, hierarchical models of the signals from the full ventral stream of the visual cortex. Yamins, Daniel L. K., and James J. DiCarlo. “Using Goal-Driven Deep Learning Models to Understand Sensory Cortex.” Nature Neuroscience, vol. 19, no. 3, Mar. 2016, pp. 356–65. www.nature.com, https://doi.org/10.1038/nn.4244.

2017: AI scientists at Google invent a new deep learning architecture that is highly scalable, yet which loses the conventional bottom-up, mechanistic interpretability of traditional neural networks. Vaswani, Ashish; Shazeer, Noam; Parmar, Niki; Uszkoreit, Jakob; Jones, Llion; Gomez, Aidan N; Kaiser, Łukasz; Polosukhin, Illia (2017). "Attention is All you Need". Advances in Neural Information Processing Systems. 30. Curran Associates, Inc. arXiv:1706.03762.

2018: The OpenWorm project comes to a culmination that falls far short of the challenges of full organism simulation, while laying conceptual foundations for how to surmount them. Sarma, Gopal P., et al. “OpenWorm: Overview and Recent Advances in Integrative Biological Simulation of Caenorhabditis Elegans.” Philosophical Transactions of the Royal Society B: Biological Sciences, vol. 373, no. 1758, Oct. 2018, p. 20170382. DOI.org (Crossref), https://doi.org/10.1098/rstb.2017.0382.

2022: Neuroscientists discover significant correlations between the signals in AI language models and the signals in language brain regions. Caucheteux, Charlotte, and Jean-Rémi King. “Brains and Algorithms Partially Converge in Natural Language Processing.” Communications Biology, vol. 5, no. 1, Feb. 2022, pp. 1–10. www.nature.com, https://doi.org/10.1038/s42003-022-03036-1.

2023: Some vision scientists question the validity of deep learning models of the visual cortex, which have become ever more popular. Bowers, Jeffrey S., et al. “Deep Problems with Neural Network Models of Human Vision.” Behavioral and Brain Sciences, vol. 46, Jan. 2023, p. e385. Cambridge University Press, https://doi.org/10.1017/S0140525X22002813.

2023: Other neuroscientists push back against the criticisms of the new deep learning models of the visual cortex, pointing out that these can be explained as limitations of the reductionism of the approach, and not the overall AI approach to making models of brain regions. Summerfield, Christopher, and Jessica A. F. Thompson. “Thinking beyond the Ventral Stream: Comment on Bowers et Al.” Behavioral and Brain Sciences, vol. 46, Jan. 2023, p. e409. Cambridge University Press, https://doi.org/10.1017/S0140525X23001723.

2023: Neuroscientists begin to interpret the deep learning paradigm as a successful paradigm for creating in silico models of brain regions, albeit, one that remains reductionist. Doerig, Adrien, et al. “The Neuroconnectionist Research Programme.” Nature Reviews Neuroscience, vol. 24, no. 7, July 2023, pp. 431–50. www.nature.com, https://doi.org/10.1038/s41583-023-00705-w.

2023: Theorists discover that the transformer, the ubiquitous architecture for AI programs, can be interpreted as implemented in biological systems by neurons and astrocytes, drawing interest from biologists and neuroscientists. Kozachkov, Leo, et al. “Building Transformers from Neurons and Astrocytes.” Proceedings of the National Academy of Sciences, vol. 120, no. 34, Aug. 2023, p. e2219150120. DOI.org (Crossref), https://doi.org/10.1073/pnas.2219150120.

2024: Neuroscientists settle on the existence of a separate, dedicated brain region for basic language processing—with the help of the first AI mockups of it. Fedorenko, Evelina, et al. “The Language Network as a Natural Kind within the Broader Landscape of the Human Brain.” Nature Reviews Neuroscience, vol. 25, no. 5, May 2024, pp. 289–312. www.nature.com, https://doi.org/10.1038/s41583-024-00802-4.

2024: Neuroscientists show that the deep neural network modeling paradigm can be used to produce one-to-one mockups between artificial and biological neurons that explain neural measurements. Lappalainen, Janne K., et al. “Connectome-Constrained Networks Predict Neural Activity across the Fly Visual System.” Nature, vol. 634, no. 8036, Oct. 2024, pp. 1132–40. www.nature.com, https://doi.org/10.1038/s41586-024-07939-3.

2024: Neuroscientists observe that AI models of language function can be interpreted, based on their functional properties, as successful, albeit reductionist models of the language network brain region. Mahowald, K., Ivanova, A. A., Blank, I. A., Kanwisher, N., Tenenbaum, J. B., & Fedorenko, E. (2024). Dissociating language and thought in large language models. Trends in Cognitive Sciences.

2024: Neuroscientists observe that AI language models can be best mapped, based on AI-to-brain signal correlations, to the specific subregion of the brain that is the language network. Ayesh, Eyas, et al. "The language network occupies a privileged position among all brain voxels predicted by a language-based encoding model." Cognitive Computational Neuroscience Conference, August 6–9, 2024. https://2024.ccneuro.org/poster/?id=274