%0 Generic %D 2023 %T BrainBERT: Self-supervised representation learning for Intracranial Electrodes %A Christopher Wang %A Vighnesh Subramaniam %A Adam Uri Yaari %A Gabriel Kreiman %A Boris Katz %A Ignacio Cases %A Andrei Barbu %K decoding %K language models %K Neuroscience %K self-supervision %K transformer %X
We create a reusable Transformer, BrainBERT, for intracranial recordings bringing modern representation learning approaches to neuroscience. Much like in NLP and speech recognition, this Transformer enables classifying complex concepts, i.e., decoding neural data, with higher accuracy and with much less data by being pretrained in an unsupervised manner on a large corpus of unannotated neural recordings. Our approach generalizes to new subjects with electrodes in new positions and to unrelated tasks showing that the representations robustly disentangle the neural signal. Just like in NLP where one can study language by investigating what a language model learns, this approach opens the door to investigating the brain by what a model of the brain learns. As a first step along this path, we demonstrate a new analysis of the intrinsic dimensionality of the computations in different areas of the brain. To construct these representations, we combine a technique for producing super-resolution spectrograms of neural data with an approach designed for generating contextual representations of audio by masking. In the future, far more concepts will be decodable from neural recordings by using representation learning, potentially unlocking the brain like language models unlocked language.
%B International Conference on Learning Representations %C Kigali, Rwanda, Africa %8 02/2023 %U https://openreview.net/forum?id=xmcYx_reUn6 %0 Generic %D 2021 %T Image interpretation by iterative bottom-up top- down processing %A Shimon Ullman %A Liav Assif %A Alona Strugatski %A Ben-Zion Vatashsky %A Hila Levi %A Aviv Netanyahu %A Adam Uri Yaari %X Scene understanding requires the extraction and representation of scene components, such as objects and their parts, people, and places, together with their individual properties, as well as relations and interactions between them. We describe a model in which meaningful scene structures are extracted from the image by an iterative process, combining bottom-up (BU) and top-down (TD) networks, interacting through a symmetric bi-directional communication between them (‘counter-streams’ structure). The BU- TD model extracts and recognizes scene constituents with their selected properties and relations, and uses them to describe and understand the image. The scene representation is constructed by the iterative use of three components. The first model component is a bottom-up stream that extracts selected scene elements, properties and relations. The second component (‘cognitive augmentation’) augments the extracted visual representation based on relevant non-visual stored representations. It also provides input to the third component, the top-down stream, in the form of a TD instruction, instructing the model what task to perform next. The top-down stream then guides the BU visual stream to perform the selected task in the next cycle. During this |
process, the visual representations extracted from the image can be combined with relevant non- visual representations, so that the final scene representation is based on both visual information extracted from the scene and relevant stored knowledge of the world. The extraction process is shown to have favourable properties in terms of combinatorial generalization, |
generalizing well to novel scene structures and new combinations of objects, properties and relations not seen during training. Finally, we compare the model with relevant aspects of the human vision, and suggest directions for using the BU-TD scheme for integrating visual and cognitive components in the process of scene understanding. |
https://hdl.handle.net/1721.1/139678
%0 Conference Paper %B International Conference on Learning Representations %D 2021 %T Multi-resolution modeling of a discrete stochastic process identifies causes of cancer %A Adam Uri Yaari %A Maxwell Sherman %A Oliver Clarke Priebe %A Po-Ru Loh %A Boris Katz %A Andrei Barbu %A Bonnie Berger %XDetection of cancer-causing mutations within the vast and mostly unexplored human genome is a major challenge. Doing so requires modeling the background mutation rate, a highly non-stationary stochastic process, across regions of interest varying in size from one to millions of positions. Here, we present the split-Poisson-Gamma (SPG) distribution, an extension of the classical Poisson-Gamma formulation, to model a discrete stochastic process at multiple resolutions. We demonstrate that the probability model has a closed-form posterior, enabling efficient and accurate linear-time prediction over any length scale after the parameters of the model have been inferred a single time. We apply our framework to model mutation rates in tumors and show that model parameters can be accurately inferred from high-dimensional epigenetic data using a convolutional neural network, Gaussian process, and maximum-likelihood estimation. Our method is both more accurate and more efficient than existing models over a large range of length scales. We demonstrate the usefulness of multi-resolution modeling by detecting genomic elements that drive tumor emergence and are of vastly differing sizes.
%B International Conference on Learning Representations %8 09/2020 %G eng %U https://openreview.net/forum?id=KtH8W3S_RE