Boris Katz

Boris Katz
Boris
Katz
Coordinator for Technology and Knowledge Transfer, Research Module Co-Leader
Massachusetts Institute of Technology (MIT)

Associated Research Module: 

Associated Research Thrust: 

Boris Katz is a Principal Research Scientist and Head of the InfoLab Group at MIT’s Computer Science and Artificial Intelligence Laboratory. His research interests include natural language understanding and generation, intelligent information access, knowledge representation, human computer interaction, and event recognition.

Boris Katz is the creator of the START information access system and the inventor of a patented method of natural language annotations, which facilitate access to multimedia information in response to questions expressed in everyday language. In 1989 during the Voyager Neptune encounter, START was used in the JPL press room to answer reporters’ questions about the Voyager 2 spacecraft. In 1993, START became the first question-answering system on the Web, and since then answered millions of questions from Web users all over the world.

Boris Katz is a member of the Open Advancement of Question Answering consortium where he contributed several technical ideas incorporated into IBM’s Watson system, which in 2011 defeated the all-time human champions on the quiz show Jeopardy. Technology created in Katz's InfoLab Group in 2006 (see press, technical paper and video) was a major inspiration for the development of Apple's personal assistant, Siri.

 

Website: 

Current Advisees

Julian Alverio - Graduate Research Assistant
Andrei Barbu - Research Scientist
Yen-Ling Kuo - Graduate Student
Xin Yu Lin - Graduate Student
Natasha Maniar - Undergraduate , UROP
Aviv Netanyahu - Graduate Student
Dana Rosenfarb - Graduate Student
Lucia Schiatti - Postdoc
Aaditya K. Singh - Graduate Student
Vighnesh Subramaniam - Research Assistant
Ravi Tejwani - Graduate Student

Past Advisees

Dalitso Banda - Research Assistant
Guy Ben-Yosef - Research Scientist
Yevgeni Berzak - Graduate Student
Run Chen - Undergraduate Researcher
Allison Fu - Graduate Student
Lila Jansen - UROP
Jessica Kenney - UROP
Casimir Lesperance - UROP
Jonathan Malmaud - Graduate Student
Cheahuychou Mao - Graduate Student
Cristina Mata - Graduate Student
David Mayo - Graduate Student
Sangeev Murty - Graduate Student
Candace Ross - Graduate Student
Amelia Smith - Undergraduate Researcher
Carrie Spadine - Graduate Student
Dalila Stanfield - Undergraduate Researcher
Adam Yaari - Graduate Student

Projects

Grounded language acquisition
Grounded question answering
Human language learning
Investigating neural signals underlying language processing in the human brain
Multi-sentence event recognition
Objects and hands in context

CBMM Publications

R. Tejwani, Kuo, Y. - L., Shu, T., Stankovits, B., Gutfreund, D., Tenenbaum, J. B., Katz, B., and Barbu, A., “Zero-shot linear combinations of grounded social interactions with Linear Social MDPs”, in Proceedings of the 37th AAAI Conference on Artificial Intelligence (AAAI), 2023.
C. Wang, Subramaniam, V., Yaari, A. Uri, Kreiman, G., Katz, B., Cases, I., and Barbu, A., “BrainBERT: Self-supervised representation learning for Intracranial Electrodes”, International Conference on Learning Representations. Kigali, Rwanda, Africa, 2023.
V. Subramaniam, Conwell, C., Wang, C., Kreiman, G., Katz, B., Cases, I., and Barbu, A., “Using Multimodal DNNs to Study Vision-Language Integration in the Brain”, in ICLR 2023, 2023.
L. Schiatti, Gori, M., Schrimpf, M., Cappagli, G., Morelli, F., Signorini, S., Katz, B., and Barbu, A., “Modeling Visual Impairments with Artificial Neural Networks: a Review”, in International Conference on Computer Vision 2023, Paris, 2023.
Y. - L. Kuo, Huang, X., Barbu, A., McGill, S. G., Katz, B., Leonard, J. J., and Rosman, G., “Trajectory Prediction with Linguistic Representations”. 2022.
R. Tejwani, Kuo, Y. - L., Shu, T., Stankovits, B., Gutfreund, D., Tenenbaum, J. B., Katz, B., and Barbu, A., “Incorporating Rich Social Interactions Into MDPs”. 2022.
A. Yaari, DeWitt, J., Hu, H., Stankovits, B., Felshin, S., Berzak, Y., Aparicio, H., Katz, B., Cases, I., and Barbu, A., “The Aligned Multimodal Movie Treebank: An audio, video, dependency-parse treebank”, in Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2022.
E. Cheng, Kuo, Y. - L., Cases, I., Katz, B., and Barbu, A., “Spontaneous sign emergence in humans and machines through an embodied communication game”, in JCoLE Workshop, 2022.
E. Cheng, Kuo, Y. - L., Correa, J., Katz, B., Cases, I., and Barbu, A., “Quantifying the Emergence of Symbolic Communication”, CogSci, 2022.
Y. - L. Kuo, Barbu, A., and Katz, B., “Compositional RL Agents That Follow Language Commands in Temporal Logic”. 2021.
I. Palmer, Rouditchenko, A., Barbu, A., Katz, B., and Glass, J., “Spoken ObjectNet: A Bias-Controlled Spoken Caption Dataset”. 2021.
A. Netanyahu, Shu, T., Katz, B., Barbu, A., and Tenenbaum, J. B., “PHASE: PHysically-grounded Abstract Social Events for Machine Social Perception”, in AAAI-21, 2021.
Y. - L. Kuo, Katz, B., and Barbu, A., “Compositional Networks Enable Systematic Generalization for Grounded Language Understanding”. 2021.
R. Tejwani, Kuo, Y. - L., Shu, T., Katz, B., and Barbu, A., “Social Interactions as Recursive MDPs”. 2021.
C. Ross, Katz, B., and Barbu, A., “Measuring Social Biases in Grounded Vision and Language Embeddings”, NAACL (Annual Conference of the North American Chapter of the Association for Computational Linguistics). 2021.
C. Conwell, Mayo, D., Buice, M. A., Katz, B., Alvarez, G. A., and Barbu, A., “Neural Regression, Representational Similarity, Model Zoology Neural Taskonomy at Scale in Rodent Visual Cortex”. 2021.
C. Ross, Barbu, A., and Katz, B., “Measuring Social Biases in Grounded Vision and Language Embeddings”. 2021.
C. Conwell, Mayo, D., Buice, M., Katz, B., Alvarez, G., and Barbu, A., “Large-scale benchmarking of deep neural network models in mouse visual cortex reveals patterns similar to those observed in macaque visual cortex”, Cosyne. 2021.
A. Netanyahu, Shu, T., Katz, B., Barbu, A., and Tenenbaum, J. B., “PHASE: PHysically-grounded Abstract Social Events for Machine Social Perception”. 2021.
A. Uri Yaari, Sherman, M., Priebe, O. Clarke, Loh, P. - R., Katz, B., Barbu, A., and Berger, B., “Multi-resolution modeling of a discrete stochastic process identifies causes of cancer”, in International Conference on Learning Representations, 2021.
Y. - L. Kuo, Katz, B., and Barbu, A., “Encoding formulas as deep networks: Reinforcement learning for zero-shot execution of LTL formulas”. 2020.
Y. - L. Kuo, Katz, B., and Barbu, A., “Deep compositional robotic planners that follow natural language commands”. 2020.
C. Wang, Ross, C., Kuo, Y. - L., Katz, B., and Barbu, A., “Learning a natural-language to LTL executable semantic parser for grounded robotics”. 2020.
C. Wang, Ross, C., Kuo, Y. - L., Katz, B., and Barbu, A., “Learning a Natural-language to LTL Executable Semantic Parser for Grounded Robotics”, Proceedings of Conference on Robot Learning (CoRL-2020), 2020.
A. Netanyahu, Shu, T., Katz, B., Barbu, A., and Tenenbaum, J. B., “PHASE: PHysically-grounded Abstract Social Eventsfor Machine Social Perception”, in Shared Visual Representations in Human and Machine Intelligence (SVRHM) workshop at NeurIPS 2020, 2020.
A. Barbu, Mayo, D., Alverio, J., Luo, W., Wang, C., Gutfreund, D., Tenenbaum, J. B., and Katz, B., “ObjectNet: A large-scale bias-controlled dataset for pushing the limits of object recognition models”, Neural Information Processing Systems (NeurIPS 2019). Vancouver, Canada, 2019.
C. Ross, Berzak, Y., Katz, B., and Barbu, A., “Learning Language from Vision.”, in Workshop on Visually Grounded Interaction and Language (ViGIL) at the Thirty-third Annual Conference on Neural Information Processing Systems (NeurIPS), Vancouver Convention Center, Vancouver, Canada, 2019.
B. Katz, Borchardt, G., Felshin, S., and Mora, F., “A Natural Language Interface for Mobile Devices”, in The Wiley Handbook of Human Computer Interaction, First., vol. 2, John Wiley & Sons , 2018, pp. 539-559.
Y. Berzak, Katz, B., and Levy, R., “Assessing Language Proficiency from Eye Movements in Reading”, 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. New Orleans, 2018.
C. Ross, Barbu, A., Berzak, Y., Myanganbayar, B., and Katz, B., “Grounding language acquisition by training semantic parsersusing captioned videos”, in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP 2018), , Brussels, Belgium, 2018.
Y. - L. Kuo, Barbu, A., and Katz, B., “Deep sequential models for sampling-based planning”, in The IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2018), Madrid, Spain, 2018.
B. Myanganbayar, Mata, C., Dekel, G., Katz, B., Ben-Yosef, G., and Barbu, A., “Partially Occluded Hands: A challenging new dataset for single-image hand pose estimation”, in The 14th Asian Conference on Computer Vision (ACCV 2018), 2018.
Y. Berzak, Nakamura, C., Flynn, S., and Katz, B., “Predicting Native Language from Gaze”, in Annual Meeting of the Association for Computational Linguistics (ACL 2017), 2017.
R. Paul, Barbu, A., Felshin, S., Katz, B., and Roy, N., “Temporal Grounding Graphs for Language Understanding with Accrued Visual-Linguistic Context”, in Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI 2017), Melbourne, Australia, 2017.
A. Morales, Premtoon, V., Avery, C., Felshin, S., and Katz, B., “Learning to Answer Questions from Wikipedia Infoboxes”, in The 2016 Conference on Empirical Methods on Natural Language Processing (EMNLP 2016), 2016.
Y. Berzak, Barbu, A., Harari, D., Katz, B., and Ullman, S., “Language and Vision Ambiguities (LAVA) Corpus”. 2016.
Y. Berzak, Kenney, J., Spadine, C., Wang, J. Xian, Lam, L., Mori, K. Sophie, Garza, S., and Katz, B., “Universal Dependencies for Learner English”. 2016.
Y. Berzak, Reichart, R., and Katz, B., “Contrastive Analysis with Predictive Power: Typology Driven Estimation of Grammatical Error Distributions in ESL”. 2016.
Y. Berzak, Kenney, J., Spadine, C., Wang, J. Xian, Lam, L., Mori, K. Sophie, Garza, S., and Katz, B., “Treebank of Learner English (TLE)”. 2016.
Y. Berzak, Barbu, A., Harari, D., Katz, B., and Ullman, S., “Do You See What I Mean? Visual Resolution of Linguistic Ambiguities”. 2016.
Y. Berzak, Huang, Y., Barbu, A., Korhonen, A., and Katz, B., “Anchoring and Agreement in Syntactic Annotations”. 2016.
B. Katz and Barbu, A., “A look back at the June 2016 BMM Workshop in Sestri Levante, Italy”. 2016.
H. Shrobe, Katz, B., and Davis, R., “Towards a Programmer's Apprentice (Again)”. 2015.
Y. Berzak, Reichart, R., and Katz, B., “Contrastive Analysis with Predictive Power: Typology Driven Estimation of Grammatical Error Distributions in ESL”, in Nineteenth Conference on Computational Natural Language Learning (CoNLL), Beijing, China, 2015.
Y. Berzak, Barbu, A., Harari, D., Katz, B., and Ullman, S., “Do You See What I Mean? Visual Resolution of Linguistic Ambiguities”, in Conference on Empirical Methods in Natural Language Processing, Lisbon, Portugal. , 2015.
Y. Berzak, Reichart, R., and Katz, B., “Reconstructing Native Language Typology from Foreign Language Usage.”. 2014.