have an interactive transcript feature enabled, which appears below the video when playing. Viewers can search for keywords in the video or click on any word in the transcript to jump to that point in the video. When searching, a dark bar with white vertical lines appears below the video frame. Each white line is an occurrence of the searched term and can be clicked on to jump to that spot in the video.
From Motor Control to Scene Perception: Using Machine Learning to Study Human Behavior and Cognition
Akram Bayat, UMass Boston
Abstract:
In this presentation, as part of my work at UMass Boston, two dimensions of implementing machine learning algorithms for solving two important real world problems are discussed. In the first part, we model human eye movements in order to identify different individuals during reading activity. As an important part of our pattern recognition process we extract multiple low-level features in the scan path including fixation features, saccadic features, pupillary response features, and spatial reading features.
While capturing eye movement during reading is desirable because it is a very common task, the text content influences the reading process, making it very challenging to obtain invariant features from eye-movement data. We address this issue with a novel idea for a user identification algorithm that benefits from extracting high level features that combines eye movements with syntactic and semantic word relationships in a text. The promising results of our identification method make eye-movement based identification an excellent approach for various applications such as personalized user interfaces.
The second part of my work focuses on scene perception and object recognition using deep convolutional neural networks. We investigate to which extent computer vision based systems for scene classification and object recognition resemble human mechanisms for scene perception. Employing global properties for scene classification, scene grammar, and top-down control of visual attention for object detection are three methodologies which we evaluate in humans and deep convolutional networks. We also evaluate the performance of deep object recognition networks (e.g., Faster R-CNN) under various conditions of image filtering in the frequency domain and compare it with the human visual system in terms of internal representation. We then show that fine-tuning the Faster-RCNN to filtered data improves network performance over a range of spatial frequencies.