Abstract: This talk will update progress on a research program which was presented in “Deep Networks and Beyond” at the CBMM AI workshop at Stanford. The goal is to develop hierarchical architectures with the same strong performance abilities as deep networks but which are also able to model the flexibility and adaptiveness of biological visual systems. These architectures are intended to be simpler and more explainable, require little supervision and few training examples, and to be adaptive to situations/environments which they have not encountered. In particular, we give new results for unsupervised learning of objects and object viewpoints, one- and few-shot learning for discriminative tasks, and the ability to deal with adversarial attacks. We conclude by speculating on how vision algorithms should be evaluated given the increasingly complexity of visual tasks and the impracticality of getting sufficient data for training and testing.