Applications of Machine Intelligence

Applications of Machine Intelligence

The Visipedia project led by Pietro Perona (Caltech) and Serge Belongie (Cornell Tech), is aimed at building machine learning systems to empower communities of experts, such as naturalists.


The enormous success of machine learning methods in tackling a wide range of classification problems has enabled the creation of intelligent machines that approach the performance of experts in some domains. This unit provides a taste of some of the successes in areas such as classification of plant and bird species, medicine, and business. Learn more about other applications on the Learning Hub and the CBMM videos page.


The Visipedia project investigates machine learning techniques for empowering communities of experts, by harnessing crowdsourced expertise and deploying that expertise in a utilitarian way. Pietro Perona highlights the success of Visipedia in working with the naturalist community, through the eBird and iNaturalist tools, to automate the classification of species in images in a way that grows the community of experts, resulting in a self reinforcing system of knowledge.
Vijay Chandrasekhar highlights important applications of image retrieval and provides an overview of traditional approaches to implementing the image analysis pipeline for solving image search problems. He then describes how deep learning has been applied to the problem of image retrieval, leading to substantial improvement in performance on image search tasks.
Caption Health (previously Baylabs) seeks to expand global access to medical imaging with low-cost ultrasound technology for diagnosing a range of health conditions. Charles Cadieu shows how the success of Caption Health’s medical diagnosis system depends on the application of deep learning methods, used in the DiCarlo Lab to study object recognition in humans and primates.
Lisa Amini: Automation of AI
Automated AI and Machine Learning technologies can now automate every step of the end-to-end AI Lifecycle, from data cleaning, to algorithm selection, and finally to model deployment and system monitoring. Lisa Amini shows how these technologies not only reduce low level coding tasks for data scientists; they also have great potential to serve non-technical users such as domain experts, to build and deploy machine learning models.
Philip Nelson highlights work of the Google Accelerated Science team, whose aim is to accelerate progress in the natural sciences through the application of Google technologies, including machine learning and large scale analysis and computation. Successes include the detection of diabetic retinopathy and prediction of cardiovascular risk factors from retinal fundus photographs and other medical imaging applications.

Further Study

Online Resources

Additional information about the speakers’ research and publications can be found at these websites:


Cadieu, C. F., Hong, H., Yamins, D. L. K., Pinto, N., Ardila, D., Solomon, E. A., Majej, N. J. & DiCarlo, J. J. (2014) Deep neural networks rival the representation of primate IT cortex for core visual object recognition, PLoS Computational Biology, 10(12):e1003963

Chandrasekhar, V., Jie, L., Morere, O., Goh, H., Veillard, A. (2016) A practical guide to CNNs and Fisher vectors for image instance retrieval, Proc. Signal Processing

Morere, O., Jie, A., Veillard, A., Duan, L., Chandrasekhar, V., Poggio, T. (2017) Nested invariance pooling and RBM hashing for image instance retrieval, ACM International Conference on Multimedia Retrieval, Bucharest, Romania

Van Horn, G., Branson, S., Loarie, S., Belongie, S., Perona, P. (2018) Lean Multiclass Crowdsourcing. Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2714-2723

Van Horn, G., Cole, E., Beery, S., Wilber, K., Belongie, S., Mac Aodha, O. (2021) Natural world image collections, Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition, 12884-12893

Wang, D., Ram, P., Weidele, D. K., Liu, S., Muller, M., Weisz, J. D., Valente, A., Chaudhary, A., Torres, D., Samulowitz, H., Amini, L. (2020), AutoAI: Automating the end-to-end AI lifecycle with humans-in-the-loop, Proc. 25th International Conference on Intelligent User Interfaces Companion, 77-78