Research Interests
  • Computer Vision: Computer Vision with Human-in-the-loop, Social Network Image Analysis, Image Retrieval, Parts and Attributes of Images, Medical Image Analysis, Image and Video Processing, Object and Scene Recognition, 3-d Reconstruction.
  • Machine Learning: Active Learning, Clustering, Deep Learning and Neural Networks
  • Others: Crowdsourcing, Human Computer Interaction, Bio-inspired Optimization


I have been working on Computer Vision with Human in the Loop, where humans collaborate/interact with computer vision algorithms to make them better and efficient (often referred as Active Learning in the Machine Learning community).

I have looked at the problem of image clustering. Although there are many excellent clustering algorithms, effective clustering remains very challenging for datasets that contain many classes. We have developed several methods to cluster images accurately.

In one of our approaches we use pairwise constraints from humans to cluster images. An algorithm provides us with pairwise image similarities. We then actively obtain selected, more accurate pairwise similarities from humans. A novel method is developed to choose the most useful pairs to show a person, obtaining constraints that improve clustering.

In another approach, instead of clustering a complete dataset, we only cluster a subset of data; we call that subclustering. This is useful for large datasets (50,000 or more), with many classes. Since large unlabeled image collections are highly prevalent these days subclustering can have a wide range of applications including browsing image databases, image search, summarizing image databases, category discovery etc.

I have worked on user modeling from social network images while I was at Xerox PARC working with Dr. Eugene Bart.

I have worked on using relative attributes for feedback in classifier learning. We have seen that we can learn classifiers fast if supervisors convey more information to the learner about an image domain in terms of attributes. (work done in TTIC with Dr. Devi Parikh).

I have worked for Leafsnap. Leafsnap is an iPhone and iPad application (released in May, 2011) for plant species identification. This is a joint project by University of Maryland, College Park, Columbia University and Smithsonian Institution.

I also collaborated with iSchool at UMD for developing a simple computer game (Odd Leaf Out) for finding labeling errors in large image databases.