My research is about building computational systems that can perceive, learn, and predict what is happening around them. The basic question that intrigues me is how can we represent and manipulate perceptual information to make it useful for intelligent systems. Over the years, I have been particularly focusing on finding novel sequence representations to model everyday human activities. More recently, I have been looking into ways to detect human actions observed at a distance for sports activity visualization.

More information about my research can be found in my Research Statement.

Research Code: You can find some of the code related to my research here.

Following are some of the specific topics I have explored so far.

Models for activity structure for unconstrained environments are generally not available a priori. Recent representational approaches to this end are limited by their computational complexity, and ability to capture activity structure only up to some fixed temporal scale. In this work, we propose the usage of Suffix Trees as an activity representation to efficiently extract structure of activities by analyzing their constituent event-subsequences over multiple temporal scales.

[PDF]

For an active environment, how can one transform semantically agnostic low-level perceptual inputs, into some mid-level abstractions that sufficiently encode the activity structure? How can one represent such activity structure over a continuum of temporal resolutions? Finally, how can one automatically detect event subsequences that are locally atypical in a structural sense? In this work, we investigate these questions in the context of understanding everyday activities.

[PDF]

A key step towards understanding what is happening in an active setting, is to discover the various kinds of frequently occurring similar activities in that domain. Equally important is the question of finding efficient characterizations for these different kinds of activities. In this work we tackle the question of activity class discovery and characterization, in the backdrop of analyzing everyday activities.

[PDF]

Anomalies are sets of rare events which, for any reasonably unconstrained situation, are hard to completely define as a prior. For the reasons of rarity and large within-class variation of anomalies, techniques which try to model them, either statistically or through a set of rules, often prove to be brittle and over-fitted. We formulate the problem of anomalous activity explanation using a novel representation of activities as bags of n-grams of discrete events.

[PDF]

A novel framework for recognizing complex multi-agent activities using probabilistic graphical models is presented. We employ statistical feature based particle filter to robustly track multiple objects in cluttered environments. Spatio-temporal features extracted from tracking are thereon used to learn graphical models for modeling these activities.

[PDF]

Due to the large variation in the physical attributes of different environments, a generic classifier trained on extensive data-sets my still perform sub-optimally in a new test environment. In this work we present a general framework for classifier adaptation that allows an already trained generic classifier to perform better in new test environments. The work was done at Microsoft Research.

[PDF]

The weighted Ensemble Boosting method combines Bayesian Averaging strategy coupled with Boosting framework, finding useful conjunctive features-combinations and achieving lower error rates than traditional Boosting algorithm. The method demonstrates a comparable level of stability with respect to the classifier selection pool. We compare its performance with different classifier combination methods, including Approximate Bayesian Combination, Boosting, Feature Stacking and the more traditional Sum and Product rules. The work was done at Mitsubishi Electronic Research Lab.

[PDF]

Context-aware applications take their context of use into account by adapting to changes in a user's activities and environments. No one has more intimate knowledge about these activities and environments than end-users themselves. Currently there is no support for end-users to build context-aware applications for these dynamic settings. To address this issue, we present a programming by demonstration context aware prototyping environment. The work was done for Intel Research Lab Berkeley.

[PDF]

The flow field of a moving sound source not only has an optical component, but also an audio component; something we call audio-visual flow. We present a common structure-tensor based variational framework for dense audio-visual flow-field estimation.

[PDF]

Decision Tree based Object Classifiers for automatic automobile detection system. The project was a collaborative effort between General Motors, & Techlogix Inc.  The project resulted in a US patent and a publication.

[PDF]

The visually impaired have limited access to the world of mobile devices. Our goal was to design a handheld mobile device to overcome limitations such as reliance on visual display and lack of audio and tactile feedback. We built a prototype handheld device using tactile feedback and auditory display information.

[PDF]

Copyright © 2010 Raffay Hamid. All rights reserved.