Md Salman Shamil

AS6-04-11,
11 Computing Dr,
Singapore, 117416

Hi, I am a second-year Ph.D. candidate at the School of Computing, National University of Singapore. I am working with Prof. Angela Yao in the CVML@NUS lab. My current research focus is utilizing hand pose data for action recognition and video understanding.

I completed my Bachelor of Science from the Department of CSE, Bangladesh University of Engineering and Technology (BUET) in February 2021. As part of my undergrad thesis, I worked with Professor M. Sohel Rahman on computer vision for medical images. We also worked on agent-based modeling and string algorithms. After graduating, I served as a lecturer at the Computer Science and Engineering department, United International University, Dhaka, before starting my Ph.D.

News

Jul 1, 2024	Our HandFormer is accepted to ECCV 2024!
Jun 17, 2024	Presented HandFormer at the EgoVis Workshop in CVPR ‘24.
Mar 16, 2024	New preprint available here!

Latest Publication

ECCV

On the Utility of 3D Hand Poses for Action Recognition

Shamil, Md Salman, Chatterjee, Dibyadip, Sener, Fadime, Ma, Shugao, and Yao, Angela

arXiv preprint arXiv:2403.09805 2024

Abs HTML PDF

3D hand poses are an under-explored modality for action recognition. Poses are compact yet informative and can greatly benefit applications with limited compute budgets. However, poses alone offer an incomplete understanding of actions, as they cannot fully capture objects and environments with which humans interact. To efficiently model hand-object interactions, we propose HandFormer, a novel multimodal transformer. HandFormer combines 3D hand poses at a high temporal resolution for fine-grained motion modeling with sparsely sampled RGB frames for encoding scene semantics. Observing the unique characteristics of hand poses, we temporally factorize hand modeling and represent each joint by its short-term trajectories. This factorized pose representation combined with sparse RGB samples is remarkably efficient and achieves high accuracy. Unimodal HandFormer with only hand poses outperforms existing skeleton-based methods at 5× fewer FLOPs. With RGB, we achieve new state-of-the-art performance on Assembly101 and H2O with significant improvements in egocentric action recognition.