Pro$^2$Assist: Continuous Step-Aware Proactive Assistance with Multimodal Egocentric Perception for Long-Horizon Procedural Tasks

Pro$^2$Assist is a novel proactive assistant designed to enhance the management of long-horizon procedural tasks by continuously monitoring user progress and providing timely assistance. Unlike existing systems that mainly offer reactive support, Pro$^2$Assist utilizes multimodal data from augmented reality glasses to track and reason over the user's evolving state, enabling a more integrated and responsive experience. This advancement in multimodal large language models could significantly improve how users interact with technology during complex tasks.