A New AI Research from Fujitsu Improves Weakly-Supervised Action Segmentation For Human-Robot Interaction With Action-Union Learning

Recent developments in the field of human action recognition have enabled some amazing breakthroughs in Human-Robot Interaction (HRI). With this technology, robots have begun to understand human behavior and react accordingly. Action segmentation, which is the process of determining the labels and temporal bounds of human actions, is a crucial part of action recognition. Robots must have this skill in order to dynamically localize human behaviors and work well with people.

Conventional methods for action-segmentation model training demand a large number of labels. For thorough supervision, it is ideal to have frame-wise labels, i.e., labels applied to every frame of action, but these labels provide two significant difficulties. First of all, it can be expensive and time-consuming to annotate action labels for each frame. Second, there may be bias in the data due to inconsistent labeling from multiple annotators and unclear time boundaries between actions.

To address these challenges, in recent research, a team of researchers has proposed a new and unique learning technique during the training phase. Their method maximizes the likelihood of action union for unlabeled frames that fall between two consecutive timestamps. The probability that a given frame has a mix of actions indicated by the labels of the surrounding timestamps is known as action union. This approach improves the quality of the training process by giving more dependable learning targets for unlabeled frames by taking the action union probability into account.

The team has developed a novel refining method during the inference step to provide better hard-assigned action labels from the model’s soft-assigned predictions. The action classes that are allocated to frames are made more precise and reliable through this refinement process. It considers not only the frame-by-frame predictions but also the consistency and smoothness of action labels over time in different video segments. This improves the model’s capacity to provide accurate action categorizations.

The techniques created in this research are intended to be model-agnostic, implying they can be utilized with various current action segmentation frameworks. These methods’ adaptability makes it possible to include them in various robot learning systems without having to make significant changes. These techniques’ effectiveness was assessed using three widely used action-segmentation datasets. The outcomes demonstrated that this method achieved new state-of-the-art performance levels by outperforming earlier timestamp-supervision techniques. The team also pointed out that their method produced similar outcomes with less than 1% of fully-supervised labels, which makes it an extremely economical solution that can equal or even outperform fully-supervised techniques in terms of performance. This illustrates how their suggested method might effectively advance the field of action segmentation and its applications in human-robot interaction.

The primary contributions have been summarized as follows.

1. Action-union optimization has been introduced into action-segmentation training, enhancing model performance. This innovative approach considers the probability of action combinations for unlabeled frames between timestamps.

2. A new and extremely beneficial post-processing technique has been introduced to improve the action-segmentation models’ output. The action classifications’ correctness and dependability are greatly increased by this refinement process.

3. The method has produced new state-of-the-art outcomes on pertinent datasets, demonstrating its potential to further Human-Robot Interaction research. 

Check out the Paper and Github. All Credit For This Research Goes To the Researchers on This Project. Also, don’t forget to join our 32k+ ML SubReddit, 40k+ Facebook Community, Discord Channel, and Email Newsletter, where we share the latest AI research news, cool AI projects, and more.

If you like our work, you will love our newsletter..

We are also on WhatsApp. Join our AI Channel on Whatsapp..