Built independently by an author, for readers. Read the story and support ChapterPal

keyword

instance-level feature learning

Instance-level feature learning is a computer vision and machine learning approach focused on extracting and encoding distinct representations for individual objects or entities within an image or scene. Unlike global image-level learning that summarizes an entire visual input as a single context, instance-level learning isolates the localized appearance, spatial coordinates, geometry, and semantic properties of specific subjects or items. Models typically achieve this through region-based pooling, bounding boxes, segmentation masks, or specialized object queries that distinguish each target entity from the background and other neighboring objects. By capturing the unique visual attributes of separate entities, instance-level feature learning provides localized representations essential for tasks such as object detection, instance segmentation, visual tracking, and identifying relationships among multiple interacting elements.

1 item

Detecting Any Human-Object Interaction Relationship: Universal HOI Detector with Spatial Prompt Learning on Foundation Models

Detecting Any Human-Object Interaction Relationship: Universal HOI Detector with Spatial Prompt Learning on Foundation Models

Yichao Cao, Qingfei Tang, Xiu Su, Song Chen, Shan You, Xiaobo Lu, Chang Xu

OrganizationsNanjing Enbo Tech.SenseTimeSoutheast UniversityUniversity of Sydney

Why you should read this

Proposes UniHOI, a framework that integrates vision-language foundation models with language-model-generated knowledge via spatial prompt learning to advance open-world and zero-shot human-object interaction detection.

Human-object interaction (HOI) detection aims to comprehend the intricate relationships between humans and objects, predicting < human, action, object > triplets, and serving as the foundation for numerous computer vision tasks. The complexity and diversity of human-object interactions in the real world, however, pose significant challenges for both annotation and recognition, particularly in recognizing interactions within an open world context. This study explores the universal interaction recognition in an open-world setting through the use of Vision-Language (VL) foundation models and large language models (LLMs). The proposed method is dubbed as UniHOI. We conduct a deep analysis of the three hierarchical features inherent in visual HOI detectors and propose a method for high-level relation extraction aimed at VL foundation models, which we call HO prompt-based learning. Our design includes an HO Prompt-guided Decoder (HOPD), facilitates the association of high-level relation representations in the foundation model with various HO pairs within the image. Furthermore, we utilize a LLM (i.e. GPT) for interaction interpretation, generating a richer linguistic understanding for complex HOIs. For open-category interaction recognition, our method supports either of two input types: interaction phrase or interpretive sentence. Our efficient architecture design and learning methods effectively unleash the potential of the VL foundation models and LLMs, allowing UniHOI to surpass all existing methods with a substantial margin, under both supervised and zero-shot settings. The code and pre-trained weights are available at: https://github.com/Caoyichao/UniHOI.

Added

2026-09-26