FSOD-VFM: few-shot object detection with graph diffusion

Abstract

FSOD-VFM uses vision foundation models for few-shot object detection. It combines a universal proposal network (UPN) for category-agnostic bounding boxes, SAM2 for precise masks, and DINOv2 features for adaptation to new object categories.

Despite foundation models’ generalization capabilities, UPN proposals often cover only parts of objects. This overfragmentation produces numerous small false positives instead of complete detections.

The authors introduce graph-based confidence reweighting. Predicted boxes form nodes in a directed graph, and graph diffusion propagates scores between them. Whole objects receive higher confidence while fragmented local parts receive lower confidence, improving detection granularity and reducing false-positive proposals.

Experiments on Pascal-5, COCO-20 and CD-FSOD demonstrate improvements without additional training. In the 10-shot CD-FSOD setting, the method reaches 31.6 AP versus 21.4 AP for a previous training-free method. Code and pretrained weights are available from the authors.

Method: from a support example to a whole object

A support example defines a novel class; a query image contains the objects to find. UPN proposes query boxes, SAM2 produces masks, and DINOv2 extracts region features. The support object supplies a class prototype against which proposals are matched.

A matching region may be only a part of the target object. The graph links proposals and redistributes confidence using relationships between regions. In the original diagram, this step sits between prototype matching and final detections.

Open original figure ↗

Visual results and new domains

The original visualization compares support examples and detections in multiple domains: animals, clipart, aerial imagery, fish, industrial defects and underwater objects. These boxes are research predictions, rather than Deepvizor demonstration annotations.

Dataset and class names in the original images are retained to make comparison with the source possible.

One supportPredictionOne supportPrediction

Open original figure ↗

Pascal: detecting novel classes

The table preserves every row for three novel-class splits and 1, 2, 3, 5 and 10 support examples. nAP50 is novel-class average precision at IoU 0.5; higher is better. Fine-tuning indicates additional detector training in this comparison.

COCO: precision at different overlap thresholds

Results use 10 and 30 examples per class. nAP evaluates novel classes across IoU thresholds; nAP50 and nAP75 use thresholds of 0.5 and 0.75. FSOD-VFM reaches 44.0 nAP at 10 shots and 45.8 at 30 shots.

CD-FSOD: transfer across domains

Each cell reports 1 / 5 / 10-shot results. FSOD-VFM averages 25.3 / 31.3 / 31.6 AP. The 10-shot value exceeds the previous training-free result by 10.2 AP points, an absolute metric difference rather than a percentage increase in system accuracy.

These values describe the research experiment. Deployment on enterprise cameras requires separate checks of classes, viewpoints, object sizes and lighting.

Related work and citation

The related FT-FSOD project examines fine-tuning and a parallel decoder for cross-domain few-shot detection. The FSOD-VFM project page also reports NJUST-KMG’s third place in the CVPR NTIRE 2026 CDFSOD Challenge using the approach; this is the research team’s result.

Related research project ↗

@inproceedings{feng2026fsod,
  title={FSOD-VFM: Few-Shot Object Detection with Vision Foundation Models and Graph Diffusion},
  author={Feng, Chen-Bin and Sha, Youyang and Liu, Longfei and Yu, Yongjun and Vong, Chi Man and Yu, Xuanlong and Shen, Xi},
  booktitle={ICLR},
  year={2026}
}