S SOS!
Accepted at BMVC 2026

SOS! A Streamlined Object-Conditional Transformer for Model-free Segmentation

Segment unseen objects from one image with a single reference view via a single feed-forward pass.

Jiaqi Hu1,2 Junwen Huang1,3 Hongli Xu1 Peter KT Yu4 Nassir Navab1,3 Benjamin Busam1,3 Slobodan Ilic1,2
1 Technical University of Munich 2 Siemens AG 3 Munich Center for Machine Learning 4 ROBOX
Paper Code
Comparison of model-based two-stage and one-stage object segmentation pipelines with the model-free SOS one-stage pipeline, which uses only single-reference views.
SOS removes CAD models, template rendering, and proposal matching from the inference pipeline. Click the figure to view it at full resolution.

Overview

Tell the model what to segment by showing it once.

Foundation segmentation models generate strong class-agnostic masks, yet they struggle to associate those masks with a specific target. SOS closes this semantic gap with a single reference image for each object. Its Object-Conditional Transformer jointly identifies the target and predicts its mask in one streamlined pass.

01

Model-free by design

No CAD model, reconstructed geometry, or exhaustive 3D template bank is required at inference time.

02

Identity-aware

Object-conditioned queries carry target identity directly into the segmentation decoder.

03

Single-stage

Mask generation and target identification are unified instead of being handled by disconnected proposal and matching stages.

Method

Object conditioning, end to end.

A frozen DINOv3 backbone provides visual features. SOS turns the reference into identity-anchored queries and decodes them against the query image to produce target-specific masks.

SOS architecture: object conditions and a scene image pass through a frozen DINO feature encoder, condition and scene embedders, an object-conditional transformer, and a mask decoder to produce object masks.
SOS architecture. Identity-anchored queries connect reference appearance to dense scene prediction. Swipe or click the figure to inspect the details.
Evaluated on
YCB-VLM-OHomebrewedDBTUD-L

Quantitative results

SOS reaches 58.8 average AP from one reference image per object, with no CAD model or onboarding cost.

58.8 Average AP
0.120 s Inference per image
1 Reference image per object
0.0 s Onboarding per object
BOP challenge datasets

Average AP

Model-based One reference SOS

Model-based · 42 CAD reference images

*CNOS
49.7
*SAM6D
52.3
*NIDS-Net
56.6
*MUSE
59.3
*OC-DiT
58.2

Model-free · one reference image

CNOS
46.0
SAM6D
47.6
SOS Ours
58.8
Average AP across YCB-V, TUD-L, LM-O, and HomebrewedDB. SOS is within 0.5 AP of the best model-based result without CAD models or onboarding. * Values sourced from the original papers.
Reported runtime

Inference time per image

Lower is better ↓
SOS Ours
0.120 s
*CNOS
0.221 s
*SAM6D
0.249 s
*NIDS-Net
0.485 s
*MUSE
0.505 s
SOS runs in 0.120 seconds per image—1.8× faster than the next-fastest reported method. * Values sourced from the original papers; methods without reported timings are omitted.

Qualitative visualizations

Predictions on LM-O, YCB-V, and TUD-L compared with CNOS, SAM6D, and ground-truth masks.

Qualitative segmentation comparisons on LM-O, YCB-V, and TUD-L, showing scene images and predictions from CNOS, SAM6D, SOS, and ground truth.
Qualitative results across three BOP benchmarks. SOS recovers the target set without CAD models. Swipe or click the figure to inspect the details.

Citation

Build on SOS.

If this project supports your research, please cite our BMVC 2026 paper. 

@inproceedings{hu2026sos,
  title     = {{SOS!}: A Streamlined Object-Conditional Transformer for Model-free Segmentation},
  author    = {Hu, Jiaqi and Huang, Junwen and Xu, Hongli and Yu, Peter KT and Navab, Nassir and Busam, Benjamin and Ilic, Slobodan},
  booktitle = {Proceedings of the British Machine Vision Conference (BMVC)},
  year      = {2026}
}