Video analytics at the edge: the EdgeCrafter study

Abstract: the challenge for compact transformers

High-performance dense prediction is difficult to deploy on edge devices with limited compute and memory. Lightweight detection, instance segmentation and pose estimation remain largely dominated by convolutional architectures such as YOLO, while compact ViTs often struggle to match their accuracy–efficiency balance even after large-scale pretraining.

The authors attribute much of this gap to insufficient task-specific representation learning in small ViTs. EdgeCrafter is a unified compact-ViT framework centered on ECDet, combining a distilled small backbone with an edge-oriented encoder–decoder design.

A large DINOv3-pretrained ViT is first adapted to detection and used as a task-specialized teacher. Knowledge is distilled into compact students using ImageNet-1K and COCO images. A lightweight convolutional stem replaces standard patch embedding, while interpolation and linear projection produce multi-scale features instead of costly pyramids.

The detection-distilled representation transfers to instance segmentation and human pose estimation through lightweight task-specific modules, reusing one representation-learning recipe across tasks.

Three stages: prepare, distill, specialize

The first stage prepares the teacher. A large DINOv3 backbone is placed inside ECDet and trained using labeled detection data. A flame in the source diagram denotes trainable weights.

The second stage distills knowledge. The teacher is frozen, marked by a snowflake, while compact ECViT-T, T+, S and S+ backbones learn from a large collection of images without labels for this stage. The transferred representation is already specialized for detection.

The third stage trains the S, M, L and X families for their target tasks: object boxes for ECDet, instance masks for ECInsSeg and human keypoints for ECPose. Distillation does not remove the need for task annotations in final training.

Original EdgeCrafter training pipeline: teacher preparation, distillation and task-specific training.

Open research figure ↗

Results: detection, pose and segmentation

Using no task annotations beyond COCO, ECDet-S reaches 51.7 box AP with fewer than 10 million parameters. ECInsSeg-X reaches 48.4 mask AP and ECPose-X reaches 74.8 keypoint AP.

Additional Objects365 detection pretraining is more compute-intensive but improves every scale. The X variants reach 59.9 box AP, 49.8 mask AP and 75.9 keypoint AP. Each table cell lists COCO-only AP first and AP with Objects365 second.

The source latency tables use NVIDIA T4, batch size 1, FP16 and TensorRT 10.6. This is a specific measurement protocol, not a speed guarantee for every AI Box. Box, mask and keypoint AP evaluate different tasks and are not directly comparable.

Deepvizor note: three representations of one scene

Detection identifies an object class and its box. Segmentation delineates the object at pixel level. Pose estimation locates human keypoints. They support different rules, but none is an operational event on its own.

Industrial scenarios connect predictions with zones, observation duration and trigger conditions. An edge computer must also handle decoding, multiple cameras, event storage and its interface. Evaluate the complete scenario rather than model size alone.

Original publication and citation

Published in TMLR 2026. The full paper, repository and extended benchmark tables are available from the authors’ source links; their citation follows.

@article{liu2026edgecrafter,
  title={EdgeCrafter: Compact ViTs for Edge Dense Prediction via Task-Specialized Distillation},
  author={Liu, Longfei and Hou, Yongjie and Li, Yang and Wang, Qirui and Sha, Youyang and Yu, Yongjun and Wang, Yinzhi and Ru, Peizhe and Yu, Xuanlong and Shen, Xi},
  journal={Transactions on Machine Learning Research},
  year={2026}
}