Benjin ZHU

selected publications

1. CVPR

   ConQueR: Query Contrast Voxel-DETR for 3D Object Detection

   Benjin Zhu, Zhe Wang, Shaoshuai Shi, Hang Xu, Lanqing Hong, and Hongsheng Li

   Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2023

   Abs Bib HTML Code

   Although DETR-based 3D detectors can simplify the detection pipeline and achieve direct sparse predictions, their performance still lags behind dense detectors with post-processing for 3D object detection from point clouds. DETRs usually adopt a larger number of queries than GTs (e.g., 300 queries v.s. 40 objects in Waymo) in a scene, which inevitably incur many false positives during inference. In this paper, we propose a simple yet effective sparse 3D detector, named Query Contrast Voxel-DETR (ConQueR), to eliminate the challenging false positives, and achieve more accurate and sparser predictions. We observe that most false positives are highly overlapping in local regions, caused by the lack of explicit supervision to discriminate locally similar queries. We thus propose a Query Contrast mechanism to explicitly enhance queries towards their best-matched GTs over all unmatched query predictions. This is achieved by the construction of positive and negative GT-query pairs for each GT, and a contrastive loss to enhance positive GT-query pairs against negative ones based on feature similarities. ConQueR closes the gap of sparse and dense 3D detectors, and reduces up to 60% false positives. Our single-frame ConQueR achieves new state-of-the-art (sota) 71.6 mAPH/L2 on the challenging Waymo Open Dataset validation set, outperforming previous sota methods (e.g., PV-RCNN++) by over 2.0 mAPH/L2.

   @article{zhu2023conquer,
     abbr = {CVPR},
     bibtex_show = {true},
     title = {ConQueR: Query Contrast Voxel-DETR for 3D Object Detection},
     author = {Zhu, Benjin and Wang, Zhe and Shi, Shaoshuai and Xu, Hang and Hong, Lanqing and Li, Hongsheng},
     journal = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
     year = {2023},
     html = {https://benjin.me/projects/2022_conquer/},
     code = {https://github.com/poodarchu/ConQueR},
     selected = {true}
   }

2. arXiv

   EqCo: Equivalent Rules for Self-supervised Contrastive Learning

   Benjin Zhu, Junqiang Huang, Zeming Li, Xiangyu Zhang, and Jian Sun

   arXiv preprint: 2010.01929 2020

   Abs Bib HTML Code

   In this paper, we propose a method, named EqCo (Equivalent Rules for Contrastive Learning), to make self-supervised learning irrelevant to the number of negative samples in InfoNCE-based contrastive learning frameworks. Inspired by the InfoMax principle, we point that the margin term in contrastive loss needs to be adaptively scaled according to the number of negative pairs in order to keep steady mutual information bound and gradient magnitude. EqCo bridges the performance gap among a wide range of negative sample sizes, so that we can use only a few negative pairs (e.g. 16 per query) to perform self-supervised contrastive training on large-scale vision datasets like ImageNet, while with almost no accuracy drop. This is quite a contrast to the widely used large batch training or memory bank mechanism in current practices. Equipped with EqCo, our simplified MoCo (SiMo) achieves comparable accuracy with MoCo v2 on ImageNet (linear evaluation protocol) while only involves 4 negative pairs per query instead of 65536, suggesting that large quantities of negative samples might not be a critical factor in InfoNCE loss.

   @article{zhu2020equal,
     abbr = {arXiv},
     bibtex_show = {true},
     title = {EqCo: Equivalent Rules for Self-supervised Contrastive Learning},
     author = {Zhu, Benjin and Huang, Junqiang and Li, Zeming and Zhang, Xiangyu and Sun, Jian},
     journal = {arXiv preprint: 2010.01929},
     year = {2020},
     html = {https://arxiv.org/abs/2010.01929},
     code = {https://github.com/poodarchu/SelfSup},
     selected = {true}
   }

3. arXiv

   AutoAssign: Differentiable Label Assignment for Dense Object Detection

   Benjin Zhu, Jianfeng Wang, Zhengkai Jiang, Fuhang Zong, Songtao Liu, Zeming Li, and Jian Sun

   arXiv preprint: 2007.03496 2020

   Abs Bib HTML Code

   Determining positive/negative samples for object detection is known as label assignment. Here we present an anchor-free detector named AutoAssign. It requires little human knowledge and achieves appearance-aware through a fully differentiable weighting mechanism. During training, to both satisfy the prior distribution of data and adapt to category characteristics, we present Center Weighting to adjust the category-specific prior distributions. To adapt to object appearances, Confidence Weighting is proposed to adjust the specific assign strategy of each instance. The two weighting modules are then combined to generate positive and negative weights to adjust each location’s confidence. Extensive experiments on the MS COCO show that our method steadily surpasses other best sampling strategies by large margins with various backbones. Moreover, our best model achieves 52.1% AP, outperforming all existing one-stage detectors. Besides, experiments on other datasets, e.g., PASCAL VOC, Objects365, and WiderFace, demonstrate the broad applicability of AutoAssign.

   @article{zhu2020auto,
     abbr = {arXiv},
     bibtex_show = {true},
     title = {AutoAssign: Differentiable Label Assignment for Dense Object Detection},
     author = {Zhu, Benjin and Wang, Jianfeng and Jiang, Zhengkai and Zong, Fuhang and Liu, Songtao and Li, Zeming and Sun, Jian},
     journal = {arXiv preprint: 2007.03496},
     year = {2020},
     html = {https://arxiv.org/abs/2007.03496},
     code = {https://github.com/Megvii-BaseDetection/AutoAssign},
     selected = {true}
   }