Position Prediction as an Effective Pretraining Strategy

15 July 2022

Papers citing "Position Prediction as an Effective Pretraining Strategy"

8 / 8 papers shown

Title
Energy-Inspired Self-Supervised Pretraining for Vision Models Ze Wang Jiang Wang Zicheng Liu Qiang Qiu 21 8 0 02 Feb 2023
Soft-Landing Strategy for Alleviating the Task Discrepancy Problem in Temporal Action Localization Tasks Hyolim Kang Hanjung Kim Joungbin An Minsu Cho Seon Joo Kim 25 5 0 11 Nov 2022
Representation Learning by Detecting Incorrect Location Embeddings Sepehr Sameni Simon Jenni Paolo Favaro ViT 26 4 0 10 Apr 2022
Relative Position Prediction as Pre-training for Text Encoders Rickard Brüel-Gabrielsson Chris Scarvelis 11 1 0 02 Feb 2022
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 302 7,434 0 11 Nov 2021
Intriguing Properties of Vision Transformers Muzammal Naseer Kanchana Ranasinghe Salman Khan Munawar Hayat F. Khan Ming-Hsuan Yang ViT 251 620 0 21 May 2021
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 303 5,773 0 29 Apr 2021
Aggregated Residual Transformations for Deep Neural Networks Saining Xie Ross B. Girshick Piotr Dollár Z. Tu Kaiming He 291 10,216 0 16 Nov 2016