The Edge of Orthogonality: A Simple View of What Makes BYOL Tick

The Edge of Orthogonality: A Simple View of What Makes BYOL Tick

9 February 2023

Pierre Harvey Richemond

Bilal Piot

Papers citing "The Edge of Orthogonality: A Simple View of What Makes BYOL Tick"

16 / 16 papers shown

Title
FreCT: Frequency-augmented Convolutional Transformer for Robust Time Series Anomaly Detection Wenxin Zhang Ding Xu Guangzhen Yao Xiaojian Lin Renxiang Guan Chengze Du Renda Han Xi Xuan Cuicui Luo AI4TS 42 0 0 02 May 2025
Representation Learning via Non-Contrastive Mutual Information Z. Guo Bernardo Avila-Pires Khimya Khetarpal Dale Schuurmans Bo Dai SSL 61 0 0 23 Apr 2025
The Hidden Pitfalls of the Cosine Similarity Loss Andrew Draganov Sharvaree P. Vadgama Erik J. Bekkers SSL 19 0 0 24 Jun 2024
On the Effectiveness of Supervision in Asymmetric Non-Contrastive Learning Jeongheon Oh Kibok Lee SSL 30 1 0 16 Jun 2024
$V_kD:$ Improving Knowledge Distillation using Orthogonal Projections Roy Miles Ismail Elezi Jiankang Deng 34 9 0 10 Mar 2024
DiffClone: Enhanced Behaviour Cloning in Robotics with Diffusion-Driven Policy Learning Sabariswaran Mani Sreyas Venkataraman Abhranil Chandra Adyan Rizvi Yash Sirvi Soumojit Bhattacharya Aritra Hazra OffRL 13 1 0 17 Jan 2024
Graph-level Representation Learning with Joint-Embedding Predictive Architectures Geri Skenderi Hang Li Jiliang Tang Marco Cristani AI4TS GNN 44 3 0 27 Sep 2023
How to Scale Your EMA Dan Busbridge Jason Ramapuram Pierre Ablin Tatiana Likhomanenko Eeshan Gunesh Dhekane Xavier Suau Russ Webb 17 17 0 25 Jul 2023
Towards a Unified Theoretical Understanding of Non-contrastive Learning via Rank Differential Mechanism Zhijian Zhuo Yifei Wang Jinwen Ma Yisen Wang 38 24 0 04 Mar 2023
Implicit variance regularization in non-contrastive SSL Manu Srinath Halvagal Axel Laborieux Friedemann Zenke 29 8 0 09 Dec 2022
Neural Eigenfunctions Are Structured Representation Learners Zhijie Deng Jiaxin Shi Hao Zhang Peng Cui Cewu Lu Jun Zhu 39 14 0 23 Oct 2022
Fast and accurate optimization on the orthogonal manifold without retraction Pierre Ablin Gabriel Peyré 40 26 0 15 Feb 2021
Understanding self-supervised Learning Dynamics without Contrastive Pairs Yuandong Tian Xinlei Chen Surya Ganguli SSL 132 278 0 12 Feb 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 293 3,683 0 11 Feb 2021
BYOL works even without batch statistics Pierre Harvey Richemond Jean-Bastien Grill Florent Altché Corentin Tallec Florian Strub ... Samuel L. Smith Soham De Razvan Pascanu Bilal Piot Michal Valko SSL 242 114 0 20 Oct 2020
Spectral Inference Networks: Unifying Deep and Spectral Learning David Pfau Stig Petersen Ashish Agarwal David Barrett Kimberly L. Stachenfeld 49 40 0 06 Jun 2018