CompRess: Self-Supervised Learning by Compressing Representations

28 October 2020

Soroush Abbasi Koohpayegani

Papers citing "CompRess: Self-Supervised Learning by Compressing Representations"

28 / 28 papers shown

Title
$Simple Semi-supervised Knowledge Distillation from Vision-Language Models via $\mathbf{\texttt{D}}$ual-$\mathbf{\texttt{H}}$ead $\mathbf{\texttt{O}}$ptimization$ Simple Semi-supervised Knowledge Distillation from Vision-Language Models via $\mathbf{\texttt{D}}$ ual- $\mathbf{\texttt{H}}$ ead $\mathbf{\texttt{O}}$ ptimization Seongjae Kang Dong Bok Lee Hyungjoon Jang Sung Ju Hwang VLM 57 0 0 12 May 2025
Dataset Distillation via Knowledge Distillation: Towards Efficient Self-Supervised Pre-Training of Deep Networks S. Joshi Jiayi Ni Baharan Mirzasoleiman DD 69 2 0 03 Oct 2024
Federated Graph Semantic and Structural Learning Wenke Huang Guancheng Wan Mang Ye Bo Du FedML 34 42 0 27 Jun 2024
Relational Self-supervised Distillation with Compact Descriptors for Image Copy Detection Juntae Kim Sungwon Woo Jongho Nang 42 1 0 28 May 2024
Retro: Reusing teacher projection head for efficient embedding distillation on Lightweight Models via Self-supervised Learning Khanh-Binh Nguyen Chae Jung Park 34 0 0 24 May 2024
Defending Against Patch-based Backdoor Attacks on Self-Supervised Learning Ajinkya Tejankar Maziar Sanjabi Qifan Wang Sinong Wang Hamed Firooz Hamed Pirsiavash L Tan AAML 30 19 0 04 Apr 2023
Learning Common Rationale to Improve Self-Supervised Representation for Fine-Grained Visual Recognition Problems Yangyang Shu A. Hengel Lingqiao Liu SSL 30 14 0 03 Mar 2023
Similarity Contrastive Estimation for Image and Video Soft Contrastive Self-Supervised Learning J. Denize Jaonary Rabarisoa Astrid Orcesi Romain Hérault SSL 14 6 0 21 Dec 2022
Distantly-Supervised Named Entity Recognition with Adaptive Teacher Learning and Fine-grained Student Ensemble Xiaoye Qu Jun Zeng Daizong Liu Zhefeng Wang Baoxing Huai Pan Zhou 24 22 0 13 Dec 2022
Pixel-Wise Contrastive Distillation Junqiang Huang Zichao Guo 37 4 0 01 Nov 2022
Effective Self-supervised Pre-training on Low-compute Networks without Distillation Fuwen Tan F. Saleh Brais Martínez 27 4 0 06 Oct 2022
Slimmable Networks for Contrastive Self-supervised Learning Shuai Zhao Xiaohan Wang Linchao Zhu Yi Yang 22 1 0 30 Sep 2022
A Pathologist-Informed Workflow for Classification of Prostate Glands in Histopathology Alessandro Ferrero Beatrice S. Knudsen Deepika Sirohi Ross T. Whitaker 19 0 0 27 Sep 2022
CMD: Self-supervised 3D Action Representation Learning with Cross-modal Mutual Distillation Yunyao Mao Wen-gang Zhou Zhenbo Lu Jiajun Deng Houqiang Li 28 38 0 26 Aug 2022
Revisiting Label Smoothing and Knowledge Distillation Compatibility: What was Missing? Keshigeyan Chandrasegaran Ngoc-Trung Tran Yunqing Zhao Ngai-man Cheung 83 41 0 29 Jun 2022
SimA: Simple Softmax-free Attention for Vision Transformers Soroush Abbasi Koohpayegani Hamed Pirsiavash 16 25 0 17 Jun 2022
A Closer Look at Self-Supervised Lightweight Vision Transformers Shaoru Wang Jin Gao Zeming Li Jian-jun Sun Weiming Hu ViT 67 41 0 28 May 2022
Generalized Knowledge Distillation via Relationship Matching Han-Jia Ye Su Lu De-Chuan Zhan FedML 22 20 0 04 May 2022
SimReg: Regression as a Simple Yet Effective Tool for Self-supervised Knowledge Distillation K. Navaneet Soroush Abbasi Koohpayegani Ajinkya Tejankar Hamed Pirsiavash 13 19 0 13 Jan 2022
On the Efficacy of Small Self-Supervised Contrastive Models without Distillation Signals Haizhou Shi Youcai Zhang Siliang Tang Wenjie Zhu Yaqian Li Yandong Guo Yueting Zhuang SyDa 21 14 0 30 Jul 2021
Simple Distillation Baselines for Improving Small Self-supervised Models Jindong Gu Wei Liu Yonglong Tian 19 8 0 21 Jun 2021
Backdoor Attacks on Self-Supervised Learning Aniruddha Saha Ajinkya Tejankar Soroush Abbasi Koohpayegani Hamed Pirsiavash SSL AAML 27 100 0 21 May 2021
DisCo: Remedy Self-supervised Learning on Lightweight Models with Distilled Contrastive Learning Yuting Gao Jia-Xin Zhuang Xiaowei Guo Hao Cheng Xing Sun Ke Li Feiyue Huang 33 40 0 19 Apr 2021
Concept Generalization in Visual Representation Learning Mert Bulent Sariyildiz Yannis Kalantidis Diane Larlus Alahari Karteek SSL 23 50 0 10 Dec 2020
Improved Baselines with Momentum Contrastive Learning Xinlei Chen Haoqi Fan Ross B. Girshick Kaiming He SSL 267 3,369 0 09 Mar 2020
Self-Supervised Feature Learning by Learning to Spot Artifacts Simon Jenni Paolo Favaro SSL 147 127 0 13 Jun 2018
Boosting Self-Supervised Learning via Knowledge Transfer M. Noroozi Ananth Vinjimoor Paolo Favaro Hamed Pirsiavash SSL 209 292 0 01 May 2018
Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results Antti Tarvainen Harri Valpola OOD MoMe 261 1,275 0 06 Mar 2017