Communities
Connect sessions
AI calendar
Organizations
Join Slack
Contact Sales
Search
Open menu
Home
Papers
2010.14713
Cited By
CompRess: Self-Supervised Learning by Compressing Representations
Neural Information Processing Systems (NeurIPS), 2020
28 October 2020
Soroush Abbasi Koohpayegani
Ajinkya Tejankar
Hamed Pirsiavash
SSL
Re-assign community
ArXiv (abs)
PDF
HTML
Github (78★)
Papers citing
"CompRess: Self-Supervised Learning by Compressing Representations"
50 / 62 papers shown
Learning Task-Agnostic Representations through Multi-Teacher Distillation
Philippe Formont
Maxime Darrin
Banafsheh Karimian
Jackie Chi Kit Cheung
Eric Granger
Ismail Ben Ayed
Mohammadhadi Shateri
Pablo Piantanida
220
2
0
21 Oct 2025
Backdooring Self-Supervised Contrastive Learning by Noisy Alignment
Tuo Chen
Jie Gui
Minjing Dong
Ju Jia
Lanting Fang
Jian Liu
AAML
160
3
0
19 Aug 2025
PCoreSet: Effective Active Learning through Knowledge Distillation from Vision-Language Models
Seongjae Kang
Dong Bok Lee
Hyungjoon Jang
Dongseop Kim
Sung Ju Hwang
VLM
459
0
0
01 Jun 2025
Simple yet Effective Semi-supervised Knowledge Distillation from Vision-Language Models via Dual-Head Optimization
Seongjae Kang
Dong Bok Lee
Hyungjoon Jang
Sung Ju Hwang
VLM
514
1
0
12 May 2025
Dataset Distillation via Knowledge Distillation: Towards Efficient Self-Supervised Pre-Training of Deep Networks
International Conference on Learning Representations (ICLR), 2024
S. Joshi
Jiayi Ni
Baharan Mirzasoleiman
DD
765
4
0
03 Oct 2024
Simple Unsupervised Knowledge Distillation With Space Similarity
European Conference on Computer Vision (ECCV), 2024
Aditya Singh
Haohan Wang
402
6
0
20 Sep 2024
Federated Graph Semantic and Structural Learning
Wenke Huang
Guancheng Wan
Mang Ye
Bo Du
FedML
462
84
0
27 Jun 2024
Lightweight Model Pre-training via Language Guided Knowledge Distillation
Mingsheng Li
Lin Zhang
Mingzhen Zhu
Zilong Huang
Gang Yu
Jiayuan Fan
Tao Chen
271
4
0
17 Jun 2024
To Distill or Not to Distill? On the Robustness of Robust Knowledge Distillation
Annual Meeting of the Association for Computational Linguistics (ACL), 2024
Abdul Waheed
Karima Kadaoui
Muhammad Abdul-Mageed
VLM
258
6
0
06 Jun 2024
Relational Self-supervised Distillation with Compact Descriptors for Image Copy Detection
Juntae Kim
Sungwon Woo
Jongho Nang
608
1
0
28 May 2024
Retro: Reusing teacher projection head for efficient embedding distillation on Lightweight Models via Self-supervised Learning
Khanh-Binh Nguyen
Chae Jung Park
196
0
0
24 May 2024
An Experimental Study on Exploring Strong Lightweight Vision Transformers via Masked Image Modeling Pre-Training
Jin Gao
Shubo Lin
Shaoru Wang
Yutong Kou
Zeming Li
Liang Li
Congxuan Zhang
Xiaoqin Zhang
Yizheng Wang
Weiming Hu
355
9
0
18 Apr 2024
Visually Grounded Speech Models have a Mutual Exclusivity Bias
Leanne Nortje
Dan Oneaţă
Yevgen Matusevych
Herman Kamper
SSL
331
2
0
20 Mar 2024
On Good Practices for Task-Specific Distillation of Large Pretrained Visual Models
Juliette Marrie
Michael Arbel
Julien Mairal
Diane Larlus
VLM
MQ
340
2
0
17 Feb 2024
Deep Clustering with Diffused Sampling and Hardness-aware Self-distillation
Hai-Xin Zhang
Dong Huang
477
2
0
25 Jan 2024
I
2
^2
2
MD: 3D Action Representation Learning with Inter- and Intra-modal Mutual Distillation
Yunyao Mao
Jiajun Deng
Wen-gang Zhou
Zhenbo Lu
Wanli Ouyang
Houqiang Li
VLM
295
0
0
24 Oct 2023
A Machine Learning-oriented Survey on Tiny Machine Learning
IEEE Access (IEEE Access), 2023
Luigi Capogrosso
Federico Cunico
D. Cheng
Franco Fummi
Marco Cristani
SyDa
MU
449
93
0
21 Sep 2023
COMEDIAN: Self-Supervised Learning and Knowledge Distillation for Action Spotting using Transformers
J. Denize
Mykola Liashuha
Jaonary Rabarisoa
Astrid Orcesi
Romain Hérault
ViT
400
21
0
03 Sep 2023
Embrace Limited and Imperfect Training Datasets: Opportunities and Challenges in Plant Disease Recognition Using Deep Learning
Frontiers in Plant Science (Front. Plant Sci.), 2023
Mingle Xu
H. Kim
Jucheng Yang
A. Fuentes
Yao Meng
Sook Yoon
Taehyun Kim
D. Park
276
39
0
19 May 2023
Multi-Mode Online Knowledge Distillation for Self-Supervised Visual Representation Learning
Computer Vision and Pattern Recognition (CVPR), 2023
Kaiyou Song
Jin Xie
Shanyi Zhang
Zimeng Luo
426
37
0
13 Apr 2023
Defending Against Patch-based Backdoor Attacks on Self-Supervised Learning
Computer Vision and Pattern Recognition (CVPR), 2023
Ajinkya Tejankar
Maziar Sanjabi
Qifan Wang
Sinong Wang
Hamed Firooz
Hamed Pirsiavash
L Tan
AAML
247
33
0
04 Apr 2023
Learning Common Rationale to Improve Self-Supervised Representation for Fine-Grained Visual Recognition Problems
Computer Vision and Pattern Recognition (CVPR), 2023
Yangyang Shu
Anton Van Den Hengel
Lingqiao Liu
SSL
253
28
0
03 Mar 2023
A Simple Recipe for Competitive Low-compute Self supervised Vision Models
Quentin Duval
Ishan Misra
Nicolas Ballas
300
11
0
23 Jan 2023
Unifying Synergies between Self-supervised Learning and Dynamic Computation
British Machine Vision Conference (BMVC), 2023
Tarun Krishna
Ayush K. Rai
Alexandru Drimbarean
Eric Arazo
Paul Albert
Alan F. Smeaton
Kevin McGuinness
Noel E. O'Connor
493
0
0
22 Jan 2023
Similarity Contrastive Estimation for Image and Video Soft Contrastive Self-Supervised Learning
Machine Vision and Applications (MVA), 2022
J. Denize
Jaonary Rabarisoa
Astrid Orcesi
Romain Hérault
SSL
297
6
0
21 Dec 2022
Distantly-Supervised Named Entity Recognition with Adaptive Teacher Learning and Fine-grained Student Ensemble
AAAI Conference on Artificial Intelligence (AAAI), 2022
Xiaoye Qu
Jun Zeng
Daizong Liu
Zhefeng Wang
Baoxing Huai
Pan Zhou
235
28
0
13 Dec 2022
ESTAS: Effective and Stable Trojan Attacks in Self-supervised Encoders with One Target Unlabelled Sample
Jiaqi Xue
Qiang Lou
AAML
224
11
0
20 Nov 2022
Pixel-Wise Contrastive Distillation
IEEE International Conference on Computer Vision (ICCV), 2022
Junqiang Huang
Zichao Guo
532
6
0
01 Nov 2022
Effective Self-supervised Pre-training on Low-compute Networks without Distillation
International Conference on Learning Representations (ICLR), 2022
Fuwen Tan
F. Saleh
Brais Martínez
292
4
0
06 Oct 2022
Attention Distillation: self-supervised vision transformer students need more guidance
British Machine Vision Conference (BMVC), 2022
Kai Wang
Fei Yang
Joost van de Weijer
ViT
182
22
0
03 Oct 2022
Slimmable Networks for Contrastive Self-supervised Learning
International Journal of Computer Vision (IJCV), 2022
Shuai Zhao
Xiaohan Wang
Linchao Zhu
Yi Yang
251
7
0
30 Sep 2022
A Pathologist-Informed Workflow for Classification of Prostate Glands in Histopathology
Alessandro Ferrero
Beatrice Knudsen
Deepika Sirohi
Ross T. Whitaker
276
0
0
27 Sep 2022
MimCo: Masked Image Modeling Pre-training with Contrastive Teacher
ACM Multimedia (ACM MM), 2022
Qiang-feng Zhou
Chaohui Yu
Haowen Luo
Zhibin Wang
Hao Li
VLM
389
29
0
07 Sep 2022
CMD: Self-supervised 3D Action Representation Learning with Cross-modal Mutual Distillation
European Conference on Computer Vision (ECCV), 2022
Yunyao Mao
Wen-gang Zhou
Zhenbo Lu
Jiajun Deng
Houqiang Li
434
73
0
26 Aug 2022
SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery
Neural Information Processing Systems (NeurIPS), 2022
Yezhen Cong
Samarth Khanna
Chenlin Meng
Patrick Liu
Erik Rozi
Yutong He
Marshall Burke
David B. Lobell
Stefano Ermon
ViT
565
470
0
17 Jul 2022
DSPNet: Towards Slimmable Pretrained Networks based on Discriminative Self-supervised Learning
Shaoru Wang
Zeming Li
Jin Gao
Liang Li
Weiming Hu
197
1
0
13 Jul 2022
Revisiting Label Smoothing and Knowledge Distillation Compatibility: What was Missing?
International Conference on Machine Learning (ICML), 2022
Keshigeyan Chandrasegaran
Ngoc-Trung Tran
Yunqing Zhao
Ngai-Man Cheung
315
52
0
29 Jun 2022
SimA: Simple Softmax-free Attention for Vision Transformers
IEEE Workshop/Winter Conference on Applications of Computer Vision (WACV), 2022
Soroush Abbasi Koohpayegani
Hamed Pirsiavash
350
43
0
17 Jun 2022
A Closer Look at Self-Supervised Lightweight Vision Transformers
International Conference on Machine Learning (ICML), 2022
Shaoru Wang
Jin Gao
Zeming Li
Jian Sun
Weiming Hu
ViT
317
51
0
28 May 2022
Generalized Knowledge Distillation via Relationship Matching
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2022
Han-Jia Ye
Su Lu
De-Chuan Zhan
FedML
201
27
0
04 May 2022
Online Continual Learning for Embedded Devices
Tyler L. Hayes
Christopher Kanan
CLL
409
68
0
21 Mar 2022
Weak Augmentation Guided Relational Self-Supervised Learning
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2022
Mingkai Zheng
Shan You
Fei Wang
Chao Qian
Changshui Zhang
Xiaogang Wang
Chang Xu
365
6
0
16 Mar 2022
SimReg: Regression as a Simple Yet Effective Tool for Self-supervised Knowledge Distillation
K. Navaneet
Soroush Abbasi Koohpayegani
Ajinkya Tejankar
Hamed Pirsiavash
177
22
0
13 Jan 2022
Constrained Mean Shift Using Distant Yet Related Neighbors for Representation Learning
K. Navaneet
Soroush Abbasi Koohpayegani
Ajinkya Tejankar
Kossar Pourahmadi
Akshayvarun Subramanya
Hamed Pirsiavash
SSL
313
8
0
08 Dec 2021
Boosting Contrastive Learning with Relation Knowledge Distillation
Kai Zheng
Yuanjiang Wang
Ye Yuan
SSL
128
14
0
08 Dec 2021
Auxiliary Learning for Self-Supervised Video Representation via Similarity-based Knowledge Distillation
Amirhossein Dadashzadeh
Alan Whone
Majid Mirmehdi
SSL
417
5
0
07 Dec 2021
A Simple Baseline for Low-Budget Active Learning
Kossar Pourahmadi
Parsa Nooralinejad
Hamed Pirsiavash
388
21
0
22 Oct 2021
Constrained Mean Shift for Representation Learning
Ajinkya Tejankar
Soroush Abbasi Koohpayegani
Hamed Pirsiavash
SSL
200
0
0
19 Oct 2021
Unsupervised Representation Learning for Binary Networks by Joint Classifier Learning
Dahyun Kim
Jonghyun Choi
SSL
MQ
423
5
0
17 Oct 2021
Towards Communication-Efficient and Privacy-Preserving Federated Representation Learning
Haizhou Shi
Youcai Zhang
Zijin Shen
Siliang Tang
Yaqian Li
Yandong Guo
Yueting Zhuang
162
7
0
29 Sep 2021
1
2
Next
Page 1 of 2
