To Cool or not to Cool? Temperature Network Meets Large Foundation Models via DRO

6 April 2024

Papers citing "To Cool or not to Cool? Temperature Network Meets Large Foundation Models via DRO"

8 / 8 papers shown

Title
Understanding Contrastive Learning via Distributionally Robust Optimization Junkang Wu Jiawei Chen Jiancan Wu Wentao Shi Xiang Wang Xiangnan He 24 23 0 17 Oct 2023
Temperature Schedules for Self-Supervised Contrastive Methods on Long-Tail Data Anna Kukleva Moritz Bohle Bernt Schiele Hilde Kuehne Christian Rupprecht 29 38 0 23 Mar 2023
Stochastic Constrained DRO with a Complexity Independent of Sample Size Q. Qi Jiameng Lyu Kung-Sik Chan E. Bai Tianbao Yang 48 14 0 11 Oct 2022
CyCLIP: Cyclic Contrastive Language-Image Pretraining Shashank Goel Hritik Bansal S. Bhatia Ryan A. Rossi Vishwa Vinay Aditya Grover CLIP VLM 160 131 0 28 May 2022
A Systematic Evaluation of Large Language Models of Code Frank F. Xu Uri Alon Graham Neubig Vincent J. Hellendoorn ELM ALM 188 624 0 26 Feb 2022
Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts Soravit Changpinyo P. Sharma Nan Ding Radu Soricut VLM 273 845 0 17 Feb 2021
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 236 1,508 0 31 Dec 2020
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 243 1,791 0 17 Sep 2019