Transformers with Learnable Activation Functions

30 August 2022

Papers citing "Transformers with Learnable Activation Functions"

5 / 5 papers shown

Title
Learnable polynomial, trigonometric, and tropical activations Ismail Khalfaoui-Hassani Stefan Kesselheim 61 0 0 03 Feb 2025
Primer: Searching for Efficient Transformers for Language Modeling David R. So Wojciech Mañke Hanxiao Liu Zihang Dai Noam M. Shazeer Quoc V. Le VLM 85 152 0 17 Sep 2021
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 245 1,817 0 17 Sep 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,956 0 20 Apr 2018
Neural Architecture Search with Reinforcement Learning Barret Zoph Quoc V. Le 264 5,326 0 05 Nov 2016