BudgetLongformer: Can we Cheaply Pretrain a SotA Legal Language Model From Scratch?

30 November 2022

Papers citing "BudgetLongformer: Can we Cheaply Pretrain a SotA Legal Language Model From Scratch?"

8 / 8 papers shown

Title
SaulLM-7B: A pioneering Large Language Model for Law Pierre Colombo T. Pires Malik Boudiaf Dominic Culver Rui Melo ... Andre F. T. Martins Fabrizio Esposito Vera Lúcia Raposo Sofia Morgado Michael Desa ELM AILaw 39 63 0 06 Mar 2024
Clinical-Longformer and Clinical-BigBird: Transformers for long clinical sequences Yikuan Li R. M. Wehbe F. Ahmad Hanyin Wang Yuan Luo VLM MedIm 135 82 0 27 Jan 2022
LexGLUE: A Benchmark Dataset for Legal Language Understanding in English Ilias Chalkidis Abhik Jana D. Hartung M. Bommarito Ion Androutsopoulos Daniel Martin Katz Nikolaos Aletras AILaw ELM 123 244 0 03 Oct 2021
Carbon Emissions and Large Neural Network Training David A. Patterson Joseph E. Gonzalez Quoc V. Le Chen Liang Lluís-Miquel Munguía D. Rothchild David R. So Maud Texier J. Dean AI4CE 239 626 0 21 Apr 2021
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 249 1,982 0 28 Jul 2020
Efficient Content-Based Sparse Attention with Routing Transformers Aurko Roy M. Saffar Ashish Vaswani David Grangier MoE 228 578 0 12 Mar 2020
Neural Legal Judgment Prediction in English Ilias Chalkidis Ion Androutsopoulos Nikolaos Aletras AILaw ELM 101 325 0 05 Jun 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,927 0 20 Apr 2018