An Exploration of Hierarchical Attention Transformers for Efficient Long Document Classification

11 October 2022

Ilias Chalkidis

Xiang Dai

Manos Fergadiotis

Prodromos Malakasiotis

Desmond Elliott

ArXiv (abs)PDF HTML Github (55★)

Papers citing "An Exploration of Hierarchical Attention Transformers for Efficient Long Document Classification"

25 / 25 papers shown

Title
HSGM: Hierarchical Segment-Graph Memory for Scalable Long-Text Semantics Dong Liu Yanxuan Yu VLM 80 2 0 17 Sep 2025
Two Heads Are Better than One: Simulating Large Transformers with Small Ones Hantao Yu Josh Alman 204 0 0 13 Jun 2025
The Strawberry Problem: Emergence of Character-level Understanding in Tokenized Language Models Adrian Cosma Stefan Ruseti Emilian Radoi Mihai Dascalu LRM 378 4 0 20 May 2025
Towards Long Context Hallucination DetectionNorth American Chapter of the Association for Computational Linguistics (NAACL), 2025 Siyi Liu Kishaloy Halder Zheng Qi Wei Xiao Nikolaos Pappas Phu Mon Htut Neha Anna John Yassine Benajiba Dan Roth HILM 252 10 0 28 Apr 2025
Graph-tree Fusion Model with Bidirectional Information Propagation for Long Document ClassificationConference on Empirical Methods in Natural Language Processing (EMNLP), 2024 Sudipta Singha Roy Xindi Wang Robert E. Mercer Frank Rudzicz 149 0 0 03 Oct 2024
InterACT: Inter-dependency Aware Action Chunking with Hierarchical Attention Transformers for Bimanual ManipulationConference on Robot Learning (CoRL), 2024 Andrew Lee Ian Chuang Ling-Yuan Chen Iman Soltani 333 21 0 12 Sep 2024
An alternative formulation of attention pooling function in translation Eddie Conti 112 0 0 23 Aug 2024
HDT: Hierarchical Document Transformer Haoyu He Markus Flicke Jan Buchmann Iryna Gurevych Andreas Geiger 200 3 0 11 Jul 2024
DAIC-WOZ: On the Validity of Using the Therapist's prompts in Automatic Depression Detection from Clinical Interviews Sergio Burdisso Ernesto Reyes-Ramírez Esaú Villatoro-Tello Fernando Sánchez-Vega Adrian Pastor Lopez-Monroy P. Motlícek 141 14 0 22 Apr 2024
Exploring Large Language Models and Hierarchical Frameworks for Classification of Large Unstructured Legal DocumentsEuropean Conference on Information Retrieval (ECIR), 2024 Nishchal Prasad M. Boughanem T. Dkaki AILaw ELM 145 10 0 11 Mar 2024
Modeling the Quality of Dialogical Explanations Milad Alshomary Felix Lange Meisam Booshehri Meghdut Sengupta Philipp Cimiano Henning Wachsmuth 160 3 0 01 Mar 2024
Towards Explainability and Fairness in Swiss Judgement Prediction: Benchmarking on a Multilingual Dataset Santosh T.Y.S.S Nina Baumgartner Matthias Sturmer Matthias Grabmair Joel Niklaus ELM AILaw 207 8 0 26 Feb 2024
A Picture is Worth More Than 77 Text Tokens: Evaluating CLIP-Style Models on Dense CaptionsComputer Vision and Pattern Recognition (CVPR), 2023 Jack Urbanek Florian Bordes Pietro Astolfi Mary Williamson Vasu Sharma Adriana Romero Soriano CLIP 3DV 320 82 0 14 Dec 2023
Recursion in Recursion: Two-Level Nested Recursion for Length Generalization with Scalability Jishnu Ray Chowdhury Cornelia Caragea 195 5 0 08 Nov 2023
VECHR: A Dataset for Explainable and Robust Classification of Vulnerability Type in the European Court of Human RightsConference on Empirical Methods in Natural Language Processing (EMNLP), 2023 Shanshan Xu Leon Staufer Santosh T.Y.S.S O. Ichim Corina Heri Matthias Grabmair 240 0 0 17 Oct 2023
An End-to-End System for Reproducibility Assessment of Source Code Repositories via Their Readmes Eyüp Kaan Akdeniz Selma Tekir Malik Nizar Asad Al Hinnawi SyDa 115 0 0 14 Oct 2023
Hallucination Reduction in Long Input Text Summarization Gregor Lenz Ronit Mandal Abhishek Agarwal Debarshi Kumar Sanyal HILM 188 10 0 28 Sep 2023
A Hierarchical Neural Framework for Classification and its Explanation in Large Unstructured Legal Documents Nishchal Prasad M. Boughanem Taoufik Dkaki ELM AILaw 182 1 0 19 Sep 2023
Large Language Model Prompt Chaining for Long Legal Document Classification Dietrich Trautmann ELM AILaw 140 18 0 08 Aug 2023
LogPrécis: Unleashing Language Models for Automated Malicious Log AnalysisComputers & security (Comput. Secur.), 2023 Matteo Boffa R. Valentim L. Vassio Danilo Giordano Idilio Drago Marco Mellia Zied Ben-Houidi 257 16 0 17 Jul 2023
Efficient Document Embeddings via Self-Contrastive Bregman Divergence LearningAnnual Meeting of the Association for Computational Linguistics (ACL), 2023 Daniel Saggau Mina Rezaei B. Bischl Ilias Chalkidis SSL MedIm 131 3 0 25 May 2023
A General-Purpose Multilingual Document Encoder Onur Galoglu Robert Litschko Goran Glavaš 171 2 0 11 May 2023
A Survey on Long Text Modeling with Transformers Zican Dong Tianyi Tang Lunyi Li Wayne Xin Zhao VLM 359 67 0 28 Feb 2023
Leveraging Task Dependency and Contrastive Learning for Case Outcome Classification on European Court of Human Rights CasesConference of the European Chapter of the Association for Computational Linguistics (EACL), 2023 Santosh T.Y.S.S Santosh T.Y.S.S Phillip Kemper Matthias Grabmair AILaw 278 18 0 01 Feb 2023
Zero-shot Transfer of Article-aware Legal Outcome Classification for European Court of Human Rights CasesFindings (Findings), 2023 Santosh T.Y.S.S O. Ichim Matthias Grabmair AILaw 450 19 0 01 Feb 2023