Do we really have to filter out random noise in pre-training data for language models?

Re-assign community