Enabling AI Scientists to Recognize Innovation: A Domain-Agnostic Algorithm for Assessing Novelty

In the pursuit of Artificial General Intelligence (AGI), automating the generation and evaluation of novel research ideas is a key challenge in AI-driven scientific discovery. This paper presents Relative Neighbor Density (RND), a domain-agnostic algorithm for novelty assessment in research ideas that overcomes the limitations of existing approaches by comparing an idea's local density with its adjacent neighbors' densities. We first developed a scalable methodology to create test set without expert labeling, addressing a fundamental challenge in novelty assessment. Using these test sets, we demonstrate that our RND algorithm achieves state-of-the-art (SOTA) performance in computer science (AUROC=0.820) and biomedical research (AUROC=0.765) domains. Most significantly, while SOTA models like Sonnet-3.7 and existing metrics show domain-specific performance degradation, RND maintains consistent accuracies across domains by its domain-invariant property, outperforming all benchmarks by a substantial margin (0.795 v.s. 0.597) on cross-domain evaluation. These results validate RND as a generalizable solution for automated novelty assessment in scientific research.

@article{wang2025_2503.01508,
  title={ Enabling AI Scientists to Recognize Innovation: A Domain-Agnostic Algorithm for Assessing Novelty },
  author={ Yao Wang and Mingxuan Cui and Arthur Jiang },
  journal={arXiv preprint arXiv:2503.01508},
  year={ 2025 }
}