ResearchTrend.AI
  • Papers
  • Communities
  • Events
  • Blog
  • Pricing
Papers
Communities
Social Events
Terms and Conditions
Pricing
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2025 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 1611.06173
78
11
v1v2 (latest)

Empirical risk minimization and complexity of dynamical models

18 November 2016
K. Mcgoff
A. Nobel
ArXiv (abs)PDFHTML
Abstract

A dynamical model consists of a continuous self-map T:X→XT: \mathcal{X} \to \mathcal{X}T:X→X of a compact state space X\mathcal{X}X and a continuous observation function f:X→Rf: \mathcal{X} \to \mathbb{R}f:X→R. Dynamical models give rise to real-valued sequences when fff is applied to repeated iterates of TTT beginning at some initial state. This paper considers the fitting of a parametrized family of dynamical models to an observed real-valued stochastic process using empirical risk minimization. More precisely, at each finite time, one selects a model yielding values that minimize the average per-symbol loss with the observed process. The limiting behavior of the minimum risk parameters is studied in a general setting where the observed process need not be generated by a dynamical model in the family and minimal conditions are placed on the loss function. We establish a general convergence theorem for minimum risk estimators and ergodic observations, showing that the limiting parameter set is characterized by the projection of the observed process onto the set of processes associated with the dynamical family, where the projection is taken with respect to a joining-based distortion between stationary processes that is a generalization of the d‾\overline{d}d-distance. We then turn our attention to observations generated from an ergodic process plus an i.i.d. noise process and study conditions under which empirical risk minimization can effectively separate the signal from the noise. The key, necessary condition in the latter results is that the family of dynamical models has limited complexity, which is quantified through a notion of entropy for families of infinite sequences. Close connections between entropy and limiting average mean widths for stationary processes are established.

View on arXiv
Comments on this paper