This book originated from a forefront R&D project pursued at Siemens Corporate Technology over the past several years. As a name for this project, we chose "Information Dynamics", which stands for information processing in complex dynamical systems. In the project, we wanted to grasp the flow of information in such systems in a quantitative manner, on the one hand by making use of an existing arsenal of methods and techniques from areas such as information theory, mathematical statistics, neural networks, nonlinear dynamics, probability theory, and statistical physics, and on the other hand by deriving new methods and techniques if required. The book contains only those contributions to the above-mentioned project which lend themselves to a unifying theoretical framework. Other important results obtained in the project, such as the extension of transport-theoretic techniques and their application to optimizing traffic flow, or the design of new neural network architectures for treating systems at the edge of chaos with applications in economics, are left out. This certainly is a sacrifice, but we think it is of benefit to the reader that we tried to be as focused and self contained as possible.
This book originated from a forefront R&D project pursued at Siemens Corporate Technology over the past several years. As a name for this project, we chose "Information Dynamics", which stands for information processing in complex dynamical systems. In the project, we wanted to grasp the flow of information in such systems in a quantitative manner, on the one hand by making use of an existing arsenal of methods and techniques from areas such as information theory, mathematical statistics, neural networks, nonlinear dynamics, probability theory, and statistical physics, and on the other hand by deriving new methods and techniques if required. The book contains only those contributions to the above-mentioned project which lend themselves to a unifying theoretical framework. Other important results obtained in the project, such as the extension of transport-theoretic techniques and their application to optimizing traffic flow, or the design of new neural network architectures for treating systems at the edge of chaos with applications in economics, are left out. This certainly is a sacrifice, but we think it is of benefit to the reader that we tried to be as focused and self contained as possible.
Zusammenfassung
This new book offers a new, theoretical approach to information dynamics, i.e., information processing in complex dynamical systems. The presentation establishes a consistent theoretical framework for the problem of discovering knowledge behind empirical, dynamical data. The book is an essential book for advanced graduates, researchers, and professionals in neural computing, information theory, nonlinear dynamics and complex systems modeling.
Inhaltsverzeichnis
l Introduction.- 2 Dynamical Systems: An Overview 7.- 2.1 Deterministic Dynamical Systems.- 2.3 Statistical Time-Series Analysis.- 3 Statistical Structure Extraction in Dynamical Systems: Parametric Formulation.- 3.1 Basic Concepts of Information Theory.- 3.2 Parametric Estimation : Maximum-Likelihood Principle.- 3.3 Linear Models.- 3.4 Nonlinear Models.- 3.5 Density Estimation.- 3.6 Information-Theoretic Approach to Time-Series Modeling: Redundancy Extraction.- 4 Applications: Parametric Characterization of Time Series.- 4.1 Feedforward Learning : Chaotic Dynamics.- 4.2 Recurrent Learning : Chaotic Dynamics.- 4.3 Dynamical Overtraining and Lyapunov Penalty Term.- 4.4 Feedforward and Recurrent Learning of Biomedical Data.- 4.5 Unsupervised Redundancy-Extraction-Based Modeling: Chaotic Dynamics.- 4.6 Unsupervised Redundancy Extraction Modeling: Biomedical Data.- 5 Statistical Structure Extraction in Dynamical Systems: Nonparametric Formulation.- 5.1 Nonparametric Detection ofStatistical Dependencies in Time Series.- 5.2 Nonparametric Characterization of Dynamics: The Information Flow Concept.- 5.3 Information Flow and Coarse Graining.- 6 Applications: Nonparametric Characterization of Time Series.- 6.1 Detecting Nonlinear Correlations in Time Series.- 6.2 Nonparametric Analysis of Time Series : Optimal Delay Selection.- 6.3 Determining the Information Flow ofDynamical Systems from Continuous Probability Distributions.- 6.4 Dynamical Characterization ofTime Signals: The Integrated Information Flow.- 6.5 Information Flow and Coarse Graining: Numerical Experiments.- 7 Statistical Structure Extraction in Dynamical Systems: Semiparametric Formulation.- 7.1 Markovian Characterization of Univariate Time Series.- 7.2 Markovian Characterization of Multivariate Time Series.- 8 Applications: Semiparametric Characterization of Time Series.- 8.1 Univariate Time Series : Artificial Data.- 8.2 Univariate Time Series: Real-World Data.- 8.3 Multivariate Time Series: Artificial Data.- 8.4 Multivariate Time Series : Tumor Detection in EEG Time Series.- 9 Information Processing and Coding in Spatiotemporal Dynamical Systems: Spiking Networks.- 9.1 Spiking Neurons.- 9.2 Information Processing and Coding in Single Spiking Neurons.- 9.3 Information Processing and Coding in Networks of Spiking Neurons.- 9.4 The Processing and Coding ofDynamical Systems.- 10 Applications: Information Processing and Coding in Spatiotemporal Dynamical Systems.- 10.1 The Binding Problem.- 10.2 Discrimination of Stimulus by Spiking Neural Networks.- 10.3 Numerical Experiments.- Epilogue.- Appendix A Chain Rules, Inequalities and Other Useful Theorems in Information Theory.- A.1 Chain Rules.- A.2 Fundamental Inequalities ofInformation Theory.- Appendix B Univariate and Multivariate Cumulants.- Appendix C Information Flow of Chaotic Systems: Thermodynamical Formulation.- Appendix D Generalized Discriminability by the Spike Response Model ofa Single Spiking Neuron: Analytical Results.- References.
