Computational properties of delay-coupled system

Abstract

In this research work we study the computational properties of delay-coupled systems. In particular, we use a machine learning technique known as reservoir computing. In machine learning, a computer learns to solve di↵erent tasks using examples and without knowing explicitly their solution. For the study of the computational properties, a numerical toolbox, written in Python, was developed. This toolbox allows a fast implementation of the di↵erent scenarios described in this thesis. Using a reservoir computer, we studied several computational properties, focusing on its kernel quality, its ability to separate di↵erent input samples and the intrinsic memory capacity. This intrinsic memory is related to the delayedfeedback of the reservoir. We used a delay-coupled system as reservoir to study its computational ability in three di↵erent kinds of tasks: system’s modeling, time-series prediction and classification tasks. The system’s modeling task was performed using the Nonlinear Autoregressive Moving Average (of ten steps), NARMA10. The NARMA10 model creates autoregressive time series from a set of normally distributed random sequences. The reservoir computer learns how to emulate the system using only the sequence of random numbers and the autoregressive time series, but without knowing the equations of the NARMA10. The results of our approach are equivalent to those published by other authors and show the computational power of our method. For the time-series prediction tasks, we used three kinds of time series: a model that gives the variations in temperature of the sea surface that provoke El Niño phenomenon, the Lorenz system and the dynamics of a chaotic laser. Di↵erent scenarios were explored depending on the nature of the time series. For the prediction of the variation in temperature of the sea surface, we perform estimations of one, three and six months in advance. The error was measured as the Normalized Root Mean Square Error (NRMSE). For the di↵erent prediction horizons, we obtained errors of 2%, 8% and 24%, respectively. The classification tasks were carried out for a Spoken Digit Recognition (SDR) task and a real-world biomedical task. The SDR was used to illustrate different scenarios of a machine learning problem. The biomedical task consists on the automatic classification of heartbeats with cardiac arrhythmias. We use the MIT-BIH Arrhythmia database, a widely used database in cardiology. For comparison purposes, we followed the guidelines of the Association for the Advancement of Medical Instrumentation for the evaluation of arrhythmia-detector algorithms. We used a biostatistical learning process named logistic regression that allowed to compute the probability that a heartbeat belongs to a particular class. This is in contrast to the commonly used linear regression. The results obtained in this work show the versatility and efficiency of our implemented reservoir computer. Our results are equivalent and show improvement over other reported results on this problem under similar conditions and using the same database. To enhance the computational ability of our delay-coupled system, we included a multivariate scheme that allows the consideration of di↵erent variables of a system. We evaluated the influence of this multivariate scenario using a timeseries prediction and the classification of heartbeat tasks. The results show improvement in the performance of the reservoir computer in comparison with the same tasks in the univariate case.