This course introduces the mathematical tool of reproducing kernel Hilbert spaces (RKHSs) and describes multiple applications of such a tool in machine learning. In particular, we first use simple linear algebra to describe finite dimensional RKHSs, then we introduce the general construction and provide illustrative examples. The second part of the course describes multiple applications of RKHSs in machine learning including kernel-based supervised learning and kernel mean embedings. For researchers/students in machine learning, the course can make more accesible RKHSs as an important tool for machine learning that has been extensively used in the last decades. For researchers/students in other mathematical fields, the course can serve to introduce/recall a highly-relevant mathematical object and as a first approximation to the field of machine learning from a familiar standpoint.

PROGRAMME:
1.Reproducing kernel Hilbert spaces (RKHSs)
1.a Motivation: supervised learning with linear rules in feature space
1.b Finite dimensional RKHSs. General RKHSs and examples

2. RKHSs for machine learning
2.a. Supervised learning with RKHSs: uniform concentration, representer theorem, and universal consistency
2.b. Approximation based on random features
2.c. Kernel mean embedding and maximum mean discrepancy

REFERENCES:
-Vladimir Vapnik, Statistical Learning Theory. Springer-Verlag, New York. 2000
-Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar. Foundations of machine learning. MIT press, Second edition, 2018.
-Jonathan H. Manton and Pierre-Olivier Amblard. "A primer on reproducing kernel Hilbert spaces." Foundations and Trends in Signal Processing, 2015.
-Charles A. Micchelli, Yuesheng Xu, and Haizhang Zhang. "Universal Kernels." Journal of Machine Learning Research, 2006.
-Krikamol Muandet, Kenji Fukumizu, Bharath Sriperumbudur, and Bernhard Schölkopf, "Kernel Mean Embedding of Distributions: A Review and Beyond," Foundations and Trends in Machine Learning, 2017.
-Alain Berlinet and Christine Thomas-Agnan. Reproducing kernel Hilbert spaces in probability and statistics. Springer Science & Business Media, 2011.