Recent trends in deep learning research, and their possible applications on deep data-driven computing

Format: Each day of the course, one professor will introduce a particular topic in deep learning and prepare a 30 minutes presentation on the topic. During the presentation, the host will ensure that it is highly interactive with the audience, who will ask questions and make comments during the presentation. Then, a short 10 minutes break will follow, during which members of the audience will prepare further questions to the professors in the presented topic and in relation to deep data-driven computing. Finally, a discussion between the audience and professors will follow in an attempt to address the questions made by the audience and to establish working topics and collaborations in advanced Deep Learning research topics for Deep Data-Driven Computing.

ABSTRACT:
The proposed course/workshop will focus on the discussion of novel deep learning techniques. For each technique, a brief presentation will be given followed up by a group discussion.
We propose to address different techniques such as (non-exhaustive):
● Domain adaptation [1]
● Geometric deep learning / graph neural networks [2], [3]
● Hybrid models [4]
● Normalizing flows [5]
● Distillation [6]
● Advanced implementation
● Forward and Inverse Problems using Deep Learning
● Data efficient methods

PREREQUISITES:
To be familiar with or have some basic knowledge about Deep Learning techniques and their implementation.


REFERENCES:
[1] M. Wang and W. Deng, “Deep visual domain adaptation: A survey,” Neurocomputing, vol. 312, pp. 135–153, Oct. 2018, doi: 10.1016/j.neucom.2018.05.083.
[2] M. M. Bronstein, J. Bruna, Y. LeCun, A. Szlam, and P. Vandergheynst, “Geometric deep learning: going beyond Euclidean data,” IEEE Signal Processing Magazine, vol. 34, no. 4, pp. 18–42, Jul. 2017, doi: 10.1109/MSP.2017.2693418.
[3] M. M. Bronstein, J. Bruna, T. Cohen, and P. Veličković, “Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges,” arXiv:2104.13478 [cs, stat], May 2021. Available: http://arxiv.org/abs/2104.13478
[4] V. Garcia Satorras, Z. Akata, and M. Welling, “Combining Generative and Discriminative Models for Hybrid Inference,” in Advances in Neural Information Processing Systems 32, 2019, pp. 13802–13812. Available: http://papers.nips.cc/paper/9532-combining-generative-and-discriminative-models-for-hybrid-inference.pdf
[5] I. Kobyzev, S. J. D. Prince, and M. A. Brubaker, “Normalizing Flows: An Introduction and Review of Current Methods,” IEEE Trans. Pattern Anal. Mach. Intell., pp. 1–1, 2020, doi: 10.1109/TPAMI.2020.2992934.
[6] Y. Tian, D. Krishnan, and P. Isola, “Contrastive Representation Distillation,” ICLR, 2020. Available: http://arxiv.org/abs/1910.10699

*Registration is free, but mandatory before 6 JUNE 2022. To sign-up go to https://forms.gle/kC8R9BafsLd6wq7N7 and fill the registration form.