My Recurrent Neighbour is Now PyTorched

My Recurrent Neighbour is Now PyTorched: a Long story Shortened in Time with deep Messages

By: Santi Pascual, PhD Researcher at TALP

PyTorch is a Python Deep Learning framework, where we can mainly build Artificial Neural Networks (ANNs or NNs), similarly to what we achieve with TensorFlow, Keras, Theano, Caffe2, and many others. There is, however, a hidden beauty behind PyTorch, and a nice level of simplicity to achieve our Deep Learning goals quickly and effectively.

This talk will cover the basic PyTorch structures to build a neural network, BUT going straight into the deeply recursive world of Recurrent Neural Networks. Thus we will play with an RNN in a way as interpretable as possible (thus understanding what's going on inside), and delve into its evolved Long Short Term Memory (LSTM) version with some toy example. The idea is to demonstrate how fun and easy and every-good-adjective it is to build the so called dynamic graphs (like RNNs) in PyTorch.

FAQ

(For other questions, contact santi.pascual@upc.edu)

1) What prior knowledge do I need for this tutorial/meetup/handson?
- I know what is a Neural Network (at least the typical MLP structure with sigmoid/tanh/relu units), and I absolutely know what a Loss function is, what does it mean to train a Machine Learning model, etc.. (Machine Learning basics in the end of the day, as we will move on to RNNs quickly).

2) Why RNNs?
- Because they are (1) challenging, (2) a key component in many nowadays Deep Learning applications, and (3) PyTorch is a very interesting framework to implement these dynamic models (e.g. models dealing with varying length data).

3) Do I have to bring my laptop with any installation?
- Yes, you should definetly bring a laptop if you want to actually learn about this (as it is a coding hands-on session). Bring the following requirements installed:

* Python 3.5+ (you can work with 2.7, but you'd be able to solve any syntax mismatch b/w both Python versions if they appear, something I don't expect to happen often)
* Latest NumPy
* Latest PyTorch for your Python version (install whatever fits your distro from http://pytorch.org/ )

-Tip: easiest thing to do is downloading an Anaconda package (https://www.anaconda.com/download/) that works for your system, and install PyTorch there, this way you won't have to worry about any extra packages required.

4) Are we working with CUDA devices? (i.e. Do we need a GPU?)
-No, we'll work with proofs of concept that don't need GPUs on our laptops, so we'll fatigue our CPUs. Nonetheless if you have a CUDA device you're absolutely free to install the PyTorch version compiled with CUDA, it'll work on CPU anyway.