Theano Tutorial @ CREI 2013

July 19, 2013, Sherbrook, Québec, Canada.

Theano is python software for evaluating complicated array expressions.

What does it do?

  • aggressive expression optimizations,

  • automatic GPU use,

  • symbolic differentiation and R op.

It complements the Python numeric/scientific software stack (e.g. NumPy, SciPy, scikits, matplotlib, PIL.)

Design and feature set has been driven by machine learning research at the University of Montreal (groups of Yoshua Bengio, Pascal Vincent, Aaron Courville and Roland Memisevic) The result is a very good library for doing research in deep learning and neural network training, and a flexible framework for many other models and algorithms in machine learning more generally.

It has proven to be useful for implementing:

  • linear and nonlinear neural network classifiers

  • convolutional models

  • Energy models: RBM, DBN, GRBM, ssRBM, AIS

  • Auto-encoders: DAE, CAE

  • GP regression

  • sparse coding

  • recurrent neural networks, echo state, (HMM?)

  • online and batch learning and optimization

  • Even SVM!

As people’s needs change this list will grow, but Theano is built around vector, matrix, and tensor expressions; there is little reason to use it for calculations on other data structures except. There is also some sparse matrix support.