## 20410557 - GE530-Linear algebra for Machine Learning

Linear algebra concepts are key for understanding and creating machine learning algorithms, especially as applied to deep learning and neural networks. This course reviews linear algebra with applications to probability and statistics and optimization–and above all a full explanation of deep learning.

Curriculum

teacher profile | teaching materials

Programme

Highlights of Linear Algebra:
Matrix-matrix multiplication; column & row space; rank
The four fundamental subspaces of linear algebra
Fundamentals of Matrix factorizations:
A=LU rows & columns point of view
A=LU elimination & factorization; permutations
A=RU=VU; Orthogonal matrices
Eigensystems and Linear ODE
Intro to PSym; the energy function
Singular Value Decomposition
Eckart-Young; derivative of a matrix norm
Principal Component Analysis
Generalized evectors;
Norms
Least Squares
Convexity & Newton’s method
Newton & L-M method; Recap of non-linear regression
Lagrange multipliers

Machine Learning:
Gradient Descend; exact line search; GD in action; GD with Matlab
Learning & Loss; Intro to Deep Neural Network; DNN with Matlab
Loss functions: Quadratic VS Cross entropy
Matlab interface for DNN
Construction of DNN: the key steps
Backpropagation and the Chain Rule
Machine Learning examples with Wolfram Mathematica
Convolutional NN + Mathematica examples of 1D convolution
Convolution and 2D filters + Mathematica examples of 2D convolution
Matlab Live Script, Network Designer, Pretrained Net

Core Documentation

G. Strang,
Linear Algebra and Learning from Data,
Wellesley-Cambridge Press

M. Nielsen,
Neural Networks and Deep Learning (free online book)
http://neuralnetworksanddeeplearning.com

Various authors,
Distill, dedicated to clear explanations of machine learning
https://distill.pub

Type of delivery of the course

Theory and practicals with computers; practicals have a noteworthy role in these lectures

Type of evaluation

Gli studenti dovranno scegliere un argomento da sviluppare tra quelli presentati durante le lezioni. Dovranno quindi preparare un testo scritto in cui viene descritto il problema, e vengono discussi i risultati degli esperimenti numerici.

teacher profile | teaching materials

Programme

Highlights of Linear Algebra:
- matrix multiplication; column & row space; rank; the four fundamental subspaces;
- elimination method; decomposition in lower and upper trinagular matrices; permutations;
- orthogonal matrices;
- eigenvalues and eigenvectors for ODE;
- positive definite symmetric matrices; the energy function;
- singular value decomposition; Eckart-Young theorem; principal component analysis; generalized evectors;
- norms; least squares method; convexity and Newton’s method; Lagrange multipliers.

Machine Learning:
- Gradient Descend; GD with Matlab;
- Learning and Loss; Deep Neural Network;
- loss functions: Quadratic VS Cross entropy;
- Stocastics Gradient Descend (SGD) & Kaczmarcz; SGD convergence rates & ADAM
- Matlab interface for DNN; Construction of DNN;
- Backpropagation and the Chain Rule;
- Machine Learning examples with Wolfram Mathematica;
- Convolutional NN + Mathematica examples of 1D convolution
- Convolution and 2D filters + Mathematica examples of 2D convolution
- Matlab Live Script, Network Designer, Pretrained Net

Core Documentation

G. Strang, Linear Algebra and Learning from Data, Wellesley-Cambridge Press (2019).

Reference Bibliography

G. Strang, Linear Algebra and Learning from Data, Wellesley-Cambridge Press (2019).

Type of delivery of the course

Lessons in presence, live streamed and registered with Microsoft Teams.

Attendance

Attending is not mandatory, but strongly recommended.

Type of evaluation

Students must choose an argument to be explored and develop among those presented during the lessons. They must then prepare a written text describing the problem, and discussing the results of numerical experiments.

teacher profile | teaching materials

Mutuazione: 20410557 GE530 - ALGEBRA LINEARE PER IL MACHINE LEARNING in Scienze Computazionali LM-40 TERESI LUCIANO, FERMI DAVIDE

Programme

Highlights of Linear Algebra:
Matrix-matrix multiplication; column & row space; rank
The four fundamental subspaces of linear algebra
Fundamentals of Matrix factorizations:
A=LU rows & columns point of view
A=LU elimination & factorization; permutations
A=RU=VU; Orthogonal matrices
Eigensystems and Linear ODE
Intro to PSym; the energy function
Singular Value Decomposition
Eckart-Young; derivative of a matrix norm
Principal Component Analysis
Generalized evectors;
Norms
Least Squares
Convexity & Newton’s method
Newton & L-M method; Recap of non-linear regression
Lagrange multipliers

Machine Learning:
Gradient Descend; exact line search; GD in action; GD with Matlab
Learning & Loss; Intro to Deep Neural Network; DNN with Matlab
Loss functions: Quadratic VS Cross entropy
Matlab interface for DNN
Construction of DNN: the key steps
Backpropagation and the Chain Rule
Machine Learning examples with Wolfram Mathematica
Convolutional NN + Mathematica examples of 1D convolution
Convolution and 2D filters + Mathematica examples of 2D convolution
Matlab Live Script, Network Designer, Pretrained Net

Core Documentation

G. Strang,
Linear Algebra and Learning from Data,
Wellesley-Cambridge Press

M. Nielsen,
Neural Networks and Deep Learning (free online book)
http://neuralnetworksanddeeplearning.com

Various authors,
Distill, dedicated to clear explanations of machine learning
https://distill.pub

Reference Bibliography

G. Strang, Linear Algebra and Learning from Data, Wellesley-Cambridge Press (2019).

Type of delivery of the course

Theory and practicals with computers; practicals have a noteworthy role in these lectures

Type of evaluation

Gli studenti dovranno scegliere un argomento da sviluppare tra quelli presentati durante le lezioni. Dovranno quindi preparare un testo scritto in cui viene descritto il problema, e vengono discussi i risultati degli esperimenti numerici.

teacher profile | teaching materials

Mutuazione: 20410557 GE530 - ALGEBRA LINEARE PER IL MACHINE LEARNING in Scienze Computazionali LM-40 TERESI LUCIANO, FERMI DAVIDE

Programme

Highlights of Linear Algebra:
- matrix multiplication; column & row space; rank; the four fundamental subspaces;
- elimination method; decomposition in lower and upper trinagular matrices; permutations;
- orthogonal matrices;
- eigenvalues and eigenvectors for ODE;
- positive definite symmetric matrices; the energy function;
- singular value decomposition; Eckart-Young theorem; principal component analysis; generalized evectors;
- norms; least squares method; convexity and Newton’s method; Lagrange multipliers.

Machine Learning:
- Gradient Descend; GD with Matlab;
- Learning and Loss; Deep Neural Network;
- loss functions: Quadratic VS Cross entropy;
- Stocastics Gradient Descend (SGD) & Kaczmarcz; SGD convergence rates & ADAM
- Matlab interface for DNN; Construction of DNN;
- Backpropagation and the Chain Rule;
- Machine Learning examples with Wolfram Mathematica;
- Convolutional NN + Mathematica examples of 1D convolution
- Convolution and 2D filters + Mathematica examples of 2D convolution
- Matlab Live Script, Network Designer, Pretrained Net

Core Documentation

G. Strang, Linear Algebra and Learning from Data, Wellesley-Cambridge Press (2019).

Reference Bibliography

G. Strang, Linear Algebra and Learning from Data, Wellesley-Cambridge Press (2019).

Type of delivery of the course

Lessons in presence, live streamed and registered with Microsoft Teams.

Attendance

Attending is not mandatory, but strongly recommended.

Type of evaluation

Students must choose an argument to be explored and develop among those presented during the lessons. They must then prepare a written text describing the problem, and discussing the results of numerical experiments.

teacher profile | teaching materials

Programme

Highlights of Linear Algebra:
Matrix-matrix multiplication; column & row space; rank
The four fundamental subspaces of linear algebra
Fundamentals of Matrix factorizations:
A=LU rows & columns point of view
A=LU elimination & factorization; permutations
A=RU=VU; Orthogonal matrices
Eigensystems and Linear ODE
Intro to PSym; the energy function
Singular Value Decomposition
Eckart-Young; derivative of a matrix norm
Principal Component Analysis
Generalized evectors;
Norms
Least Squares
Convexity & Newton’s method
Newton & L-M method; Recap of non-linear regression
Lagrange multipliers

Machine Learning:
Gradient Descend; exact line search; GD in action; GD with Matlab
Learning & Loss; Intro to Deep Neural Network; DNN with Matlab
Loss functions: Quadratic VS Cross entropy
Matlab interface for DNN
Construction of DNN: the key steps
Backpropagation and the Chain Rule
Machine Learning examples with Wolfram Mathematica
Convolutional NN + Mathematica examples of 1D convolution
Convolution and 2D filters + Mathematica examples of 2D convolution
Matlab Live Script, Network Designer, Pretrained Net

Core Documentation

G. Strang,
Linear Algebra and Learning from Data,
Wellesley-Cambridge Press

M. Nielsen,
Neural Networks and Deep Learning (free online book)
http://neuralnetworksanddeeplearning.com

Various authors,
Distill, dedicated to clear explanations of machine learning
https://distill.pub

Type of delivery of the course

Theory and practicals with computers; practicals have a noteworthy role in these lectures

Type of evaluation

Gli studenti dovranno scegliere un argomento da sviluppare tra quelli presentati durante le lezioni. Dovranno quindi preparare un testo scritto in cui viene descritto il problema, e vengono discussi i risultati degli esperimenti numerici.

teacher profile | teaching materials

Programme

Highlights of Linear Algebra:
- matrix multiplication; column & row space; rank; the four fundamental subspaces;
- elimination method; decomposition in lower and upper trinagular matrices; permutations;
- orthogonal matrices;
- eigenvalues and eigenvectors for ODE;
- positive definite symmetric matrices; the energy function;
- singular value decomposition; Eckart-Young theorem; principal component analysis; generalized evectors;
- norms; least squares method; convexity and Newton’s method; Lagrange multipliers.

Machine Learning:
- Gradient Descend; GD with Matlab;
- Learning and Loss; Deep Neural Network;
- loss functions: Quadratic VS Cross entropy;
- Stocastics Gradient Descend (SGD) & Kaczmarcz; SGD convergence rates & ADAM
- Matlab interface for DNN; Construction of DNN;
- Backpropagation and the Chain Rule;
- Machine Learning examples with Wolfram Mathematica;
- Convolutional NN + Mathematica examples of 1D convolution
- Convolution and 2D filters + Mathematica examples of 2D convolution
- Matlab Live Script, Network Designer, Pretrained Net

Core Documentation

G. Strang, Linear Algebra and Learning from Data, Wellesley-Cambridge Press (2019).

Reference Bibliography

G. Strang, Linear Algebra and Learning from Data, Wellesley-Cambridge Press (2019).

Type of delivery of the course

Lessons in presence, live streamed and registered with Microsoft Teams.

Attendance

Attending is not mandatory, but strongly recommended.

Type of evaluation

Students must choose an argument to be explored and develop among those presented during the lessons. They must then prepare a written text describing the problem, and discussing the results of numerical experiments.