Linear algebra concepts are key for understanding and creating machine learning algorithms, especially as applied to deep learning and neural networks. This course reviews linear algebra with applications to probability and statistics and optimization–and above all a full explanation of deep learning.
Curriculum
teacher profile teaching materials
Matrix-matrix multiplication; column & row space; rank
The four fundamental subspaces of linear algebra
Fundamentals of Matrix factorizations:
A=LU rows & columns point of view
A=LU elimination & factorization; permutations
A=RU=VU; Orthogonal matrices
Eigensystems and Linear ODE
Intro to PSym; the energy function
Gradient and Hessian
Singular Value Decomposition
Eckart-Young; derivative of a matrix norm
Principal Component Analysis
Generalized evectors;
Norms
Least Squares
Convexity & Newton’s method
Newton & L-M method; Recap of non-linear regression
Lagrange multipliers
Machine Learning:
Gradient Descend; exact line search; GD in action; GD with Matlab
Learning & Loss; Intro to Deep Neural Network; DNN with Matlab
Loss functions: Quadratic VS Cross entropy
Stocastics Gradient Descend (SGD) & Kaczmarcz; SGD convergence rates & ADAM
Matlab interface for DNN
Construction of DNN: the key steps
Backpropagation and the Chain Rule
Machine Learning examples with Wolfram Mathematica
Convolutional NN + Mathematica examples of 1D convolution
Convolution and 2D filters + Mathematica examples of 2D convolution
Matlab Live Script, Network Designer, Pretrained Net
Linear Algebra and Learning from Data,
Wellesley-Cambridge Press
M. Nielsen,
Neural Networks and Deep Learning (free online book)
http://neuralnetworksanddeeplearning.com
Various authors,
Distill, dedicated to clear explanations of machine learning
https://distill.pub
Programme
Highlights of Linear Algebra:Matrix-matrix multiplication; column & row space; rank
The four fundamental subspaces of linear algebra
Fundamentals of Matrix factorizations:
A=LU rows & columns point of view
A=LU elimination & factorization; permutations
A=RU=VU; Orthogonal matrices
Eigensystems and Linear ODE
Intro to PSym; the energy function
Gradient and Hessian
Singular Value Decomposition
Eckart-Young; derivative of a matrix norm
Principal Component Analysis
Generalized evectors;
Norms
Least Squares
Convexity & Newton’s method
Newton & L-M method; Recap of non-linear regression
Lagrange multipliers
Machine Learning:
Gradient Descend; exact line search; GD in action; GD with Matlab
Learning & Loss; Intro to Deep Neural Network; DNN with Matlab
Loss functions: Quadratic VS Cross entropy
Stocastics Gradient Descend (SGD) & Kaczmarcz; SGD convergence rates & ADAM
Matlab interface for DNN
Construction of DNN: the key steps
Backpropagation and the Chain Rule
Machine Learning examples with Wolfram Mathematica
Convolutional NN + Mathematica examples of 1D convolution
Convolution and 2D filters + Mathematica examples of 2D convolution
Matlab Live Script, Network Designer, Pretrained Net
Core Documentation
G. Strang,Linear Algebra and Learning from Data,
Wellesley-Cambridge Press
M. Nielsen,
Neural Networks and Deep Learning (free online book)
http://neuralnetworksanddeeplearning.com
Various authors,
Distill, dedicated to clear explanations of machine learning
https://distill.pub
Type of delivery of the course
Theory and practicals with computers; practicals have a noteworthy role in these lecturesType of evaluation
Gli studenti dovranno scegliere un argomento da sviluppare tra quelli presentati durante le lezioni. Dovranno quindi preparare un testo scritto in cui viene descritto il problema, e vengono discussi i risultati degli esperimenti numerici. teacher profile teaching materials
- matrix multiplication; column & row space; rank; the four fundamental subspaces;
- elimination method; decomposition in lower and upper trinagular matrices; permutations;
- orthogonal matrices;
- eigenvalues and eigenvectors for ODE;
- positive definite symmetric matrices; the energy function;
- gradient and Hessian matrix;
- singular value decomposition; Eckart-Young theorem; principal component analysis; generalized evectors;
- norms; least squares method; convexity and Newton’s method; Lagrange multipliers.
Machine Learning:
- Gradient Descend; GD with Matlab;
- Learning and Loss; Deep Neural Network;
- loss functions: Quadratic VS Cross entropy;
- Stocastics Gradient Descend (SGD) & Kaczmarcz; SGD convergence rates & ADAM
- Matlab interface for DNN; Construction of DNN;
- Backpropagation and the Chain Rule;
- Machine Learning examples with Wolfram Mathematica;
- Convolutional NN + Mathematica examples of 1D convolution
- Convolution and 2D filters + Mathematica examples of 2D convolution
- Matlab Live Script, Network Designer, Pretrained Net
Programme
Highlights of Linear Algebra:- matrix multiplication; column & row space; rank; the four fundamental subspaces;
- elimination method; decomposition in lower and upper trinagular matrices; permutations;
- orthogonal matrices;
- eigenvalues and eigenvectors for ODE;
- positive definite symmetric matrices; the energy function;
- gradient and Hessian matrix;
- singular value decomposition; Eckart-Young theorem; principal component analysis; generalized evectors;
- norms; least squares method; convexity and Newton’s method; Lagrange multipliers.
Machine Learning:
- Gradient Descend; GD with Matlab;
- Learning and Loss; Deep Neural Network;
- loss functions: Quadratic VS Cross entropy;
- Stocastics Gradient Descend (SGD) & Kaczmarcz; SGD convergence rates & ADAM
- Matlab interface for DNN; Construction of DNN;
- Backpropagation and the Chain Rule;
- Machine Learning examples with Wolfram Mathematica;
- Convolutional NN + Mathematica examples of 1D convolution
- Convolution and 2D filters + Mathematica examples of 2D convolution
- Matlab Live Script, Network Designer, Pretrained Net
Core Documentation
G. Strang, Linear Algebra and Learning from Data, Wellesley-Cambridge Press (2019).Reference Bibliography
G. Strang, Linear Algebra and Learning from Data, Wellesley-Cambridge Press (2019).Type of delivery of the course
Lessons in presence, live streamed and registered with Microsoft Teams.Attendance
Attending is not mandatory, but strongly recommended.Type of evaluation
Students must choose an argument to be explored and develop among those presented during the lessons. They must then prepare a written text describing the problem, and discussing the results of numerical experiments. teacher profile teaching materials
Matrix-matrix multiplication; column & row space; rank
The four fundamental subspaces of linear algebra
Fundamentals of Matrix factorizations:
A=LU rows & columns point of view
A=LU elimination & factorization; permutations
A=RU=VU; Orthogonal matrices
Eigensystems and Linear ODE
Intro to PSym; the energy function
Gradient and Hessian
Singular Value Decomposition
Eckart-Young; derivative of a matrix norm
Principal Component Analysis
Generalized evectors;
Norms
Least Squares
Convexity & Newton’s method
Newton & L-M method; Recap of non-linear regression
Lagrange multipliers
Machine Learning:
Gradient Descend; exact line search; GD in action; GD with Matlab
Learning & Loss; Intro to Deep Neural Network; DNN with Matlab
Loss functions: Quadratic VS Cross entropy
Stocastics Gradient Descend (SGD) & Kaczmarcz; SGD convergence rates & ADAM
Matlab interface for DNN
Construction of DNN: the key steps
Backpropagation and the Chain Rule
Machine Learning examples with Wolfram Mathematica
Convolutional NN + Mathematica examples of 1D convolution
Convolution and 2D filters + Mathematica examples of 2D convolution
Matlab Live Script, Network Designer, Pretrained Net
Linear Algebra and Learning from Data,
Wellesley-Cambridge Press
M. Nielsen,
Neural Networks and Deep Learning (free online book)
http://neuralnetworksanddeeplearning.com
Various authors,
Distill, dedicated to clear explanations of machine learning
https://distill.pub
Mutuazione: 20410557 GE530 - ALGEBRA LINEARE PER IL MACHINE LEARNING in Scienze Computazionali LM-40 TERESI LUCIANO, FERMI DAVIDE
Programme
Highlights of Linear Algebra:Matrix-matrix multiplication; column & row space; rank
The four fundamental subspaces of linear algebra
Fundamentals of Matrix factorizations:
A=LU rows & columns point of view
A=LU elimination & factorization; permutations
A=RU=VU; Orthogonal matrices
Eigensystems and Linear ODE
Intro to PSym; the energy function
Gradient and Hessian
Singular Value Decomposition
Eckart-Young; derivative of a matrix norm
Principal Component Analysis
Generalized evectors;
Norms
Least Squares
Convexity & Newton’s method
Newton & L-M method; Recap of non-linear regression
Lagrange multipliers
Machine Learning:
Gradient Descend; exact line search; GD in action; GD with Matlab
Learning & Loss; Intro to Deep Neural Network; DNN with Matlab
Loss functions: Quadratic VS Cross entropy
Stocastics Gradient Descend (SGD) & Kaczmarcz; SGD convergence rates & ADAM
Matlab interface for DNN
Construction of DNN: the key steps
Backpropagation and the Chain Rule
Machine Learning examples with Wolfram Mathematica
Convolutional NN + Mathematica examples of 1D convolution
Convolution and 2D filters + Mathematica examples of 2D convolution
Matlab Live Script, Network Designer, Pretrained Net
Core Documentation
G. Strang,Linear Algebra and Learning from Data,
Wellesley-Cambridge Press
M. Nielsen,
Neural Networks and Deep Learning (free online book)
http://neuralnetworksanddeeplearning.com
Various authors,
Distill, dedicated to clear explanations of machine learning
https://distill.pub
Reference Bibliography
G. Strang, Linear Algebra and Learning from Data, Wellesley-Cambridge Press (2019).Type of delivery of the course
Theory and practicals with computers; practicals have a noteworthy role in these lecturesType of evaluation
Gli studenti dovranno scegliere un argomento da sviluppare tra quelli presentati durante le lezioni. Dovranno quindi preparare un testo scritto in cui viene descritto il problema, e vengono discussi i risultati degli esperimenti numerici. teacher profile teaching materials
- matrix multiplication; column & row space; rank; the four fundamental subspaces;
- elimination method; decomposition in lower and upper trinagular matrices; permutations;
- orthogonal matrices;
- eigenvalues and eigenvectors for ODE;
- positive definite symmetric matrices; the energy function;
- gradient and Hessian matrix;
- singular value decomposition; Eckart-Young theorem; principal component analysis; generalized evectors;
- norms; least squares method; convexity and Newton’s method; Lagrange multipliers.
Machine Learning:
- Gradient Descend; GD with Matlab;
- Learning and Loss; Deep Neural Network;
- loss functions: Quadratic VS Cross entropy;
- Stocastics Gradient Descend (SGD) & Kaczmarcz; SGD convergence rates & ADAM
- Matlab interface for DNN; Construction of DNN;
- Backpropagation and the Chain Rule;
- Machine Learning examples with Wolfram Mathematica;
- Convolutional NN + Mathematica examples of 1D convolution
- Convolution and 2D filters + Mathematica examples of 2D convolution
- Matlab Live Script, Network Designer, Pretrained Net
Mutuazione: 20410557 GE530 - ALGEBRA LINEARE PER IL MACHINE LEARNING in Scienze Computazionali LM-40 TERESI LUCIANO, FERMI DAVIDE
Programme
Highlights of Linear Algebra:- matrix multiplication; column & row space; rank; the four fundamental subspaces;
- elimination method; decomposition in lower and upper trinagular matrices; permutations;
- orthogonal matrices;
- eigenvalues and eigenvectors for ODE;
- positive definite symmetric matrices; the energy function;
- gradient and Hessian matrix;
- singular value decomposition; Eckart-Young theorem; principal component analysis; generalized evectors;
- norms; least squares method; convexity and Newton’s method; Lagrange multipliers.
Machine Learning:
- Gradient Descend; GD with Matlab;
- Learning and Loss; Deep Neural Network;
- loss functions: Quadratic VS Cross entropy;
- Stocastics Gradient Descend (SGD) & Kaczmarcz; SGD convergence rates & ADAM
- Matlab interface for DNN; Construction of DNN;
- Backpropagation and the Chain Rule;
- Machine Learning examples with Wolfram Mathematica;
- Convolutional NN + Mathematica examples of 1D convolution
- Convolution and 2D filters + Mathematica examples of 2D convolution
- Matlab Live Script, Network Designer, Pretrained Net
Core Documentation
G. Strang, Linear Algebra and Learning from Data, Wellesley-Cambridge Press (2019).Reference Bibliography
G. Strang, Linear Algebra and Learning from Data, Wellesley-Cambridge Press (2019).Type of delivery of the course
Lessons in presence, live streamed and registered with Microsoft Teams.Attendance
Attending is not mandatory, but strongly recommended.Type of evaluation
Students must choose an argument to be explored and develop among those presented during the lessons. They must then prepare a written text describing the problem, and discussing the results of numerical experiments. teacher profile teaching materials
Matrix-matrix multiplication; column & row space; rank
The four fundamental subspaces of linear algebra
Fundamentals of Matrix factorizations:
A=LU rows & columns point of view
A=LU elimination & factorization; permutations
A=RU=VU; Orthogonal matrices
Eigensystems and Linear ODE
Intro to PSym; the energy function
Gradient and Hessian
Singular Value Decomposition
Eckart-Young; derivative of a matrix norm
Principal Component Analysis
Generalized evectors;
Norms
Least Squares
Convexity & Newton’s method
Newton & L-M method; Recap of non-linear regression
Lagrange multipliers
Machine Learning:
Gradient Descend; exact line search; GD in action; GD with Matlab
Learning & Loss; Intro to Deep Neural Network; DNN with Matlab
Loss functions: Quadratic VS Cross entropy
Stocastics Gradient Descend (SGD) & Kaczmarcz; SGD convergence rates & ADAM
Matlab interface for DNN
Construction of DNN: the key steps
Backpropagation and the Chain Rule
Machine Learning examples with Wolfram Mathematica
Convolutional NN + Mathematica examples of 1D convolution
Convolution and 2D filters + Mathematica examples of 2D convolution
Matlab Live Script, Network Designer, Pretrained Net
Linear Algebra and Learning from Data,
Wellesley-Cambridge Press
M. Nielsen,
Neural Networks and Deep Learning (free online book)
http://neuralnetworksanddeeplearning.com
Various authors,
Distill, dedicated to clear explanations of machine learning
https://distill.pub
Programme
Highlights of Linear Algebra:Matrix-matrix multiplication; column & row space; rank
The four fundamental subspaces of linear algebra
Fundamentals of Matrix factorizations:
A=LU rows & columns point of view
A=LU elimination & factorization; permutations
A=RU=VU; Orthogonal matrices
Eigensystems and Linear ODE
Intro to PSym; the energy function
Gradient and Hessian
Singular Value Decomposition
Eckart-Young; derivative of a matrix norm
Principal Component Analysis
Generalized evectors;
Norms
Least Squares
Convexity & Newton’s method
Newton & L-M method; Recap of non-linear regression
Lagrange multipliers
Machine Learning:
Gradient Descend; exact line search; GD in action; GD with Matlab
Learning & Loss; Intro to Deep Neural Network; DNN with Matlab
Loss functions: Quadratic VS Cross entropy
Stocastics Gradient Descend (SGD) & Kaczmarcz; SGD convergence rates & ADAM
Matlab interface for DNN
Construction of DNN: the key steps
Backpropagation and the Chain Rule
Machine Learning examples with Wolfram Mathematica
Convolutional NN + Mathematica examples of 1D convolution
Convolution and 2D filters + Mathematica examples of 2D convolution
Matlab Live Script, Network Designer, Pretrained Net
Core Documentation
G. Strang,Linear Algebra and Learning from Data,
Wellesley-Cambridge Press
M. Nielsen,
Neural Networks and Deep Learning (free online book)
http://neuralnetworksanddeeplearning.com
Various authors,
Distill, dedicated to clear explanations of machine learning
https://distill.pub
Type of delivery of the course
Theory and practicals with computers; practicals have a noteworthy role in these lecturesType of evaluation
Gli studenti dovranno scegliere un argomento da sviluppare tra quelli presentati durante le lezioni. Dovranno quindi preparare un testo scritto in cui viene descritto il problema, e vengono discussi i risultati degli esperimenti numerici. teacher profile teaching materials
- matrix multiplication; column & row space; rank; the four fundamental subspaces;
- elimination method; decomposition in lower and upper trinagular matrices; permutations;
- orthogonal matrices;
- eigenvalues and eigenvectors for ODE;
- positive definite symmetric matrices; the energy function;
- gradient and Hessian matrix;
- singular value decomposition; Eckart-Young theorem; principal component analysis; generalized evectors;
- norms; least squares method; convexity and Newton’s method; Lagrange multipliers.
Machine Learning:
- Gradient Descend; GD with Matlab;
- Learning and Loss; Deep Neural Network;
- loss functions: Quadratic VS Cross entropy;
- Stocastics Gradient Descend (SGD) & Kaczmarcz; SGD convergence rates & ADAM
- Matlab interface for DNN; Construction of DNN;
- Backpropagation and the Chain Rule;
- Machine Learning examples with Wolfram Mathematica;
- Convolutional NN + Mathematica examples of 1D convolution
- Convolution and 2D filters + Mathematica examples of 2D convolution
- Matlab Live Script, Network Designer, Pretrained Net
Programme
Highlights of Linear Algebra:- matrix multiplication; column & row space; rank; the four fundamental subspaces;
- elimination method; decomposition in lower and upper trinagular matrices; permutations;
- orthogonal matrices;
- eigenvalues and eigenvectors for ODE;
- positive definite symmetric matrices; the energy function;
- gradient and Hessian matrix;
- singular value decomposition; Eckart-Young theorem; principal component analysis; generalized evectors;
- norms; least squares method; convexity and Newton’s method; Lagrange multipliers.
Machine Learning:
- Gradient Descend; GD with Matlab;
- Learning and Loss; Deep Neural Network;
- loss functions: Quadratic VS Cross entropy;
- Stocastics Gradient Descend (SGD) & Kaczmarcz; SGD convergence rates & ADAM
- Matlab interface for DNN; Construction of DNN;
- Backpropagation and the Chain Rule;
- Machine Learning examples with Wolfram Mathematica;
- Convolutional NN + Mathematica examples of 1D convolution
- Convolution and 2D filters + Mathematica examples of 2D convolution
- Matlab Live Script, Network Designer, Pretrained Net
Core Documentation
G. Strang, Linear Algebra and Learning from Data, Wellesley-Cambridge Press (2019).Reference Bibliography
G. Strang, Linear Algebra and Learning from Data, Wellesley-Cambridge Press (2019).Type of delivery of the course
Lessons in presence, live streamed and registered with Microsoft Teams.Attendance
Attending is not mandatory, but strongly recommended.Type of evaluation
Students must choose an argument to be explored and develop among those presented during the lessons. They must then prepare a written text describing the problem, and discussing the results of numerical experiments.