Optimization for Learning
Optimization for Learning
What is this course about?
This is a mathematical course that focuses on optimization theory and the role of optimization as a foundation for classical and contemporary supervised learning applications. This means that if you are interested in this type of mathematical theory, this is a course for you. If you are interested in applying (deep) learning to real-world problems, this is not a course for you.
The course program contains the schedule and relevant information regarding lectures, exercises, deadlines, etc. All relevant information will also be posted on this canvas page.
Before the course
Mathematical prerequisites. The course is fairly mathematical. We will in the teaching assume that you feel reasonably comfortable with the content of this mathematical prerequisites document.
Coding environments. We will program in Python. Have a look at this introduction to Python if you are unfamiliar.
Check TimeEdit for the most up-to-date schedule.
Some lecture videos (that are short flipped classroom type videos, not lecture recordings) can be found here. The videos are intended for active listening. This means pausing, skipping 15 s back (or forward) within video to repeat, e.g., an argument, skipping between videos to recall concepts, changing playback speed, and maybe taking notes and verifying calculations while watching,
The slides and videos may be updated during the course.
- L0 - Intro
- L1 - Convex sets (videos)
- L2 - Convex functions (videos)
- L3 - Subdifferentials and the proximal operator (videos/videos)
- L4 - Conjugate functions and duality (videos/videos)
- L5 - Proximal gradient method - Basics (videos)
- L6 - Algorithms and convergence (videos)
- L7 - Proximal gradient method - Theory (videos)
- L8 - Least squares (lecture video from last year) / Logistic regression (lecture video from last year)
- L9 - Support vector machines (lecture video from last year)
- L10 - Deep learning
- L11 - SGD - Qualitative convergence
- L12 - SGD - Implicit regularization
All lecture slides are collected in two different pdfs that both contain eight slides per page. In the first one, the "animation slides" have been condensed to take up one slide in the pdf. This pdf can be downloaded here and contains 83 pages. The other one is the same except that it shows all slides of all animations in sequence. This pdf can be downloaded here and contains 122 pages. You will not loose important information by using the shorter pdf, but some slides may look strange.
Extra material (not in the course this year):
- Exercise compendium (New exercises will be added on a weekly basis.)
- Mathematical prerequisites
- Introduction to Python
|Before||Mathematical prerequisites and introduction to Python|
|2022-08-30 E1||Ch 0: 3 and Ch 1: 1-10|
|2022-09-01 E2||Ch 2: 1-11|
|2022-09-06 E3||Ch 2: 12-19, 22, 23, 27, 30|
|2022-09-08 E4||Ch 3: 1-14|
|2022-09-13 E5||Ch 4: 1-2 and Ch 5: 1-9|
|2022-09-15 E6||Ch 5: 10-16 and Ch 6: 1-5|
|2022-09-20 E7||Ch 7: 1-5|
|2022-09-22 E8||Ch 7: 6-9|
|2022-09-27 E9||Ch 8: 1-5|
|2022-09-29 E10||Ch 8: 6-10|
|2022-10-04 E11||Ch 9: 1-4|
|2022-10-06 E12||Ch 9: 5-7|
|2022-10-11 E13||Ch 10: 1-4|
|2022-10-13 E14||Ch 10: 5-9|
|2022-10-18 Recap||Previous exams|
|2022-10-20 Recap||Previous exams|
- Assignment 1 (Deadline for the second resubmission: 1 November)
- Assignment 2 (Deadline for the resubmission: 1 November)
If your submission is not passed: Two resubmissions are allowed on the first assignment. Only one resubmission is allowed on the second assignment. The resubmission deadlines will be posted as notifications here in Canvas.
- 2022-10-25 (with solutions)
- 2021-10-26 (with solutions)
- 2020-10-26 (with solutions)
- 2019-10-28 (with solutions)
|Mika Nishimura||Ladok email@example.com|
|Pontus Giselsson||Course firstname.lastname@example.org|
|Manu Upadhyaya||Teaching email@example.com|
|Max Nilsson||Teaching firstname.lastname@example.org|
The syllabus page shows a table-oriented view of course schedule and basics of course grading. You can add any other comments, notes or thoughts you have about the course structure, course policies or anything else.
To add some comments, click the 'Edit' link at the top.