FRTN50 - Optimization for Learning
What is this course about?
This is a mathematical course that focuses on optimization theory and the role of optimization as a foundation for classical and contemporary supervised learning applications. This means that if you are interested in this type of mathematical theory, this is a course for you. If you are interested in applying (deep) learning to real-world problems, this is not a course for you. We advise that you have already taken an optimization course, a machine learning course, and have a strong mathematical background.
The course program contains the schedule and relevant information regarding lectures, exercises, deadlines, etc. All relevant information will also be posted on this canvas page.
Before the course
Mathematical prerequisites. We will in the teaching assume that you feel comfortable with the content of this mathematical prerequisites document.
Coding environments. We will program in Python. Have a look at this introduction to Python if you are unfamiliar.
Check TimeEdit for the most up-to-date schedule.
Some lecture videos (that are short flipped classroom type videos, not lecture recordings) can be found here. The videos are intended for active listening. This means pausing, skipping 15 s back (or forward) within video to repeat, e.g., an argument, skipping between videos to recall concepts, changing playback speed, and maybe taking notes and verifying calculations while watching,
The slides and videos may be updated during the course.
- L0 - Intro
- L1 - Convex sets (videos)
- L2 - Convex functions (videos)
- L3 - Subdifferentials and the proximal operator (videos/videos)
- L4 - Conjugate functions and duality (videos/videos)
- L5 - Proximal gradient method - Basics (videos)
- L6 - Algorithms and convergence (videos)
- L7 - Proximal gradient method - Theory (videos)
- L8 - Least squares (lecture video from a previous year) / Logistic regression (lecture video from a previous year)
- L9 - Support vector machines (lecture video from a previous year)
- L10 - Deep learning
- L11 - SGD - Qualitative convergence
- L12 - SGD - Implicit regularization
All lecture slides are collected in two different pdfs that both contain eight slides per page. In the first one, the "animation slides" have been condensed to take up one slide in the pdf. This pdf can be downloaded here and contains 83 pages. The other one is the same except that it shows all slides of all animations in sequence. This pdf can be downloaded here and contains 122 pages. You will not loose important information by using the shorter pdf, but some slides may look strange.
Extra material (not in the course this year):
The course does not have an official course book although we recommend for the interested student: First-Order Methods in Optimization by A. Beck and Convex Optimization by S. Boyd and L. Vandenberghe, both available via LUBsearch.
|Before||Mathematical prerequisites and introduction to Python|
|2022-08-29 E1||Ch 0: 3 and Ch 1: 1-10|
|2022-08-31 E2||Ch 2: 1-11|
|2022-09-05 E3||Ch 2: 12-19, 22, 23, 27, 30|
|2022-09-07 E4||Ch 3: 1-14|
|2022-09-12 E5||Ch 4: 1-2 and Ch 5: 1-6|
|2022-09-14 E6||Ch 5: 7-13|
|2022-09-19 E7||Ch 5: 14-16 and Ch 6: 1-5|
|2022-09-21 E8||Ch 7: 1-4|
|2022-09-26 E9||Ch 7: 5-9 and Ch 8: 1-2|
|2022-09-27 E10||Ch 8: 3-10|
|2022-10-03 E11||Ch 9: 1-4|
|2022-10-05 E12||Ch 9: 5-7|
|2022-10-10 E13||Ch 10: 1-4|
|2022-10-12 E14||Ch 10: 5-9|
|2022-10-17 Recap||Previous exams|
|2022-10-19 Recap||Previous exams|
Weekly problems for bonus points
Throughout the seven-week course, weekly challenging problems offer the chance to earn bonus points. By successfully tackling these problems, you can accumulate up to 0.5 points each week, leading to a potential 3.5 bonus points added to your final exam score. Note that submission is not mandatory.
- Problems for week 1 (deadline: 2023-09-03)
- Problems for week 2 (deadline: 2023-09-10)
- Problems for week 3 (deadline: 2023-09-17)
- Problems for week 4 (deadline: 2023-09-24)
- Problems for week 5 (deadline: 2023-10-01)
- Problems for week 6 (deadline: 2023-10-08)
- Problems for week 7 (deadline: 2023-10-15)
Submission is mandatory. If your submission is not passed: Two resubmissions are allowed for the first assignment. Only one resubmission is allowed for the second assignment.
- 2023-10-25 (with solutions)
- 2022-10-25 (with solutions)
- 2021-10-26 (with solutions)
- 2020-10-26 (with solutions)
- 2019-10-28 (with solutions)
|Mika Nishimura||Ladok email@example.com|
|Pontus Giselsson||Course firstname.lastname@example.org|
|Manu Upadhyaya||Teaching email@example.com|
|Max Nilsson||Teaching firstname.lastname@example.org|
|Anton Åkerman||Teaching email@example.com|
Kursöversiktssidan visar en tabellorienterad vy av kursschemat och grunderna för kursens bedömning. Du kan lägga till kommentarer, anteckningar eller tankar som du har om kursens struktur, kursens regler eller något annat.
Klicka på länken ”Redigera” längst upp för att lägga till kommentarer.