FRTN50 - Optimization for Learning

What is this course about?

This is a mathematical course that focuses on optimization theory and the role of optimization as a foundation for classical and contemporary supervised learning applications. This means that if you are interested in this type of mathematical theory, this is a course for you. If you are interested in applying (deep) learning to real-world problems, this is not a course for you. We advise that you have already taken an optimization course, a machine learning course, and have a strong mathematical background.

Course program

The course program Download course program contains the schedule and relevant information regarding lectures, exercises, deadlines, etc. All relevant information will also be posted on this canvas page.

Before the course

Mathematical prerequisites. We will in the teaching assume that you feel comfortable with the content of this Download this mathematical prerequisites document.

Coding environments. We will program in Python. Have a look at this introduction to Python if you are unfamiliar.

Schedule

Check TimeEdit Links to an external site. for the most up-to-date schedule.

Lecture videos

Some lecture videos (that are short flipped classroom type videos, not lecture recordings) can be found here Links to an external site.. The videos are intended for active listening. This means pausing, skipping 15 s back (or forward) within video to repeat, e.g., an argument, skipping between videos to recall concepts, changing playback speed, and maybe taking notes and verifying calculations while watching,

Lecture slides

The slides and videos may be updated during the course.

All lecture slides are collected in two different pdfs that both contain eight slides per page. In the first one, the "animation slides" have been condensed to take up one slide in the pdf. This pdf can be downloaded here Download here and contains 83 pages. The other one is the same except that it shows all slides of all animations in sequence. This pdf can be downloaded here Download here and contains 122 pages. You will not loose important information by using the shorter pdf, but some slides may look strange.

Extra material (not in the course this year):

Course book

The course does not have an official course book although we recommend for the interested student: First-Order Methods in Optimization by A. Beck and Convex Optimization by S. Boyd and L. Vandenberghe, both available via LUBsearch.

Exercise material

Suggested exercises

Session Exercise set
Before Mathematical prerequisites Download Mathematical prerequisites and introduction to Python
E1 Ch 0: 3 and Ch 1: 1-10
E2 Ch 2: 1-11
E3 Ch 2: 12-19, 22, 23, 27, 30
E4 Ch 3: 1-14
E5 Ch 4: 1-2 and Ch 5: 1-6
E6 Ch 5: 7-13
E7 Ch 5: 14-16 and Ch 6: 1-5 
E8 Ch 7: 1-4 
E9 Ch 7: 5-9 and Ch 8: 1-2
E10 Ch 8: 3-10
E11 Ch 9: 1-4
E12 Ch 9: 5-7
E13 Ch 10: 1-4
E14 Ch 10: 5-9
Recap Previous exams

Weekly problems for bonus points

Throughout the seven-week course, weekly challenging problems offer the chance to earn bonus points. By successfully tackling these problems, you can accumulate up to 0.5 points each week, leading to a potential 3.5 bonus points added to your final exam score. Note that submission is not mandatory.

Assignments

Submission is mandatory. If your submission is not passed: Two resubmissions are allowed for the assignments.

Exam

The regular exam will take place on October 30. You may bring the following to the exam:

You may use the results in the lecture slides and the mathematical prerequisites document unless the opposite is explicitly stated. You may also make some notes on the lecture slides that you bring.

Previous exams

Contact information

Mika Nishimura Links to an external site. Ladok administrator mika.nishimura@control.lth.se
  Pontus Giselsson Links to an external site. Course responsible pontusg@control.lth.se
Manu Upadhyaya Links to an external site. Teaching assistant manu.upadhyaya@control.lth.se
Max Nilsson Links to an external site. Teaching assistant max.nilsson@control.lth.se
Anton Åkerman Links to an external site. Teaching assistant anton.akerman@control.lth.se

Additional material