Lecture 13: Adaptive Control

(Big slides) Download Big slides) (Small slides Download Small slides)

Learning for Dynamics and Control

For recent research performed in the field of Learning for dynamics and control (L4DC) you might find these conference links interesting, where many presentations are freely available

Short-list of recommended talks:

---------------------------------------------------------------------------------------------------------------------

Deep RL control of Tokamak Fusion Reactor (from Feb 2022):  ITER Tokamak reactor

Personal reflection[BoB]: A main achievement is to have developed a so good model of the physical system that it can be used to get RL to work, and to then be trusted to run online.Note that it was built "on top of" a realtime control system doing most of the heavy work, so it was not learnt from scratch. The paper illustrates the power of combining new learning techniques with traditional modeling+control methods.

---------------------------------------------------------------------------------------------------------------------

RL in power systems.  Hybrid_Power_System.gif

---------------------------------------------------------------------------------------------------------------------

Solving the protein folding problem      AlphaFold_2_block_design.png

Maybe unclear if it fits into "learning for control" yet, but it might in the future, when people control how to change and build these "molecules of life". It again illustrates the fact that the progress is possible by combining traditional domain expert knowledge, in feature representation and score function definition etc, with novel methods for learning.

---------------------------------------------------------------------------------------------------------------------

DeepMind's AlphaStar (2019)

---------------------------------------------------------------------------------------------------------------------

Learning to play Poker

Interesting because it is involves  game theory with imperfect information

---------------------------------------------------------------------------------------------------------------------

Iterative Learning Control for Beam Control at ESS

  • This presentation needs to wait until next year. Will be based on some work done here in Lund, where also people from our department have been involved.

 

--------- Not applications, but the big picture : -------------------------

NY Times February 2023: Bing's new chatbot left me deeply unsettled.

NY Times January 2023: ChatGPT Hijacks Democracy Links to an external site.

Nature February 2023: How will AI change mathematics? Links to an external site.

On the upcoming EU legislation for AI Download On the upcoming EU legislation for AI

--------- Not applications, but also worth a watch : -------------------------

Video about control of POMDPs: https://www.youtube.com/watch?v=yrC2O7T7aKU Links to an external site.

Video about Model Free RL and TD Learning: https://www.youtube.com/watch?v=0iqz4tcKN58 Links to an external site.

And while you have a moment over, why not take this course in POMDPs using Julia and Pluto.jl Links to an external site. :-)