Lab 1: Abstract neural networks
Abstract neural network lab
To goal of this lab is to get hands on experience of how a neural network works. In particular the students should understand how a distributed representation work (including cells, activation and weights), how encoding can be done though Hebbian learning, how to retrieval of patterns can occur by input from weighted activation from other cells, and how forgetting occurs in neutral networks.
In particular we will look at simple Hopfield neural network that is implemented in a short Matlab code.
Preparation before the lab
1. Read up on how a Hopfield neural network works. This is also covered in the lecture preceding the lab.
2. Make sure that you can run Matlab and bring your computer to the lecture. If you are a Window user then you should be able to run Matlab online from this page: https://apps.lu.se/ (you may need to download the Citris-App first: https://www.citrix.com/products/receiver.html) Links to an external site.. As a Mac user you should be able to run Matlab online from this page https://se.mathworks.com/products/simulink-online.html#license-types. Links to an external site.It is also possible to download the full version Matlab from this page: https://program.ddg.lth.se/en/ Links to an external site.. If you do not have a laptop computer, or can not get Matlab working, team up with a student that can do that.
3. Download the Matlab script(s) that you will be working on from here Download here [This script may be updated later]. You can also upload a set of more complex files here Download here.
Getting started
Open a new script file in Matlab. Paste in the downloaded Matlab code (. Save the file with name, for example name it: NNLab.m.
Run the script by pasting in the file name (e.g., NNLab <enter> in the command window. Look at the figures and try to understand what is represented. Notice every time that you run the code you get different random patterns and somewhat different results,
Change a parameter in the file, e.g., set the number of cells to 30 (N.p.Ncells=10), save the file, and run the code again. How did the result change? Did the number of correctly retrieved cells/patterns change?
How you should think about the network
You can think this Hopfield neural network as a simulation of a memory study. For example, you are in the memory lab and you are instructed to encode (i.e., apply the learning rule in the network) words appearing on the screen (i.e., patterns in the network). Then there is a recognition test of the encoded list. The words reappear on the screen and your task it say if they were in the list or not (i.e., by applying the retrieval rule and the activation function. If the retrieved activity is the same as the encoded pattern then you have correctly recognised it).
Connect the simulation to findings in human memory
Can you simulate the following memory phenomena in the network? If you can not simulate it, how could understand them in a neural network framework?
- Forgetting by interference, or forgetting by decay, theories.
- List length effect, i.e., longer list of words are more difficult to retrieve than short lists.
- How can you simulate object recognition, episoding recognition, and episodic free recall
- Forgetting curves, i.e. newer memories are remembered better than older memories.
- Can you simulate a short-term/working memory ?
- The isolation effect, i.e. that a different item is remembered better than similar items in a word list
The lab
You are welcome to do the lab in groups of two, but it also possible to do it on your own. Run the script and answer the following questions in your lab report:
1. How are patterns represented in the network.
2. How is encoding conducted in the network.
3. How is retrieval conducted in the network.
4. How is performance (i.e. accuracy of retrieval measured in the network.
5. Carefully study in experiments how performance depends on:
a. The number of cells in the network
b. The number of patterns in the network.
c. The activity in the network
d. The activation function of the network
6. Try different types of network by changing the cue and target cells:
a. Fully interconnected network (default settings)
b. Feedforward network
c. Can you make it a recurrent network?
The lab report
Write a lab report of what you have done and your experience of the lab. Try to finish the lab rapport no later than the week after the lab.
Write according to the APA-style, see here:
https://libguides.lub.lu.se/apa_short
1. For each task above, answer with a paragraph that shows that you understand the function of the network well and how it relates to human memory. The lab report should a few pages long. Write it in Google Docs
2. Put a link of your lab report in the Course Sheet.
3. Review another groups report on the Course Sheet and give them constructive feedback in another Google Docs document and put the link to this document in the Course Sheet.
4. Wait for feedback on your report and make changes to report given the feedback.