Jin Hyun Park / Qual Exam Prep / AI / ML basics
AI / ML basics
The exam touches AI only lightly. This is the one concept that came up.
Capacity
- In general machine learning, the capacity of a model is how complex a set of functions it can represent. Higher capacity means the model can fit more, which also means a greater risk of overfitting. (The VC dimension is one formal measure of capacity.)
- For an associative memory such as a Hopfield network, capacity has a more specific meaning: the number of patterns that can be stored in the weights and later recalled correctly. This is the sense used in the exam question I saw.
- With Hebbian learning, a Hopfield network of N neurons can store roughly 0.14 N patterns (about N / (2 ln N) for error-free recall), so capacity grows roughly linearly with the number of neurons.
Overview ·
Useful materials ·
Sample questions ·
Algorithms ·
Architecture ·
OS ·
Networking ·
Databases ·
Automata ·
Things to remember