These are sample questions like they might be asked in an exam.
Lecture 1 What are major levels of description for digital systems?
 What levels of neural organization correspond to transistors, gates, circuits, block diagrams, ...?
 Label the major parts of a neuron.
 Describe how information is transmitted in a sensory neuron from the sensing element to the postsynaptic cell?
 Describe briefly how a chemical synapse works.
 What is the knee jerk reflex and how does the reflex arc work?
 What is "synaptic integration" and what kinds are there?
Lecture 2 How are voltages measured inside neurons?
 What is depolarization?
 What is patch clamping and what does it measure?
 What are the major ions involved in generating neural activity?
 What are the approximate Nernst potentials of the major ions in neural activity?
 What is the Nernst equation?
 Describe the steps involved in action potential generation.
 What is the HodgkinHuxley model?
 Describe how differential equations are solved numerically using difference equation approximations.
 What is "cable theory" and how does it relate to neurons?
 What is the FitzHughNagumo Model and what is its significance?
 What is the "phase space" of a differential equation?
 What does the phase space for the FitzHughNagumo model look like?
 How can we use kmeans clustering to categorize different behaviors of a neuron model?
Lecture 3 What is the integrateandfire model?
 What is the Izhikevich model?
 What does the phase space for the Izhikevich model look like?
 What is a McCullochPitts neuron?
 How is the McCullochPitts neuron justified using a discretization of time in the HodgkinHuxley model?
 How is the McCullochPitts neuron justified using a rate coding model?
 How can McCullochPitts neurons be used to represent logic gates?
 What is computaitonal universality?
 What is a cellular automaton?
 What is Rule 110?
 What is a tag system?
 What is a recurrent neural network?
 How can Rule 110 be implemented as a recurrent neural network?
Lecture 4 Explain why a refractory period is important for achieving synchronization in a network of weakly coupled integrateandfire neurons.
 State the equation for the van der Pol oscillator.
 Draw the phase space and nullclines for the van der Pol oscillator.
 Which neural model does the van der Pol oscillator relate to, and how?
 What is entrainment?
 Give a mathematical example of entrainment based on the van der Pol oscillator.
 What phenomenon is the strongly forced van der Pol oscillator an example of?
 What is a Lorenz system and what is it an example of?
 Explain the meaning of "strong sensitivity to initial conditions".
 What is the difference between a limit cycle and a strange attractor?
 Can the LotkaVolterra equations produce chaos? How?
 What realworld phenomena are described by LotkaVolterra equations (give several examples)?
 Give an example of a chaotic system based on a difference equation.
 Give an example of a chaotic 1D cellular automaton.
 Give an example of a class 4 1D cellular automaton.
 Define the different classes of behavior in 1D cellular automata.
 How does computational universality relate to chaotic behavior?
 How does computational universality relate to the classes of 1D cellular automata?
 What is Willshaw's associative memory? State the equations and the training method.
 What is a Hopfield network? State the equations and the training method.
Lecture 5 What is Bayes formula?
 What is zeroone loss?
 What is the Bayes error?
 What is the optimal decision rule under zeroone loss?
 What is the relationship between a linear threshold unit and similarity measures between vectors?
 What are augmented vectors?
 State the perceptron learning algorithm.
 What is the perceptron criterion function?
 How can a linear least square algorithm be used for classification?
 What is the pseudoinverse and how is it used for linear least square problems?
 What is a linearly separable problem?
 Does a perceptron learning algorithm always find a lowest error solution for a linearly separable problem?
 Does a perceptron learning algorithm always find a lowest error solution for a problem that is not linearly separable?
 What is logistic regression?
 How are the parameters of logistic regression estimated with gradient descent?
Lecture 6 How are multilayer perceptrons defined?
 Are the perceptrons in multilayer perceptrons the same as in single layer perceptrons?
 What was the book "Perceptrons" by Minsky and Papert about?
 What is the criterion function usually used for training MLPs?
 What is the training algorithm usually used for training MLPs?
 Derive the MLP learning algorithm.
 Why is the MLP learning algorithm called "backpropagation"?
 What is stochastic gradient descent?
 What is the learning rate?
 What is the number of hidden units?
 How do you choose learning rates and the number of hidden units?
 What is a test set?
 What is a training curve?
 Why would test set error be larger than training set error?
 What is crossvalidation?
Lecture 7 If you implement layers in an MLP library as classes, what are the three primary methods needed during training?
 Describe a design for a modular MLP library.
 Describe how deltas are passed around in an MLP library.
 Assuming you have multiple perceptron layers being trained using backpropagation. In what order should the forward, backward, and update methods be invoked on the three layers?
 How are classes encoded for training an MLP?
 How does an MLP have to be trained so that its outputs approximate posterior probabilities?
 Why is it important that the outputs of an MLP approximate posterior probabilities?
 Describe the training of a single layer convolutional neural network with one output.
 What is the relationship between "convolution" and a convolutional neural network?
Lecture 8 Describe the architecture of a multilayer convolutional neural network.
 What is unsupervised learning?
 What is the MNIST database?
 How do you center a dataset?
 What is the covariance matrix?
 What does the covariance matrix of a centered dataset mean? How do you visualize it?
 What is an eigenvector?
 If you sample from a Gaussian density, what do the eigenvectors of the covariance matrix represent?
 Explain what PCA accomplishes?
 If you project onto the eigenvectors corresponding to the first n largest eigenvalues, what are you doing?
 How can PCA be used to separate signal from noise?
 How is PCA used to preprocess data for pattern recognition?
 How does PCA speed up nearest neighbor classification?
 What kind of matrix is the covariance matrix?
 Describe a simple, iterative algorithm for finding the eigenvector corresponding to the largest eigenvalue of a covariance matrix.
 Describe a simple, iterative algorithm for finding the top n eigenvectors of a symmetric positive definite matrix.
Lecture 9
 Describe and explain the kmeans algorithm.
 Explain the mixture density estimation problem.
 Describe the EM algorithm for mixture density estimation.
 Explain "incremental" or "neural" analogs of the kmeans algorithm.
 Describe how to use the kmeans algorithm for improving MNIST classification.
 Describe and explain the SelfOrganizing Map algorithm.
 What are simple cells, what are complex cells, where are they found? What is the difference?
 What is a feature hierarchy?
 What is the Neocognitron?
 Be able to answer questions about the paper "GradientBased Learning Applied to Document Recognition" by Y. LeCun, L. Bottou, Y. Bengio, P. Haffner
 Be able to answer questions about the paper "Learning Methods for Generic Object Recognition with Invariance to Poste and Lighting" by Yann LeCun et al.
 Describe how convolutional neural networks are used for handwriting recognition.
 Describe how convolutional neural networks are used for object recognition.
 Describe how convolutional neural networks are trained for object recognition.
 Be able to answer questions about the paper "Hierarchical models of object recognition in cortex" by Riesenhuber and Poggio
 Describe the HMAX model of visual object recognition.
 Describe experiments supporting analogies between the HMAX model and human visual object recognition.
 Be able to answer questions about the paper "Comparing StateoftheArt Visual Features on Invariant Object Recognition" by Pinto et al.
 How did Pinto generate test cases for invariant object recognition?
 What should object recognition be invariant to?
 What does invariant object recognition mean?
 Broadly summarize the benchmark results from Pinto's paper.
Lecture 11 Describe how 3D modelbased reocgnition works.
 Explain the difficulties encountered in implementing 3D modelbased recognition.
 Explain componentbased 3D object recognition.
 What is a geon?
 What is viewbased recognition?
 Summarize the paper "Psychophysical support for a twodimensional view interpolation theory of object recognition." and be able to answer questions about it.
 What is a "2AFC experiment"?
 What are the different predictions of 2D view interpolation and 3D model based recognition in visual object recognition experiments?
 Summarize the results of the paper "Recognizing DepthRotated Objects" and be able to answer questions about it.
 What is Biederman's major criticism of Bülthoff's approach and conclusions?
 Give examples of viewpoint dependent parts changes and explain how they are important in experiments trying to prove a geonbased object recognition theory.
 How did Biederman modify Bülthoff's experiments by adding a distinctive geon? What results did he find?
Lecture 14  What is a time series?
 Give examples of time series?
 What is a stationary process?
 What is the trend of a time series? How might you find it?
 What is a random walk?
 What is the normal density?
 What is the Cauchy distribution? What unusual properties does it have?
 What is the autocorrelation function? What does it tell you?
 What is an autoregressive process?
 What is a moving average process?
 How do autoregressive and moving average processes relate to image/signal filtering?
 Give some common sequence classification tasks.
 Give some common language recognition tasks used for testing recurrent neural networks.
 What is a recurrent neural network?
 What is backpropagation through time?
