| Entrambe le parti precedenti la revisioneRevisione precedenteProssima revisione | Revisione precedente |
| magistraleinformatica:aa2:midterm14_15 [13/03/2015 alle 17:19 (10 anni fa)] – Davide Bacciu | magistraleinformatica:aa2:midterm14_15 [13/03/2015 alle 17:35 (10 anni fa)] (versione attuale) – [AA2 - Midterm Reading List A.A. 2014-15] Davide Bacciu |
|---|
| ====== Machine Learning: Neural Networks and Advanced Models (AA2) ====== | ====== AA2 - Midterm Reading List 2014-15 ====== |
| | |
| ===== Midterm Reading List A.A. 2014-15 ===== | |
| |
| In the following, it is a list of the topics and articles for the midterm assignment. | In the following, it is a list of the topics and articles for the midterm assignment. |
| ==== 1. Self-Organizing Map for sequences ==== | ==== 1. Self-Organizing Map for sequences ==== |
| |
| **Reading Material:** T. Voegtlin "Recursive self-organizing maps." Neural Networks 15.8 (2002): 979-991. [[magistraleinformatica:aa2:rsom02.pdf| pdf]] | **Reading Material:** T. Voegtlin "Recursive self-organizing maps." Neural Networks 15.8 (2002): 979-991. {{magistraleinformatica:aa2:rsom02.pdf| pdf}} |
| |
| **Questions:** Describe the recursive encoding of sequences in the RSOM. Report and discuss the network error and the update equations for the network weights. Provide a comparison between RSOM, temporal SOM and recurrent SOM (also showing the differences in the respective activation functions). | **Questions:** Describe the recursive encoding of sequences in the RSOM. Report and discuss the network error and the update equations for the network weights. Provide a comparison between RSOM, temporal SOM and recurrent SOM (also showing the differences in the respective activation functions). |
| ==== 2. Echo State Networks for indoor localization ==== | ==== 2. Echo State Networks for indoor localization ==== |
| |
| **Reading Material:** D. Bacciu, P. Barsocchi, S. Chessa, C. Gallicchio, A. Micheli, An experimental characterization of reservoir computing in ambient assisted living applications, Neural Computing and Applications, vol. 24 (6), pag. 1451–1464, 2014 | **Reading Material:** D. Bacciu, P. Barsocchi, S. Chessa, C. Gallicchio, A. Micheli, An experimental characterization of reservoir computing in ambient assisted living applications, Neural Computing and Applications, vol. 24 (6), pag. 1451–1464, 2014 {{magistraleinformatica:aa2:localization.pdf| pdf}} |
| |
| **Questions:** Describe the application and the experimental scenario: highlight the differences between the homogenous and heterogeneous settings. Describe the leaky integrator echo state network: discuss changes (also with equations) with respect to the standard ESN. Why is the leaky integrator needed? | **Questions:** Describe the application and the experimental scenario: highlight the differences between the homogenous and heterogeneous settings. Describe the leaky integrator echo state network: discuss changes (also with equations) with respect to the standard ESN. Why is the leaky integrator needed? |
| ==== 3. Minimum complexity Echo State Networks ==== | ==== 3. Minimum complexity Echo State Networks ==== |
| |
| **Reading Material:** Rodan, P. Tino, Minimum complexity echo state network, IEEE Transactions on Neural Networks, vol. 22(1), pag. 131-144, 2011 | **Reading Material:** Rodan, P. Tino, Minimum complexity echo state network, IEEE Transactions on Neural Networks, vol. 22(1), pag. 131-144, 2011 {{magistraleinformatica:aa2:minCompESN.pdf| pdf}} |
| |
| **Questions:** Describe the DLR, DLRB and SCR topologies of an ESN. Sketch the demonstration of the memory capacity MC for an SCR (theorem 1). Summarize the experimental results: what minimal topology/parameterization has performance levels comparable to standard ESNs? | **Questions:** Describe the DLR, DLRB and SCR topologies of an ESN. Sketch the demonstration of the memory capacity MC for an SCR (theorem 1). Summarize the experimental results: what minimal topology/parameterization has performance levels comparable to standard ESNs? |
| ==== 4. Long-Short term memory networks ==== | ==== 4. Long-Short term memory networks ==== |
| |
| **Reading Material:** Hochreiter, Sepp, and Jürgen Schmidhuber. "Long short-term memory." Neural computation 9.8 (1997): 1735-1780. | **Reading Material:** Hochreiter, Sepp, and Jürgen Schmidhuber. "Long short-term memory." Neural computation 9.8 (1997): 1735-1780. {{magistraleinformatica:aa2:lstm.pdf| pdf}} |
| |
| **Questions:** Explain the vanishing gradient problem. Describe the LSTM architecture and main equations. What is the role of the gate units? | **Questions:** Explain the vanishing gradient problem. Describe the LSTM architecture and main equations. What is the role of the gate units? |
| ==== 5. Structure finding in Bayesian Networks ==== | ==== 5. Structure finding in Bayesian Networks ==== |
| |
| **Reading Material:** D.Bacciu, T.A. Etchells, P.J.G. Lisboa and J. Whittaker, "Efficient identification of independence networks using mutual information", Computational Statistics, Springer, vol 28, no. 2, pp 621-646, Apr. 2013 | **Reading Material:** D.Bacciu, T.A. Etchells, P.J.G. Lisboa and J. Whittaker, "Efficient identification of independence networks using mutual information", Computational Statistics, Springer, vol 28, no. 2, pp 621-646, Apr. 2013 {{magistraleinformatica:aa2:pcAlgo.pdf| pdf}} |
| |
| **Questions:** Summarize the standard PC algorithm: describe the test of conditional independence and how it is computed with Mutual Information. Explain what is a False Negative in this scenario and describe the idea of power correction for reducing false negatives. Describe the concept of strong and weak edges and how/why this is used for the test-the-weakest-first policy. | **Questions:** Summarize the standard PC algorithm: describe the test of conditional independence and how it is computed with Mutual Information. Explain what is a False Negative in this scenario and describe the idea of power correction for reducing false negatives. Describe the concept of strong and weak edges and how/why this is used for the test-the-weakest-first policy. |
| ==== 6. Image-Denoising with the Ising model ==== | ==== 6. Image-Denoising with the Ising model ==== |
| |
| **Reading Material:** Section 8.3.3 from Bishop chapter. | **Reading Material:** Section 8.3.3 from Bishop chapter ([[http://research.microsoft.com/en-us/um/people/cmbishop/prml/pdf/Bishop-PRML-sample.pdf|pdf]]). |
| |
| **Questions:** Describe the problem and the associated Markov random field. Provide the energy function equations and discuss their interpretation for the particular application. | **Questions:** Describe the problem and the associated Markov random field. Provide the energy function equations and discuss their interpretation for the particular application. |
| ==== 7. Bi-Directional Hidden Markov Models ==== | ==== 7. Bi-Directional Hidden Markov Models ==== |
| |
| **Reading Material:** Baldi, P., Brunak, S., Frasconi, P., Pollastri, G., & Soda, G. (2001). Bidirectional dynamics for protein secondary structure prediction. In Sequence Learning (pp. 80-104). Springer Berlin Heidelberg. | **Reading Material:** Baldi, P., Brunak, S., Frasconi, P., Pollastri, G., & Soda, G. (2001). Bidirectional dynamics for protein secondary structure prediction. In Sequence Learning (pp. 80-104). Springer Berlin Heidelberg. {{magistraleinformatica:aa2:bidir-hmm.pdf| pdf}} |
| |
| **Questions:** Describe the bi-diretional IO-HMM and discuss the equation for its joint distribution factorization: identify the model parameters and what are the stationariety assumptions. Summarize how the transition functions can be implemented using MLP neural networks. | **Questions:** Describe the bi-diretional IO-HMM and discuss the equation for its joint distribution factorization: identify the model parameters and what are the stationariety assumptions. Summarize how the transition functions can be implemented using MLP neural networks. |
| ==== 8. Max-product Algorithm ==== | ==== 8. Max-product Algorithm ==== |
| |
| **Reading Material:** Section 5.2.1 of David Barber’s Book. | **Reading Material:** Section 5.2.1 of David Barber’s Book [BRML]. |
| |
| **Questions:** Describe what is the typical max-product inference problem: why is different from sum-product? Describe the variable elimination idea in max-product. Describe the max-product message passing using factor graphs. | **Questions:** Describe what is the typical max-product inference problem: why is different from sum-product? Describe the variable elimination idea in max-product. Describe the max-product message passing using factor graphs. |
| ==== 9. Markov Chains and Pagerank ==== | ==== 9. Markov Chains and Pagerank ==== |
| |
| **Reading Material:** Jia Li, Markov Chain Interpretation of Google Page Rank, Tech Report. Integrate with David Barber’s Book, pages from 461 to 463. | **Reading Material:** Jia Li, Markov Chain Interpretation of Google Page Rank, Tech Report ({{magistraleinformatica:aa2:pagerankMC.pdf| pdf}}). Integrate with David Barber’s Book [BRML], pages from 461 to 463. |
| |
| **Questions:** Describe the Pagerank algorithm from a Markov Chain point of view. Define the concepts of stationary and equilibrium distribution and discuss their interpretation in terms of Pagerank. | **Questions:** Describe the Pagerank algorithm from a Markov Chain point of view. Define the concepts of stationary and equilibrium distribution and discuss their interpretation in terms of Pagerank. |
| |
