====== Elaborazione del Linguaggio Naturale ======
** Laurea Magistrale**: [[http://compass2.di.unipi.it/didattica/wif18/|Informatica]].

**Docente**: [[http://www.di.unipi.it/~attardi|Giuseppe Attardi]]
**Ricevimento**: Friday, 11:00

^  Schedule  ^^^
^ Day ^  Hour ^ Room ^ 
| Thursday| 11-13 | N1, Polo Fibonacci |
| Friday| 16-18 | N1, Polo Fibonacci |

====== Forum ======

Forum on [[https://piazza.com/class/itzo486y1fz2e1|Piazza]]

====== Prerequisites ======

  - Basic Probability and Statistics
  - Programming


====== Syllabus======

  - Introduction
    - History
    - Present and Future
    - NLP and the Web
  - Mathematical Background
    - Probability and Statistics
    - Language Model
    - Hidden Markov Model
    - Viterbi Algorithm
    - Generative vs Discriminative Models
  - Linguistic Essentials
      - Part of Speech and Morphology
      - Phrase structure
      - Collocations
      - n-gram Models
      - Word Sense Disambiguation
      - Word Embeddings
  - Preprocessing
    - Encoding
    - Regular Expressions
    - Segmentation
    - Tokenization
    - Normalization
  - Machine Learning Basics
  - Text Classification and Clustering
  -Tagging
    - Part of Speech
    - Named Entity
  - Sentence Structure
    - Constituency Parsing
    - Dependency Parsing
  - Semantic Analysis
  - Statistical Machine Translation
  - Deep Learning
  - Libraries
    - NLTK
    - Theano/Keras
    - Tensorflow
  - Applications
    - Opinion Mining
    - Entity Linking
    - Semantic Search
    - Question Answering
    - Language Inference

====== Lecture Notes ======

^ Date ^ Lecture ^ Notes ^
| 22/9/2016 | L'età della parola| {{:magistraleinformatica:eln:etadellaparola.pptx|L'età della parola}} |
| 23/9/2016 | Introduction |{{:magistraleinformatica:eln:1-intro.pptx|Introduction}} |
| 29/9/2016 | Introduction to probability ({{:magistraleinformatica:eln:2-probability.ppt|Probability}})| |
| 30/9/2016 | Language Modeling ({{:magistraleinformatica:eln:2-languagemodeling.ppt|Language Modeling}})| {{:magistraleinformatica:eln:languagemodel.ipynb.zip|Jupyter Notebook}}|
| 6/10/2016 | Python Tutorial ({{:magistraleinformatica:eln:3-python.ppt|Tutorial}}) | [[https://github.com/rajathkumarmp/Python-Lectures|Python Tutorial Notebook]]|
| 7/10/2016 | Python Tutorial and examples ({{:magistraleinformatica:eln:3-python-generators.pdf|Python: Functionals and Generators}}) |{{:magistraleinformatica:eln:homework_12-1.pdf|Homework 1}} |
| 13/10/2016 | Introduction to NLTK ({{:magistraleinformatica:eln:4-nltk.ppt|slides}}) | |
| 14/10/2016 | Segmentation and Tokenization ({{:magistraleinformatica:eln:5-tokenization.ppt|slides}}) | |
| 20/10/2016 | Text Classification ({{:magistraleinformatica:eln:4-textclassification.pptx|slides}})| |
| 21/10/2016 | Naive Bayes Classifier ({{:magistraleinformatica:eln:4-textclassification.pptx|slides}}) |{{:magistraleinformatica:eln:homework_16-2.pdf|Homework n. 2}} |
| 27/10/2016 | Maximum Entropy Models ({{:magistraleinformatica:eln:8-classifiers.ppt|slides}}) | |
| 28/10/2016 | Hidden Markov Model ({{:magistraleinformatica:eln:6-hmm.ppt|slides}}) | |
| 10/11/2016 | Named Entity Recognition ({{:magistraleinformatica:eln:7-ner.ppt|slides}}) | |
| 11/11/2016 | MEMM ({{:magistraleinformatica:eln:7-ner.ppt|slides}}) | |
| 17/11/2016 | Perceptron, SVM ({{:magistraleinformatica:eln:8-classifiers.ppt|}}) | |
| 19/11/2016 | Deep Learning Libraries | {{:magistraleinformatica:eln:keras-mnist-tutorial.zip|}}|
| 24/11/2016 | Dependency Formalism ({{:magistraleinformatica:eln:esslli1.pdf|slides}}) | |
| 25/11/2016 | Dependency Parsing ({{:magistraleinformatica:eln:11-parsing.ppt|Transition Based}})| {{:magistraleinformatica:eln:topicsandprojects.ppt|Topics for Seminars and Projects}} |
| 1/12/2016 | Dependency Parsing ({{:magistraleinformatica:eln:esslli4.pdf|Graph Based }})| |
| 2/12/2016 | Sentiment Analysis{{:magistraleinformatica:eln:13-opinionmining.ppt|}} |  |
|  | Deep Learning [[http://nlp.stanford.edu/courses/NAACL2013/NAACL2013-Socher-Manning-DeepLearning.pdf|Deep Learning Tutorial at NAACL 2013]] | [[https://github.com/jdwittenauer/ipython-notebooks/tree/master/notebooks|ML Course by Andrew Ng]]|
|  | Deep Learning for NLP {{:magistraleinformatica:eln:deepnl.pptx| DL and the DeepNL Library}} | |
| 15/12/2016 | Machine Translation ({{:magistraleinformatica:eln:13-smt.ppt|MT}}) ({{:magistraleinformatica:eln:14-smt.ppt|PBMT}})| |
|  | PB SMT ({{:magistraleinformatica:eln:phrase-model.pdf|Phrase Tables}}, {{:magistraleinformatica:eln:phrase-decoding.pdf|Decoding}}, {{:magistraleinformatica:eln:evaluation.pdf|Evaluation}}) | |
| 16/12/2016 | {{:magistraleinformatica:eln:universaldependencies.ppt| Universal Dependencies}} | |
|  | State of the Art in Sentiment Analysis of Tweets [[http://www.saifmohammad.com/WebDocs/web-Atlanta-Talk-SemEval2013-Sentiment.pdf|NRC Canada at SemEval 2013]] | |
|  | Relation Extraction {{:magistraleinformatica:eln:12-relextraction.ppt|}} | |
|  | {{:magistraleinformatica:eln:tsunamidlovernlp.pptx|The tsunami of Deep Learning over NLP }}| |
|  | Andrew Ng | [[https://www.dropbox.com/s/dyjdq1prjbs8pmc/NIPS2016%20-%20Pages%202-6%20(1).pdf|Nuts and Bolts of Building Applications using Deep Learning]] |
====== Textbooks ======

  - C. Manning, H. Schutze. [[http://nlp.stanford.edu/fsnlp/promo/|Foundations of Statistical Natural Language Processing]]. MIT Press, 2000.
  - D. Jurafsky, J.H. Martin, [[http://www.cs.colorado.edu/~martin/slp2.html|Speech and Language Processing]]. 2nd edition, Prentice-Hall, 2008. See also upcoming 3rd edition.
  - S. Kubler, R. McDonald, J. Nivre. [[http://www.morganclaypool.com/doi/abs/10.2200/S00169ED1V01Y200901HLT002|Dependency Parsing]]. 2010.
  - P. Koehn. [[http://www.statmt.org/book/|Statistical Machine Translation]]. Cambridge University Press, 2010.
  - S. Bird, E. Klein, E. Loper. [[http://www.nltk.org/book| Natural Language Processing with Python]].
  - I. Goodfellow, Y. Bengio, A. Courville. [[http://www.deeplearningbook.org/| Deep Learning]]. MIT Press, 2016.
  - M. Nielsen. [[http://neuralnetworksanddeeplearning.com| Neural Networks and Deep Learning]].

====== Related Courses ======

[[http://web.stanford.edu/class/cs224n/|Natural Language Processing with Deep Learning]], Stanford.

[[https://www.youtube.com/playlist?list=PL3FW7Lu3i5Jsnh1rnUwq_TcylNr7EkRe6|Natural Language Processing with Deep Learning, video lectures]], Stanford.
====== Exam ======
Project or seminar.

{{:magistraleinformatica:eln:topicsandprojects.ppt|Topics for projects and seminars}}

====== Evalita 2015 Projects ======

   * SentiPolc: Sentiment Polarity in Twitter. Primo classificato.
   * NeelIt: Named Entity Recognition and Linking in Twitter. Primo classificato
====== Affine courses ======

  - Apprendimento Automatico: Fondamenti
  - Data Mining: fondamenti
  - Information Retrieval
  - Sistemi Basati sulla Conoscenza

====== Edizioni Precedenti ======

  * [[http://didawiki.di.unipi.it/doku.php/magistraleinformatica/eln/start?rev=1307549824|2010-2011]]
  * [[http://didawiki.di.unipi.it/doku.php/magistraleinformatica/eln/start?rev=1335197471|2011-2012]]
  * [[http://didawiki.di.unipi.it/doku.php/magistraleinformatica/eln/start?rev=1370280570|2012-2013]]
  * [[http://didawiki.di.unipi.it/doku.php/magistraleinformatica/eln/start?rev=1400671809|2013-2014]]
  * [[http://didawiki.di.unipi.it/doku.php/magistraleinformatica/eln/start?rev=1433408423|2014-2015]]
  * [[http://didawiki.di.unipi.it/doku.php/magistraleinformatica/eln/start?rev=1474461123|2015-2016]]