Questa è una vecchia versione del documento!
Indice
<html> <!– Google Analytics –> <script type=“text/javascript” charset=“utf-8”> (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) })(window,document,'script','//www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-34685760-1', 'auto', 'personalTracker', {'allowLinker': true}); ga('personalTracker.require', 'linker'); ga('personalTracker.linker:autoLink', ['pages.di.unipi.it', 'enforce.di.unipi.it', 'didawiki.di.unipi.it'] );
ga('personalTracker.require', 'displayfeatures'); ga('personalTracker.send', 'pageview', 'ruggieri/teaching/smd/'); setTimeout(“ga('send','event','adjusted bounce rate','30 seconds')”,30000); </script> <!– End Google Analytics –> <!– Global site tag (gtag.js) - Google Analytics –> <script async src=“https://www.googletagmanager.com/gtag/js?id=G-LPWY0VLB5W”></script> <script>
window.dataLayer = window.dataLayer || [];
function gtag(){dataLayer.push(arguments);}
gtag('js', new Date());
gtag('config', 'G-LPWY0VLB5W');
</script> <!– Capture clicks –> <script> jQuery(document).ready(function(){
jQuery('a[href$=".pdf"]').click(function() {
var fname = this.href.split('/').pop();
ga('personalTracker.send', 'event', 'SMD', 'PDFs', fname);
});
jQuery('a[href$=".r"]').click(function() {
var fname = this.href.split('/').pop();
ga('personalTracker.send', 'event', 'SMD', 'Rs', fname);
});
jQuery('a[href$=".zip"]').click(function() {
var fname = this.href.split('/').pop();
ga('personalTracker.send', 'event', 'SMD', 'ZIPs', fname);
});
}); </script> </html>
Statistical Methods for Data Science A.Y. 2017/18
Instructors
- Daniele Tantari
- Scuola Normale Superiore
- Salvatore Ruggieri
- Università di Pisa
Classes
| Day of Week | Hour | Room |
|---|---|---|
| Monday | 16:00 - 18:00 | Fib-L1 |
| Tuesday | 9:00 - 11:00 | Fib-N1 |
Office hours
- Prof. Tantari: Tuesday h 11:00 - 15:00, Scuola Normale Superiore, room 93 (please send an email in advance)
- Prof. Ruggieri: Tuesday h 14:00 - 17:00, Department of Computer Science, room 321/DO.
Text Books
The following are mandatory text books:
- [B1] F.M. Dekking C. Kraaikamp, H.P. Lopuha, L.E. Meester. A Modern Introduction to Probability and Statistics. Springer, 2005.
- [B2] P. Dalgaard. Introductory Statistics with R. 2nd edition, Springer, 2008.
The following is an optional text book for recalling mathematics pre-requisites of the course:
- [B3] J. Ward, J. Abdey. Mathematics and Statistics. University of London, 2013. Chapters 4-8 of Part 1 present basic calculus (derivatives and integrals).
Software
Preliminary program and calendar
Project
- Project can be done in groups of at most 3 students.
- Project final delivery dates: 11 June h 9, 2 July h 9, 23 July h 9
- Oral dates: 12 June, 3 July, 24 July
- Students doing the project will skip the written exam but they have to register for the written dates in order to fill the student's questionnaire.
- AIDA database (max 2 concurrent connections)
- Ateco 2007 classification: .xls, note esplicative, volume integrale
- Script for loading and transforming data Updated 15/5/2018
Written exam
Written exam consists of open questions and exercises. Example text: sample1, sample2. The exam lasts 2 hours. No teaching material can be consulted during the exam. Registration is mandatory.
| Date | Hour | Room |
|---|---|---|
| 22/1/2019 | 9:00 - 11:00 | Fib-L1 |
| 12/2/2019 | 9:00 - 11:00 | Fib-L1 |
Class calendar (final)
| Day | Room | Topic | Learning material | Instructor | |
|---|---|---|---|---|---|
| 1. | 19.02 16:00-18:00 | L1 | Introduction. Probability and independence. | [B1] Chpts. 1-3 | Tantari |
| 2. | 20.02 9:00-11:00 | N1 | R basics. | [B2] Chpts. 1,2.1,2.4 slides script1.R | Ruggieri |
| 3. | 27.02 9:00-11:00 | N1 | Discrete and continuous random variables. | [B1] Chpts. 4-5 | Tantari |
| 4. | 06.03 9:00-11:00 | N1 | Simulation. Expectation and variance | [B1] Chpts. 6-7 noteSim | Tantari |
| 5. | 12.03 16:00-18:00 | L1 | R basics and distributions. | [B2] Chpts. 2.2,3-4 script2.R | Ruggieri |
| 6. | 13.03 9:00-11:00 | N1 | R programming and graphics. | [B2] Chpts. 2.3,3-4 exercise.R script3.R | Ruggieri |
| 7. | 19.03 16:00-18:00 | L1 | Computations with random variables. Covariance | [B1] Chpts. 8-10 | Tantari |
| 8. | 20.03 9:00-11:00 | N1 | Sum of random variables. Law of large numbers | [B1] Chpts. 11,13 | Tantari |
| 9. | 26.03 16:00-18:00 | L1 | The central limit theorem. Graphical summaries | [B1] Chpts. 14,15 | Tantari |
| 10. | 27.03 9:00-11:00 | N1 | Numerical summaries. Poisson process | [B1] Chpts. 12,16 Rcode slides | Tantari |
| 11. | 16.04 16:00-18:00 | L1 | Examples on CLT. Data preprocessing. | [B2] Chpt. 10 dataprep.r script4.R | Ruggieri |
| 12. | 17.04 9:00-11:00 | N1 | Unbiased estimators. Efficiency and MSE | [B1] Chpts. 17,19, 20 | Tantari |
| 13. | 23.04 16:00-18:00 | L1 | Maximum likelihood. | [B1] Chpt. 21 | Tantari |
| 14. | 24.04 9:00-11:00 | N1 | Fisher Information. Linear Regressions and Least Squares. | [B1] Chpt. 22 fisher | Tantari |
| 15. | 30.04 16:00-18:00 | L1 | Examples on and MSE. Power-laws | Newman's paper, roc_adult.R script5.R | Ruggieri |
| 16. | 02.05 14:00-16:00 | A1 | Project and data presentation | Tantari+Ruggieri | |
| 17. | 07.05 16:00-18:00 | L1 | Confidence Intervals: Gaussian, T-student, large sample method. | [B1] Chpt. 23,24 | Tantari |
| 18. | 08.05 9:00-11:00 | N1 | Empirical and parametric bootstrap. Application to confidence intervals. | [B1] Chpts. 18,23 | Tantari |
| 19. | 14.05 16:00-18:00 | L1 | Hypotheses testing. | [B1] Chpts. 25-26 | Tantari |
| 20. | 15.05 9:00-11:00 | N1 | Hypotheses testing. Bootstrap. Project tutoring. | [B2] Chpt. 5.1, script6.R | Ruggieri |
| 21. | 21.05 16:00-18:00 | L1 | Hypotheses testing. t-test and application to linear regressions | [B1] Chpts. 27 | Tantari |
| 22. | 22.05 9:00-11:00 | N1 | Hypotheses testing: correlation and Fisher transformation, comparing samples | [B1] Chpt. 28 CorrNotes | Tantari |
| 23. | 28.05 16:00-18:00 | L1 | Hypotheses testing: F-test, K-S, chi-square | K-S | Tantari |
| 24. | 29.05 9:00-11:00 | N1 | Hypotheses testing, parameter estimation. | [B2] Chpts. 5.2-5.7, 6, script7.R | Ruggieri |