DidaWiki

Questa pagina è in sola lettura. Puoi visualizzare il sorgente, ma non puoi modificarlo. Contatta l'amministratore se pensi che ci sia un errore.
====== Data Analytics for Digital Health (DAD) - 9 CFU A.Y. 2025/2026======
**Instructors:**
  * **Anna Monreale**
    * KDDLab, Università di Pisa
    * [[anna.monreale@unipi.it]]   
  * **Francesca Naretto**
    * KDDLab, Università di Pisa
    * [[francesca.naretto@unipi.it]]   



====== News ======

  * [08.09.2025] - Lecture of the first week will be canceled, so they will start on 22nd September 2025
  * [30.09.2025] - All Students must fill  [[https://docs.google.com/spreadsheets/d/1HFdrGb7PjjKglAFYHR1T0RJHjg5C2ufyPuysCSsfuho/edit?pli=1&gid=0#gid=0| this document ]] for exam information

====== Learning Goals ======
     * Fundamental concepts of data knowledge and discovery.
     * Data Types in Healthcare Data and Public Databases
     * Data understanding
     * Data preparation
     * Clustering
     * Classification
     * Rule-based methods
     * Outlier Detection
     * Time Series Analysis
     * Sequential Pattern Mining
====== Hours and Rooms ======

**Classes**

^  Day of Week  ^  Hour  ^  Room  ^ 
|  Monday   |   09:00 - 11:00  |  Room FIB PS4  | 
|  Tuesday  |  14:00 - 16:00    |  Room C1  | 
|  Friday   |  11:00 - 13:00 |  Room FIB PS4  | 



**Office hours - Ricevimento:**
Anna Monreale: TBD - Online using Teams or in my Office (Appointment by email).
Francesca Naretto: TBD  - Online using Teams or in my Office (Appointment by email).


A [[https://teams.microsoft.com/l/team/19%3AaixkwjuGSoUvrBNsO88NiDZsr8C2yIucNEonmj8ssSY1%40thread.tacv2/conversations?groupId=bfaf6e19-deca-4d53-921c-65b44db73608&tenantId=c7456b31-a220-47f5-be52-473828670aa1|Teams Channel]] will be used ONLY to post news, Q&A, and other stuff related to the course. The lectures will be only in presence and will **NOT** be live-streamed.
====== Learning Material -- Materiale didattico ======

===== Textbook -- Libro di Testo =====

  * Pang-Ning Tan, Michael Steinbach, Vipin Kumar. **Introduction to Data Mining**. Addison Wesley, ISBN 0-321-32136-7, 2006
    * [[http://www-users.cs.umn.edu/~kumar/dmbook/index.php]]
    * Chapters 4,6 and 8 are also available at the publisher's Web site.
  *  Jake VanderPlas. **[[http://shop.oreilly.com/product/0636920034919.do| Python Data Science Handbook: Essential Tools for Working with Data.]]** 1st Edition. 
  *  For Python Notions: {{ :magistraleinformatica:dmi:python_basics.ipynb.zip | Very basic notions on Python}} 


===== Slides =====

  * The slides used in the course will be inserted in the calendar after each class. Most of them are part of the slides provided by the textbook's authors [[http://www-users.cs.umn.edu/~kumar/dmbook/index.php#item4|Slides per "Introduction to Data Mining"]].

===== Past Excercises and past exams of similar courses  =====
  * Exercises on Clustering: {{ :dm:ex._clustering.pdf |}}
  * Excercise for DT learning simulation: {{ :magistraleinformatica:dmi:dt-learning-simulation.pdf |}} {{ :magistraleinformatica:dmi:learnedtree.pdf |}}
  * Some text of past exams of a similar course: {{ :dm:2017-1-19.pdf |}}, {{ :dm:2017-9-6.pdf |}}, {{ :dm:2016-05-30-dm1-seconda.pdf |}}, {{ :dm:dm2_exam.2017.06.13_solutions.pdf |}}, {{ :dm:dm2_exam.2017.07.04_solutions.pdf |}}, {{ :dm:dm2_mid-term_exam.2017.06.06_solutions.pdf |}}, {{ :dm:dm2_exam.2015.04.13.results.pdf|}}, {{ :dm:dm2_exam.2016.04.4_sol.pdf |}}, {{ :dm:dm2_exam.2016.04.5_sol.pdf |}}, {{ :dm:dm2_exam.2016.06.20_sol.pdf |}}, {{ :dm:dm2_exam.2016.07.08_sol.pdf |}}
   * Some very old exercises (part of them with solutions) are available here, most of them in Italian, not all of them on topics covered in this year program: {{tdm:verifica2006.pdf|Verifica 2006}}, {{tdm:verifica2005.pdf|Verifica 2005 (con soluzioni)}}, {{tdm:verifica2004.pdf|Verifica 2004}}, {{dm:verifica.05.06.2007.pdf|Verifica 5 giugno 2007}}, {{dm:verifica.26.06.2007.pdf|Verifica 26 giugno 2007}}, {{dm:verifica.24.07.2007_corretto.pdf|Verifica 24 luglio 2007}} (e {{:dm:soluzioni.2008.04.03.pdf|Soluzioni}}), {{:dm:dm-tdm.appello_2008_07_18_parte1.pdf|Verifica 18 luglio 2008 - parte 1}}, {{:dm:dm-tdm.appello_2008_07_18_parte2.pdf|Verifica 18 luglio 2008 - parte 2}},{{:dm:appello.2010.06.01_soluzioni.pdf| Exam with solution 2010-06-01}},{{:dm:appello.2010.06.22_soluzioni.pdf|Exam with solution 2010-06-22}}, {{:dm:appello.2010.09.09_soluzioni.pdf|Exam with solution 2010-09-09}},{{:dm:appello.2010.07.13_soluzioni.pdf| Exam with solution 2010-07-13}}
   

  
===== Software=====

  * Python - Anaconda (at least 3.7 version!!!): Anaconda is the leading open data science platform powered by Python. [[https://www.anaconda.com/distribution/| Download page]] (the following libraries are already included)
  * Scikit-learn: python library with tools for data mining and data analysis [[http://scikit-learn.org/stable/ | Documentation page]]
  * Pandas: pandas is an open source, BSD-licensed library providing high-performance, easy-to-use data structures and data analysis tools for the Python programming language. [[http://pandas.pydata.org/ | Documentation page]]

 
====== Class Calendar (2025/2026) ======

c
===== First Semester  =====

^ ^ Day ^ Topic ^ Learning material ^ References  ^ Teacher ^
|  |  22.09  | Strike | | | |
|  |  23.09  | CANCELED for Teacher's health issues | | | | 
|1.  |  26.09  | Overview. Introduction to Data Analyics for DH  + Data Types | {{ :digitalhealth:0-overview-2025.pdf | Overview}} {{ :digitalhealth:1-intro-da-dm-tecs.pdf |}}|Chap. 1 Kumar Book |Monreale |
|2.  |  29.09  | Data Understanding TD | {{ :digitalhealth:2-Data_Understanding.pdf | Data Understanding}}|Chap.2 Kumar Book and additioanl resource of Kumar Book: [[https://www-users.cs.umn.edu/~kumar001/dmbook/data_exploration_1st_edition.pdf|Data Exploration Chap.]] If you have the first ed. of KUMAR this is the Chap 3  |Naretto |
|3.  |  30.09  | Data Preparation TD | {{ :digitalhealth:3-data_preparation_dad.pdf | Data Preparation}} | Chap.2 Kumar Book and additional resource of Kumar Book: [[https://www-users.cs.umn.edu/~kumar001/dmbook/data_exploration_1st_edition.pdf|Data Exploration Chap.]] If you have the first ed. of KUMAR this is the Chap 3 | Monreale|
|4.  |  01.10 - Room I | Python Lab: Data Understanding & Preparation TD | | | Naretto|
|  |  03.10  | Strike | | | Naretto|
|5.  |  06.10  | Project Presentation + Data Understanding and Preparation for TD  |{{ :digitalhealth:PIMA-DU.zip | Zip file for DU e DP for TD}} {{ :digitalhealth:Integrazione.zip | Zip file for Python}}| | Naretto|
|6.  |  07.10  | Clustering: intro and k-means |{{ :digitalhealth:6-basic_cluster_analysis-intro.pdf | Intro clustering}} {{ :digitalhealth:6-basic_cluster_analysis-kmeans.pdf | kmeans}}| Chapter 7, Introduction to Data Mining, 2nd Edition by Tan, Steinbach, Karpatne, Kumar	| Naretto|
|7.  |  08.10 Room I | Clustering: hierarchical and db-scan  | {{ :digitalhealth:7.basic_cluster_analysis-hierarchical.pdf | hierarchical}} {{:digitalhealth:10-basic_cluster_analysis-dbscan.pdf | DB-scan}}| Chapter 7, Introduction to Data Mining, 2nd Edition by Tan, Steinbach, Karpatne, Kumar	| Naretto|
|     | 10.10 | suspension of teaching activities | | | |
|8.   | 13.10 | Density-based clusering + Clustering Validity | {{ :digitalhealth:12-basic_cluster_analysis-validity.pdf |}}| Chapter 7, Introduction to Data Mining, 2nd Edition by Tan, Steinbach, Karpatne, Kumar	 |Naretto |
|     | 14.10 | Canceled: No Lecture| | | |
|9.   | 17.10 | Clustering Validity + Data Warehouse | {{ :digitalhealth:6-dw.pdf |}} | | Monreale |
|10.  | 20.10 | Data Warehouse| {{ :digitalhealth:6-dw.pdf |}}|   | Monreale|
|11.  | 21.10 | Data Warehouse + PowerBI Demo|Same Slides of the previous lecture | | Monreale|
|12.  | 22.10 Room Lab I| Pre-processing for Image | {{ :digitalhealth:2.1-data-understanding_images.pdf |}}| | Naretto|
|13.  | 24.10 |Python Lab: Clustering | {{ :digitalhealth:clustering_diabetes.zip}} | |Naretto |
|14.  | 27.10 | Pre-processing for Image | {{ :digitalhealth:2.1-data-understanding_images.pdf |}} {{ :digitalhealth:kernel1.pdf |}} |Digital Image processing (Gonzales, Woods) | Naretto|
|15.  | 28.10 |Time series pre-processing | {{ :digitalhealth:5-data-understanding_ts.pdf|}}| |Monreale |
|16.  | 31.10 |Time series pre-processing, similarities and project presentation (task n.2) | {{ :digitalhealth:8_time_series_similarity_2024.pdf |}} {{digitalhealth:timeseries_similarity_clustering.zip}}| |Naretto |
|17.  | 03.11 |Image clustering and presentation of the project (task 3)| {{ :digitalhealth:3.2-clustering_images.pdf |}} {{ :digitalhealth:ecg-first-analysis.ipynb.zip}}| |Naretto |
|18.  | 04.11 |Image clustering |{{ :digitalhealth:3.2-clustering_images.pdf |}} {{digitalhealth:images_similarity.zip}}| |Naretto |
|19.  | 07.11 |Project work |  | |Monreale, Naretto |
|20.  | 10.11 |KNN and Logistic regression  | {{ :digitalhealth:10-KNN.pdf |}} {{ :digitalhealth:10-lg2025.pdf |}} | |Naretto |
|21.  | 11.11 |LG  |{{ :digitalhealth:10-lg2025.pdf |}} | |Naretto |
|22.  | 14.11 |Rule-based classifier |{{ :digitalhealth:10-Rule-Based-Classifiers.pdf |}} | |Naretto |
|23.  | 17.11 | Naive Bayes | {{:digitalhealth:11_2021-naive_bayes.pdf}}| |Naretto
|24.  | 18.11 | Naive Bayes and Ensemble methods| {{:digitalhealth:11_2021-naive_bayes.pdf}} {{:digitalhealth:13_ensemble_2023.pdf}}| |Naretto |
|25.  | 21.11 | Ensemble methods | {{:digitalhealth:13_ensemble_2023.pdf}}| |Naretto |
|26.  | 24.11 |Imbalanced learning  | {{:digitalhealth:imbalanced-learning.pdf}}| | |
|27.  | 25.11 | Python Lab on classification and presentation of the project, task 4|{{:digitalhealth:classification-diabetes.ipynb.zip}}{{:digitalhealth:imbalanced-classification.zip}} | |Naretto |
|28.  | 28.11 |GSP and Apriori |{{:digitalhealth:18_sequential_patterns_2024.pdf}} | |Monreale |
|29.  | 01.12 |GSP and Apriori |{{:digitalhealth:17_association_analysis.pdf}} | |Monreale |
|30.  | 02.12 |Time series |{{:digitalhealth:23_time_series_motif-2024.pdf}}{{:digitalhealth:matrixprofile.pdf}} {{ :digitalhealth:shaplets.pdf |}}| |Monreale |
|31.  | 05.12 |Time series lab |{{:digitalhealth:23_time_series_motif-2024.pdf}} | |Naretto |
|32.  | 09.12 |Anomaly detection tabular data |{{:digitalhealth:21_anomaly_detection_2020.pdf}} | |Naretto |
|  | 12.12 |Strike | | | |
|33.  | 15.12 |Anomaly detection ts + Python lab |{{:digitalhealth:21_anomaly_detection_2020.pdf}} {{digitalhealth:anomalydetection-1.ipynb.zip}} | | Naretto |
|34.  | 16.12 |Linear regression and forecasting |{{:digitalhealth:16_linear_regression_2020.pdf}} {{:digitalhealth:ts_forecasting_2025.pdf}} | |Naretto |
|35.  | 19.12 | Project CHECK - mandatory | | | |



====== Exams ======
The exam consists of: a **group project** (in teams of two or three) and an **oral exam** that includes a discussion of the project and an assessment of the theoretical knowledge acquired, for those who complete the project during the course and meet all intermediate and final deadlines set by the instructors. 

Alternatively, students who do not complete or submit the project within the established deadlines will be required to take a **written exam** and an **oral exam** covering all course topics.


**PROJECT**

A project consists in data analyses based on the use of data mining tools. 
The project has to be performed by a team of 2 max 3 students. It has to be performed by using Python. The guidelines require to address specific tasks. Results must be reported in a unique paper. The total length of this paper must be max 25 pages of text including figures. The students must deliver both: paper (single column) and  well commented Python Notebooks.


Deadline. January 5th, 2026.
Delivery instructions. The final deadline of the project is 5th January 2026 at 23:59. This deadline is STRICT. No extension is possible because then the winter session of exams starts. Groups that will not deliver the project by 5th January will need to do the written exam during the exam sessions. Each group must deliver by email to anna.monreale@unipi.it, francesca.naretto@unipi.it a zipped folder named DM_GroupID.zip and containing 4 folders and 1 pdf file: a folder named DM_GroupID_TASK_DU, containing source code of data understanding; a folder named DM_GroupID_TASK_CLU, containing source code of data clustering; a folder named DM_GroupID_TASK_CLA, containing source code of classification; a folder named DM_GroupID_TASK_TS, containing source code of time series analysis; a pdf file with maximum 25 pages including figures discussing the results of the tasks. The name of this file must be: DM_Report_GroupID.pdf. The file must contain the list of authors (i.e., members of the group). The subject of the email must be “DADHProject25_GroupID”

====== Previous years =====
[[DAD 2024-2025]]