Differenze

Queste sono le differenze tra la revisione selezionata e la versione attuale della pagina.

--- magistraleinformatica:dmi:start [18/09/2024 alle 09:33 (13 mesi fa)] – Update teaching material, and office hours Mattia Setzu
+++ magistraleinformatica:dmi:start [22/10/2025 alle 16:54 (33 ore fa)] (versione attuale) – added corels lecture Mattia Setzu
@@ Linea 1: / Linea 1: @@
-====== Data Mining (309AA) - 9 CFU A.Y. 2024/2025 ======
+====== Data Mining (309AA) - 9 CFU A.Y. 2025/2026 ======
 **Instructors:**
@@ Linea 12: / Linea 12: @@
   * * **Lorenzo Mannocci**
     * University of Pisa
-    * [[lorenzo.mannocci@phd.unipi.it]]
+    * [[lorenzo.mannocci@di.unipi.it]]
 ====== News ======
-  * [14.09.2024] ** The lectures will start on 19th September 2024**
+[23-09-2025]: Please register yourself and your group for the project .Group registration available  [[https://docs.google.com/spreadsheets/d/1Xl8Hd-giIuJQw0x2NDkXjbGZ2REGF-OukqC5XGU6pzA/edit?gid=0#gid=0|here]].
 ====== Learning Goals ======
-     * Fundamental concepts of data knowledge and discovery.
+The Data Mining course tackles the analysis of large collections of data, and the extraction of information and patterns. It aims to explore core components of the Knowledge Discovery from Data (KDD) process, and focuses on:
-     * Data understanding
+  * Data understanding
-     * Data preparation
+  * Data cleaning, preparation, and transformation
-     * Clustering
+  * Data analysis: outlier detection and data representation
-     * Classification
+  * Data clustering
-     * Pattern Mining and Association Rules
+  * Pattern extraction: itemset, rules, association rules, and sequential patterns
-     * Outlier Detection
+  * Inference models: trees, and ensemble models
-     * Time Series Analysis
+  * Responsible data use: privacy and interpretability
-     * Sequential Pattern Mining
-     * Ethical Issues
 ====== Schedule ======
@@ Linea 34: / Linea 32: @@
 ^  Day of Week  ^  Hour  ^  Room  ^
-|  Tuesday   |  11:00 - 13:00  |  Room C1  |
+|  Tuesday   |  11:00 - 13:00  |  Room C  |
-|  Thursday  |  09:00 - 11:00  |  Room C  |
+|  Wednesday |  14:00 - 16:00  |  Room C  |
-|  Friday    |  09:00 - 11:00  |  Room C1  |
+|  Thursday  |  14:00 - 16:00  |  Room A1  |
 **Office hours - Ricevimento:**
-  * Anna Monreale: TBD
+  * Anna Monreale:TBD- Online using Teams or in my Office (Appointment by email).
   * Mattia Setzu: Infos on [[https://unimap.unipi.it/cercapersone/dettaglio.php?ri=177323&template=dett_didattica.tpl|Unimap]]
-A [[https://teams.microsoft.com/l/team/19%3Aq8IK5DrzMwEE5TxVhuw4QdYEVFJ06KVITI5jSJTmaJ81%40thread.tacv2/conversations?groupId=5fae2fa6-38fd-414f-a0c9-ffbd8e6f0710&tenantId=c7456b31-a220-47f5-be52-473828670aa1|Teams Channel]] will be used ONLY to post news, Q&A, and other stuff related to the course. The lectures will be only in presence and will **NOT** be live-streamed, but recordings of the lecture or of the previous years will be made available here for non-attending students.
+A [[ https://teams.microsoft.com/l/team/19%3Ai_Ge38xXm8FdnepLNud6ddbz_OECbBPRKfA1UKbUsQo1%40thread.tacv2/conversations?groupId=41e56778-e965-462a-9fef-250df0ee7055&tenantId=c7456b31-a220-47f5-be52-473828670aa1|Teams Channel]] will be used ONLY to post news, Q&A, and other stuff related to the course. The lectures will be only in presence and will **NOT** be live-streamed.
 ====== Teaching Material ======
@@ Linea 68: / Linea 67: @@
 The slides used in the course will be inserted in the calendar after each class. Some are part of the slides provided by the textbook's authors [[http://www-users.cs.umn.edu/~kumar/dmbook/index.php#item4|Slides per "Introduction to Data Mining"]].
+====== Class Calendar (2025/2026) ======
+===== First Semester  =====
+^ ^ Day ^ Topic ^ Teaching material ^ References ^ Teacher ^
+|1.  |  18.09  | Course Overview. Introduction to Data Mining |  {{ :magistraleinformatica:dmi:intro_dm.pdf |Introduction to DM}} | Chap. 1 Kumar Book | Setzu  |
+|    |  23.09  | Canceled for Teacher's health issues         |   |  | |
+|2.  |  24.09  | Data Understanding + Data Preparation        | {{ :magistraleinformatica:dmi:data_understanding.pdf |}} {{ :magistraleinformatica:dmi:data_preparation_and_cleaning.pdf | Data Preparation}}| Chap. 2 Kumar Book and additioanl resource of Kumar Book: [[https://www-users.cs.umn.edu/~kumar001/dmbook/data_exploration_1st_edition.pdf|Data Exploration Chap.]] If you have the first ed. of KUMAR this is the Chap 3 |Setzu |
+|3.  |  25.09  | Data representation      |{{ :magistraleinformatica:dmi:data_representation.pdf |}} | References: Introduction to linear algebra (Sections 1, 3.1, 4.2, 6.1, 6.4, 6.5, 7.3), [[https://www.jmlr.org/papers/volume9/vandermaaten08a/vandermaaten08a.pdf|t-SNE paper]], [[https://arxiv.org/abs/1802.03426 | UMAP paper (Section 3)]]  |Setzu |
+|4.  |  30.09  | Data Cleaning + Transformations. PyLab: Data Understanding     | {{ :magistraleinformatica:dmi:5-data_cleaning_transformation.pdf | Data Cleaning & Transformations}}| | Monreale, Mannocci |
+|5.  |  01.10  | PyLab: Data Understanding + Preparation    |{{ :magistraleinformatica:dmi:1_basics_and_understanding.ipynb.zip |}} {{ :magistraleinformatica:dmi:2_feature_engineering_and_data_representation.ipynb.zip |}} {{ :magistraleinformatica:dmi:data_notebook.zip |}}| | Monreale, Mannocci |
+|6.  |  02.10  | Similarities + Introduction to Clustering and Centroid-based clustering  | {{ :magistraleinformatica:dmi:6-data_similarity.pdf |}} {{ :magistraleinformatica:dmi:6-basic_cluster_analysis-intro.pdf |}} {{ :magistraleinformatica:dmi:8-basic_cluster_analysis-kmeans.pdf |}}| | Monreale |
+|7.  |  07.10  | K-means   | {{:magistraleinformatica:dmi:8-basic_cluster_analysis-kmeans.pdf |}}}| | Monreale |
+|8.  |  08.10  | Hierarchical Clustering + Density Based Clustering + Validity   | {{ :magistraleinformatica:dmi:9-basic_cluster_analysis-hierarchical.pdf |}} {{ :magistraleinformatica:dmi:8.basic_cluster_analysis-dbscan-validity.pdf |}} |  | Monreale |
+| 9. | 14.10 | Clustering evaluation and Python notebooks | {{ https://didawiki.cli.di.unipi.it/lib/exe/fetch.php/magistraleinformatica/dmi/12-basic_cluster_analysis-validity.pdf | Clustering validation}} {{ :magistraleinformatica:dmi:3_clustering.ipynb.zip |}} | | Setzu, Mannocci |
+| 10. | 15.10 | Anomaly detection | {{ https://github.com/data-mining-UniPI/teaching25/blob/lectures/anomaly%20detection/Anomaly%20detection.html.pdf | Slides }} | | Setzu |
+| 11. | 16.10 | Anomaly detection | {{ https://github.com/data-mining-UniPI/teaching25/blob/lectures/anomaly%20detection/Anomaly%20detection.html.pdf | Slides }}, {{ https://github.com/data-mining-UniPI/teaching25/blob/main/notebooks/outliers.ipynb | Notebook }}, {{ https://github.com/data-mining-UniPI/teaching25/blob/main/notebooks/isolation_forest.py | Rule extraction from isolation forests }} | | Setzu |
+|12.  |  21.10  | Variants of K-means + Association Rule Mining | {{ :magistraleinformatica:dmi:11-basic_cluster_analysis-kmeans-variants.pdf |}} {{ :magistraleinformatica:dmi:17_association_analysis2023.pdf |}} | | Monreale  |
+|14.  |  23.10  | Association Rule Mining: CORELS | {{ https://github.com/data-mining-UniPI/teaching25/blob/lectures/rule_mining/Rule%20extraction.html.pdf | Slides }}, {{ https://corels.cs.ubc.ca/corels/index.html | Online tool }} | | Setzu  |
-**Software**
-Software material available in the [[https://github.com/data-mining-UniPI/teaching23|Github repository]] (available in the coming days).
-====== Class Calendar (2024/2025) ======
-===== First Semester  =====
+====== Exam ======
+The exam can be taken in one of two ways:
+**Project track**:
+  * Project (70% of the final score) to be delivered after the end of the course
+  * Oral exam (30% of the final score)
+During the course, you will have some “Project presentation” sessions wherein you’ll briefly (~3 minutes) present your work, and receive feedback from the lecturers. These sessions do not contribute to your grade.
+**Written test track**
+  * Written exam (70% of the final score): to be delivered after the end of the course during the exam sessions and can include both theoretical questions and exercises.
+  * Oral exam (30% of the final score)
+Note that a passing grade for the project/written exam is required to be admitted to the oral exam.
+**Project Guidelines:**
+A project consists in data analyses based on the use of data mining tools.
+The project has to be performed by a team of 3 students. It has to be performed by using Python. The guidelines require to address specific tasks. Results must be reported in a unique paper. The total length of this paper must be max 25 pages of text including figures. The students must deliver both: paper (single column) and  well commented Python Notebooks.
+Specifically, if any of these tasks appear in the project track, make sure to focus on the following:
+**Data understanding**
+  * An analysis of all variables, their relations, distributions, and quality
+  * An eventual feature imputation and/or selection
+  * The engineering of additional features, including the aforementioned analyses
+**Clustering Analysis**
+  * A properly justified feature selection phase
+  * Tackling all clusternig families, exploring their respective hyperparameters
+  * An analysis of the best clusterings per family, including cluster description
+  * A comparison of the best clusterings per family
+**Anomaly detection**
+  * A selection of outliers through appropriate algorithms
+  * An interpretation of such outliers
+  * An analysis of the impact of the outliers on the previously performed data understanding
+**Time series analysis**
+  * Appropriate representation choice for the task at hand
+**Supervised learning**
+  * Feature selection
+  * Test different families of models
+  * Proper model validation, including both model performance and model complexity
+  * Comparison of the best models of each family
+**Explainability**
+  * Justified selection of instances to explain
+  * Analysis of the explanations
+**Project and Deadlines**
+Information about the dataset to be analyzed and project description:
+  * **Dataset.** https://drive.google.com/file/d/1K9garfm03-PFUMYyOenH9kqEJ7D5RrmD/view?usp=sharing
+  * **Project description.** {{ :magistraleinformatica:dmi:data_mining_project.pdf |}}
+  * **Project Question & Answers.** https://docs.google.com/spreadsheets/d/1D6lMKJTGNtMiUuNGFsrQwMPQgAjgAEflw8F5_LNQlXM/edit?usp=sharing
+  * **Deadline.**
+  * **Delivery instructions.**
-^ ^ Day ^ Topic ^ Learning material ^ References ^ Video Lectures ^ Teacher ^
-|    |  17.09  | Candeled   |  |   |
-|1.  |  19.09  | Overview. Introduction to KDD   |  |Chap. 1 Kumar Book | | |
-====== Exams ======
-TBD
 ====== Previous years =====
+[[DM-INF 2024-2025]]
 [[DM-INF 2023-2024]]