DidaWiki

Questa pagina è in sola lettura. Puoi visualizzare il sorgente, ma non puoi modificarlo. Contatta l'amministratore se pensi che ci sia un errore.
====== Strumenti di programmazione per sistemi paralleli e distribuiti ======

** Teacher: ** Dr. Massimo Coppola

**Contact info**\\
Official CNR-ISTI web page [[https://www.isti.cnr.it/en/about/people-detail/71/Massimo_Coppola|Massimo Coppola at ISTI]]\\
e-mail : massimo.coppola@isti.cnr.it \\
Phone :  050 315 2992 \\
Office location :  CNR Reseach Area, ISTI-CNR, Building B, room C-33\\ 

^ **IMPORTANT: the first lesson will be on THURSDAY 19/02/2026**   |
^  **Interested students are kindly asked to contact the teacher via email.**  |


**Course info**\\
  * This page concerns the SPD course (535AA) for the academic year 2025-2026, which gives you **6 credits**.
  * Be sure to alway check the NEWS section in this page.
  * Information about latest past editions of the SPD course can be found at the following links.

^ [[magistraleinformaticanetworking:spd:spd-18-19|SPD 2018-2019]] ^ [[magistraleinformaticanetworking:spd:spd-20-21|SPD 2020-2021]] ^ [[magistraleinformaticanetworking:spd:spd-21-22|SPD 2021-2022]]  ^ [[magistraleinformaticanetworking:spd:spd-22-23|SPD 2022-2023]] ^ [[magistraleinformaticanetworking:spd:spd-24-25|SPD 2024-2025]]  |

==== Timetable ====


{{  http://fpga.org/wp-content/uploads/2017/01/1-32-1680-RISC-Vs.jpg?250|1-core, 32-core and 1680-core RISC-V development boards.
See http://fpga.org/2017/01/12/grvi-phalanx-joins-the-kilocore-club/ }}


^ Thursday |  **09:00-11:00** | Room FIB X3 |
^ Friday |  **14:00-16:00** | Room FIB X3 |   

[[https://unipi.prod.up.cineca.it/calendarioPubblico/linkCalendarioId=6319dae0e209821793111be9|Here the official time table]] 

  * See also the Dept. [[https://didattica.di.unipi.it/laurea-magistrale-in-informatica-e-networking/orario-magistrale-informatica-e-networking/|official timetable]] 
  * In case of exceptions, notice will be given in the News section below in this page and by email to the students. 

** Question time **\\
Please contact the teacher to set up either individual or joint telco sessions, or in person meetings.

=== News and Updates ===
   * 26/02/2024 First lesson will be held next Thursday 19/2, 9:00-11:00.
   * 06/03/2026 Starting next week the lessons will be recorded and uploaded to the official MS Teams channel

==== Course Journal ====

The [[magistraleinformaticanetworking:spd:lezioni25.26|course journal]] is a separate page in this wiki.

--------
==== Aim of the Course ====
The course will provide a description and analysis of a few key parallel and distributed programming platforms and models, starting from their theoretical foundations, where not covered by previous courses, and focusing on (1) existing standards for platforms and programming systems and (2) State-of-the-art solutions. The course will include  practical use of those systems to develop simple applications.

^ **The overall program reported below is a preliminary one** |  

  - The course this year will be more flexible in the topics and in the definition of the students' projects. 
  - Specific topics can be expanded according to the interest of the students,
  - Interdisciplinary projects proposed by the student will be evaluated for the final assignment.

==== Overall Program ====

The course will cover the following topics.

  * Parallel Programming systems/frameworks
     * **MPI** message-passing programming (the core part of the MPI standard)
     * **ONEAPI** as a unifying meta-approach toward programming several architectural layers, encapsulating other existing programming frameworks
    * **TBB** Thread Building Blocks. Multicore oriented, shared-memory programming framework
    * **SYCL** - Common source CPU/GPU C++ programming framework
    * Massively Multicore computation and GPU programming frameworks: mainly **OpenCL** but also references to **CUDA** 
  * other topics and parallel/distributed formalisms we may partially address
    * Software defined processors: FPGA-based open source processors, OpenCL to FPGA compilation
    * High-Level SPP languages for Clusters/Clouds, dynamic and autonomic management
    * SPC++
    * BSP-based approaches (e.g. Apache Hama / Giraph, or MulticoreBSP)
  * Example Applications may include 
    * Data mining (K-means, classification…), 
    * machine learning algorithms
    * computational simulation algorithms
  * Further technology topics 
    * Managing HPC experiments with SLURM
    * Scripting HPC applications with Python

We may cover other Foundational aspects and technologies, depending on the specific year and students' interests or that may be exploited in the project work.

  * Elementary mechanisms and theory of distributed computation: message passing, shared memory, massive multicore. 
  * Programming frameworks or libraries for parallelising/distributing computation: e.g. AMD ROCm, NVIDIA CUDA.
  * Basics of scheduling algorithms and resource management in different field (HPC; Cloud , Continuum)
  * Basics of Service Oriented Architectures SOA
  * Service oriented Platforms, Cloud Computing and Cloud Federations

--------

==== Project Work ====

TBD

--------

==== Teaching material ====
=== Books ===

  * Latest **MPI Standard** from [[https://www.mpi-forum.org/docs/| MPI Forum]]\\The course mostly relies on MPI 3.1, 2015 features, and of those only some parts are needed, that we will specify during the lessons. Please always check the related MPI standard errata.\\ On the [[http://www.mpi-forum.org/|MPI forum web site]] you will find //alternate formattings//  and //translations// of the same material. 
  * **M. Voss, J. Reinders -  Today's TBB (C++ Parallel Programming with Threading Building Blocks)**.\\ Springer Open access book available at [[https://link.springer.com/book/10.1007/979-8-8688-1270-5|Today's TBB]]
  * //Older reference//  //M. Voss, R. Asenjo, J. Reinders - Pro TBB (C++ Parallel Programming with Threading Building Blocks)//. Open access OAPEN book available at [[http://library.oapen.org/handle/20.500.12657/22838| OAPEN library]]
  * **B. Wilkinson, M. Allen -- Parallel Programming**, 2nd edition. 2005, Prentice-Hall.\\ It is available in the University Library of Math/Comp.Sc./Physiscs, under code C.1.2 w74 INF .
  * **Michael Mc Cool, Arch D. Robinson and James Reinders -- Structured Parallel Programming (patterns for Efficient Computation)** 2012, Morgan Kaufmann. \\ Chapters 1 to 3 cover background topics which should be already known from previous courses (SPA, SPD courses). Stundents need to focus on the TBB material throughout the book: Appendix C and D, and the TBB examples in the book that appendix C references from chapters 4, 5, 8 and 9. Check also Chapter 11 on k-means. 
  *  **J. Reinders et al. - Data Parallel C++ (Programming Accelerated Systems Using C++ and SYCL)**\\ Open access book on Springer Open Access, available at the URL [[https://link.springer.com/book/10.1007/978-1-4842-9691-2|Data Parallel C++]]
  * //Alternate book:// An introduction to TBB is also found in //James Reinders -- Intel Threading Building Blocks// 2007, O'Reilly Media.\\ More focused on TBB alone, but describes a quite old release of the framework, hence you need to look at online documentation for some of the features.

=== Papers and reading material ===

Examples: 
  * **Parallel k-means** Dhillon & Modha Technical Report about parallel K-means, published as: A Data-Clustering Algorithm On Distributed Memory Multiprocessors. I.S. Dhillon, D.S.Modha, LNAI 1759, pag 245. Published version has issues, refer to the Tech. rep. 
  * **Parallel Clustering** Large-Scale Parallel Data Clustering. Dan Judd, Philip K. Mckinley, Anil K. Jain. Ieee Transaction On Pattern Analysis And Machine Intelligence, Vo. 20, No. 8 August 1998.