DidaWiki

The course covers the programming models and the paradigms used with distributed and parallel systems, for both the application and support tool software. The main focus of the course is on structured programming models (parallel design pattern, algorithimc skeletons) as the main API provided to the application programmers to develop parallel applications. The implementation of structured parallel programming models will be detailed in terms of models and support mechanisms. The models and mechanisms considered include classical models (client/server, master/worker, map/reduce) and mechanisms (POSIX TCP/IP, OpenMP, MPI, etc). While presenting models and paradigms, special emphasis will be given to different aspects related to either the programming model presented to the final user or the particular implementation mechanism chosen, namely

• separation of concerns (application vs. system programmers, functional vs. non functional concerns)
• program refactoring techniques
• modularity, software reuse, incremental design
• performance modelling and portability
• fault tolerance and adaptivity
• targeting of different, possibly heterogeneous, architectures, including co-processor exploitation.

The program of the course is divided into parts:

• Part I: Introduction Motivation of the course, hardware advances and software gap.
• Part II: Foundations Parallel and distributed computing measures & mechanisms (control, communications, synchronization).
• Part III: Parallel patterns and algorithmic skeletons Parallelism classification, parallel design, Skeleton frameworks, implementation models.
• Part IV: Implementing parallel and distributed patterns Performance modelling, support paradigms, parallelism related problems (cache and memory management, load balancing, communication management, accelerator management).
• Part V: Optimizations Refactoring, Adaptivity, Portability, Energy management.

One third of the total course hours will be related to practical exercises, using state-of-the-art structured parallel programming frameworks. This year two of the nine total credits will be given by the assistant prof. Massimo Torquati, covering a large part of this “practical” lessons of the course.

The whole program will be covered by the course notes material.