Strumenti di programmazione per sistemi paralleli e distribuiti

Teacher: Dr. Massimo Coppola

Contact info
e-mail : massimo.coppola@isti.cnr.it
Phone : 050 315 2992
Office : CNR Reseach Area, ISTI-CNR, Building C, room 33

Course info

• This page concerns the SPD course (535AA) for the academic year 2012-2013, which gives you 6 credits.
• The previous SPD course (308AA, 9 credits) can still be taken by those who had it in their study plan since previous years. Different program and possibly different rules for the final test will be applied in that case.
• Be sure to alway check the NEWS section in this page.
• Information about past editions of the SPD course can be found at the following links.

This is the provisional timetable from 24/09/2012

• Lesson schedule is subject to change in the first weeks to reduce time conflict with other courses, according to the needs of students.
• In case of exceptions, notice will be given in the News section below in this page.

Question time
Not yet defined for this year. Students should contact the teacher in advance by phone / e-mail. A different time can be agreed upon if needed.

• 17/12/12 Lesson for tomorrow is canceled
• 10/12/12 Extra lesson confirmed for today, Monday 10/12, 16-18 Room L, as well as for next Monday 17/12, 15-18 Lab Room I.
• 04/12/12 Lesson canceled due to illness. I apologize for the short notice.
• 03/12/12 Additional lesson today is 16-18 in Room H. The Room is reserved from 14 to 18 though, due to an error in the reservation, so it is available to the students.
• 19/11/12 Lesson canceled. Today's lesson has to be canceled. Notice was given this morning via the Polo Fibonacci personnel. Tomorrow there will be a lab session as scheduled.
• 07/11/12 Lesson schedule changes concerning November. As anticipated, we will have additional lessons on 12/11, 19/11, 26/11 in room N1 from 16 to 18. Lesson on 27/11 is canceled. See also the course journal page.
• 25/10/12 Also the lesson on 26/10 is canceled. Due to travel issues, I won't be able to be back in Pisa on Friday Morning. Lessons will be moved to a different time in November. Next lesson will be on Tuesday 30th.
• 12/10/12 Lessons on 16/10, and 23/10 were moved. We are considering the slot on Monday 16-18 for Monday 22/10, please report any conflict with other courses.
• 04/10/12 Final timetable confirmed, valid from now on (including tomorrow 05/10).
• 02/10/12 Next lesson will be on Friday 05/10/2012, 11:00-13:00, room N1. Lesson on Thursday 04/10/2012 is cancelled due to work constraints.
• 30/9/12 As of today, the course schedule is still provisional. It seems likely the Monday lesson will move to Tuesday 9-11, but we have no confirmation yet. Unless otherwise noticed, we will have lesson tomorrow Monday 1/10/12.
• 27/9/12 Second Lesson. Lesson schedule is likely to change next week.
• 24/9/12 First lesson

The course journal is a separate page in this wiki.

The course will provide a description and analysis of a few key parallel and distributed programming platforms and models, starting from their theoretical foundations, where not covered by previous courses, and focusing on (1) existing standards for platforms and programming systems and (2) State-of-the-art solutions. The course will include practical use of those systems to develop simple applications.

The course will cover the following topics.

• Parallel Programming systems/frameworks
  • MPI message-passing programming (the core part of the MPI 2.2 standard)
  • TBB Thread Building Blocks Multicore oriented, shared-memory programming framework
  • ASSIST a framework for parallel high-level parallel programming with autonomic management
  • Massively Multicore computation and GPGPU frameworks
    • OpenCL
    • CUDA, Brooks+ possibly
• Foundation, Technologies
  • Elementary mechanisms to distribute computation: message passing, shared memory, massive multicore
  • Basics of scheduling algorithms and resource management
  • Basics of Service Oriented Architectures SOA
• Service oriented Platforms
  • Cloud Computing and Cloud Federations
  • Contrail
  • OpenNebula
  • Rackspace
• Example Applications

• Standard MPI 2.2 Only those parts that we will specify during the lessons.
  On the MPI forum web site you will find alternate formattings and some translations; version 2.2 of the MPI standard collects all the material from previous versions of the standard (1.0, 1.1, 2.0 and so on) within a single document.
• B. Wilkinson, M. Allen Parallel Programming, 2nd edition. 2005, Prentice-Hall. This book will be also used; at least the 1st edition is available in the University Library of Math/Comp.Sc./Physiscs, under code C.1.2 w74 INF .
• Tutorial on the Assist Compiler. Old version:Tutorial ASSIST Febbraio 2007
  Slightly newer version (in English) Assist Tutorial Nov 2007
  Tutorial of the deployment server GEA GEA documentation, November 2006.

Virtualization

• An Overview of Virtual Machine Architectures J. E. Smith and Ravi Nair
• The architecture of virtual machines J.E. Smith, Ravi Nair. IEEE Computer Vol. 38, Issue 5, 2005, Pages 32 - 38.

Data Mining Algorithms

• Dhillon & Modha Technical Report su K-means parallelo A Data-Clustering Algorithm On Distributed Memory Multiprocessors. I.S. Dhillon, D.S.Modha, LNAI 1759, pag 245. Nota: la versione disponibile online via LNCS riporta un algoritmo errato, il technical report è corretto.
• Ottimizzazioni sequenziali e parallele per K-means Large-Scale Parallel Data Clustering. Dan Judd, Philip K. Mckinley, Anil K. Jain. Ieee Transaction On Pattern Analysis And Machine Intelligence, Vo. 20, No. 8 August 1998.

Specific prerequisite knowledge from the first-level degree, which we assume known, includes in particular the basics of Computer Architecture (AE) as well as programming in C/C++ (AIL, PRL). Knowledge of basic techniques for testing program performance and behaviour is also assumed known.

Typical prerequisites of this course within the Master Degree are presented in the SPA course, and some topics are closely related to the SPM course, which is a suggested prerequisite although within the same year.

For transfer students and students belonging to a different Master course, we shall say that while all the topics in the SPA and SPM courses are related to SPD, a minimal subset of topics the student shall master includes

• Parallelization methodology and models
• Support to parallel programming models and software development tools
• Parallel Architectures
• Performance evaluation (cost models)
• Basics of parallel architectures
  • Parallel processors
  • Multiprocessors
  • Multicore / manycore / … / GPU
  • Shared vs distributed memory architectures
• Modularity, software reuse, incremental design
• Performance modelling and portability

Final test: Project + short colloquium to be held during one of the exam sessions. Topic to be previously agreed with the teacher.

The project will typically involve the use of one or more of the frameworks presented during the course, possibly developing more in-depth knowledge on one of the topics of the course.

• Individual projects are assigned to the students by the teacher, but can be proposed by the students.
• For difficult projects, 2 students, no more than that, can work on the same project.
• The projects can be assigned during the second half of the lesson period, or later on, during the academic year, when the student ask for it.
• Project can be experimental work (coding + testing + written relation), to be discussed with the teacher, or seminarial (relation and public talk about assigned papers) to be presented during one exam session. As of year 2011-12 and following, experimental projects are the norm.
• As soon as the project topic is agreed with the teacher, you are expected to send a two-page summary of the problem to tackle / application to write, with enough insight to explain it, and stating which parallel/distributed tools you are going to use and how the code produced will be evaluated.
• As the project is completed, you have to send a short report (about 20 pages) about project development and analysis of the results, including test results.
• Contact the teacher if you have issues, or if you are unsure how to deal with the rules in a specific case (e.g. you need to write a longer report, test with specific hardware etc. )