Lab 1 - "Non-cooperative" multithreading
Lab 2 - Producer-consumer synchronization
Lab 3 - Simple parallel tasks
Lab 4 - Futures and continuations
Lab 5 - Parallelizing techniques
Lab 6 - Parallelizing techniques (2 - parallel explore)
Lab 7 - MPI programming
Lab 8 - Distributed protocols

Laboratory 1 requirements:

The problems will require to execute a number of independent operations, that operate on shared data. There shall be several threads launched at the beginning, and each thread shall execute a lot of operations. The operations to be executed are to be randomly choosen, and with randomly choosen parameters. The main thread shall wait for all other threads to end and, then, it shall check that the invariants are obeyed. The operations must be synchronized in order to operate correctly. Write, in a documentation, the rules (which mutex what invariants it protects). You shall play with the number of threads and with the granularity of the locking, in order to asses the performance issues. Document what tests have you done, on what hardware platform, for what size of the data, and what was the time consumed.

Bank accounts

At a bank, we have to keep track of the balance of some accounts. Also, each account has an associated log (the list of records of operations performed on that account). Each operation record shall have a unique serial number, that is incremented for each operation performed in the bank.

We have concurrently run transfer operations, to be executer on multiple threads. Each operation transfers a given amount of money from one account to someother account, and also appends the information about the transfer to the logs of both accounts.

From time to time, as well as at the end of the program, a consistency check shall be executed. It shall verify that the amount of money in each account corresponds with the operations records associated to that account, and also that all operations on each account appear also in the logs of the source or destination of the transfer.

Laboratory 2 requirements:

Create two threads, a producer and a consumer, with the producer feeding the consumer.

Requirement: Compute the scalar product of two vectors.

Create two threads. The first thread (producer) will compute the products of pairs of elements - one from each vector - and will feed the second thread. The second thread (consumer) will sum up the products computed by the first one. The two threads will behind synchronized with a condition variable and a mutex. The consumer will be cleared to use each product as soon as it is computed by the producer thread.

Laboratory 3 requirements:

Write several programs to compute the product of two matrices.

Have a function that computes a single element of the resulting matrix.

Have a second function whose each call will constitute a parallel task (that is, this function will be called on several threads in parallel). This function will call the above one several times consecutively to compute several elements of the resulting matrix. Consider the following ways of splitting the work betweeb tasks (for the examples, consider the final matrix being 9x9 and the work split into 4 tasks):

Each task computes consecutive elements, going row after row. So, task 0 computes rows 0 and 1, plus elements 0-1 of row 2 (20 elements in total); task 1 computes the remainder of row 2, row 3, and elements 0-3 of row 4 (20 elements); task 2 computes the remainder of row 4, row 5, and elements 0-5 of row 6 (20 elements); finally, task 3 computes the remaining elements (21 elements).

Each task computes consecutive elements, going column after column. This is like the previous example, but interchanging the rows with the columns: task 0 takes columns 0 and 1, plus elements 0 and 1 from column 2, and so on.

Each task takes every k-th element (where k is the number of tasks), going row by row. So, task 0 takes elements (0,0), (0,4), (0,8), (1,3), (1,7), (2,2), (2,6), (3,1), (3,5), (4,0), etc.

For running the tasks, also implement 2 approaches:

Create an actual thread for each task (use the low-level thread mechanism from the programming language)

Use a thread pool.

Laboratory 4 requirements:

The goal of this lab is to use C# TPL futures and continuations in a more complex scenario, in conjunction with waiting for external events.

Write a program that is capable of simultaneously downloading several files through HTTP. Use directly the BeginConnect()/EndConnect(), BeginSend()/EndSend() and BeginReceive()/EndReceive() Socket functions, and write a simple parser for the HTTP protocol (it should be able only to get the header lines and to understand the Content-lenght: header line).

Try three implementations:

• Directly implement the parser on the callbacks (event-driven);
• Wrap the connect/send/receive operations in tasks, with the callback setting the result of the task;
• Like the previous, but also use the async/await mechanism.

Laboratory 5 requirements:

The goal of this lab is to implement a simple but non-trivial parallel algorithm.

Perform the multiplication of 2 polynomials. Use both the regular O(n2) algorithm and the Karatsuba algorithm, and each in both the sequencial form and a parallelized form. Compare the 4 variants.

The documentation will describe:

• the algorithms
• the synchronization used in the parallelized variants
• the performance measurements

Laboratory 6 requirements:

The goal of this lab is to implement a simple but non-trivial parallel algorithm.

Given a directed graph, find a Hamiltonean cycle, if one exists. Use multiple threads to parallelize the search.

The documentation will describe:

• the algorithms
• the synchronization used in the parallelized variants
• the performance measurements

Laboratory 7 requirements:

The goal of this lab is to implement a distributed algorithm using MPI.

Perform the multiplication of 2 polynomials, by distributing computation across several nodes using MPI. Use both the regular O(n2) algorithm and the Karatsuba algorithm. Compare the performance with the "regular" CPU implementation from lab 5.

The documentation will describe:

• the algorithms
• the distribution and communication
• the performance measurements

Laboratory 8 requirements:

The goal of this lab is to deal with the consistency issues of the distributed programming.

Implement a simple distributed shared memory (DSM) mechanism. The lab shall have a main program and a DSM library. There shall be a predefined number of communicating processes. The DSM mechanism shall provide a predefined number of integer variables residing on each of the processes. The DSM shall provide the following operations:

write a value to a variable (local or residing in another process);
a callback informing the main program that a variable managed by the DSM has changed. All processes shall receive the same sequence of data change callbacks; it is not allowed that process P sees first a change on a variable A and then a change on a variable B, while another process Q sees the change on B first and the change on A second.
a "compare and exchange" operation, that compares a variable with a given value and, if equal, it sets the variable to another given value. Be careful at the interaction with the previous requirement. 

Notes:

Only nodes that subscribe to a variable will receive notifications about changes of that variable, and only those nodes are allowed to change (set) that variable;
The subscriptions are static and each node knows, for each variable it is subscribed to, which are the other subscribers for that variable.
We assume that most variables are accessed locally (within a small group of a few computers); we don't want a centralized server that hold all variables, because it would be a central bottleneck of the system. Therefore, as a result of changing a variable, all the messages should be exchanged only between the subscribers of that variable.
The computers are not faulty.