S25, CSCI 415, Test 1 Review

Topic Review:

• Brief History
• Basic definitions (Ch 1)
• parallel vs distributed
• thread, PE, ...
• Why use parallelism?
• Scope of Parallel Computing (see ch 1.2 in text)
• Architectures (Ch 2)
• SIMD, MIMD, SPMD
• Global memory, PRAM
• basic ideas
• multiple models of sharing
• Non-shared Memory machines
• basic ideas
• interconnection topologies
• N.O.W.s
• Communication Costs
• Processor Mapping Techniques
• Parallel Program design (Ch 3)
• sequential -> parallel problems
• need to start over with parallel view
• data placement
• load balancing
• other issues
• Baisc Communication Operations (Ch 4)
• One to all broadcast, all to one reduction
• all to all broadcast and reduction
• All reduce, prefix-sum
• scatter, gather
• all to all personalized communication
• mapping to hardware
• Analysis of parallel alogrithms (Ch 5)
• areas of overhead
• measures
• speedup
• efficiency
• cost
• ...
• Mapping algorithms to machines
• data partitioning
• execution partitioning
• Snyder's Type architure paper
• Jacobi's sequential to parallel data dependency
• PRAM
• maximum algorithm by Valiant
• true cost on hardware
• Matrix algorithms (Grama Ch 8, Eijkhout Ch 6)
• Matrix Transpose
• Matrix/Vector multiply
• Matrix/Matrix multiply
• "simple" communicate then calculate algorithm
• Cannon's Algorithm
• Nelson's Algorithm