Cover Image

Parallel computing on heterogeneous networks /

Show other versions (1)
Bibliographic Details
Main Author: Lastovetsky, Alexey, 1957-
Format: eBook
Language:English
Published: Hoboken, NJ : Wiley-Interscience, ©2003.
Series:Wiley series on parallel and distributed computing.
Subjects:
Parallel programming (Computer science)
Heterogeneous computing.
Computer networks.
Internetworking (Telecommunication)
Computer Communication Networks
Online Access:Connect to the full text of this electronic book
Table of Contents:
  • Part I Evolution of Parallel Computing 9
  • 1.1. Serial Scalar Processor and Programming Model 11
  • 1.2. Basic Program Properties 11
  • 2. Vector and Superscalar Processors 15
  • 2.1. Vector Processor 15
  • 2.3. Programming Model 21
  • 2.4. Optimizing Compilers 22
  • 2.5. Array Libraries 33
  • 2.5.1. Level 1 BLAS 34
  • 2.5.2. Level 2 BLAS 35
  • 2.5.3. Level 3 BLAS 40
  • 2.5.4. Sparse BLAS 43
  • 2.6. Parallel Languages 44
  • 2.6.1. Fortran 90 45
  • 2.6.2. The C Language 50
  • 2.7. Memory Hierarchy and Parallel Programming Tools 59
  • 3. Shared Memory Multiprocessors 65
  • 3.1. Shared Memory Multiprocessor Architecture and Programming Models 65
  • 3.2. Optimizing Compilers 67
  • 3.3. Thread Libraries 68
  • 3.3.1. Operations on Threads 69
  • 3.3.2. Operations on Mutexes 71
  • 3.3.3. Operations on Condition Variables 73
  • 3.3.4. Example of MT Application: Multithreaded Dot Product 75
  • 3.4. Parallel Languages 78
  • 3.4.1. Fortran 95 78
  • 3.4.2. OpenMP 80
  • 4. Distributed Memory Multiprocessors 95
  • 4.1. Distributed Memory Multiprocessor Architecture: Programming Model and Performance Models 95
  • 4.2. Message-Passing Libraries 103
  • 4.2.1. Basic MPI Programming Model 104
  • 4.2.2. Groups and Communicators 106
  • 4.2.3. Point-to-Point Communication 111
  • 4.2.4. Collective Communication 120
  • 4.2.5. Environmental Management 127
  • 4.2.6. Example of an MPI Application: Parallel Matrix-Matrix Multiplication 127
  • 4.3. Parallel Languages 130
  • 5. Networks of Computers: Architecture and Programming Challenges 141
  • 5.1. Processors Heterogeneity 142
  • 5.1.1. Different Processor Speeds 142
  • 5.1.2. Heterogeneity of Machine Arithmetic 146
  • 5.2. Ad Hoc Communication Network 147
  • 5.3. Multiple-User Decentralized Computer System 150
  • 5.3.1. Unstable Performance Characteristics 150
  • 5.3.2. High Probability of Resource Failures 150
  • Part II Parallel Programming for Networks of Computers with mpC and HMPI 157
  • 6. Introduction to mpC 159
  • 6.1. First mpC Programs 159
  • 6.2. Networks 164
  • 6.3. Network Type 170
  • 6.4. Network Parent 173
  • 6.5. Synchronization of Processes 178
  • 6.6. Network Functions 182
  • 6.7. Subnetworks 186
  • 6.8. A Simple Heterogeneous Algorithm Solving an Irregular Problem 190
  • 6.9. The RECON Statement: A Language Construct to Control the Accuracy of the Underlying Model of Computer Network 196
  • 6.10. A Simple Heterogeneous Algorithm Solving a Regular Problem 199
  • 6.11. Principles of Implementation 206
  • 6.11.1. Model of a Target Message-Passing Program 207
  • 6.11.2. Mapping of the Parallel Algorithm to the Processors of a Heterogeneous Network 209
  • 7. Advanced Heterogeneous Parallel Programming in mpC 215
  • 7.1. Interprocess Communication 215
  • 7.2. Communication Patterns 233
  • 7.3. Algorithmic Patterns 241
  • 7.4. Underlying Models and the Mapping Algorithm 244
  • 7.4.1. Model of a Heterogeneous Network of Computers 244
  • 7.4.2. The Mapping Algorithm 247
  • 8. Toward a Message-Passing Library for Heterogeneous Networks of Computers 255
  • 8.1. MPI and Heterogeneous Networks of Computers 255
  • 8.2. HMPI: Heterogeneous MPI 257
  • Part III Applications of Heterogeneous Parallel Computing 263
  • 9. Scientific Applications 265
  • 9.1. Linear Algebra 265
  • 9.1.1. Matrix Multiplication 265
  • 9.1.2. Matrix Factorization 288
  • 9.1.3. Heterogeneous Distribution of Data and Heterogeneous Distribution of Processes Compared 295
  • 9.2. N-Body Problem 298
  • 9.3. Numerical Integration 300
  • 9.3.1. Basic Quadrature Rules 301
  • 9.3.2. Adaptive Quadrature Routines 304
  • 9.3.3. The quanc8 Adaptive Quadrature Routine 307
  • 9.3.4. Parallel Adaptive Quadrature Routine for Heterogeneous Clusters 313
  • 9.4. Simulation of Oil Extraction 323
  • 10. Business and Software Engineering Applications 331
  • 10.1. Acceleration of Distributed Applications 331
  • 10.1.2. Distributed Application of a Supermarket Chain 332
  • 10.1.3. Parallel Implementation of the Remote Operation getDistribution 334
  • 10.1.4. Experimental Results 337
  • 10.2. Parallel Testing of Distributed Software 338
  • 10.2.1. Motivation 338
  • 10.2.2. Parallel Execution of the Orbix Test Suite on a Cluster of Multiprocessor Workstations 339
  • 10.2.3. Experimental Results 350
  • Appendix A. The mpC N-Body Application 353
  • Appendix B. The Block Cyclic Matrix Multiplication Routine for Heterogeneous Platforms 371
  • Appendix C. The Parallel Adaptive Quadrature Routine 385
  • C.1. Source Code 385
  • Appendix D. The mpC User's Guide 397
  • D.2. Outline of the mpC Programming Environment 397
  • D.3. Supported Systems 398
  • D.4. The mpC Compiler 399
  • D.4.1. Options 399
  • D.4.2. Pragmas 401
  • D.5. How to Start Up 401
  • D.6. Virtual Parallel Machine 402
  • D.6.1. VPM Description File 403
  • D.7. Environmental Variables 404
  • D.7.1. WHICHMPI 404
  • D.7.2. MPIDIR 404
  • D.7.3. MPCHOME 404
  • D.7.4. MPCLOAD 405
  • D.7.5. MPCTOPO 405
  • D.8. How to Run mpC Applications 405
  • D.8.1. mpccreate 406
  • D.8.2. mpcopen 406
  • D.8.3. mpcbcast 407
  • D.8.4. mpcload 407
  • D.8.5. mpcrun 408
  • D.8.6. mpctouch 408
  • D.8.7. mpcclose 408
  • D.8.8. mpcclean 408
  • D.8.9. mpcmach 408
  • D.8.10. mpcdel 409
  • D.9. How to Debug mpC Applications 409
  • D.10. Sample mpC Sessions 409.