Programming Massively Parallel Processors: A Hands-on Approach
Original price was: $89.99.$19.95Current price is: $19.95.
✔️ (PDF) • Pages : 556
Programming Massively Parallel Processors: A Hands-on Approach shows both students and professionals alike the basic concepts of parallel programming and GPU architecture. Concise, intuitive, and practical, it is based on years of road-testing in the authors’ own parallel computing courses. Various techniques for constructing and optimizing parallel programs are explored in detail, while case studies demonstrate the development process, which begins with computational thinking and ends with effective and efficient parallel programs. The new edition includes updated coverage of CUDA, including the newer libraries such as CuDNN. New chapters on frequently used parallel patterns have been added, and case studies have been updated to reflect current industry practices.
- Parallel Patterns Introduces new chapters on frequently used parallel patterns (stencil, reduction, sorting) and major improvements to previous chapters (convolution, histogram, sparse matrices, graph traversal, deep learning)
- Ampere Includes a new chapter focused on GPU architecture and draws examples from recent architecture generations, including Ampere
- Systematic Approach Incorporates major improvements to abstract discussions of problem decomposition strategies and performance considerations, with a new optimization checklist
11 reviews for Programming Massively Parallel Processors: A Hands-on Approach
You must be logged in to post a review.
Related products
-
The Cartoon Guide to Geometry
Original price was: $26.00.$11.95Current price is: $11.95. -
Schaum’s 3,000 Solved Problems in Calculus (Schaum’s Outlines)
Original price was: $32.99.$19.00Current price is: $19.00. -
Artificial Intelligence: A Systems Approach from Architecture Principles to Deployment (MIT Lincoln Laboratory Series)
Original price was: $108.22.$19.99Current price is: $19.99.
Steve Cole (verified owner) –
I liked the reorganizations and updates from the 3rd edition to this one. As an instructor, I also very much appreciated the section in the front material describing the typical usage and course context of the book’s chapters, and dependencies and pathways through the chapters.
Arthur Morgan (verified owner) –
The cover and some pages in the first chapter are folded. Clearly the previous owner returned it before finishing reading the first chapter for some reason
Arslan (verified owner) –
Just finished reading the first chapter and I am already impressed.
Mark Saroufim (verified owner) –
I’ve been using this book to teach my sessions in the cuda mode discord community, it’s been by far the best reference I’ve found on the market to learn CUDA.
In particular chapters 1-6 will give you the core foundation to be able to start working on your own CUDA kernels and if you supplement those chapters with learning how to integrate your kernels in pytorch using features like load_inline but also the ncu profiler you’ll be well on your way to writing real-world kernels that are performant. There’ a long glossary of confusing concepts like grids, blocks, threads, warps which you won’t remember if you’re browsing the occasional medium blogpost or Wikipedia article, even ChatGPT often makes subtle mistakes. Learning CUDA or at least the basics of it is very much a open the textbook and do the problems sort of exercise.
Starting chapter 7 the book goes into various case studies of popular algorithms and how to optimize them , the lessons are generically helpful even if you’re not interested in those subproblems but my point is more that the book becomes significantly easier to understand after the initial struggle from chapter 1 through 6. This is also a natural point for you to experiment with your own CUDA kernels in maybe a workload you’re trying to accelerate at work, whenever you get stuck you can browse the book for inspiration on common CUDA patterns that accelerate performance.
Before this book I’d been stuck in tutorial hell with cuda for many years but this book gave me the right foundation to start using kernels at my day job and it’s been a fantastic level up. Keep in mind that with tools like ChatGPT or code generators like torch.compile you can focus on only learning CUDA as opposed to also having to learn about makefiles and c++
Granted the main gap the book seems to have is that it doesn’t really cover CUDA C++ so reading codebases like CUTLASS will still be a struggle but more importantly the book doesn’t cover how to program with tensor cores or have a treatment of lower precision dtypes and with modern ML workloads. CUDA streams are also briefly covered but spending a bit more on NCCL would be really nice to see in future editions.
Eduardo Hiroshi Nakamura (verified owner) –
Excelente
Brandon Awbrey (verified owner) –
Very clear and thoughtful, covers not only the programming abstractions needed to use CUDA to develop applications, but uses that context to explain the hardware differences and challenges. One of the best programming books I’ve ever read.
Brian L. (verified owner) –
Covers CUDA programming and then has several chapters discussing massively parallel algorithms.
Al – NW2M (verified owner) –
A great overview along with deep-dives. College & Masters level content.
Nitin Deshpande (verified owner) –
A great book for beginners. The fundamentals are explained crisply and clearly. A highly recommended book.
Seripis (verified owner) –
Very happy with the quality and delivery
Vamshi Balanaga (verified owner) –
It’s good