Cuda-120-Days-Challenge

A 120-day CUDA learning plan covering daily concepts, exercises, pitfalls, and references (including “Programming Massively Parallel Processors”). Features six capstone projects to solidify GPU parallel programming, memory management, and performance optimization skills.

120-Day CUDA Curriculum

A structured, day-by-day plan to master NVIDIA CUDA programming over 120 days. Each day includes:

• Core Topic
• Practical Exercise / Mini-Project
• Debugging Pitfalls
• Resource Suggestions

Six Capstone Projects are spread out at Days 20, 40, 60, 80, 100, and 120 to synthesize the skills acquired.

---

Day-by-Day CUDA Curriculum

Day	Core Topic	Practical Exercise / Mini-Project	Debugging Pitfalls	Resource Suggestions
1	Introduction to GPU Computing & CUDA(daily-updates/day-01-intro-cuda.md)	Write a simple “Hello GPU” kernel that prints a message from one thread.	Missing <cuda_runtime.h> or incorrect compiler flags.	CUDA C Programming Guide, Introduction
2	Setting Up the Development Environment	Install CUDA Toolkit & drivers; compile and run sample codes.	Incompatible driver/toolkit versions.	Official “Getting Started” Guide
3	GPU vs. CPU Architecture Foundations	Compare GPU SMs and CPU cores; discuss throughput vs. latency.	Mixing CPU vs. GPU roles can lead to design inefficiencies.	CUDA C Programming Guide, “Hardware Model Overview”
4	Thread Hierarchy: Grids & Blocks	Launch a kernel using different grid/block dimensions, experiment with thread indexing.	Off-by-one errors in thread indexing.	CUDA C Programming Guide, “Thread Hierarchy”
5	Thread Hierarchy: Warps (Intro)	Inspect warp size and how threads are grouped (no direct warp programming yet).	Divergence issues in the same warp.	CUDA C Best Practices Guide, “Warps and SIMT Model”
6	Basic Kernel Launch & Execution	Write a kernel for element-wise addition on a small array.	Mixing up cudaMemcpyHostToDevice vs. cudaMemcpyDeviceToHost.	CUDA Samples, Vector Addition
7	Memory Model (Global Memory Basics)	Transfer data to global memory, run a simple computation, transfer results back.	Mismatched data sizes or forgetting to free GPU memory.	CUDA C Programming Guide, “Global Memory Access”
8	Memory Allocation & Pointers	Use cudaMalloc/cudaFree; practice error checking.	Memory leaks if forgetting to free device pointers.	CUDA C Programming Guide, “Memory Allocation”
9	Memory Alignment & Coalescing	Benchmark coalesced vs. non-coalesced memory accesses in a kernel.	Non-contiguous access patterns hurt performance.	CUDA C Best Practices Guide, “Memory Coalescing”
10	Shared Memory Fundamentals	Implement tile-based matrix multiplication using shared memory.	Race conditions without thread sync.	CUDA C Programming Guide, “Shared Memory”
11	Thread Synchronization (__syncthreads())	Extend tile-based multiplication with sync calls; measure performance.	Missing __syncthreads() => partial updates.	CUDA C Programming Guide, “Synchronization”
12	Bank Conflicts in Shared Memory	Test an access pattern that causes bank conflicts; measure performance impact.	Overlooking bank conflict in shared memory accesses.	CUDA C Programming Guide, “Shared Memory Bank Conflicts”
13	Basic Atomic Operations	Use atomicAdd to sum an array in parallel.	Large-scale sums using atomics can degrade performance.	CUDA C Programming Guide, “Atomic Functions”
14	Progress Checkpoint	Quick recap or quiz: global vs. shared memory usage.	Not consolidating learning => repeated mistakes.	Review Days 1–13 notes; create a “lessons learned” list
15	Advanced Atomic Operations	Experiment with atomicCAS, atomicExch, etc.	Overusing atomics => performance bottlenecks.	CUDA C Programming Guide, “Atomic Functions”
16	Kernel Configuration Tuning	Adjust block sizes for the same kernel; observe occupancy changes.	Non-multiples of warp size can lower occupancy.	CUDA Occupancy Calculator (within Nsight Compute)
17	Host-Device Synchronization Patterns	Use cudaDeviceSynchronize() for timing; measure kernel durations.	Missing sync => partial results readback.	CUDA C Programming Guide, “Device Synchronization”
18	Error Handling & cudaGetErrorString()	Implement robust error checks after each CUDA call.	Ignoring errors => hard-to-trace bugs.	CUDA Error Handling
19	Unified Memory (UM) Intro	Use cudaMallocManaged; run simple vector addition.	Forgetting that UM still migrates data.	CUDA C Programming Guide, “Unified Memory”
20	Capstone Project #1	Image Convolution Pipeline: 2D convolution (e.g., edge detection) on GPU.	–	–
21	Streams & Concurrency (Basics)	Launch two kernels in different streams; check overlap.	Default stream is blocking if concurrency not enabled.	CUDA C Programming Guide, “Streams”
22	Events & Timing	Use CUDA events for precise kernel timing.	Misplacing events => skewed time measurements.	CUDA C Programming Guide, “Events”
23	Asynchronous Memory Copy	Copy data using streams asynchronously.	Not using pinned host memory => limited async.	CUDA C Programming Guide, “Asynchronous Transfers”
24	Pinned (Page-Locked) Memory	Compare pinned vs. pageable host memory transfers.	Excessive pinned memory => system performance hit.	CUDA C Programming Guide, “Pinned Memory”
25	Double Buffering Technique	Implement a two-buffer pipeline to overlap compute and transfer.	Improper sync => reading incomplete data.