Computer Science Self-Learning Notes

📌 Overview

This repository archives my notes and materials during my computer science self-learning jouney. Currently, I mainly focus on LLM/VLM inference engine and GPU/NPU computing, thus I have gathered many technical blogs for AI infra beginners and MLSys papers for researchers.

🔍 Contents:

• 📚 Learning Notes
• 📚 Technical Blogs
• 📚 Papers
• 📚 Learning Projects

📚 Learning Notes

🧱 Basic Knowledges

• Programming Languages:
  • Python
  • C/C++
  • Java
  • Go
• Data Structure & Algorithm:
  • Data Structure & Algorithm
  • LeetCode Practices
• Network
• Operating System
• Design Pattern

🤖 AI

• Mathematics:
  • Algebra
  • Differentiation
  • Probability
  • Optimization
• Deep Learning:
  • Basic Knowledges
  • PyTorch Tutorial
• LLM:
  • Basic Knowledges
    • Transformer
  • MoE
• Multi-Modality:
  • ViT
  • Diffusion
    • Basic Knowledges
    • Inference Acceleration
• AI Infra:
  • Environment Preparation
  • Basic Knowledges
  • Inference Engine
    • vLLM
  • Distributed System
    • Ray
  • HPC
    • CUDA

🚀 Backend & Big Data

• Roadmap
• Backend Development:
  • Spring
  • MySQL
  • Redis
  • Oracle
• Big Data Development:
  • ElasticSearch
  • Flink
  • Hudi

🛠️ Tools

• Git
• Docker
• CMake
• Documentation:
  • Markdown
  • reStructuredText
• IDE:
  • VSCode Shortcut Key
• AI Agent:
  • Skills

🔗 Others

• Open Source Best Practices
• Research:
  • Research Notes
  • Popular MLSys Papers
• Employment:
  • Interview Experience
  • Working Insights

📚 Technical Blogs

📖 Basic Knowledges

Title	Category	Author	Note	Rec	Read
[The Illustrated Transformer][2016]	Transformer	[@Jay Alammar][3008]	Transformer 原理详解	⭐️⭐️⭐️⭐️⭐️	✅
[The Illustrated GPT-2 (Visualizing Transformer Language Models)][2021]	Transformer	[@Jay Alammar][3008]	Transformer 推理过程	⭐️⭐️⭐️⭐️⭐️	✅
[图文详解 LLM inference：KV Cache][2024]	KV Cache	[@季叶][3011]		⭐️⭐️⭐️	✅
[Mixture of Experts Explained][2038]	MoE	[@HuggingFace Blog][3017]	MoE 综述	⭐️⭐️⭐️⭐️	✅
[MoE 并行负载均衡：EPLB 的深度解析与可视化][2040]	MoE	[@kaiyuan][3005]		⭐️⭐️⭐️	✅
[LLM 推理并行优化的必备知识][2032]	Parallel Strategy	[@kaiyuan][3005]
[分布式推理优化思路][2051]	Parallel Strategy	[@kaiyuan][3005]
[The Ultra-Scale Playbook: Training LLMs on GPU Clusters][2039]	Parallel Strategy	[@HuggingFace Blog][3017]
[图解大模型计算加速系列：分离式推理架构 1，从 DistServe 谈起/u>][2049]	PD Disaggregation	[@猛猿][3003]	PD 分离原理详解	⭐️⭐️⭐️⭐️	✅

📖 Dive into vLLM

Title	Category	Author	Note	Rec	Read
[Inside vLLM: Anatomy of a High-Throughput LLM Inference System][2045]	Overview	[@vLLM Blog][3019]	vLLM 全面详解	⭐️⭐️⭐️⭐️⭐️	✅
[vLLM V1 整体流程｜从请求到算子执行][2003]	Architecture	[@SSS不知-道][3001]	vLLM 推理流程	⭐️⭐️⭐️⭐️⭐️	✅
[图解 vLLM V1 系列 1：整体流程][2011]	Architecture	[@猛猿][3003]		⭐️⭐️⭐️	✅
[图解 vLLM V1 系列 2：Executor-Workers 架构][2012]	Architecture	[@猛猿][3003]		⭐️⭐️⭐️	✅
[图解 vLLM V1 系列 3：KV Cache 初始化][2013]	KV Cache	[@猛猿][3003]		⭐️⭐️⭐️	✅
[图解 vLLM V1 系列 4：加载模型权重][2005]	Model	[@猛猿][3003]		⭐️⭐️	✅
[vLLM 模型权重加载：使用 setattr][2007]	Model	[@风之魔术师][3004]		⭐️⭐️	✅
[ColumnParallelLinear 和 RowParallelLinear][2008]	Model	[@风之魔术师][3004]		⭐️⭐️	✅
[图解 vLLM V1 系列 5：调度器策略][2006]	Scheduler	[@猛猿][3003]		⭐️⭐️⭐️⭐️	✅
[Introducing vLLM Hardware Plugin, Best Practice from Ascend NPU][2010]

📖 Dive into PyTorch

Title	Category	Author	Note	Rec	Read
[PyTorch 显存管理介绍与源码解析（一）][2061]	Memory	[@kaiyuan][3005]		⭐️⭐️⭐️⭐️	✅
[PyTorch 显存可视化与 Snapshot 数据分析][2062]	Memory	[@kaiyuan][3005]		⭐️⭐️⭐️⭐️	✅
[OOM 解决利器 —— PyTorch Expandable Segments 全链路剖析][2081]	Memory			⭐️⭐️⭐️	✅
[A guide on good usage of non_blocking and pin_memory() in PyTorch][2072]	Data Transfer	[@Vincent Moens][3030]		⭐️⭐️⭐️⭐️⭐️	✅

📖 CUDA Programming

Title	Category	Author	Note	Rec	Read
[CUDA 内核优化策略][2027]	Performance	[@Zhang][3010]		⭐️⭐️⭐️	✅
[从啥也不会到 CUDA GEMM 优化][2028]	Performance	[@猛猿][3003]		⭐️⭐️⭐️⭐️	✅

📖 Communication

Title	Category	Author	Note	Rec	Read
[NCCL: Collective Operations][2050]	Collective Communication	[@NVIDIA Developer][3022]	集合通信常用操作	⭐️⭐️⭐️⭐️	✅
[一文读懂｜RDMA 原理][2035]	Network	[@Linux内核库][3015]		⭐️⭐️⭐️⭐️	✅
[PyTorch 中基于 CUDA IPC 的进程间 Tensor 共享简介][2080]	IPC	[@kaiyuan][3005]		⭐️⭐️⭐️	✅

📖 Multi-Modality

Title	Category	Author	Note	Rec	Read
[多模态技术梳理：ViT 系列][2067]	ViT	[@姜富春][3014]	ViT 研究综述	⭐️⭐️⭐️	✅
[LLaVA 系列模型结构详解][2019]	ViT	[@Zhang][3010]		⭐️⭐️⭐️	✅
[文生图模型之 Stable Diffusion][2077]	Diffusion	[@小小将][3033]		⭐️⭐️⭐️⭐️	✅
[DiT 推理加速综述: Caching][2078]	Diffusion	[@DefTruth][3002]		⭐️⭐️⭐️⭐️	✅

📖 Dive into Qwen

Title	Category	Author	Note	Rec	Read
[多模态技术梳理：Qwen-VL 系列][2068]	VL	[@姜富春][3014]		⭐️⭐️⭐️⭐️	✅
[Qwen2-VL 源码解读：从准备一条样本到模型生成全流程图解][2069]	VL	[@姜富春][3014]		⭐️⭐️⭐️⭐️⭐️	✅
[万字长文图解 Qwen2.5-VL 实现细节][2020]	VL	[@猛猿][3003]		⭐️⭐️⭐️⭐️⭐️	✅
[Qwen3-VL 解剖][2073]	VL	[@Plunck][3031]		⭐️⭐️⭐️	✅
[Qwen3-Omni：统一多模态大模型再升级]