Artificial Intelligence Hardware Architectures

Lecture 1: Introduction to Al, Course objectives, Course Expectations, Grading Lecture 2: Introduction to the building blocks of digital integrated circuits for computing : * Fundamentals of digital electronics * Basic elements and digital building blocks (Diode, Transistor, AND Gate, OR Gate, Flip/Flop) * Building basic electronic functionality such as counter and arithmetic logic unit (ALU) using synthesis tools. Lectures 3 and 4: Introduction to CPU (central processing unit), pre-AI GPU (graphics processing unit) and OS (operating system) (Part 1 and Part 2) : * Explain the fundamentals of pre-AI HW and SW Architecture * Architecture Overview of CPU, pre-AI GPU and OS * Understand the concept of SW-HW virtuous cycle of progress Lecture 5: Introduction to Al Hardware * Overview of Al and its hardware demands, Historical evolution * Key challenges, Real-world applications * Performance metrics, Introduction to benchmarks Lecture 6: Fundamentals of Deep Neural Networks (DNN) (Part 1) * Review of neural network basics, DNN and Convolutional Neural Networks (CNN) architectures * Inference processes, Training processes (forward/backward propagation, SGD (stochastic gradient decent)) * Number representations for hardware * Computational kernels Lecture 7: Fundamentals of Deep Neural Networks (DNN) (Part 2) * Review of Recurrent Neural Networks (RNN), Long Short Term Memory Transformers (LSTM) * Training challenges and advanced optimizers, Deep dive into computational kernels * Precision accuracy tradeoff details, Hardware implications Lectures 8 and 9: AI Workload Characterization (Part 1 and Part 2) * Introduction to workload characterization, Compute requirements for training and inference * Memory requirements, Roofline models for performance analysis * Bottlenecks in traditional hardware (CPU vs. GPU), Profiling tools and techniques Lecture 10: Traditional and General-Purpose Al Architectures (Part 1) * Concepts of SISD (Single Instruction Single Data) and SIMD (Single Instruction Multiple Data) * Role of CPU architectures in Al * Role of GPU architectures in AI: Fundamentals, optimizations (SIMD versus SISD), tensor cores) * Case studies: NVIDIA Volta/Ampere V1, Intel Xeon Lecture 11: Traditional and General-Purpose Al Architectures (Part 2) * Advanced GPU Architectures, Deep dive into optimizations: (kernel fusion, batching) * Case studies: NVIDIA Ampere V2, AMD Instinct * Performance comparisons and trade-offs Lectures 12 and 13: AI Accelerators (Parts 1 and 2) * Introduction to Specialized Accelerators, Design Principles Overview, Dataflow Models: Weight Stationary, Output Stationary Systolic Arrays, * Case Studies: Google Tensor Processing Units (TPUs), MIT Eyeriss, DianNao Family * Performance comparisons and trade-offs, Energy-Delay Trade-offs * Mapping Algorithms to Hardware Lectures 14 and 15: Optimized Al Accelerators (Part 1 and Part 2) * Introduction to the Al accelerator optimization * Case Studies: Google TPU (tensor processing unit) V2/V3, Eyeriss V2, ShiDianNao * Scalability and Flexibility, Energy Optimization Techniques, Mapping Algorithms Lectures 16 and 17: Memory Systems and Data Reuse Content (Part 1 and Part 2) * Memory hierarchies in Al hardware. * Exploration of on-chip memory and buffers, off-chip memory (Dynamic Random Access Memory (DRAM), High Bandwidth Memory (HBM)). * Learn techniques for data reuse and locality optimization. Analyze in-memory and near-memory computing. * Study case studies: Hybrid Memory Cube (HMC), High Bandwidth Memory (HBM). Lectures 18 and 19: Neuromorphic Computing (Part 1 and Part 2) * Neuromorphic computing principles and architectures * Case Studies: IBM TrueNorth, Intel Loihi, Akida Brainchip, Sysense Speck * Applications and Conclusions Lectures 20, 21 and 22: Technologies for AI (Part 1, Part 2 and Part 3) * Resistive memory (Resistive Random Access Memory (RRAM), Phase Change Memory (PCM)) for in-memory acceleration * Analog computing, mixed signal computing * Challenges and Future Lecture 23: Software/Hardware Co-design Lecture 24: Future directions and relevant topics * Explore programmable Al architectures for flexibility. * Understand Shannon-inspired limits in Al hardware. Analyze beyond-CMOS fabrics for next-gen computing. * Discuss ethical considerations: energy, accessibility, Review recent advancements via guest lectures or papers.