Channel Title: MIT HAN Lab Channel Description: MIT HAN Lab: Hardware, AI and Neural-nets Accelerate Deep Learning Computing Group website: hanlab.mit.edu TinyML Project: tinyml.mit.edu TinyML Course: efficientml.ai, streamed via live.efficientml.ai every Tuesday/Thursday 3:35-5:00pm ET, fall 2022. <video> EfficientML.ai Lecture 21 - On-device Training (MIT 6.5940, Fall 2024) Description: EfficientML.ai Lecture 21 - On-device Training and Transfer Learning (MIT 6.5940, Fall 2024) Lecture 21: On-device Training and Transfer Learning Instructor: Prof. Song Han Slides: https://efficientml.ai Outline: - Deep leakage from gradients, gradient is not safe to share - Memory bottleneck of on-device training - Tiny transfer learning (TinyTL) - Sparse back-propagation (SparseBP) - Quantized training with quantization aware scaling (QAS) - PockEngine: system support for sparse back-propagation </video> <video> EfficientML.ai Lecture 21 - On-device Training (Zoom Recording) (MIT 6.5940, Fall 2024) Description: EfficientML.ai Lecture 21 - On-device Training and Transfer Learning (Zoom Recording) (MIT 6.5940, Fall 2024) Lecture 21: On-device Training and Transfer Learning Instructor: Prof. Song Han Slides: https://efficientml.ai Outline: - Deep leakage from gradients, gradient is not safe to share - Memory bottleneck of on-device training - Tiny transfer learning (TinyTL) - Sparse back-propagation (SparseBP) - Quantized training with quantization aware scaling (QAS) - PockEngine: system support for sparse back-propagation </video> <video> EfficientML.ai 2024 | Introduction to Deep Compression Autoencoder Description: EfficientML.ai 2024 | Introduction to Deep Compression Autoencoder Lecture 18: Diffusion Models Instructor: Prof. Song Han Slides: https://efficientml.ai </video> <video> EfficientML.ai Lecture 22: Quantum Machine Learning Part 1 (Zoom Recording) (MIT 6.5940, Fall 2024) Description: EfficientML.ai Lecture 22: Quantum Machine Learning Part 1 (Zoom Recording) (MIT 6.5940, Fall 2024) Lecture 22: Quantum Machine Learning Part 1 Instructor: Prof. Hanrui Wang Slides: https://efficientml.ai Outline: - Course Summary - Introduce single qubit state and gates - Introduce multiple-qubit state and gates - Introduce quantum circuit - Introduce the NISQ era and compilation problems - Introduce the example workflow and compiler on neutral atom quantum computer </video> <video> TinyML Demo on Microcontrollers (short) Description: </video> <video> EfficientML.ai Lecture 20 - Distributed Training Part 2 (Zoom Recording) (MIT 6.5940, Fall 2024) Description: EfficientML.ai Lecture 20 - Distributed Training Part 2 (Zoom Recording) (MIT 6.5940, Fall 2024) Lecture 20: Distributed Training Part 2 Instructor: Prof. Song Han Slides: https://efficientml.ai Outline: - Hybrid (mixed) parallelism and how to auto-parallelize - Understand the bandwidth and latency bottleneck of distributed training - Gradient compression: overcome the bandwidth bottleneck - Gradient Pruning: Sparse Communication, Deep Gradient Compression - Gradient Quantization: 1-Bit SGD, TernGrad - Delayed gradient update: overcome the latency bottleneck </video> <video> EfficientML.ai Lecture 19 - Distributed Training Part 1 (MIT 6.5940, Fall 2024) Description: EfficientML.ai Lecture 19 - Distributed Training Part 1 (MIT 6.5940, Fall 2024) Lecture 19: Distributed Training Part 1 Instructor: Prof. Song Han Slides: https://efficientml.ai Outline: - Background and motivation - Parallelization methods for distributed training - Data parallelism - Communication primitives - Reducing memory in data parallelism: ZeRO-1 / 2/ 3 and FSDP - Pipeline parallelism - Tensor parallelism - Sequence parallelism </video> <video> EfficientML.ai Lecture 18 - Diffusion Models (Zoom Recording) (MIT 6.5940, Fall 2024) Description: EfficientML.ai Lecture 18 - Diffusion Models (Zoom Recording) (MIT 6.5940, Fall 2024) Lecture 18: Diffusion Models Instructor: Prof. Song Han Slides: https://efficientml.ai Outline: - Basics of diffusion model - Denoising diffusion models - Conditional diffusion models - Latent diffusion models - Image editing - Model personalization - Fast sampling techniques - Denoising diffusion implicit models - Distillation - Acceleration techniques - Sparsity - Quantization - Parallelism </video> <video> EfficientML.ai Lecture 20 - Distributed Training Part 2 (MIT 6.5940, Fall 2024) Description: EfficientML.ai Lecture 20 - Distributed Training Part 2 (MIT 6.5940, Fall 2024) Lecture 20: Distributed Training Part 2 Instructor: Prof. Song Han Slides: https://efficientml.ai Outline: - Hybrid (mixed) parallelism and how to auto-parallelize - Understand the bandwidth and latency bottleneck of distributed training - Gradient compression: overcome the bandwidth bottleneck - Gradient Pruning: Sparse Communication, Deep Gradient Compression - Gradient Quantization: 1-Bit SGD, TernGrad - Delayed gradient update: overcome the latency bottleneck </video> <video> EfficientML.ai Lecture 18 - Diffusion Models (MIT 6.5940, Fall 2024) Description: EfficientML.ai Lecture 18 - Diffusion Models (MIT 6.5940, Fall 2024) Lecture 18: Diffusion Models Instructor: Prof. Song Han Slides: https://efficientml.ai Outline: - Basics of diffusion model - Denoising diffusion models - Conditional diffusion models - Latent diffusion models - Image editing - Model personalization - Fast sampling techniques - Denoising diffusion implicit models - Distillation - Acceleration techniques - Sparsity - Quantization - Parallelism </video> <video> EfficientML.ai Lecture 23: Quantum Machine Learning Part 2 (Zoom Recording) (MIT 6.5940, Fall 2024) Description: EfficientML.ai Lecture 23: Quantum Machine Learning Part 2 (Zoom Recording) (MIT 6.5940, Fall 2024) Lecture 23: Quantum Machine Learning Part 2 Instructor: Prof. Hanrui Wang Slides: https://efficientml.ai Outline: - Parameterized Quantum Circuit (PQC) - PQC Training - Quantum Classifiers - Noise Aware On-Chip Training of PQC - TorchQuantum Library for QML - Robust Quantum Architecture Search </video> <video> EfficientML.ai 2024 | Introduction to SVDQuant for 4-bit Diffusion Models Description: EfficientML.ai 2024 | SVDQuant for 4-bit Diffusion Models Lecture 18: Diffusion Models Instructor: Prof. Song Han Slides: https://efficientml.ai </video> <video> EfficientML.ai Lecture 22: Course Summary + Quantum Machine Learning Part 1 (MIT 6.5940, Fall 2024) Description: EfficientML.ai Lecture 22: Course Summary + Quantum Machine Learning Part 1 (MIT 6.5940, Fall 2024) Lecture 22: Course Summary + Quantum Machine Learning Part 1 Instructor: Prof. Song Han and Prof. Hanrui Wang Slides: https://efficientml.ai Outline: - Course Summary - Introduce single qubit state and gates - Introduce multiple-qubit state and gates - Introduce quantum circuit - Introduce the NISQ era and compilation problems - Introduce the example workflow and compiler on neutral atom quantum computer </video> <video> EfficientML.ai Lecture 19 - Distributed Training Part 1 (Zoom Recording) (MIT 6.5940, Fall 2024) Description: EfficientML.ai Lecture 19 - Distributed Training Part 1 (Zoom Recording) (MIT 6.5940, Fall 2024) Lecture 19: Distributed Training Part 1 Instructor: Prof. Song Han Slides: https://efficientml.ai Outline: - Background and motivation - Parallelization methods for distributed training - Data parallelism - Communication primitives - Reducing memory in data parallelism: ZeRO-1 / 2/ 3 and FSDP - Pipeline parallelism - Tensor parallelism - Sequence parallelism </video> <video> EfficientML.ai Lecture 23: Quantum Machine Learning Part 2 (MIT 6.5940, Fall 2024) Description: EfficientML.ai Lecture 23: Quantum Machine Learning Part 2 (MIT 6.5940, Fall 2024) Lecture 23: Quantum Machine Learning Part 2 Instructor: Prof. Hanrui Wang Slides: https://efficientml.ai Outline: - Parameterized Quantum Circuit (PQC) - PQC Training - Quantum Classifiers - Noise Aware On-Chip Training of PQC - TorchQuantum Library for QML - Robust Quantum Architecture Search </video>

Contacts and Links

Description

MIT HAN Lab: Hardware, AI and Neural-nets

Accelerate Deep Learning Computing

Group website: hanlab.mit.edu
TinyML Project: tinyml.mit.edu
TinyML Course: efficientml.ai, streamed via live.efficientml.ai every Tuesday/Thursday 3:35-5:00pm ET, fall 2022.

Videos from channel