1 Preliminaries Mixture-of-Experts is an essential architecture choice when building LLMs. Since the prevalence of DeepSeekV3, companies will consider whether to use MoE structure before LLM pretrain

This is a lecture note of the course CSE 234 - Data Systems for ML - LE [A00].From UC SanDiegoProf. Zhang HaoWinter, 2025Link: https://podcast.ucsd.edu/watch/wi25/cse234_a00/1

This is a lecture note of the course CSE 234 - Data Systems for ML - LE [A00].From UC SanDiegoProf. Zhang HaoWinter, 2025Link: https://podcast.ucsd.edu/watch/wi25/cse234_a00/1

This is a lecture note of the course CSE 234 - Data Systems for ML - LE [A00].From UC SanDiegoProf. Zhang HaoWinter, 2025Link: https://podcast.ucsd.edu/watch/wi25/cse234_a00/1

This is a lecture note of the course CSE 234 - Data Systems for ML - LE [A00].From UC SanDiegoProf. Zhang HaoWinter, 2025Link: https://podcast.ucsd.edu/watch/wi25/cse234_a00/1

The series of tutorial is based on Flow Matching Guide and CodearXiv: 2412.06264Thank you, META Flow Matching Problem Instead of learning the likelihood of the target like the

The series of tutorial is based on Flow Matching Guide and CodearXiv: 2412.06264Thank you, META Flow Models Before flow matching, flow models are hard to train, it entails sol

The series of tutorial is based on Flow Matching Guide and CodearXiv: 2412.06264Thank you, META 1 The Definition of the Velocity ODE Let’s consider a particle moving in the sp

The Beginning of Everything Today marks exactly one year since I became an amateur AI researcher. I still remember how disheartened I was with my circumstances back then. I was originally supposed to

What happens when using LoRA? It starts with the error RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn when you have part of the parameter