Your browser isn’t supported anymore. Update it to get the best YouTube experience and our latest features. Learn more
Read the original article
Comments
By vimgrinder 2025-02-062:323 reply So much respect for this guy. He is like Neo of the matrix, bridging the gap between humans and machines. I have so far learned the following for free from his repos/videos:
1. minGPT, nanoGPT (transformers)
2. NLP (make more series)
3. tokenizers (his youtube)
4. RNN (from his blog)
There are many domains which don't have a karpathy and we don't hear about them. So glad we have this guy to spread his intuitions on ML.
By DoingIsLearning 2025-02-068:302 reply There are also different styles of teaching and learning. Karpathy always like to start from first principles and increment the building blocks.
Whereas for example Jeremy Howard's style resonates a lot more with how I enjoy learning, very much a "let's build it" and then tinker around to gain intuition on how things inside the box are working.
I see the benefit in both approaches and perhaps Karpathy is more methodical and robust. But I just find Howard's top-down style a lot easier to stay motivated with when I am learning on my own time.
Definitely agree. I think a lot of people get hung up on the math in ML but honestly there are so many other things you could spend time on, and there are opportunity costs for everything.
So I say, build the thing, figure out where the shortcomings in your knowledge are, and continue refining. One of those things will inevitably be math. Maybe it will be signals processing the next week or fundamentals of Spark the next. And there are always interesting papers coming out.
By vimgrinder 2025-02-0610:24 I definitely avoided ML for years just due to math. But having a chatbot who can explain math with examples in any style you want defintely changed my opinion about math and ML in general. A big barrier to math is how it's written imo and not explained in a fun way with lot of examples. I certainly don't have a mathy brain, but I do get things when explained with examples (and certainly find it hard to come up with my own examples while fighting with the symbols).
By attentionmech 2025-02-0610:21 Will checkout jeremy's lectures. I actually use his fastbook notebooks a lot to self-study.
Karpathy's style, for me is more like at the right abstraction to bring out curiosity in me towards the subject. After watching his lectures, i go on to more materials generally, and never really stop there.
By bamboozled 2025-02-0613:111 reply "Neo of the matrix", what great analogy! Made my day and gave me a good laugh, thanks.
He is for sure a cool guy.
By attentionmech 2025-02-0619:15 he has earned it haha.
By attentionmech 2025-02-0815:24 saw that video just now, thanks for this.
By levocardia 2025-02-060:19 I tell all my friends that Andrej was the best instructor I had in grad school, even though I didn't even go to Stanford--I just watched his CS321n videos on YouTube. Really thrilled that he's still making videos.
He's made more than 5 videos covering basically the same topic, of transformer architecture and training. Wonder whats different about this one?
My YouTube videos fall into two tracks:
1. technical track (all the GPT repro series)
2. general audience track
For (2), I had a 1hr video from 1 year ago, but I didn't actually expect that video to be some kind of authoritative introduction to LLMs. The history is that I was invited to give an LLM talk (to general audience), prepared some random slides for a day, gave the talk, and then re-recorded the talk in my hotel room later in a single take, and that become the video. It was quite random and haphazard. So I wanted to loop back around more formally and do a more comprehensive intro to LLMs for general audience; Something I could for example give to my parents, or a friend who uses ChatGPT all the time and is interested in it, but doesn't have the technical background to go through my videos in (1). That's this video.
By gnunez 2025-02-0522:44 Great work! I love your videos; they've taught me so much. Any plans for a Mixture of Experts (MoE) video? My understanding is that starting from GPT4 most advance models use MoE to some extent. For example, can I take the model from your GPT2 video and just change the feed forward layer to an MoE layer like the one found here (1)? I guess I can just try it myself but I enjoy the expert guidance you provide in your videos. Please don't stop! great content!
1. https://github.com/mistralai/mistral-inference/blob/main/src...
By gdjdkslslp 2025-02-0520:502 reply I haven’t watched this video yet, but do you plan to create any technical videos in the (1) series on RL in LLMs?
By MasterScrat 2025-02-060:03 His intro to RL (not for LLM) blog post is a great read FYI
By amelius 2025-02-0521:23 This would be very welcome as it brings us closer to understanding the secret sauce behind training a real, practical LLM.
By pj_mukh 2025-02-065:30 "Something I could for example give to my parents,"
Not for nothing, Andrej's parents maybe different from mine, but I definitely can't send this to my parents.
By yingliu4203 2025-02-065:01 Both are the best videos in NN and LLM. Your kindness is highly appreciated.
By gardenhedge 2025-02-0522:31 I watched that one after finding it randomly! Will give the new one a watch too
By carbine 2025-02-0617:35 thank you SO much for all your hard work, we sincerely appreciate it
By ks2048 2025-02-0519:29 From the description:
I think he has videos on building GPT2 from scratch, but this seems more high-level.I have one "Intro to LLMs" video already from ~year ago, but that is just a re-recording of a random talk, so I wanted to loop around and do a lot more comprehensive version.By hustwindmaple1 2025-02-0523:421 reply When he drops a vid, you don't ask questions. You watch first and then ask questions :)
By lolinder 2025-02-064:20 I appreciated the question, because the video is 3hr30min long, and karpathy's answer here was helpful to know that it's not directed at me.
By thomassmith65 2025-02-0611:25 Among other things, this video is recent enough to discuss DeepSeek R1.
