I've been using ChatGPT to teach myself all sorts of interesting fields of mathematics that I've wanted to learn but never had the time previously. I use the Pro version to pull up as many actual literature references as I can.

I don't use it at all to program despite that being my day job for exactly the reason you mentioned. I know I'll totally forget how to program. During a tight crunch period, I might use it as a quick API reference, but certainly not to generate any code. (Absolutely not saying it's not useful for this purpose—I just know myself well enough to know how this is going to go haha)

I’ve always thought the issue was a bit less “Find the interesting research problem” and more “Find the resources, network, or skills that get you into the position of being able to work on the interesting research problem.”

If you asked a bunch of researchers working on the “boring” stuff to predict what the hot papers of the year will be about, do we really think they’ll be that far off base? I’m not talking about groundbreaking or truly novel ideas that seem to come out of nowhere, but rather the high impact research that’s more typical of a field.

Even in big tech companies, it’s quite obvious what the interesting stuff to work on is. But there are limited spots and many more people who want those spots than are available.

“You’re not [X]—you’re [Y]” is the one that drives me nuts. [X] is typically some negative characterization that, without RLHF, the model would likely just state directly. I get enough politics/subtext from humans. I’d rather the LLM just call it straight.

It’s kind of intriguing that predicting the future state of any quantum system becomes almost trivial—assuming you can diagonalize the Hamiltonian. But good luck with that in general. (In other words, a “simple” reference frame always exists via unitary conjugation, but finding it is very difficult.)

“Live next to your relatives because if they get in a car accident and you live across the country you won’t be there to tell them goodbye.”

^Another one I’ve never understood. Like geez, hopefully my daughter doesn’t give up her life dreams just based on the possibility I might be in a freak accident one day...