For me it is an active question if coding training data "purity" matters. Python has Go on volume, but within that is a ton of API changes, language changes, etc. Is that free regularization or does it poison the dataset? As the author points out Go code is nominal because basically all published Go code looks the same and the library APIs are frozen in time to some degree.

I actually spent some time trying to get to the bottom of what a logical extension of this would be. An entirely made up language spec for an idealized language it never saw ever, and therefore had no bad examples of it. Go is likely the closest for the many reasons people call it boring.

I find the dependency creep for both rust and node unfortunate. Almost anything I add explodes the deps and makes me sweat for maintenance, vulnerabilities, etc. I also feel perpetually behind, which I think is basically frontend default mode. Go does the one thing I wish Rust had more of which is a pretty darn great standard library with total backwards compatibility promises. There are awkward things with Go, but man, not needing to feel paranoid and how much can be built with so little feels good. But I totally understand just getting crap done and taking off the tin foil. Depends on what you prioritize. Solo devs don't have the luxury.

I have run into a surprising number of basic syntax errors on this one. At least in the few runs I have tried it's a swing and a miss. Wonder if the pressure of the Claude release is pushing these stop gap releases.

I have run into that a lot which is annoying. Even though all the code compiles because go is backwards compatible it all looks so much different. Same issue for python but in that case the API changes lead to actual breakage. For this reason I find go to be fairly great for codegen as the stability of the language is hard to compete with and the standard lib a powerful enough tool to support many many use cases.

I find it fascinating to give the LLMs huge stacks of reflective context. It's incredible how good they are at feeling huge amounts of csv like data. I imagine they would be good at trimming their context down.

I did some experiments by exposing the raw latent states, using hooks, of a small 1B Gemma model to a large model as it processed data. I'm curious if it is possible for the large model to nudge the smaller model latents to get the outputs it wants. I desperately want to get thinking out of tokens and into latent space. Something I've been chasing for a bit.