A delightful Ruby way to work with AI. No configuration madness, no complex callbacks, no handler hell – just beautiful, expressive Ruby code. - crmne/ruby_llm
Read the original article
Comments
This interface needs to have a better relationship with streaming, there is always a lag in response and a lot of people are going to want to stream the response in non blocking threads instead of hanging the process waiting for the response. Its possible this is just a documentation issue, but either way streaming is a first class citizen on anything that takes more than a couple seconds to finish and uses IO.
Aside from that the DSL is quite excellent.
By bradgessler 2025-03-1517:28 There’s a whole world of async IO in Ruby that doesn’t get enough attention.
Checkout the async gem, including async-http, async-websockets, and the Falcon web server.
By earcar 2025-03-1521:40 Thank you for your kind words!
Valid point. I'm actually already working on testing better streaming using async-http-faraday, which configures the default adapter to use async_http with falcon and async-job instead of thread-based approaches like puma and SolidQueue. This should significantly improve resource efficiency for AI workloads in Ruby - something I'm not aware is implemented by other major Ruby LLM libraries. The current approach with blocks is idiomatic Ruby, but the upcoming async support will make the library even better for production use cases. Stay tuned!
From https://rubyllm.com/#have-great-conversations
# Stream responses in real-time chat.ask "Tell me a story about a Ruby programmer" do |chunk| print chunk.content endThis will synchronously block until ‘chat.ask’ returns though. Be prepared to be paying for the memory of your whole app tens/low hundreds of MB of memory being held alive doing nothing (other than handling new chunks) until whatever streaming API this is using under the hood is finished streaming.
By andrewmutz 2025-03-1520:081 reply Threads?
Rails is a hot ball of global mutable state. Good luck with threads.
By andrewmutz 2025-03-163:021 reply The default rails application server is puma and it uses threads
Yes, it does. Ruby has a global interpreter lock (GIL) that prevents multiple threads to be executed by the interpreter at the same time, so Puma does have threads, they just can’t run Ruby code at the same time. They can hide IO though.
By andrewmutz 2025-03-1713:351 reply The GIL is released during common IO operations like the HTTP requests that power LLM communication
By jupp0r 2025-03-1714:36 The Rails documentation has lots of info about this: https://guides.rubyonrails.org/tuning_performance_for_deploy...
Concurrency support is missing from the language syntax and this particular library as a concept. This is by design, to not distract from beautiful code. Your request will make zero progress and take up memory while waiting for the LLM answer. Other threads might make progress on other requests, but in real world deployments this will be a handful (<10). This server will get 10s of requests per second when something written in JS or Go will get many 1000s.
It’s amazing how the Ruby community argues against their own docs and doesn’t acknowledge the design choices their language creators have made.
By kyledrake 2025-03-1523:05 That looks good, I didn't see that earlier.
Such a breath of fresh air compared to poor DX libraries like langchain
By nullpoint420 2025-03-153:024 reply I’ve found the Ruby community really cares about DUX. Not sure why it’s not in other language communities
By toasterlovin 2025-03-155:512 reply I don’t really mean this to be derogatory toward people who enjoy other things, but Ruby is a language and ecosystem by and for people who have taste.
By continuational 2025-03-157:004 reply Certainly a taste for global state, it seems.
Dogmatically rejecting global state even if it simplifies things in some particular case _is_ poor taste.
Even a goto can be elegant sometimes.
By zelphirkalt 2025-03-158:261 reply Usually though it just creates long term issues, putting off important work to later.
Sometimes later never comes, so it's a net win compared to languages where you are forced do this important work now.
But when later does come, it can take dev-years to fully disentangle the global state and allow code reuse. Did you gain dev-years in productivity by using it in the first place? Probably not.
If you have good reason to believe that an app will stick around for more than a year, be maintained by more than 3 people, or grow to more than 500k lines of code (sub in whatever metrics make sense to you), don't put off removing global state for later. You will regret it eventually, and it doesn't cost much to do it right the first time.
(Also, no mainstream language I'm aware of forces you to not use global state. Even Java, famed for its rigidity, has global state readily available if you really do need it.)
By trevorhinesley 2025-03-1514:011 reply You’re describing the pain of poor architecture rather than the pain of global state. The tool itself is neutral. Sharp knives and all that.
Global state is a tool that will almost always lead to bad architecture in an app where architecture matters. I'm sure you can point to a counterexample or two where a set of devs managed to keep disciplined indefinitely, but that doesn't change the fact that allowing people to reach into a mutable variable from anywhere in the system enables trivially accessible spooky action at a distance, and spooky action at a distance is a recipe for disaster in a medium to large code base.
In a project with more than a few people on it, your architecture will decay if it can decay. Avoiding global state removes one major source of potential decay.
By trevorhinesley 2025-03-1515:401 reply “Almost” is key there. I respect your position, but it’s an always/never take, and the longer I am in this industry, the more I find myself leaning into “it depends.” Here’s a take that articulates this being done well on a large codebase better than I can in a short comment: https://dev.37signals.com/globals-callbacks-and-other-sacril...
> it’s an always/never take
No, it isn't—I'm the one who inserted the word "almost" into that sentence! Where did you get the idea that I meant always/never?
Like I said, you can point to exceptions but that doesn't change the rule. It's better to teach the rule and break it when you really know what you're doing—when you understand that you're breaking a rule and can articulate why you need to and why it's okay this time—than it is to spread the idea that globals are really just fine and you need to weigh the trade-offs. The odds are strongly against you being the exception, and you should act accordingly, not treat globals as just another tool.
Sometimes amputation is the right move to save someone's life, but you certainly should not default to that for every papercut. It's a tool that comes out in extreme circumstances only when a surgeon can thoroughly justify it.
By trevorhinesley 2025-03-160:25 Respectfully, your response further qualifies what I meant by your take being an always/never. I’m aware you’re the one who put “almost” in there, and I didn’t meant to imply you were being stubborn with that take, that’s why I said (and genuinely meant) that I respect it.
But I’m also aware that you’re comparing using global state to amputating a human limb. I don’t think it’s nearly that extreme. I certainly wouldn’t say global state “almost always leads to bad architecture,” as evidenced by my aligning with a framework which has a whole construct for globals baked into it (Rails’ Current singleton) that I happen to enjoy using.
Sure, global state is a sharp knife, which I already said. It can inflict pain, but it’s also a very useful tool in certain scenarios (more than would equate to “almost [never]” IMO).
So your response aligns with how I took your original post, and what I inferred “almost” really meant: basically never. My point is that I don’t agree with your take being a “rule.” While I understand your perspective, instead of saying basically never, I would say, “it depends.”
By IshKebab 2025-03-1510:08 Don't forget ungreppable code! And what are type hints anyway?
By techscruggs 2025-03-1513:44 Well, taste for a Global Interpreter Lock, at least.
By simpaticoder 2025-03-1513:261 reply Global state is wonderful when the world is small. Rubyfolk then keep the world small, which has many other benefits.
By lolinder 2025-03-1513:32 Sorry, I like Ruby, but this is nonsense. Rails apps get enormous very quickly, like apps written in every other framework. In most work you can't just declare that your world will be small, your world is as big as your problem is.
By madeofpalk 2025-03-1512:18 For a certain type of taste.
Matz said he designed Ruby to optimize for developer happiness, it’s just a core principle of the language since it was created
Happiness of a developer writing code can be a misery of a one having to read / debug it. I worked in ruby for a couple years around 2009 and having to deal with a code that implemented most of its logic via method missing is still one of the strongest negative memories I have about coding.
By MatthiasPortzel 2025-03-1511:501 reply `binding.irb` and `show_source` have been magical in my Ruby debugging experience. `binding.irb` to trigger a breakpoint, and `show_source` will find the source code for a method name, even in generated code somehow.
By richjdsmith 2025-03-1518:12 … I’ve been using Ruby for years and never thought to use show_source like this in a debugger. Thanks kind stranger, you just made my day!
By viraptor 2025-03-159:07 Another annoying one from that category is Ruby's forwarded methods. Since they're created via generated, injected code, you can't query which method it forwards to at runtime. Or not easily anyway.
By RangerScience 2025-03-1519:58 Yep yep, that's the whole "sharp knives" thing.
What I advise (and aim for) is only pulling out the sharp knives for "library" code, but application code should stay "simple" (and this much more easily navigable). Otherwise you can absolutely make a bloody mess!
By RangerScience 2025-03-1519:551 reply Every language prioritizes something (or somethings) because every language was made by a person (or people) with a reason; python and correctness; Java and splitting up work; Go and something like "simplicity" (not that these are the only priorities for each language). As another comment points out, Matz prioritized developer happiness.
My favorite example of this is the amazing useful and amazing whack Ruby array arithmetic; subtraction (`arr1 - arr2`) is element-wise removal, but addition (`arr1 + arr2`) is a simple append. These are almost always exactly what you want to do when you reach for them, but they're completely "incorrect" mathematically.
By okeuro49 2025-03-1521:40 > python and correctness
I thought it was Python and readability and "one way of doing things".
Umm, doesn’t Go do so as well? Personally, I’ve had a better experience working with Go tooling.
By danenania 2025-03-157:40 I'd say they both optimize for DX, but they come at it from very different angles. Ruby is focused on actually writing the code: making it feel expressive, intuitive, and fun.
Go is more about making it easier to build fast and robust systems. But it really doesn't care if the code itself is ugly and full of boilerplate.
As I've gotten more experience, I've come to really appreciate Go's tradeoffs. It's not as fun up front, but on the other hand, you're less likely to get server alerts at 4am. It really depends what you're building though.
Go ecosystem is generally good. However, given that Go as a language doesn't have any "fancy" (for the lack of a better word) syntactical features you can't create DSL's like this
though Ruby's expressiveness comes at a cost and I'd personally stick with Go in a team but use something like RubyLLM for personal projects
I'm wary of equating "ability to create dsls like this" with "prioritizing developer experience".
Ruby and go each prioritize different parts of the developer experience. Ruby prioritizes the experience of the author of the initial code at the expense of the experience of the maintainer who comes later. Go prioritizes the experience of a maintainer over the experience of the initial author. Both prioritize a developer, both de-prioritize a different developer, and which one makes more sense really depends on the ratio of time spent on writing greenfield code versus maintaining something another human wrote years ago who's long gone.
I disagree with the idea that Go prioritizes the maintainer. More lines of code typically makes maintenance more difficult. Go is easy to read line by line, but the verbosity makes it more challenging to understand the bigger picture.
I find changes in existing Go software often end up spreading far deeper into the app than you'd expect.
The runtime is fantastic, though, so I don't see it losing it's popularity anytime soon.
> More lines of code typically makes maintenance more difficult.
That’s kind of just the surface level of maintenance though. Go is not so much focused on making it easy to read a single file, but on minimizing the chains of abstraction and indirection you need to follow to understand exactly how things work.
It’s much more likely that all the logic and config to do something is right there in that file, or else just one or two “Go to definition” clicks away. You end up with way more boilerplate and repetition, but also looser coupling between files, functions, and components.
Contrast that to a beautiful DSL in Ruby. It’s lovely until it breaks or you need to extend it, and you realize that a small change will require refactoring call sites across a dozen different files. Oh and now this other thing that reused that logic is broken, and we’ve got to update most of the test suite, and so on.
> or else just one or two “Go to definition” clicks away
This is the biggest part of it: maintainers need static analysis and/or (preferably and) very good grepability to help them navigate foreign code. Ruby by its nature makes static analysis essentially impossible to do consistently, whereas Go leans to the opposite extreme.
By smw 2025-03-2021:33 I've found for years that ctrl-b works pretty damn well in Rubymine as well as Goland. Huge advantage vs other editors 5+ years ago. I wonder if the difference is still as stark in the age of lsps?
By drdaeman 2025-03-1521:59 Surely you can have semantically the same API in Go:
And so on. Syntax is different, of course, but semantics (save for language-specific nuances, like error handling and lack of optional arguments) are approximately the same, biggest difference being WithSomething() having to precede Ask()// Must[T](T, error) T is necessary because of Go error handling differences chat := Must(gollm.Chat().WithModel("claude-3-7-sonnet-20250219")) resp := Must(chat.Ask("What's the difference between an unexported and an exported struct field?")) resp = Must(chat.Ask("Could you give me an example?")) resp = Must(chat.Ask("Tell me a story about a Go programmer")) for chunk := range resp { // Requires Go 1.23+ for iterators fmt.Print(chunk.Content) } resp = Must(chat.WithImages("diagram1.png", "diagram2.png").Ask("Compare these diagrams")) type Search struct { Query string `description:"The search query" required:"true"` Limit int `description:"Max results" default:"5"` } func (s Search) Execute() ([]string, error) { ... } resp = Must(chat.WithTool[Search]().Ask("Find documents about Go 1.23 features"))
By jasongill 2025-03-1515:06 I was an early contributor to Langchain and it was great at first - keep in mind, that's before chat models even existed, not to mention tools, JSON mode, etc.
Langchain really, I think, pushed the LLM makers forward toward adding those features but unfortunately it got left in the dust and became somewhat of a zombie. Simultaneously, the foundational LLM providers kept adding things to turn them more into a walled garden, where you no longer needed to connect multiple things (like scraping websites with one tool, feeding that into the LLM, then storing in a vector datastore - now that's all built in).
I think Langchain has tried to pivot (more than once perhaps) but had they not taken investor $$ early on (and good for them) I suspect that it would have just dried up and the core team would have gone on to work at OpenAI, Anthropic, etc.
langchain and llamaindex are such garbage libraries: not only they never document half of the features they have, but they keep breaking their APIs from one version to the next.
By brokegrammer 2025-03-1510:35 I was about to mention those. I decided a while ago to build everything myself instead of relying on these libraries. We could use a PythonLLM over here because it seems like nobody cares about developer experience in the Python space.
By earcar 2025-03-1521:42 Thank you! This is what the Ruby community has always prioritized - developer experience. Making complex things simple and joyful to use isn't just aesthetic preference, it's practical engineering. When your interface matches how developers think about the problem domain, you get fewer bugs and more productivity.
Would anyone happen to know of a similar library with as good DX but for JavaScript or TypeScript?
Perhaps something like https://llmjs.themaximalist.com/
By olegp 2025-03-1511:07 Great, thank you!
By kakasoo 2025-03-172:37 [dead]
By SkyPuncher 2025-03-1518:05 IMO, the samples look great because they're ridiculously simple.
It doesn't deal with any of the hard problems you'll routine face with implementation.
By someothherguyy 2025-03-155:07 What about it is a breath of fresh air? What do the other libraries do that this doesn't?
By gregmolnar 2025-03-1515:042 reply Be careful with the examples though: https://github.com/crmne/ruby_llm/issues/25
