Hey HN, Chris and Yuhong here from Onyx (https://github.com/onyx-dot-app/onyx). We’re building an open-source chat that works with any LLM (proprietary + open weight) and gives these LLMs...
Demo: https://youtu.be/2g4BxTZ9ztg
Two years ago, Yuhong and I had the same recurring problem. We were on growing teams and it was ridiculously difficult to find the right information across our docs, Slack, meeting notes, etc. Existing solutions required sending out our company's data, lacked customization, and frankly didn't work well. So, we started Danswer, an open-source enterprise search project built to be self-hosted and easily customized.
As the project grew, we started seeing an interesting trend—even though we were explicitly a search app, people wanted to use Danswer just to chat with LLMs. We’d hear, “the connectors, indexing, and search are great, but I’m going to start by connecting GPT-4o, Claude Sonnet 4, and Qwen to provide my team with a secure way to use them”.
Many users would add RAG, agents, and custom tools later, but much of the usage stayed ‘basic chat’. We thought: “why would people co-opt an enterprise search when other AI chat solutions exist?”
As we continued talking to users, we realized two key points:
(1) just giving a company secure access to an LLM with a great UI and simple tools is a huge part of the value add of AI
(2) providing this well is much harder than you might think and the bar is incredibly high
Consumer products like ChatGPT and Claude already provide a great experience—and chat with AI for work is something (ideally) everyone at the company uses 10+ times per day. People expect the same snappy, simple, and intuitive UX with a full feature set. Getting hundreds of small details right to take the experience from “this works” to “this feels magical” is not easy, and nothing else in the space has managed to do it.
So ~3 months ago we pivoted to Onyx, the open-source chat UI with:
- (truly) world class chat UX. Usable both by a fresh college grad who grew up with AI and an industry veteran who’s using AI tools for the first time.
- Support for all the common add-ons: RAG, connectors, web search, custom tools, MCP, assistants, deep research.
- RBAC, SSO, permission syncing, easy on-prem hosting to make it work for larger enterprises.
Through building features like deep research and code interpreter that work across model providers, we've learned a ton of non-obvious things about engineering LLMs that have been key to making Onyx work. I'd like to share two that were particularly interesting (happy to discuss more in the comments).
First, context management is one of the most difficult and important things to get right. We’ve found that LLMs really struggle to remember both system prompts and previous user messages in long conversations. Even simple instructions like “ignore sources of type X” in the system prompt are very often ignored. This is exacerbated by multiple tool calls, which can often feed in huge amounts of context. We solved this problem with a “Reminder” prompt—a short 1-3 sentence blurb injected at the end of the user message that describes the non-negotiables that the LLM must abide by. Empirically, LLMs attend most to the very end of the context window, so this placement gives the highest likelihood of adherence.
Second, we’ve needed to build an understanding of the “natural tendencies” of certain models when using tools, and build around them. For example, the GPT family of models are fine-tuned to use a python code interpreter that operates in a Jupyter notebook. Even if told explicitly, it refuses to add `print()` around the last line, since, in Jupyter, this last line is automatically written to stdout. Other models don’t have this strong preference, so we’ve had to design our model-agnostic code interpreter to also automatically `print()` the last bare line.
So far, we’ve had a Fortune 100 team fork Onyx and provide 10k+ employees access to every model within a single interface, and create thousands of use-case specific Assistants for every department, each using the best model for the job. We’ve seen teams operating in sensitive industries completely airgap Onyx w/ locally hosted LLMs to provide a copilot that wouldn’t have been possible otherwise.
If you’d like to try Onyx out, follow https://docs.onyx.app/deployment/getting_started/quickstart to get set up locally w/ Docker in <15 minutes. For our Cloud: https://www.onyx.app/. If there’s anything you'd like to see to make it a no-brainer to replace your ChatGPT Enterprise/Claude Enterprise subscription, we’d love to hear it!
Comments
This is great, the value is there. I work for a F100 company that is trying (and failing) to build this in house because every product manager fundamentally misunderstands that users just want a chat window for AI, not to make their own complicated agents. Your biggest competition in the enterprise space, Copilot, has terrible UI and we only put up with it because it has access to email, SharePoint and Teams.
By paool 2025-11-2617:36 > every product manager fundamentally misunderstands that users just want a chat window for AI,
I would actually argue chat windows are terrible ui/ux for most cases and users. It does the opposite of `don't make me think`. Too much potential for user error.
Not saying there shouldn't be any LLM integration/features, just that it should be in the form of a button press or something (familiar ux), not the same chatgpt interface that all the early apps are trying to mimic for no good reason.
By Weves 2025-11-2515:42 Haha, yea we've seen that exact story many times! Dissatisfied with Copilot and building a (not great) internal solution that is missing polish + most of the "advanced" feature set.
By echelon 2025-11-265:45 > I work for a F100 company that is trying (and failing) to build this in house because every product manager fundamentally misunderstands that users just want a chat window for AI
If you're a non-tech company, why doesn't your org dictate a single model provider? How do these decisions work internally, and how to the departments consume them? (Are they consuming the tools?)
> make their own complicated agents.
Asking a non-tech employee to make an agent sounds like hell.
> we only put up with it because it has access to email, SharePoint and Teams.
Ah, that's how a third party can make money. Bake in external org-wide knowledge and enable search.
I immediately thought of Google's Agentspace when I saw this product. The value for me sits in its ability to do RAG via connectors.
RAG + connectors is a huge reason why people deploy Onyx (enterprise search roots means we do a pretty good job there).
Also, open-source works really here, since connectors are a long-tail game. We've tried to make it easy to add connectors (a single python interface), and as a result over half of our connectors are contributed by the community. We expect that percentage to grow over time. This means that compared to something like Agentspace, we'll very likely be connected to all of the key tools at your company (and if we aren't, you can easily add an integration).
By adam2221 2025-11-2822:14 Do you accept connectors? For instance, I saw that someone wished to develop a file system connector (which I saw that many people need) by himself, but there didn't appear to be much cooperation.
I was pretty excited for Onyx as a way to stand up a useful open source RAG + LLM at small scale but as of two weeks ago it was clearly full of features ticked off a list that nobody has actually tried to use. For example, you can scrape sites and upload docs but you can’t really keep track of what’s been processed within the UI or map back to the documents cleanly.
It’s nice to see an attempt at an end to end stack (for all that it seems this is “obvious” … there are not that many functional options) but wow we’ve forgotten the basis of making useful products. I’m hoping it gets enough time to bake.
Really appreciate the feedback (and glad to hear the core concept resonated with you).
The admin side of the house has been missing a bit of love, and we have a large overhaul coming soon that I'm hoping addresses some (most?) of your concerns. For now, if you'd like to view documents that have been processed, you can check out the `Explorer` panel on the left.
In general, I'd love to hear more about what gives it that "unbaked" feel for you if you're up for a quick chat.
By rao-v 2025-11-2620:02 Hey - good response!
I'm sure you guys are thinking about this, but please just go through the steps of setting up via docker, uploading say a grad student's worth of papers and docs, scrape a small topic off wikipedia, try and use it for three days and take a look at the ergonomics. It's not easy to regroup sets of documents, get results that link to the document to view post indexing for RAG etc. etc. etc.
In general there are a lot of low hanging RAG optimizations that you could do to make this usable for people who don't want to write their own bits of code to make it usable. I ended up fiddling a bit more with anythingllm which, while having fewer features, understands the workflows a bit more.
By CuriouslyC 2025-11-2518:043 reply Something like this has a very limited shelf life as a product. What users need from chat is very user specific, trying to be the one chat to rule them all is not gonna end well, and as models get more capable each chat experience is going to need to be more customized.
Something like this could have a nice future as an open source chat framework for building custom UIs if it's well made and modular, but that isn't gonna work well with a SaaS model.
I've been using Onyx (and Danswer before it) for over a year, and I'd push back on this. We have Freshdesk, Bookstack, Google Drive, YouTrack, and Slack all connected. It seamlessly answers questions like:
"What's Max's GitHub username?" "I need wire transfer instructions for an incoming wire"
We also index competitors' helpdesks and KB articles to track new features they're rolling out. Our tech support team uses it daily because Freshdesk's AI is terrible and their internal KB search is lackluster. Onyx actually finds things. The value isn't in being "one chat to rule them all" — it's in unified search across disparate systems with citations. That's not getting commoditized anytime soon. Keep up the good work, team.
By nrhrjrjrjtntbt 2025-11-264:251 reply > "I need wire transfer instructions for an incoming wire"
Ooooof. Careful with that.
By DANmode 2025-11-275:44 Definitely follow the sourced permalink as a default on that one.
By jstummbillig 2025-11-2518:251 reply I disagree. This has both APIs as well as connectors. One of the reasons I use Google Workspace as SaaS is because of the extensive API, that gives me the flexibility I need with a great starting point (and continued development, that I continue to benefit from).
By CuriouslyC 2025-11-2518:481 reply Yes, but imagine a chat app that's designed for accountants, that has widgets for accounting, and it's set up for accounting workflows. That's _HUGE_ but not something that a "one chat to rule them all" is going to just go and do. You could use that same example for lab technicians and any other role.
Same "verticalization" argument people made for search in the early 2000s. There's a lot more use cases for tech than just vertical SaaS apps.
By CuriouslyC 2025-11-2614:071 reply I don't think that comparison holds, search is a task you do as part of a workflow, there isn't a big difference in search across verticals other than curation of the data set you're searching from. If chat is becoming how people do their work, I don't see how product proliferation across verticals isn't going to be a thing.
By motoxpro 2025-12-027:09 That's only obvious in hindsight. Back in the day, people said how in the world am I going to use the same search box for buying a car (color, make, model, photos) as I do for finding the latest concerts in my area (map, seating charts, etc.)
And you're right vertical SaaS DID become a thing, and so will vertical AI, but the horizontal versions of both (search and SaaS) crush the vertical ones (Google for search Microsoft for SaaS), and I believe it will be the same. Theres a layer above what you are talking about (e.g. Teams as the product vs Slack as the company).
Horizontal has a higher TAM and Vertical is easier to execute.
But this is besides the point. My point was that productivity is a minority of the TAM.
Hmm, will have to disagree here. I think "one chat to rule them all" is the way it will end.
It does requires having UI components for many different types of interactions (e.g. many ways to collect user input mid-session + display different tools responses like graphs and interactives). With this, people should be able to easily build complex tools/flows on top of that UI, and get a nice, single interface (no siloed tools/swapping) for free. And having this UI be open-source make this easier.
By CuriouslyC 2025-11-2518:50 I agree with an end state something like you describe, but I don't think it will be a chat app, I think you'll have an agent lives outside your apps, that managers your apps.
