I'm working on Damon[1], a Nomad Events stream operator that automates cluster operations and eliminates repetitive DevOps tasks. It's a lightweight Go binary that monitors the Nomad events stream and triggers actions based on configurable providers.

A few examples of what it can currently do:

- Automated data backup: Listens for Nomad job events and spawns auxiliary jobs to back up data from services like PostgreSQL or Redis to your storage backend based on job meta tags. The provider for this is not limited to backups, as it allows users to define their custom job and ACL templates, and expected tags. So it can potentially run anything based on the job registration and de-registration events.

- Cross-namespace service discovery: Provides a lightweight DNS server that acts as a single source of truth for services across all namespaces, solving Nomad's limitation of namespace-bound services. Works as a drop-in resolver for HAProxy, Nginx, etc.

- Event-driven task execution: Allows defining custom actions triggered by specific Nomad events; perfect for file transfers, notifications, or kicking off dependent processes without manual intervention. This provider takes in a user-defined shell script and executes it as a nomad job based on any nomad event trigger the user defines in the configuration.

Damon uses a provider-based architecture, making it extensible for different use cases. You can define your own providers with custom tags, job templates, and event triggers. There's also go-plugin support (though not recommended for production) for runtime extension.

I built this to eliminate the mundane operational tasks our team kept putting off. It's already saving us significant time and reducing gruntwork in our clusters.

Check out the repository[1] if you're interested in automating your Nomad operations. I'd love to hear your thoughts or answer any questions about implementation or potential use cases!

[1]: https://github.com/Thunderbottom/damon

No, unfortunately our company’s and external regulatory compliance policies require us to host all data within the country itself, alongside it being required to be run on an infrastructure that is easily auditable. So as a policy within the company, all our internal services are open source and self hosted.

We had separate database and app instances, the DB instance had 32 cores and 64GB memory, which we doubled to keep up with our requirements. We have tens of millions of visits daily, and our database was close to ~300GB within the first few months.

For plausible I believe that since it runs on Postgres, scaling should not be a problem as long as you scale the resources with it.

We had huge servers, with the database and the application itself running on separate instances. IIRC, we had a 32 core, 64GB instance just for the DB itself which we doubled when we started adding more sites to our configuration and it still wasn’t enough. As for the numbers, our site(s) get heavy traffic everyday, in millions daily, since we are a stock broker.

You’re right about MySQL performing alright for 10-20m rows, but from our perspective those numbers are not that big for a company this size.