Go is my hammer, and everything is a nail

2024-08-1212:59479798www.maragu.dev

Always choose the right tool for the job? Nah. I know Go quite well, and I use it wherever I can. Want to find out why?

You know that old saying:

Always choose the right tool for the job.Shakespeare, or someone like that

Well, I’ve turned that one on its head. I’m using one tool for all possible tasks, for what I believe are good reasons. But I’m getting ahead of myself.

Let’s start somewhere else. I’m Markus, a solo developer and digital product builder. It’s just me. I have no team or subcontractors, and I intend to keep it that way. I’ve pursued a lifestyle that includes having my own business, because it gives me the most flexibility of how and when I want to work. To structure my work around the rest of my life, so to speak. I want to collaborate, but not manage other people.

As a solo developer, I believe it’s a good idea to choose your tools wisely. Even more so than when in a team setting, having a tech stack that is too complex can bury you in unnecessary busy-work; you’ll end up building infrastructure and nursing your layers of technology all day, instead of, you know, talking to customers and building what they want. Delivering value to someone, to anyone, to the world.

In 2016, I started building stuff in Go. I originally didn’t like it, but I started “getting it”, and it stuck with me. So much that I eventually turned my back on other programming languages. I was writing a lot of Python and Javascript at the time, but I dropped that. I ended up making a course for Go developers, as well as sort-of kind-of reinventing React for backend Go HTML generation. I now build basically all my software in Go, from tiny one-off tools, to web services, CLIs, and everything in between.

But why? When the common wisdom is to always take the problem at hand, analyze it, and then choose the tools, why would I ignore that and go: “nah, I’ll just use Go again”?

Well, I present to you: REASONS.

Reason 1: Go can do basically anything

Yeah, Go can do basically anything. Actually, I’ll widen that statement to all popular programming languages can do basically anything. Yes, most are known for one thing or another, and they definitely have their strengths and weaknesses. But I think at the end of the day, it basically comes down to opinion and taste.

I like Go. It’s simple enough, easy to read, consistent, and a lot of other good things that align well with how I want to build software. There are certainly also things I don’t like, but I don’t care enough to let it bother me. But you probably know all of those things already.

Go is good for the obvious things: building CLIs, cloud infrastructure, HTTP servers, network stuff.

But people also build games for the Nintendo Switch in Go. They build GUI apps using Go. I’ve even tried a string synthesizer in the terminal built in Go, which I thought was beyond cool. You get the point: it’s often possible to do what you want, regardless of the tool you use.

Reason 2: Less context switching

Do you know the feeling when you start writing syntax for one programming language, realize that, nope, you’re supposed to be using a different one, and try to switch your brain into another mode while feeling slightly dumb? I know I used to.

But I don’t anymore, because I only use one programming language! Ha! Joke’s on past me.

But it goes much deeper than that. A programming language isn’t just a bunch of syntax to build stuff. It’s also a compiler, an IDE, a toolchain. A community to engage with. A never-ending news machine to keep up with. An ecosystem of libraries, tools, programs, idioms, styles. It can be quite a mouthful on its own, so I found it overwhelming to try to keep up with several. Is it possible? Sure. Will it be more superficial? Also yep.

Which brings me to my third reason.

Reason 3: Depth of knowledge

When I don’t have to keep up with everything around several languages, and build everything in just one language, I can go deep. Learn stuff I wouldn’t have gone into otherwise. Investigate nooks and crannies I wouldn’t have given a thought. Learn esoteric language features, history, read obscure blog posts on details I wouldn’t necessarily have bothered with.

That doesn’t mean I ignore the rest of the world, obviously. You and I both know there’s always more to learn in this field. Cutting away programming languages doesn’t make software development less complex in a multitude of other aspects, but it does so in one very concrete setting, and one which I spend a lot of time on.

So, what, I’m going to limit my career options?

The world is laaaarge. The number of projects are basically infinite. Even if I carve out a tiny subset of infinite, that’s still infinite. Good enough for me and my career ambitions.

And I’ll be really good at at least this one thing. I’m way more productive in Go than when I was just starting out (obviously, otherwise I should probably have switched to plumbing), and I hope to stay on that path, and maybe even increase the slope. EXPONENTIALLY. Okay, less will do as well.

Anyhoo, these are my thoughts. Bring on the nails, and I’ll beat them with a bloody, determined gopher who is screaming “IS THAT ALL YOU GOT?!?”.

A picture of me, Markus.

I’m Markus, a professional software consultant and developer. 🤓✨ You can reach me at markus@maragu.dk, or check social links in the page footer.


Read the original article

Comments

  • By bborud 2024-08-1214:3120 reply

    People always under-estimate the cost of properly learning a language. At any given time I tend to have a "main go-to language". I typically spend 2-4 years getting to the point where I can say I "know" a language. Then I try to stick to it long enough for the investment to pay off. Usually 8-10 years.

    A surprising number of people think this is a very long time. It isn't. This is typically the time it takes to understand enough of the language, the compiler, the runtime, the standard library, and idiomatic ways to do things. It is the time it takes to where you can start to meaningfully contribute to evolving how the language is used and meaningfully coach architects, programmers and system designers. It is also what you need to absorb novices into the organization and train them fast.

    • By m_a_g 2024-08-1311:402 reply

      Here is the thing: many software engineers don't need to learn a language "properly."

      When you start a new job, there will almost always be an existing code base, and you'll have to make contributions to it. Pattern recognition will get you a long way before you need to dive deep into the language internals.

      • By bborud 2024-08-1314:151 reply

        You are mixing two things. Learning a language and learning a codebase. Those are not the same thing. Besides, you are also just talking about what happens during the first N months. Most companies I've worked for allow for time to learn the language(s) being used. Some of them will fire you if you can't show sufficient progress over time.

      • By rkangel 2024-08-1312:302 reply

        This is true, and it makes the people who do actually know the language in depth extra valuable.

        • By tekknik 2024-08-1412:011 reply

          Not so much. I know quite a few languages very well, more than the average programmer, and more often than not a senior programmer overrides your recommendation and don’t know the language as well as you or some other red tape. What has never happened is getting an increase in pay for knowing a language well.

          So I share the sentiment that learning a language well is valuable. It’s not, and with the newer AI tools coming out there will soon be no reason to learn any language in depth.

          • By sevensor 2024-08-1413:11

            I think you have that exactly backwards. LLM-based tools do best on shallow-knowledge tasks. It’s depth where they struggle. Show me the syntax for constructing a lazy sequence in your language? LLM. Reason about its performance characteristics and interactions with other language features? Human, for the foreseeable future.

        • By lnenad 2024-08-1313:23

          Only for situations where that knowledge is required.

    • By saghm 2024-08-132:281 reply

      > It is the time it takes to where you can start to meaningfully contribute to evolving how the language is used and meaningfully coach architects, programmers and system designers. It is also what you need to absorb novices into the organization and train them fast.

      I think these criteria in particular are much more than a lot of people mean when they say "learn a language", which would explain why their estimates are lower than yours. You're talking about expertise, whereas plenty of people are speaking of basic competency. This seems more like (natural) language semantics about what it means to "know" and the definition of "properly learn" than about the accuracy of people's estimates.

      • By bb88 2024-08-133:394 reply

        2012: Python is Awesome!

        2014: Python is a great language, but there are a few pitfalls

        2016: Python is a good language with the right IDE, tooling, and process. The people are pretty cool though.

        2018: I like python, but I wish more people used type annotations.

        2020: You know, metaclasses are freaking awesome! They saved me so much work!

        2022: Why can't people code the most obvious solution in python?

        2024: Celery! Jesus H. Christ! What were you thinking?!

        • By sph 2024-08-136:268 reply

          2012: Python is Awesome!

          2014: Python is a pain in the butt to manage packages and dependencies, how the hell am I gonna deploy this?

          It's still a mess 10 years later unless you live deep in the ecosystem and know what third-party solutions du jour to manage that complexity. At least we have Docker now.

          Also let's not forget the Python 3 migration fiasco that lasted ~2008-2018 and I still find myself porting libraries to this day.

          I haven't used Python in a personal project in 10 years because of these painful paper cuts .

          • By derriz 2024-08-1310:034 reply

            At least for Python dependency management, I'd hardly call it a mess these days and yes it was a horrible 10 years ago. The de-facto standard of venv and pip (using a requirements.txt file) is generally painless these days. To the extent that moving between MacOS, Linux and Windows is feasible for most (of my) Python work.

            And the Python 2/3 pain stopped being an issue a few years back. I haven't been forced because of library requirement to use Python 2 in years.

            Your criticisms WERE valid 5 or 10 years ago. In my experience, the ecosystem has improved greatly since then. I wouldn't use Python for large and shared developer code base but for one-man experiments and explorations, Python is pretty compelling these days in my opinion.

            • By shakna 2024-08-1310:412 reply

              > The de-facto standard of venv and pip (using a requirements.txt file) is generally painless these days.

              Except Python themselves now recommend the third party, pipenv. [0] Which means that there are multiple authoritative voices saying different things, and that produces confusion.

              [0] https://packaging.python.org/en/latest/tutorials/managing-de...

              • By aragilar 2024-08-1312:15

                Now being from 2017 ;) I suspect no-one has touched that file properly since then because of old scars :(

                Part of the issue is there's not a lot of differentiation between most of the "project management" tools (apart from their age, which in some cases explains certain design choices), and the coupling of most of these to their PEP 517 backend doesn't help. There are backends where there is differentiation, such scikit-build, meson-python/mesonpy and enscons for wiring into pre-existing general purpose build systems (cmake, meson and scons respectively), maturin for rust, and flit-core as a bootstrapper, but given setuptools can be used without a setup.py now, I'm not clear on what some of the others provide.

              • By derriz 2024-08-1312:011 reply

                That's why I deliberately described the pip/venv solution as a "de-facto" standard.

                Is Python perfect in this regard? Absolutely not but the situation is FAR less painful than it was 10 years ago. The original comment claimed nothing has changed in 10 years. I used Python 10 years ago and disliked it intensely but I've recently (in the last 3 years) used it again and it is simply not true that nothing has improved in 10 years.

                • By bb88 2024-08-1319:06

                  Poetry is a reasonable solution and has been for a few years now. We use it in production for deployments, and it's worked well for us.

            • By theshrike79 2024-08-1310:373 reply

              It still won't beat the deployment speed of

                 scp executable user@host:direc/tory/
              
              that you get with Go.

              I still use python for stuff that never leaves my computer, but in most cases if I know I need to run it on Someone Else's Machine (or even a server of mine) I'll reach for Go instead.

              • By okasaki 2024-08-1311:542 reply

                At the expense of absolutely massive, gargantuan, entire-OS-sized binaries.

                Just looking at my bin/ folder eg

                    k9salpha - a relatively simple curses app - 56MB
                    argocd - a cli for magaging argocd - 155MB

                • By sph 2024-08-1321:201 reply

                  It's like what, $0.01 per GB these days? I bought a 2 TB NVMe for Linux 4 years ago, on which I also game, and it's 50% full.

                  Disk space hasn't been a serious concern in two decades.

                  • By pjmlp 2024-08-1512:40

                    Until you start paying for it on cloud deployments per GB transfer.

                • By anthk 2024-08-1315:48

                          strip --strip-all k9salpha argocd

              • By sph 2024-08-1310:58

                I agree with you, though if you live in a Linux+podman world, you can literally scp whole containers, so at least we made some progress.

              • By derriz 2024-08-1311:522 reply

                Sure but that's not going to work across platforms, right? So not really an apples to apples comparison.

                • By thrwaway1985882 2024-08-1313:062 reply

                  Even the most ardent of Go-haters have to be fair and admit it makes cross-compilation really nice.

                      GOOS=linux GOARCH=amd64 go build && scp executable user@host:direc/tory/
                  
                  Out of the box, no toolchains to manage.

                  • By theshrike79 2024-08-1316:12

                    This is pretty much what I do.

                    I have a Taskfile for local compilation and running (macOS) and when I need to release I can just say `task publish` and it'll cross-compile a Linux binary and copy it to my server.

                  • By bb88 2024-08-1319:121 reply

                    Sure, it's nice in that regard.

                    But honestly running poetry install for python is not bad either.

                    • By theshrike79 2024-08-1413:13

                      Except in 3 years when poetry is bad for some reason and now there's a new better(?) tool again :D

                • By theshrike79 2024-08-1316:11

                  Like the sibling comment said, cross-compiling is literally setting two environment variables.

            • By entropyie 2024-08-1313:051 reply

              Eh, poetry, pipenv, conda, setup.py, GCC and all dev tools to possibly compile some C shite you didn't know you needed? It's a complete shit show still...

            • By ryanjshaw 2024-08-1315:301 reply

              I started using Dagster a few days ago, which runs on Python.

              As far as I can tell, I have to install the packages with pip. Wait - don't forget venv first. Oh but I'm not supposed to commit the venv, so how is anybody to know what I did? Okay I have to setup a requirements.txt... but now what did I install in the venv? Okay there's a way to dump the venv.. but it's giving me all the dependencies, not the root package I installed?!

              Okay there's something called pipenv that seems smart... Oh wait no this Dagster has a pyproject.toml, maybe I should use poetry? Wait poetry doesn't work, it seems like I'm supposed to use setup.py... so what's the point of pyproject.toml.

              Okay I put the package into the setup.py. Great now I have to figure out the correct version dependencies by hand?! I don't get it, python is a nightmare.

              Compare to .NET: dotnet add package

              On build: dotnet run # automatically restores packages to a project level scope

              • By Daishiman 2024-08-1321:441 reply

                Setting up a requirements.txt file and venv is _not_ a difficult ask. Seems like a skills issue here frankly.

                • By ryanjshaw 2024-08-1416:251 reply

                  Your attitude towards a new developer in the community is disappointing.

                  I just wrote in detail why I found it difficult/unintuitive and you dismissed everything without addressing it.

                  • By Daishiman 2024-08-196:22

                    If that's too difficult then you might as well not be in the profession, because it's one of the simplest things in the world.

          • By mtalantikite 2024-08-1313:56

            I've always wanted to like Python, but I got out of undergrad in 2006 and pretty much the entire first part of my career was the ridiculous Python 3 migration nonsense. Every time I wanted or needed to use Python there was some dependency management nightmare to deal with. For like a decade or so.

            I think I'm like you, I have some sort of Python ptsd from those years and don't really care to venture into it again. It sounds like it's gotten a lot better recently, but also after years of using Go and TypeScript I can't imagine working on a codebase without static types.

          • By bb88 2024-08-1317:31

            We use poetry for dependency management in production, and it's fine.

          • By underdeserver 2024-08-1313:521 reply

            Hah, even at Google, the Python 3 migration lasted well into 2020.

            • By bb88 2024-08-1318:58

              I remember a large bank building a major new project in 2017 with python 2.7.

          • By imtringued 2024-08-1311:55

            The only answer to is to use rye and I mean it. pyenv, pip, venv, pipenv, poetry, it's all crap.

            (yes rye uses venv, etc under the hood, I know)

          • By sunrunner 2024-08-1313:54

            > At least we have Docker now

            I'm not so sure this is a desirable state to be in long-term.

            While Docker (and other container solutions) does provide solutions to some of the challenges around environment consistency, build artefact management, development environments with linked services, etc., it's also one of numerous de-facto tools that in some ways ends up putting more effort on the end-user to adopt. Less effort than installing all dependencies yourself? Absolutely. More effort (and for no real reward) than a single binary built natively for the target platform? Yep.

            This feels like a sort of 'shift right' approach to releasing an application in terms of developer effort. As a developer you can just bundle everything into a container image and now the end user needs Docker and the install is going to leave a mess of image layers on their system. You don't need all the dependencies installed system-wide, that's true, but why can't they just be bundled with the application?

            It's also not a great cross-platform tool, as Docker on anything except Linux just hides the requisite virtualisation. It's mostly transparent, true, but is still plagued by platform-specific issues such as write performance on mounted volumes on macOS being ~10x as slow (there's a fairly old thread on the Docker forums that's still open with this as a reported issue, and it does't look like it's going away).

            Additionally, while disk storage _is_ (comparably) cheap nowadays, the buildup of image layers that contain endless copies of binary dependencies doesn't feel great. I don't want or need hundreds of copies of libc et al. all with slightly different versions sitting around. Even the 'slim' Debian images are min. 120MB, let alone the 1GB full ones. As Python is on topic for this thread the Alpine images are smaller, but then Alpine image builds for Python don't work with the standard PyPI wheels (I'm not actually sure if this is still true, need to check).

            As languages like Python are just source releases, some tooling is definitely needed to make that more palatable in terms of handling releases of Python tools (just big collections of scripts), but being able to build and copy a single binary for a Go application feels great having spent a lot of time practically requiring Docker for release Python tools.

            And all of this is really just getting us back to where we were years ago, where it was actually feasible to release a native binary for a target platform. The difference is that Go's tooling around related areas such as dependency management and cross-platform compilation is more standardised, and as a result the developer experience for this is much nicer than with other comparable languages that also compile to native code.

            Arguably a lot of the development decisions around how to manage and release tools comes down to how good the experience is for developers. Docker ends up being favourable for Python because there's no 'standard' Python way to bundle applications with their dependencies and Docker basically solves this problem, even if in a messy way.

            Does Docker have its place? Definitely. Do I hope it sticks around in favour of improved cross-platform developer tooling? Absolutely not.

          • By aduwah 2024-08-139:572 reply

            chore(omfg): fix print to print()

            • By dotancohen 2024-08-1310:34

              I have a feeling that you've never dealt with sequences of bytes (encoded strings) in Python 2.

            • By bb88 2024-08-143:16

              Fix print to logger.info() why not?

          • By andreif 2024-08-139:403 reply

            Dependencies is a sign of weakness.

            • By tmountain 2024-08-139:46

              As opposed to writing everything from scratch? Python has a huge standard library. Making it bigger to avoid dependencies would add more bloat to the distribution.

            • By markhahn 2024-08-1314:33

              and automatic internet-supplied dependencies are a sign of supplychain vulnerability

            • By pjc50 2024-08-1312:44

              Ah yes, all applications should be a single very large source file that issues OS syscalls directly.

        • By radicalbyte 2024-08-137:343 reply

          I have a nice way of testing a language - I download a couple of projects written by beginner / mid-level developers and not having commercial dependencies and see how much effort it takes to get them running.

          Python is one of the worst performers (and Java is shockingly bad too, although a lot of that is down to the way the JVM/language have been mismanaged). At least unless you're comparing it to very low-level languages (VDL, C).

          • By munksbeer 2024-08-138:392 reply

            > and Java is shockingly bad too, although a lot of that is down to the way the JVM/language have been mismanaged

            This is interesting and not what I would expect. Generally to get a java app running from github you'd install the correct JDK and that should be about it. Many projects will use maven, which will know how to obtain dependencies. Could you describe a typical issue?

            • By radicalbyte 2024-08-1312:343 reply

              Dependencies requiring you edit some random XML file somewhere on your machine to get them working.

              That and the whole JVM fragmentation and not being able to touch Oracle because it's Oracle and they'll use it as an excuse to sue you (at least that's the impression I had, the entire lawsuits thing is why I left Java back in the day).

              You can say the same about other package managers, but it's not very often that I have to add another source to apt unless I'm running in a locked-down environment.

              • By munksbeer 2024-08-1411:08

                > Dependencies requiring you edit some random XML file somewhere on your machine to get them working.

                No it doesn't. That's only if you're dealing with private maven repos and authentication etc. You don't need to do that at all for the use case in this thread, running a project from github.

              • By pjmlp 2024-08-1512:42

                Like the C and C++ compiler fragmentation, each with their own flavour of ISO support, UB and language extensions?

                Or the Go compiler fragmentation, where only the reference compiler supports everything, tinygo a subset, and gccgo left to die in Go pre-generics era.

              • By blandflakes 2024-08-1313:52

                Editing your dependencies in maven is not some "random XML file somewhere", come on. It's a standard file that is situated like any build declaration.

            • By andreasmetsala 2024-08-139:242 reply

              Does Maven come bundled with the SDK?

              • By imtringued 2024-08-1311:58

                No but you can just use ./mvnw instead of installing it. I have never had to install gradle as a package on my computer in my entire lifetime. I always used ./gradlew.

              • By callmeal 2024-08-1310:321 reply

                It's an additional install, like the jdk. Typically on a brand new os install the equivalent of these steps would be done:

                    sudo apt-get install <jdk>  
                    sudo apt-get install maven
                
                And then you'll be good to go.

                • By radicalbyte 2024-08-2014:02

                  I develop on a Windows machine so you're left working out which version of which JDK you need to install.

                  That isn't really a problem with other languages.

          • By quectophoton 2024-08-139:09

            My usual test for choosing a new programming language, is to see how easy or difficult it is to read documentation without having to open a web browser.

            For languages/tools/frameworks/whatever that I'm already using, the main way I compare between them is looking at the ratio of "problems I was able to solve from their official documentation" vs "problems where I needed to use random blog posts (or worse, Stack Overflow) to find the solution".

            Makes it a lot easier for me to choose tech for my hobby projects, where I sometimes don't even touch projects for several years because stuff just runs smoothly.

          • By DarkNova6 2024-08-138:371 reply

            In which way do you believe the JVM/language has been mismanaged?

            • By pjc50 2024-08-1312:451 reply

              Oracle requiring commercial licensing for their JVM has made it so radioactive that my workplace firewalls "oracle.com" to prevent anyone from accidentally using it.

              • By DarkNova6 2024-08-1322:20

                Does anybody actually use OracleJDK? OpenJDK has been the de-facto standard for years.

        • By sgt 2024-08-139:013 reply

          What's so bad about Celery?

          • By gen220 2024-08-1314:191 reply

            The docs are surprisingly unspecific for such a mature project, enough that you often need to read source code to confirm your suspicions one way or another. The style has been redone a few different times now, and most of the old stuff is there for backwards compatibility so it's unclear what The Correct Way is to use it (once you step outside the obvious classes and methods).

            The source code is like an Escher drawing, I'm sure it makes sense to the maintainer but its logic is incredibly abstract and scattered. From my notes on it ~1 year ago: "I took a dive into Celery source to better understand their native way for inspecting workers... holy shit. Celery is an overgrown jungle of abstractions."

            It's honestly kind of surprising to me that Celery is still the undisputed #1 message queue abstraction library. I guess it's a simple enough problem that companies using Python at scale roll their own. I wonder what Instagram uses instead of Celery.

            I found this [1] to be a useful comparison between queues in Python, if anybody has found a similarly-terse and exhaustive one that's more up to date for 2024 I'd love to see it.

            [1]: https://dramatiq.io/motivation.html

            • By sgt 2024-08-1319:20

              dramatiq seems like a true gem. I didn't know it existed. I don't see why this can't just be an immediate Celery replacement for your typical Django project that already uses Celery - with a bit of time spent on refactoring into dramatiq.

          • By infecto 2024-08-1312:41

            Its powerful but good luck reading the source. Its a bit of a tangled over-engineered mess at this point and the reason there are a number of "newer" libraries to try and replace it. I could not recall specifics since it has been maybe a decade since I last used it but it works until it does not and then its incredibly hard to debug.

            Its been a decade and maybe its gotten better but that sentiment is why a number of people do not want to use celery. I would also add that it was created in a different mindset of delayed job and I believe there are better patterns these days without as much complexity.

          • By jacomoRodriguez 2024-08-1310:451 reply

            curious as well!

            • By bb88 2024-08-143:17

              Go look at the source code. Everything is abstracted to the nth degree.

              If you have a bug, good luck figuring it out!

        • By gigatexal 2024-08-1310:421 reply

          hah my journey was -- man im 100s of lines in what is the type of this thing that I am passing? ... man i really wish i had types or at least hints

          and then we got type hints but then I think more of my work will be moving to go anyway

          why is packaging still broken?

          • By maples37 2024-08-1316:101 reply

            I only dabbled with Python occasionally in college, and the lack of types was my biggest obstacle. Even reading the official documentation of libraries, I still couldn't figure out what I should expect from whatever function I was calling.

            I recall the Selenium library I was using as a particularly gnarly offender -- pretty much every function I called, I had to debug or `print(typeof(x))` to figure out what I'd gotten back.

            • By gigatexal 2024-08-1316:17

              yeah ... i've become far more disenchanted with loosely typed languages. It's amazing to be able to quickly script things together and focus on the happy path in one-off scripts and bash-replacements but ... in a huge system when reasoning about what's going on -- I find it far more rewarding to know exactly what the type of a thing is and what it can do, etc., etc., and here with Python I rely on the editor a ton and I don't really like that.

    • By BiteCode_dev 2024-08-1222:274 reply

      And then you have the ever changing ecosystem, that can take years so sort out on it's own and must be constantly studied.

      E.G: if you arrive in Python right now, numpy is in version 2 and polars is stable. uv is all the rage, pydantic gained so much perf it's not even funny, toml is part of the stdlib and textual is looking very good. Type hints are much better than 2 years ago, htmx is used a lot in the web department, fasthtml is having a moment in the light and pyscript got a platform. Is any of those information worth acting on? Should you ignore them all for the sake of productivity? Should you carefully evaluate each proposition?

      Now if you have to do that with rust and erlang as well, it's a lot of time you could spend on coding instead.

      • By jaapz 2024-08-1311:201 reply

        Instead of constantly trying to keep up with everything in an ecosystem (which is simply impossible), just keep up with the stuff you are using to get your product working. Keep a side eye on the rest of the ecosystem to see if something neat is developing that you might use, but no need to study every hyped package coming out. In the end they are all tools to get something that works.

        We still use flask and it's been great all the way, and it still holds up really well, there's no reason to move to any of the other packages just because that's the next new shiny thing.

        • By BiteCode_dev 2024-08-1313:31

          There is a middle ground.

          Not getting ruff, uv, django-ninja or diskcache would be just plain waste of productivity and ease of life for me.

      • By aragilar 2024-08-1312:421 reply

        I would think the best option is to always be suspicious of hype, unless you understand why something is being hyped? I'd also argue its worth understanding where your stack falls down, so you know when you need to look for alternative stacks.

        The other bit is it's worth understanding how your stack interacts with related stacks (to use your examples, does uv and pydantic using rust vs. being pure python cause issues), but your OS is also changing, are your tools still going to work?

        • By BiteCode_dev 2024-08-147:371 reply

          First, you don't have to follow hype, but listening to it is how you know where your ecosystem is heading.

          Second, there is way more to an ecosystem that hype. Api, compat, deprecation, new features...

          If you never created an web api and have to do it now, you still have to make a choice, hype or not. This choice will have very different consequences if you know only about flask and drf, if you heard of fastapi, or if you get told about django-ninja.

          It's the same for every single task.

          And even without new things, just learning the old ones is work. In python you can encounter -o ir PYTHONBREAKPOINT, be put on a project with one of 10 packaging systems, have to deal with pth files, etc.

          And it's the same for all langages. It's a lot of work.

          • By aragilar 2024-08-1412:44

            Sure, languages have depth, and there is at some level some need to keep up with the Joneses (though to mix metaphors, different ecosystems have their red queens running at very different paces, and is it wise to be in or try to join the fastest paced race?), but I feel there is a distinction between following the latest hype blindly (and using the latest tool because of hype), and evaluating tools based on some criteria (and choosing the latest tool because it actually solves the problem on hand best). The latter is a skill that can be gained and taught in one ecosystem, and applied in another (and the former causes issues far too often).

            I think this skill is very similar to what highly productive academics have, they evaluate and examine ideas, techniques and concepts from a wide variety of fields, and combine them to build new ones.

      • By Daishiman 2024-08-1321:471 reply

        Python has _remarkably little_ ecosystem churn compared to a lot of languages. New packages come and go all the time but the older packages are very well maintained and nobody's forcing you to switch to the new ones.

        • By BiteCode_dev 2024-08-147:281 reply

          Depends on your bubble. AI circuls have huge churn.

          And I'm an advocate of not chasing new shiny toy, but it's also a problem when after so many years people don't know -m exist or stick to setup.py.

          In fact, I'm willing to bet half of HN can code in Python, yet can't tell the difference between setup.py, setup.cfg and pyproject.toml.

          That's part of the things the ecosystem dumps on you.

          • By Daishiman 2024-08-196:21

            I'm sorry to say if you can't google the difference between setup.py and pyproject.toml then it's a skills issues, because in 14 years of using Python that has never been an impediment to getting anything done in a timely manner.

      • By otteromkram 2024-08-133:121 reply

        IMHO, that's not part of learning the language. You don't really even need half of the packages you mentioned unless you don't know the language and, instead, rely upon third-party crutches.

        • By dagw 2024-08-139:58

          Depends on what your goal is. If your primary purpose for learning python is to be able to use numpy and pandas, then you should probably start learning them from day 1. Learning how to inefficiently implement parts of pandas in pure python on an ad-hoc basis is a waste of time for most people compared to just spending that time learning how to efficiently use pandas.

    • By cube2222 2024-08-1214:39

      Agreed. Having a passing knowledge of many languages is of course easy (even being moderately productive with them, esp. with Copilot now), but knowing the ins and outs of a language and its ecosystem is a very nontrivial time investment.

      Thus, I'm mostly an anti-fan of the idea of polyglot environments.

    • By pjmlp 2024-08-1222:07

      Naturally one can't know all the languages of the world.

      Even so, I have done consulting since 1999, with a small 2 years pause in research, and have several languages that I keep switching, every couple of months when project reassignment takes place.

      The usual T-shape kind of consultant, know some stuff really well, and others good enough to deliver.

    • By Klonoar 2024-08-1221:482 reply

      It’s not just the time to understand those nuances, but to keep up with the pace of changes. There’s an aspect of navigating those in some languages (JS…) that can be time consuming depending on where it’s deployed.

      • By bborud 2024-08-1612:12

        I think JS is a degenerate case. The reason I stay as far away from JS/TS as possible is that everything seems sloppy and therefore people try to "fix" things by replacing them every N months.

        Indeed, when you see so many commenters advocating not bothering to learn a language deeply, you kind of get a taste for how prevalent the sloppiness mindset is. JS seems to attract a very high proportion of unskilled programmers.

        I would struggle to name a single project I have been involved in where the JS/TS based front-end hasn't been reimplemented multiple times, from what I can tell, because people get lost.

      • By tengbretson 2024-08-131:34

        Not only is it the nuances and pace of changes, but theres also a temporal element of seeing cause and effect over time from within an ecosystem. Is there a way to just parachute in and learn how to judiciously use classes in JavaScript the way one would if they were out there assigning Thing.prototype.method = function () {}? Can Udemy give you the same a-ha moment learning Promises today that someone got when they were drowning in callback pyramids in their Express v3 app?

    • By p0w3n3d 2024-08-136:10

      I totally agree with you. Only after min 2 years of fully coding a viable production solution can one learn most of the pitfalls of given language. I would add frameworks which give another level of complexity and sometimes a framework is like a separate language.

    • By port19 2024-08-139:162 reply

      I believe people over-estimate the benefit of properly learning a language.

      3-12 months seems to be sufficient for the specialisation benefits to outweigh the small hit to language competency.

      My strategy is to be a generalist superficially learning specialized languages.

      • By tmountain 2024-08-139:482 reply

        Whatever time frames you apply, the only way to really learn a language is to use it. Reading a book, blog articles, and watching videos on YouTube won't give you the hands on experience necessary to internalize how everything works.

        • By port19 2024-08-1312:461 reply

          Of course. My methodology is usually this:

          Day 1: read the introduction of a book, set up editor and environment

          Day 2: (superficially) learn flow control and type system

          Day 3: practice with coding problems on codewars

          Day 4-7: reimplement one of my projects

          Week 2: read the second quarter of the book and continue with puzzles

          Week 3+: Just projects, maybe invest a day into tooling somewhere in between

          That is assuming 1-3 hour days

          • By sunrunner 2024-08-1314:141 reply

            > read the second quarter of the book

            And then never get to the third and fourth quarters because the practise you get from actually typing stuff is far more valuable? :)

            • By port19 2024-08-1315:15

              Exactly. There usually is some stuff about the deeper, darker corners of a language in the type of book I pick.

              I usually pick the authoritative book / advanced book. Think "Programming PHP", "The joy of clojure", "The C Programming Language", etc. I don't necessarily need to know all about performance optimization, the async model or non-html templating.

              Reading a technical book cover to cover feels wasteful imo

        • By Ma8ee 2024-08-1314:21

          True, but actually reading a book about the language to understand the philosophy behind it and its trade offs speeds up the process enormously.

      • By bborud 2024-08-1511:26

        The way to become good at something is to do it for a long time and to be willing to think about what you are doing and continually improve.

        There's much more to learning a language properly than just learning the language spec itself. Especially if we're talking about languages such as Java, where you have to know quite a bit about how the runtime works in order to be able to make good design decisions. And Go isn't really that simple either when we start to get into the weeds of performant concurrent systems that also need to be robust. It requires quite a bit of work to be able to make full use of what Go offers. It may look simple, but concurrency never is.

        You're essentially arguing in favor of not being good at what you do, or at best being mediocre at lots of things. And I'm sure there are lots of jobs where that is okay. The thing is: why would you want to spend a lifetime being mediocre, to work on mediocre projects with other people who are mediocre?

    • By nemo1618 2024-08-1222:295 reply

      One of the most important qualities of Go is that it is actually possible to fully understand the language -- in the sense that you never see a snippet of Go code and think "wtf, you can do that?" or "hang on, why does that work?"

      Getting to that point takes many years, to be sure. But the language is simple enough, and changes slowly enough, that it is not an unrealistic goal -- the way it would be in C++, or Rust, or just about any other mainstream language.

      • By Dylan16807 2024-08-136:412 reply

        Why is that important?

        Let's say one language has me find a weird rabbit hole every 50 hours of use, and I spend half an hour learning about it.

        Let's say another language has no rabbit holes, but I'm 5% slower at coding in it.

        Why would I not prefer the first language?

        (And 5% is supposed to be an intentional lowball. I'm confident I can find language pairs where my productivity differs by significantly more.)

        • By bborud 2024-08-1714:03

          That's an odd assumption. That you either have to choose between rabbit holes or development speed. Usually languages with lots of rabbit holes tend to require a lot more work.

          Take C++ vs Go for instance. C++ is an endless series of deep, complex rabbit holes where you get enough rope to hang not only yourself but your entire team. Serious C++ projects tend to require a style guide for either the company (most companies don't have that kind of discipline) or at least some consensus of what subset of the language to use and how to structure problems for a particular project. Look at the guides Google use or the Joint Strike Fighter. They're massive. C++ is not a particularly fast language to develop quality code in unless you have a small team and/or you are supremely well aligned and disciplined. And even when you work on a disciplined team, it takes time.

          Go has fewer rabbit holes. It gives you an easier path to concurrency. It has managed memory. It is sufficiently performant for a very wide range of projects. It has a very capable and practically oriented standard library that even includes HTTP, cryptography, support for some common formats and a decent networking abstraction that actually allows you design flexibility. And importantly: it comes with a shared code style which is tooling enforced, so you don't have to waste time on mediating between various opinionated hothead programmers on your project.

        • By cloudrkt 2024-08-1312:301 reply

          Because not all code is yours. In a team, the time spent on “rabbit holes” adds up, increasing the risk of bugs. A `slower` but predictable language can lead to more consistent, maintainable code, which is often more valuable in the long run or last but not least, running in production.

          • By Dylan16807 2024-08-1314:04

            > In a team, the time spent on “rabbit holes” adds up, increasing the risk of bugs.

            It adds up with the number of people on the team? Or the number of people on the team squared? Cubed? nlogn? Because a lot of those options would still favor the former language.

            And if it's happening particularly often, that means the rate will fall off drastically as mastery is achieved.

            I see a risk when code does something different from expectations. I don't see any risk when code has some kind of novel syntax that requires looking it up. Or when you learn about a feature from the documentation or a blog post.

            Being predictable is quite valuable, but predictability is different from memorizing every feature.

      • By mingusrude 2024-08-135:54

        This is similar to the experience that I had with Erlang. After having spent time with it, I was hardly every surprised looking at any code and my brain could deal with the actual problems at hand without having to figure out how to apply what I knew about the language.

      • By EE84M3i 2024-08-130:581 reply

        Personally my biggest issue with reading go code has come from the //go: magic comments that I never seem to fully understand.

        For example, what the heck is a //go:cgo_import_dynamic? As far as I can tell this is only documented on GitHub issues and mailing list comments.

      • By bborud 2024-08-1511:17

        I've been doing Go for a long time now and I still occasionally come across snippets of code that I don't understand or understand the purpose of. So I think "never" is too strong a word. I'd say "rarely" rather than "never".

      • By tialaramex 2024-08-130:012 reply

        > in the sense that you never see a snippet of Go code and think "wtf, you can do that?" or "hang on, why does that work?"

        I am extremely doubtful that this is true and would like evidence.

        • By monkeywork 2024-08-131:212 reply

          kind of hard for someone to prove a negative - feel free to find some Go code that meets that definition.

          • By shepherdjerred 2024-08-132:052 reply

            I feel this way... very often.

            * Init functions

            * Top-level variables being shared between all files in a package

            * For-loop sharing (fixed in 1.22 [0])

            * ldflags (this is more of build behavior, but it took me a while to figure out how some variables were being set [1]. Note: Go does embed some data by default, but an app I was working on introduced more metadata)

            * Go build directives prevent IDEs/linters from analyzing a file at all. E.g. if you edit a linux-only file on macOS, you get _zero_ help.

            I dunno, there are a lot of other weird, confusing things that Go does. It is less than most other languages, though.

            [0] https://go.dev/wiki/LoopvarExperiment

            [1] https://www.digitalocean.com/community/tutorials/using-ldfla...

            • By bb88 2024-08-134:141 reply

              Needing to cast nil to a type was weird. C/C++/Java/Python don't do that.

              They also take on the weird Java-ish culture that everything is better if it's written in Go.

              They also had this weird take on the C++ vtable, so they eschewed anything class like to keep everything statically linked for speed.

              It also took them a long time to figure out their loop variable syntax was broken. So ... hrm.

              • By commodoreboxer 2024-08-1413:16

                C++ and Java do make you cast null/nullptr, if you need to resolve an ambiguous overload based on pointer type.

            • By funny_falcon 2024-08-135:342 reply

              First three are clearly documented in language reference, which could be read in several hours (at least before generics were introduced).

              • By shepherdjerred 2024-08-1315:38

                You're misunderstanding what I'm saying. How the language works is relatively simple. Understanding what's going on when problems arise isn't.

                * "how is my Cobra command being registered when I'm not calling any code?" -> "oh, there's an init function that registers the command when imported"

                * "why is this variable not working?" -> "oh, it's that for loop thing again"

                * "why does the result of my unit tests depend on the order of test execution?" -> "oh, we have 150 Cobra commands in one package with variables shared between _all_ of those commands"

                * "how is this variable being set?" -> "oh, we have to call go build with a monstrous ldflags argument

                * "why is CI not passing for this oneliner when this looked fine locally?" -> "oh, it's a linux-only file so the IDE isn't even showing me simple syntax errors"

              • By lifthrasiir 2024-08-136:35

                I like to read language references (not just Go's, for example I have also fully read the ECMAScript standard as a part of my research in the past) and yet I will never say I can remember everything from those references. "Documented" isn't same as "easy to remember or recall".

          • By tialaramex 2024-08-1310:03

            I looked around for something like CppQuiz or its Rust analogue, where you're presented puzzling "Why would anybody do that?" fragments and asked what happens -- something that's often extremely difficult. If I had found one of those for Golang I could maybe see whether there are people who just ace the quiz first time.

            But what I found were the sort of softball "Is Go Awesome, Brilliant, Cool or all of the above?" type questions, or maybe some easy syntax that you'd see in day 1. So I have no evidence whether in fact many, some or even a few Go programmers have this deep comprehensive understanding.

        • By kylecazar 2024-08-130:291 reply

          It's known for being a small and simple language.

          • By kaba0 2024-08-136:57

            Just like Brainfuck.

    • By brightball 2024-08-130:091 reply

      This is actually how I feel about Ruby. Not full on Rails because there is a lot more to learn there which comes with the time investment (which is also worth it in many cases).

      But for most scripting work especially, just raw Ruby and maybe the Sequel library for a database just makes my productivity soar.

      • By jalopy 2024-08-139:11

        Do you have any open source examples (either your own or others) of simple Ruby + Sequel that are particularly elegant + productive that you could point to? Would love to see this.

        I agree Ruby is the most elegant and enjoyable language to develop in.

    • By Etheryte 2024-08-137:54

      Agreed, there's such a wide knowledge gap between "I read about this language design choice online" and "I know this from first principles". Truly, thoroughly understanding the languages and tools you work with every day unlocks a deeper level of understanding that you can't substitute with anything else. It's what people often describe when they first learn Lisp, but the same actually holds for every language, tool, etc. There is tremendous value in depth. However, since there is considerable opportunity cost in choosing what you put your time and effort into, do be mindful of the tools you decide to acquire this amount of knowledge in.

    • By lifthrasiir 2024-08-136:50

      It takes forever to "properly learn" a language in my opinion, because it would mean the total emersion into the ecosystem and most prominent libraries built around that language as well, and they will be always moving targets even though the language itself might be static. Go is no exception, and in fact there are only a handful of languages that can be ever "properly learned" if we are strict about that. The trick is not to properly learn, but to know how to learn enough of anything you need and make the best use out of that.

    • By simiones 2024-08-139:54

      I completely disagree. I have been successful in building a new project for my company with myself and the entire team being completely new to Go when we started. Did we have growing pains, and some sub-par decisions? Absolutely. Do we right better Go code today, 4 years later? Absolutely. Did this mean that the project was in any way significantly slowed down by our use of Go compared to the C# or Java we knew before after the first 1-2 months? No, not really. The ways in which Go fit better for our chosen domain completely made up for that (and this all despite my continued belief that Go as a language is inferior to Java and C#).

      Now, there are other languages where I'm sure that we would have had significant issues if we tried to adopt them. In particular, switching from the GC languages we were used to to a native memory language like C++ or Rust or C would have surely been a much bigger problem. The same would have probably been true if we had switched to a dynamic language.

      But generally, deep language proficiency has less value in my experience than general programming proficiency, and much less than relevant domain knowledge. I would hire a complete Go novice that had programmed in Python in our specific domain over someone who has worked in Go for a decade but has only passing domain knowledge any day.

    • By georgeecollins 2024-08-1223:41

      I also think it is a good idea to learn a few very different languages when you are starting out so that you have a sense of the tradeoffs and the different ways things can be done. When I was in college they taught C and Lisp (I am dating myself) which seemed very different to me at the time but also useful in different ways. Close to the hardware, vs supporting objects, lambda functions, etc. Later I learned a number of other languages and now I try hard to avoid switching languages because of the lost productivity.

      Now if I were starting out I think I would try to focus on two different languages, but more modern ones. Maybe Rust and Python?

    • By ramijames 2024-08-132:48

      I don't think it is that long. Think of learning any spoken language. We expect that it takes AT LEAST ten years to be able to master it, and typically it takes much longer to meaningfully add to the compendium of literature. Usually we are talking about decades. It isn't just about the tool (speaking, writing) but being so fluent that you can successfully express yourself in a way that has impact.

      If you can do it in ten years with a software stack, I'd say that's pretty impressive.

    • By irjustin 2024-08-136:062 reply

      ChatGPT accelerates this timeline significantly. As a senior programmer you can largely skip the "junior" phase of the learning, almost like pair programming with a very proficient junior programmer.

      • By bborud 2024-08-1511:37

        I disagree. ChatGPT is like pair programming with someone who is on drugs and tends to lie a lot. Co-Pilot is even worse as it constantly bombards you with poor suggestions and interrupts your flow when you should be left alone to think about the problem you are solving.

        What separates a senior developer from a junior is that the language is no longer a barrier so your spend most of your time thinking about the problem you are solving and how you balance requirements, take into account human elements, and try to think ahead without prematurely spending time on things you aren't going to need.

        Being shown a bunch of possible solutions to choose from doesn't help you develop the ability to think about problems and come up with better solutions.

        Yes, ChatGPT can occasionally help you when you need a nudge to pick a direction or you come across something you don't know anything about. But it is not a substitute for knowing stuff and it can't actually teach you how to think.

      • By pjmlp 2024-08-137:55

        I am fully convinced that programming language generation out of stuff like ChatGPT is an intermediate step on their evolution, just like early compiler adopters wouldn't trust compilers that wouldn't show the Assembly output.

        Eventually we will achieve another milestone in what it means to program computers, where tools like ChatGPT will directly produce executables, or perform actions, and only when asked it will produce something akin to source code for cross-checking.

    • By vouaobrasil 2024-08-1312:07

      That's true. I agree. Python is the only language I ever really learned to a deep level and properly, and it makes a huge difference! (And I'm still learning of course.)

    • By machiaweliczny 2024-08-1313:391 reply

      Using esoteric lang constructs is bad idea IMO. Makes software harder to read/port/maintain. It’s useful for some hack & high perf stuff but not typical apps.

      • By bborud 2024-08-1714:10

        Agreed. But knowing a bit about how a language (and runtime) actually works can make a huge difference when you design stuff.

        For instance, I've seen lots of examples of people implementing LRU caches in Java, and then scratching their heads when performance drops because they have no idea how the GC works. Or complicating things immensely because they realize memory is tricky and then ending up with some complex pooling scheme that constantly breaks rather than realizing that often more naive code that exploits the low cost of ultra short-lived local objects.

        If you know what you are doing it becomes easier to choose solutions that are performant and understandable.

    • By oooyay 2024-08-1219:32

      By now I know three major languages: Go, Python, and Typescript. I know tradeoffs at-a-glance, I deeply understand the syntax and its various forms, the full array of tooling and what they do, and lastly (but maybe most importantly) I can estimate more accurately because I can architect in my head.

      I can work in a myriad of other languages. I may be able to do some of the things above in Java or Rust but not nearly to the degree to which I can in languages I know. I think the difference is I'm probably not going to be leading a Java project or producing anything really innovative at a code level.

      To me, more important than picking a hammer, is knowing a variety of hammers that are good at certain tasks. I don't focus on Rust or Java as much because, frankly, I can build most things that are pertinent to the constraints of my work environment with those tools and most people I encounter also know them. The other considerable factor I have is that most things I work on can be horizontally scaled so my need for Rust is very niche. With respect to Java, I have a lot of workarounds that are cleanly abstracted enough before I need its dynamism and subsequent mental overhead.

    • By jamwt 2024-08-1315:33

      IME, engineers and "the discourse" when we argue on the Internet often conflate "ease of use" and "ease of learning". When we say "ease of use", we usually mean "ease of learning."

      If you're a hobbyist, yeah, you should heavily value "ease of learning." If you're a professional, the learning curve is worth it if the tool's every day leverage is very high once you're ramped up. Too many developers don't put those 3-4 months in, in part due to the over-emphasis on "ease of learning" in our discussions/evaluations of things.

      I was a part of a very large go project (https://news.ycombinator.com/item?id=11282948) and go-based company infra generally some years ago, and go is emblematic of the classic tool that is amazing at ease of learning, and quite mediocre at "ease of use" as time goes on.

      I personally end up resenting those tools because I feel tricked or condescended to. (This is a little silly, but emotions are silly.)

      I'd wager this is also why Rust is a perennial "most loved" winner in the surveys: it gets better as your relationship with it deepens, and it keeps its promises. Developers highly value integrity over trickery, and hard-earned but deep value feels like integrity and wins in the long run. (other examples: VIM, *nix, git)

  • By afiodorov 2024-08-138:3312 reply

    I used to work for a Go shop. We dealt with financial data. I found it so annoying that many of my colleagues would use Go for one-off tasks such as aggregating CSV files, updating the database with some data, or fetching data from the database, and then trying to make a plot. I saw my colleagues again and again implementing basic algorithms such as rolling median, or finding a maximum. Instead of loading data into Pandas and doing a group by, they would create some kind of loopy solution that would use maps.

    I totally understand why some people prefer Go in production to Python, but I could never understand why people wouldn't just learn the standard data science tools instead of reinventing the wheel in Go, always debugging their own off-by-one errors. It was difficult for me to trust the results of such analyses, given that I knew how many of the basic functions were written on the fly and probably not even tested.

    In the end, I didn't think it was a good use of the company's time. It felt more like an ego thing - thinking and showing that Go is sufficient. It reminds me of how people try to use iPads to code - only to show that they can do it.

    • By umvi 2024-08-1314:092 reply

      I've mostly seen the opposite - pandas and jupyter notebooks shipped directly to production because the data scientists and AI guys didn't know how to do anything but python. As a result, the solutions were not performant and often had lots of runtime crashes due to python's more loose typing

      • By philote 2024-08-1314:461 reply

        If they're shipping notebooks to production and having so many crashes, I'd question that they even know how to do Python.

        • By Narhem 2024-08-1316:373 reply

          When do you ship notebooks to production? Jupiter was never meant for external clients.

          • By pclmulqdq 2024-08-1317:152 reply

            While terrifying, it is not uncommon to see python notebooks make it to production.

            • By EuAndreh 2024-08-1317:271 reply

              Oh god why. I thought (and hoped) that GP didn't actually mean this.

              I see how a team or an organization can eventually get to this point. It just saddens me that they got there.

              • By nsonha 2024-08-1318:36

                why not, many data related tasks are rather ad-hoc, it's a waste of time to make a long lasting software out of every ad-hoc request

            • By Narhem 2024-08-1323:19

              Seen the same with exposed matlab web apps. Not just the Python guys, kind of shocking how much group think exists on any platform.

          • By mathgeek 2024-08-1317:16

            Quite a number of AI edtech sites use notebooks in production for assignments, as an example of when it happens.

          • By umvi 2024-08-1319:211 reply

            Usually what happens is:

            1. "We got it all working in the notebook"

            2. "Great, ship it"

            3. (data scientist takes the notebook code almost verbatim, wraps it in a basic CLI or HTTP API and it gets shipped off in a docker container for other services to consume)

            • By Narhem 2024-08-1323:25

              In practice that’s not too crazy. It’s fast easy to debug and works with most CI/CD tools.

              If you ever want to work in teams that kind of setup works extremely well.

      • By Spivak 2024-08-1314:282 reply

        I'm not sure how you're going to fix this given the data science tools are in Python. Are you gonna implement a half-broken 20% of numpy/scipy for a one-off program and then try to port?

        These libraries hide a lot of complexity and implementing even a few operators is a project.

        • By supermatt 2024-08-1314:43

          i dont think they are complaining about the libraries, but rather the scratchpads and notebooks that people use for ideation and evaluation being moved directly into a production environment because the authors don't have the experience or time to build more structured, efficient and maintainable code.

        • By grahamjameson 2024-08-1314:57

          Rust + Polars comes to mind.

    • By physicsguy 2024-08-139:292 reply

      I've had exactly the same experience, it's a nice language but using it for things it's not suited for like data exploration makes no sense to me.

      Production data pipelines on the other hand, but only after testing them well and as you say, making sure there's good testing if you're implementing things like numerical routines.

      • By Lyngbakr 2024-08-139:503 reply

        Do you have experience implementing ETL pipelines in Go? I think it'd be a better fit for us over our current language, but I'm curious to hear from people who've actually done it.

        • By physicsguy 2024-08-1314:24

          Yes. It works fairly well. With that said, I've got a feeling that life would be a lot easier to change around if we weren't using it, we end up writing a lot of code to do relatively simple things.

        • By brushyamoeba 2024-08-1313:33

          I do this at my job.. Disclaimer: I’m a web dev (“architect”) who does some lightweight data engineering tasks to facilitate views in some of my apps.

          My pipelines are very simple (no DAG-like dependencies across pipelines). I could just have separate scripts, but instead I have a monorepo of pipelines that implement an interface with Extract, Transform, Load methods. I run this as a single process that runs pipelines on a schedule and has an HTTP API for manually triggering pipelines.

          At some point I felt guilty that I am doing something nobody else seems to do, and that I had rolled my own poor-man’s orchestrator. I played around with Dagster and it was pretty nice but I decided it was overkill for my needs (however I definitely think the actual data analysis team at my company should switch from Jenkins to Dagster heh…)

          On a separate note, all of my pipelines Load into Elasticsearch, which I’m using as a data warehouse. I’ve realized this is another unconventional decision I’ve made, but it also seems to work well for my use-cases.

        • By fifilura 2024-08-1310:121 reply

          What is current language and have considered doing it in SQL?

          I don't think go will be the right choice. It is just not its strength.

          • By physicsguy 2024-08-1314:261 reply

            It depends on what you're doing right? The commenter here replied to me, and we're processing really large data files that are deliberately not in a SQL database due to size, only artefacts of these files eventually make it into a time series DB. For us Go works well and is performant without any great difficulty. For domain specific analytics we generally use Python, and Go just calls out to an API to do them.

            • By fifilura 2024-08-1315:05

              You are right. And I am mostly a "T" guy so I guess the answer was mostly about the transform.

              For extracting the data, go is probably a very good choice. But for transforming, pretty often not, although your use case may be suitable.

              In the end, the question was very open ended.

      • By caeril 2024-08-1318:121 reply

        > it's a nice language but using it for things it's not suited for like data exploration

        Pedantic point, but this is an issue of library support, not the language.

        For whatever reason, data scientists and ML researchers decided to write their libraries and tools in a Kindergartner's language with meaningful whitespace, dynamic typing, and various other juvenile PL features specifically aimed at five year olds and the mentally infirm.

        • By physicsguy 2024-08-146:01

          Nobody would really use a compiled language for this, the compile-run-edit-cycle just takes too long. Prior to Python people really just used MATLAB and Mathematica for that sort of work in the physics/engineering side, and R and Stata/SPSS on the bio + maths side. MATLAB, Mathematica, Stata and SPSS are all commercial and R has exactly the same problems to Python in environment management and compiled binaries, if you use it today you end up doing a lot of manual compilation of dependencies and putting them in the PATH on Linux at least.

          Python became popular because the key scientific ecosystem libraries copied the libraries from MATLAB closely which made it easy to pick up, and because it was free. Anaconda made a distribution that was easy to install with all the dependencies compiled for you, which worked on Linux/Mac/Windows which made it much easier to use than R. The other interactive languages around at the time were Ruby which was heavily web dev focused, and Perl. Node didn’t yet exist.

          Once you have an ecosystem in a language it’s very hard to supplant. You need big resources to go against the grain. That no big company has decided to pour lots of money into alternatives even despite the problems probably tells us that it’s not viewed as being worth it.

    • By theshrike79 2024-08-1310:369 reply

      > I used to work for a Go shop. We dealt with financial data.

      People dealing with finance are THE MOST risk-averse people I know. No new tool will be used without years of vetting and it'll still be blamed if something goes wrong - even if the new tool never touched that bit of the process =)

      Source: Consulted for a finance company and it was Ye Olde Java and COBOL all the way down =)

      • By Someone 2024-08-1311:15

        If they’re risk averse, they shouldn’t, as the OP claimed, implement basic algorithms such as rolling median, or finding a maximum instead of loading data into Pandas and doing a group by, would they?

      • By sanderjd 2024-08-1312:25

        Using Go for data analysis is not the risk averse choice. If the comment were "everyone was using excel and maybe R and I couldn't get them to use python", then the risk aversion comment would be spot on :)

      • By zerkten 2024-08-1313:39

        >> People dealing with finance are THE MOST risk-averse people I know. No new tool will be used without years of vetting and it'll still be blamed if something goes wrong - even if the new tool never touched that bit of the process =)

        What exactly do you mean by "finance" in your context? Even within a huge organization like Bank of America you can have an extremely conservative part and an innovative part. Obviously, most employees are likely working for the conservative part, but that doesn't mean that there aren't some people playing using modern technology. It just doesn't get evenly distributed, and you can be many layers from that usage in your department.

      • By dagw 2024-08-1315:04

        Depends what you mean with "dealing with finance". If you're talking about core back office 'plumbing' the yes. Other parts are far more adventurous. Poke around any huge financial institution and you'll probably find everything from COBOL to some experimental in-house variant of Haskell.

      • By lolinder 2024-08-1312:31

        It sounds like every single data operation in this shop was done with a "new tool" written on the spot in Go.

      • By noisy_boy 2024-08-142:52

        People who are saying Finance == assembly/COBOL/Java either have never worked in the area or have decided about the elephant by touching a very small part it.

        Within a bank, you can have the accounting department using tested and tried approaches and/or vendor products whereas the machine learning group may be using the latest tech. But finance is not all banks - you can have non-banking financial orgs, rating firms, investment consultancies, private equity, hedge funds and on and on. A hedge fund probably uses super cutting edge stuff in one team while the other has been chugging along with stored procedures and Java.

        It is too big and varied to generalize.

      • By murray-buttchin 2024-08-1314:57

        I don't think this is true of every financial institution. Nubank runs almost entirely on Clojure except for some of its Data Science tooling which is in Scala. Sure it's JVM, but it doesn't exactly scream risk-averse to me.

      • By martinclayton 2024-08-1314:22

        My experience is different to that.

        I'd say people and businesses that best understand and balance risks against potential rewards are the most successful. I know a couple of skydiving accountants...

        Use of innovative tech in finance is one way for companies to gain an "edge" which can make them a lot of money. I wouldn't characterise people working in those particular areas as more risk averse.

        OTOH finance businesses (esp. those that manage other peoples money) are regulated and can't put new tech into Production without careful change control. But this evolves over time, and controls were looser in the past.

        There's a steady creep of regulation and control into tech used in finance outside and around IT: so called EUC (End-User Computing). This can be a source of "wrong tool for the job" syndrome. I've seen some hellish SQL written by non-IT people, but it got the job done. It's also an area where source code control, testing, and release processes are inevitably more human and error-prone.

        There is a culture clash internal to large finance between "move fast and break things!" and "nothing can change, ever - too risky!" This leads to a mixture of the very old and very new, and inevitably, more complexity. Achieving homogeneity once the heterogeneous is out of the bottle is very hard - attempts to do so often (always?) lead to a variation on the xkcd "Standards" situation https://xkcd.com/927/

      • By classified 2024-08-1311:361 reply

        I'm surprised they were adventurous enough to use Java.

        • By theshrike79 2024-08-1318:56

          Someone picked Java at the start of the millennium-ish and they never dared to move away. =)

    • By whateveracct 2024-08-1314:291 reply

      > I used to work for a Go shop. We dealt with financial data.

      Hopefully you never forgot to initialize a numeric field and had it default to 0!

      • By ratorx 2024-08-140:302 reply

        I think you are implying that this is incorrect behaviour. Could you explain why?

        The alternative would be to have it be undefined, which seems clearly worse. Or do you mean that it should be a compiler warning/error instead?

        • By sapiogram 2024-08-1410:551 reply

          I've worked in Go for two years, and I hate zero values with a passion. I would much prefer undefined/nil/whatever as a default value. At least that obviously represents an invalid value, and will crash on use, become `null` in the database and when serialized to JSON. `0`, however, is indistinguishable from "missing data", and will just sit there and slowly poison your production data over time.

          • By ratorx 2024-08-1411:091 reply

            I don’t think you mean that. By undefined, I was referring to C-style undefined behaviour stemming from reading uninitialised memory (which is the only alternative to not setting the memory to a known value when it is declared but not initialised). This is clearly worse than zero values because at least zero values won’t result in initialisation from memory which could be anything.

            If “null” is better for your use case, that’s not too difficult to emulate in Go (although perhaps not as ergonomic as it is in other languages). What you want is effectively an “optional” type rather than the type itself. The easiest way to represent this in Go is to use pointers to the type, which have the zero-value of nil. Combining this with JSON encoding ‘omitempty’ would get you the properties you were looking for.

            I don’t think the lack of default nullability is bad (look at all the languages that are trying to slowly phase it out by supporting non-nullable types).

            • By sapiogram 2024-08-1414:54

              > I don’t think you mean that. By undefined, I was referring to C-style undefined behaviour stemming from reading uninitialised memory

              You're right, I was referring to Javascript-style `undefined`, where it's basically a 2nd `null` value.

              > The easiest way to represent this in Go is to use pointers to the type, which have the zero-value of nil.

              This certainly works, but it has some serious drawbacks imo:

              * Performance overhead

              * The integer field is no longer copied along with the struct. The resulting aliasing shenanigans can easily trip up experienced developers, because "why would anyone store an `*int` in a struct instead of a plain `int`?"

              * Other silly mistakes that no linters catch, such as comparing the pointers when you meant to compare the raw values.

        • By whateveracct 2024-08-1413:58

          > Or do you mean that it should be a compiler warning/error instead?

          Yes - it's pretty trivial to enforce that all fields be initialized.

    • By RandomThoughts3 2024-08-1315:272 reply

      > rolling median, or finding a maximum

      The real question is why are you writing code to make a plot when you can load a csv in excel and get a clean plot in 2 minutes.

      • By noisy_boy 2024-08-145:101 reply

        Maybe because they need to do that every day with variations and automating away the monotonous chore is one of the main points of writing code.

        • By RandomThoughts3 2024-08-148:39

          > I saw my colleagues again and again implementing basic algorithms such as rolling median, or finding a maximum.

          Emphasis mine so I will go with no regarding your explanation. It’s obvious the discussion was about one shot plotting from the start. The whole Go vs Python discussion for graphing doesn’t make sense in the context of automation.

      • By rcxdude 2024-08-169:33

        Because Excel sucks at plotting, especially if you have non-trivial amounts of data.

    • By bitbasher 2024-08-1317:17

      For that kind of processing, I prefer to use the database itself. I mean, it is a "data" base and it's pretty good at it. I get it-- sometimes you have a half cooked spreadsheet someone sends you and you need to run some analysis on it.. in that case I don't see any harm in using Go, using visidata, importing it in sqlite or using pandas or whatever.

    • By wejick 2024-08-1313:33

      I have done the same like your colleagues. Recently I discovered that pandas + jupyter notebook is much better tools. However as gopher that needs occasional data cruncher, I totally relying on LLM chat agent to do my work.

      But the rest of the work is still go, so the context switching is there but bare able. Imagine a go crawler dumping csv and a python script crunching the csv.

    • By pantsforbirds 2024-08-1317:101 reply

      I've almost entirely replaced Pandas with DuckDB in my day-to-day work. I wonder if it would be an easier lift for someone who is familiar with SQL, and doesnt want to pick up Pandas/Polars.

      • By ansgri 2024-08-1319:271 reply

        This comment might finally push me to try DuckDB, thanks. I’m quite proficient with scientific python in general, but Pandas API is just alien (and slow). SQL is more natural for its domain.

        • By Daishiman 2024-08-1321:391 reply

          How's DuckDB better than using SQLite on an in-memory DB?

          • By pantsforbirds 2024-08-1418:41

            It's designed for analytics work. Some examples: * You could run a bunch of analytics queries directly on a jsonlines, csv, or parquet file (or even glob of files). * You can output directly to a pandas or polars dataframe * You can use numeric python functions inside of queries * You can also attach to S3 buckets, postgres databases, or sqlite databases and query them directly.

    • By richrichie 2024-08-143:26

      > I saw my colleagues again and again implementing basic algorithms such as rolling median, or finding a maximum. Instead of loading data into Pandas and doing a group by, they would create some kind of loopy solution that would use maps.

      Isn’t this trivial? I use these functionalities and maintain my own Go package for such utilities. A software firm should be able to do this without much issue. In fact, it is better done in house. Don’t have to worry about dependency management and downloading the entire Pandas library for just rolling means, etc.

    • By classified 2024-08-1311:342 reply

      > Instead of loading data into Pandas and doing a group by

      SQL can do `group by`. No need for overkill with "data science tools".

      • By sanderjd 2024-08-1312:271 reply

        If you have a csv, pandas is the "less overkill" route than loading it into a database to use sql.

        • By chrisandchris 2024-08-1316:161 reply

          Each their own tool, but that's nothing awk can't do [1]. I prefer a shell script, because shell is everywhere.

          [1] https://stackoverflow.com/a/75073649

          • By sanderjd 2024-08-1318:18

            Sorry, but no, awk and shell scripts are not a good choice for this kind of work. I'm sorry, but they just aren't!

            Sure, go for it if you're just doing stuff for your own personal interest. But if you're doing serious data-driven work in a professional environment, this is going to be an awful choice.

      • By infecto 2024-08-1312:553 reply

        While definitely doable, using a SQL database to do data discovery is obtuse.

        • By zerkten 2024-08-1313:461 reply

          It's obtuse, but it's effective for some people. I know someone who hires devs from the .NET and Python space for finance. When asked to suggest a solution to any kind of data problem like this the interviewees split down the middle with the .NET ones almost entirely using a relational database and the Python ones using either the library du jour or suggesting something on the command line (since they often have Linux experience.)

          In both parts of their org, the .NET area tends to use SQL exclusively for analysis and the Python folks use Pandas and bunch of other stuff. These departments are also in significantly different parts of their organization with their own mandates and culture.

          • By hermitdev 2024-08-1314:091 reply

            As someone with +20 years in finance (hedge funds/trading) and knows .Net, Java, Python, C++, shell, etc, the first question I'd ask is: where's the data?

            If I'm being asked to do data analysis, it's because we need the answer yesterday. So, the tool I choose always going to be a matter of which gets me the answer fastest and with the least amount of friction. That's almost always dictated by where the data is _now_. Not where it'd ideally be.

            • By ryanjshaw 2024-08-1315:251 reply

              It's in this email attachment right here. Now what?

              • By dotancohen 2024-08-1318:34

                To me that sounds like a csv file and it'll likely be a python script from me.

        • By pjmlp 2024-08-1512:52

          Depends, that is what OLAP is all about, naturally most of the good tools happen to be commercial for the enterprise space.

        • By orthoxerox 2024-08-1313:221 reply

          Not really. DuckDB can operate on raw csv without ingesting them. This is as easy if not easier than setting up python and pandas.

          • By infecto 2024-08-1321:32

            Totally agree that it depends on the tools available but given a typical toolset (sql and python). I would lean towards python because some types of analysis are easier to express in code, especially when working on top of the multiple data sets.

    • By senkora 2024-08-1317:29

      XTX? I know that they use a lot of Go.

    • By arberavdullahu 2024-08-139:57

      You don’t need to spend time learning standard data science tools. These tasks are well-defined, thoroughly documented in different blogs, and with today’s AI-driven code generation, basic programming knowledge is sufficient. Attempting to manipulate CSV data and generate visualizations in Go would be a waste of time, delivering subpar results.

  • By daghamm 2024-08-1214:0117 reply

    The author lists multiple reasons for this, but for me the biggest one is the first one: Go is good for almost everything.

    I have extremely good productivity when using Go. Once your project exceeds 100 lines it is usually even better than python. And yes, I am aware that Rustians did a survey where Rust was crowned as the most efficient language but in my reality (which may differ from yours) Go is simply the best tool for most tasks.

    Why is that? Well, for me there are 3 reasosns:

    1. The language is extremely simple. If you know 20% of C you are already a Go expert.

    2. The core libraries are extremly well thought.

    3. Batteries are included. The toolchain and the core libraries alone can do 90% of what most people will ever need.

    When people argue about the validity of these claims, I simply point them to this talk https://go.dev/talks/2012/concurrency.slide#42

    • By conor- 2024-08-1214:546 reply

      Go is the only language I've ever felt highly productive working in. Oftentimes in other stacks I find myself in analysis paralysis on meta things that don't matter:

      - what design patterns/language features make sense to use

      - what is the best lib to accomplish X

      - how do you keep things up to date

      With Go, the language is so simple that it's pretty difficult to over engineer or write terse code. Everything you need is in stdlib. The tooling makes dependency management and upgrades trivial because of strong backwards compatibility.

      • By pjmlp 2024-08-1222:092 reply

        Sure, if everything one does is either CLI stuff, or UNIX daemons, containers, ....

        Because in the reign of graphics , GUI, GPGPU, HPC, HFT, ML, game engines,numeric analysis, ... there is hardly any library that really stands out.

        • By dartos 2024-08-1222:134 reply

          Or servers, and about anything that really benefits from concurrency.

          Even a lot of games could be made with go. The gc wouldn’t really kill the frame rate of a game unless you really push it.

          • By pjmlp 2024-08-135:111 reply

            That is what "UNIX daemons, containers means" on my comment.

            Gamedev in Go only for those that rather spend their time doing engines from scratch.

            Additionally Go's lack of support for dynamic linking is a no Go (pun intended) for big A game studios.

            • By l33t7332273 2024-08-135:341 reply

              My Go is a little rusty by now, but I thought they supported some type of dynamic linking(although if I recall correctly it comes with a number of free footguns)

              • By pjmlp 2024-08-135:59

                It does a very crude one, where one is bound to expose C ABI types, all shared objects have to be linked with the same runtime, and there are still issues making this rather basic support work on Windows, land of game developers.

          • By kbolino 2024-08-130:341 reply

            Gamedev in Go will require cgo (and not just the easy parts), which ups the complexity quite a bit, unless you're already very familiar with C.

            I think it's pretty viable nonetheless, but more for the experienced developer with specific goals outside of the nice parts of common engines, or for a hobbyist who knows the language and wants to tinker and learn.

            • By tslocum 2024-08-132:012 reply

              Sorry, this comment is so incorrect that I have to ask, what are you basing it on?

              You can create games today using Go without cgo, and there are numerous examples of shipped games of varying complexity and quality. I do this to ship the bgammon.org client to Windows, Linux and WebAssembly users, all compiled using a Linux system without any cgo.

              https://ebitengine.org

              https://github.com/sedyh/awesome-ebitengine#games

              • By wokwokwok 2024-08-136:491 reply

                https://ebitengine.org/en/documents/install.html

                For anything other than windows:

                > Installing a C compiler

                > A C compiler is required as Ebitengine uses not only Go but also C.

                I mean, even on platforms without cgo, it's it working magically?

                No; it's using https://github.com/ebitengine/purego, which is:

                > A library for calling C functions from Go without Cgo.

                Like... I mean.... okaaaay, it's not cgo, but it's basically cgo? ...but it's not cgo so you can say 'no cgo' on your banner page?

                If you're calling c functions, it's not pure go.

                If calls some C library, and it doesn't work on any other platform, its like 'pure go, single platform'.

                hmm.

                Seems kind of like... this is maybe not the right hammer for gamedev; or, perhaps, maybe not quite mature yet...

                Certainly for someone in the 'solo dev pick your tools carefully' team, like the OP, I don't think this would be a good pick for people; even if they were deeply familiar with go.

                • By hajimehoshi 2024-08-1918:10

                  Ebitengine author here.

                  PureGo is not fully used in Ebitengine for Linux, macOS, and so on yet. You still need Cgo for such environment.

              • By kbolino 2024-08-133:23

                It was based on my own experience (with e.g. sdl2) and, clearly, some ignorance.

                I didn't mean to imply that cgo was an insurmountable barrier. But apparently it was a big enough deal for the authors of this engine that they copied over large parts of major API surface to Go to avoid it. Impressive.

                However, AFAICT avoiding cgo means using unsafe tricks and trusting that struct layout will stay compatible. Nevertheless, it's a proven solution and as you say used by many already.

          • By neonsunset 2024-08-1317:10

            Note that Go has very different GC behavior to what .NET GC and likely Unreal GC do. At low allocation rates the pauseless-like behavior might be desirable, but it has poor scaling with allocation rate and cores and as the object graph and allocation patterns become more complex it will start pausing and throttling allocations, producing worse performance[0].

            It also has weaker compiler that prevents or makes difficult efficient implementation of performance-sensitive code paths the way C# allows you to. It is unlikely game studios would be willing to compensate for that with custom Go ASM syntax.

            Almost every game is also FFI heavy by virtue of actively interacting with user input and calling out to graphics API. Since the very beginning, .NET was designed for fast FFI and had runtime augmentations to make it work well with its type system and GC implementations. FFI call that does not require marshalling (which is the norm for rendering calls as you directly use C structs) in .NET costs ~0.5-2ns, sometimes as cheap as direct C call + branch. In GoGC it costs 50ns or more. This is a huge difference that will dominate a flamegraph for anything that takes, for example, 30ns to execute and is called in a loop.

            It is also more difficult to do optimal threading with Go in FFI context as it has no mechanism to deal with runtime worker threads being blocked or spending a lot of time in FFId code - .NET threadpool has hill-climbing algorithm which scales active worker thread count (from 1 to hundreds) and blocked workers detection to make it a non-issue.

            Important mention goes to .NET having rich ecosystem of media API bindings: https://github.com/dotnet/Silk.NET and https://github.com/terrafx cover practically everything (unless you are on macOS) you would ever need to call in a game or a game engine, and do so with great attention paid to making the bindings efficient and idiomatic.

            For less intensive 2D games none of these are a dealbreaker. It will work, but unless the implementation details change and Go evolves in these areas, it will remain a language that is poorly suited for implementing games or game engines.

            [0]: https://gist.github.com/neon-sunset/72e6aa57c6a4c5eb0e2711e1...

          • By tacticus 2024-08-134:032 reply

            both unity and unreal have GC these days.

            • By pjmlp 2024-08-135:15

              And dynamic linking, plugins....

              Which Go doesn't do really well, and there is no interest in improving.

            • By dartos 2024-08-1912:49

              Yeah, but their GC are tuned for the specific task of being a game engine. (Idk the specifics, but I doubt they are stop-the-world GCs for example)

        • By cdelsolar 2024-08-1312:42

          wails, raylib, ebiten

      • By randomdata 2024-08-1216:262 reply

        > it's pretty difficult to over engineer

        I don't know about that. Every programmer's first Go program seems to like to go to channel city. Perhaps more accurately: Over-engineering your Go program is going to quickly lead to pain. It doesn't have the escape hatches that help you paper over bad design decisions like some other languages do.

        • By arp242 2024-08-1222:023 reply

          Also: interfaceiritus. Someone saw "accept interfaces, return structs" somewhere and now EVERYTHING accepts an interface, whether or makes sense or not. Many (sometimes even all) of these interfaces have just one implementation.

          • By hellcow 2024-08-1222:203 reply

            Doing this allows you to mock out that implementation in unit tests.

            • By tengbretson 2024-08-131:374 reply

              A lot of times you want to be able to cmd+click on something and actually see what the hell the code actually does and not get dead-ended at an interface declaration.

              • By simiones 2024-08-1310:00

                What are you using that can cmd+click to take you to a definition, but can't also take you to an interface implementation? I develop Go in Emacs with the built-in eglot + gopls, and M-. takes me to the definition, C-M-. takes me to the implementation(s). It's a native feature of gopls. Sure, it's one extra button, but hardly impossible.

              • By randomdata 2024-08-134:581 reply

                Sounds like a UI bug more than anything.

                The compiler certainly knows how to determine if there is only one implementation of an interface and remove the interface indirection when so. There is nothing really stopping the cmd+click tooling from doing the same.

                • By anonymoushn 2024-08-135:351 reply

                  Does the compiler do that? That sounds extremely unlikely, especially because an interface with only one implementation can store the nil type tag or a tagged pointer to an instance of that implementation.

                  • By randomdata 2024-08-135:552 reply

                    The nil interface is another implementation. I mean, unless it is being used as the sole implementation, but I think we can assume that isn't the implementation being talked about given that it isn't a practical implementation. We're talking about where there is one implementation.

                    • By anonymoushn 2024-08-136:321 reply

                      Right. Can you cite anything that says that the go compiler does this sort of whole-program analysis to try to prove that a certain argument to a function is always non-nil, so that it can change the signature of that function and the types of variables declared in other functions?

                      • By randomdata 2024-08-1314:17

                        Uh. No. Why would I ever waste my time proving something I said? If I'm right, I'm right. If I'm wrong, you'll be sure to tell me. No reason for my involvement.

                    • By Dylan16807 2024-08-1314:251 reply

                      If a nil is another implementation then interfaces with a single implementation don't exist.

                      • By randomdata 2024-08-1314:571 reply

                        Given the following, where is the nil implementation found?

                            package main
                        
                            type FooInterface interface {
                                Baz()
                            }
                        
                            func bar(fizz FooInterface) {
                                bizz.Baz()
                            }
                        
                            type MyFoo struct{}
                        
                            func (*MyFoo) Baz() {}
                        
                            func main() {
                                var foo FooInterface = &MyFoo{}
                                bar(foo)
                            }

                        • By Dylan16807 2024-08-1316:061 reply

                          Nil is built-in. You just have to write the code to instantiate it and the compiler gives you one. The coder does not need to create an implementation, it's there for free.

                          I would not have called it a "second implementation" myself, but that's your claim to defend, not mine.

                          • By randomdata 2024-08-1316:171 reply

                            map is also built-in. Where do you find the hash map in the given program?

                            By your logic some nebulous package in a random GitHub repository that happens to satisfy an interface is also another implementation, but you would have to be completely out to lunch to think that fits with the topic of discussion.

                            • By Dylan16807 2024-08-1316:281 reply

                              > map is also built-in. Where do you find the hash map in the given program?

                              If you told me a type can be optimized because the compiler knows it can only have non-hash-map uses, but I could put that type into a hash map with a single line, I think I would be right to be skeptical.

                              > By your logic some nebulous package in a GitHub repository that happens to satisfy an interface is also another implementation, but you would have to be completely out to lunch to think that fits with the topic of discussion.

                              I expect the compiler to have a list of implementations somewhere. I don't know if I can expect it to track if nil is ever used with an interface. I could believe the optimization exists with the right analysis setup but you called the idea of finding a citation a "waste of time" so that's not very convincing.

                              • By randomdata 2024-08-1316:452 reply

                                > but you called the idea of finding a citation a "waste of time" so that's not very convincing.

                                Not only a waste of time, but straight up illogical. If one wants to have a discussion with someone else, they can go to that someone else. There is no logical reason for me to be a pointless middleman even if time were infinite.

                                Now, as fun as that tangent was, where is the nil implementation and hash map found in the given program?

                                • By anonymoushn 2024-08-1317:23

                                  You can head over to godbolt.org and see for yourself that changing the value to nil doesn't change the implementation of `bar`, though it does cause `main` to gain a body rather than returning immediately.

                                • By Dylan16807 2024-08-1317:231 reply

                                  The implementation is preexisting. Even if it was directly used, there would not be an implementation in the snippet. So it not being implemented in the snippet proves nothing.

                                  And what do you mean "someone else"? You're the one that said the compiler "certainly knows" how to do that.

                                  • By randomdata 2024-08-1317:431 reply

                                    > So it not being implemented in the snippet proves nothing.

                                    It doesn't prove anything, but is what we've been talking about. Indeed, there is nothing to prove. Never was. What is it with this weird obsession you have with being convinced by something? Nobody was ever trying to convince you of anything, nor would there be any reason to ever try to. That would be a pointless endeavour.

                                    > And what do you mean "someone else"?

                                    He who wrote the "citation".

                                    • By Dylan16807 2024-08-1317:581 reply

                                      > there is nothing to prove. Never was.

                                      What was the point of your question, if not to prove something?

                                      If you were trying to imply that the implementation doesn't exist, that implication was fatally flawed.

                                      If you were asking to waste time, then it worked.

                                      If you had another motive, what was it?

                                      Are we having a 5d chess game? I thought it was a normal conversation.

                                      > He who wrote the "citation".

                                      Nobody? Nobody wrote a citation.

                                      Do you mean the person that asked for a citation? If so, you're wrong. Finding evidence for your own claims would not make you a middleman. They didn't want to have a discussion with someone else, they wanted a discussion with you, and for that discussion to have evidence. Citing evidence is not passing the buck to someone else, it's an important part of making claims.

                                      • By randomdata 2024-08-1318:272 reply

                                        > What was the point of your question, if not to prove something?

                                        My enjoyment. For what other reason would you spend your free time doing something?

                                        > If you were trying to imply that the implementation doesn't exist, that implication was fatally flawed.

                                        And if I weren't trying?

                                        > If you were asking to waste time, then it worked.

                                        I ask nothing, but if you feel wasted your time, why? Why would you do such a thing?

                                        > If you had another motive, what was it?

                                        As before, my enjoyment. Same as you, I'm sure. What other possible reason could there be?

                                        > Nobody? Nobody wrote a citation.

                                        There was a request for me to refer another party who was willing to talk about the subject that was at hand – one that you made reference to ('you called the idea of finding a citation a "waste of time"'). Short memory?

                                        > Finding evidence for your own claims would not make you a middleman.

                                        There wasn't a request for evidence, there was a request for a citation. Those are very different things. A citation might provide some kind of pointer to help find evidence, which is what I suspect you mean, but, again, if that's what you seek then you're back to wanting to talk to someone else. If you want to talk to someone else, just go talk to them. There is no reason for me to serve as the middleman.

                                        > it's an important part of making claims.

                                        Nonsense. If my claim does not hold merit on its own, it doesn't merit further discovery. It's just not valuable at all. It can be left at that, or, if still enjoyable, can be talked about to the extent that remains enjoyable.

                                        Perhaps you are under the mistaken impression that we are writing an academic research paper here? I can assure you that is not the case.

                                        • By anonymoushn 2024-08-1318:381 reply

                                          It's great that in your reply upthread you actually understood that it was a request for any kind of evidence, including evidence you just created on the spot, but now you pretend not to understand that.

                                          • By randomdata 2024-08-1319:24

                                            What ever do you mean? There was no change in understanding. You spoke to seeking a proof in addition to a citation, the parent did not originally speak to the proof bit, only to a citation. Entirely different contexts.

                                            In fact, you would have noticed, if you read it, that the "upstream" comment doesn't even touch on the citation at all. It is focused entirely on the proof aspect. While the parent wanted to talk about citations exclusively, at least at the onset. Very different things, very different topics.

                                        • By Dylan16807 2024-08-1318:351 reply

                                          Okay so confirmed you're going meta to dodge any actual points being made, and grind any possible progress of the conversation to a halt. Bye!

                                          • By randomdata 2024-08-1318:361 reply

                                            Confused by academic paper writing confirmed. What lead you down that path?

                                            • By Dylan16807 2024-08-1318:371 reply

                                              Normal conversations have people back up their claims, dude...

                                              You're getting way more academic with this meta waste.

                                              • By randomdata 2024-08-1318:39

                                                You could just go to godbolt.org, as others have already said, and as any normal person would do. Evidence is neither here nor there, though. We're talking about citations, which nobody of sound mind does. Why on earth would you have a conversation using someone else's words? That's the stupidest thing I have ever heard of.

              • By cdelsolar 2024-08-1312:45

                Right click > view implementations

              • By zinodaur 2024-08-134:06

                yeah gotta use an IDE for that

            • By throwaway2037 2024-08-131:09

              I agree with your point. OP wrote:

                  > Many (sometimes even all) of these interfaces have just one implementation.
              
              They are missing that mocks are the second implementation. (It took me years to see this point.) I would say that in most of my code at work, 95+% of my interfaces only have a single implementation for the production code, but any/all of them can have a second implementation when mocking for unit tests.

            • By arp242 2024-08-1223:421 reply

              > Many (sometimes even all) of these interfaces have just one implementation.

              • By kbolino 2024-08-130:371 reply

                The point of using a mockable interface, even if there's only one real implementation, is to test the behavior of the caller in isolation without also testing the behavior of the callee.

                This can be overdone of course, not everything needs this level of separation, but if it makes testing one or both sides easier, then it's usually worth it. It's especially useful for testing nontrivial interactions with other people's code, such as libraries for services that connect to real infrastructure.

                • By randomdata 2024-08-131:072 reply

                  Did you miss "just one implementation"? A mock is literally defined by being another implementation. If the 'mock' is your sole implementation, we don't call it a mock, that's just a plain old regular implementation.

                  • By kbolino 2024-08-133:361 reply

                    I think my comment was clear on the distinction between real and mock implementations. If the code was testable with no need for mocks then certainly remove the interface and devirtualize the method calls.

                    • By randomdata 2024-08-134:141 reply

                      Your comment was clear about mocks, but not why mocks are relevant to the topic at hand. The original comment was equally clear that it was in reference to where there is only one implementation. In fact, just to make sure you didn't overlook that bit amid the other words, the author extracted that segment out into a secondary comment about that and that alone.

                      Mocks, by definition, are always a supplemental implementation – in other words, where there is two or more implementations. What you failed to make clear is why you would bring up mocks at all. Where is the relevance in a discussion about single implementations the other commenter has observed? I wondered if you had missed (twice!) the "one implementation" part, but it seems you deny that, making this ordeal even stranger.

                      • By kbolino 2024-08-1312:521 reply

                        It is easy to generate mock implementation code (GoMock has mockgen, testify has mockery, etc.) The lack of a hand-rolled mock implementation doesn't mean that much. For example, many people do not like to put generated code under source control. So, just because you don't see a mock implementation right away doesn't mean one isn't meant to be there. Also, the original author of the function that consumed the apparently unnecessary interface type may have intended to test it, but not had the time to write the tests or generate the mocks. If we are going to be this pedantic, I did say "mockable" interface, implying the usefulness and possibility, but not necessarily existence, of a mock implementation.

                        Since we are examining code we can't see, we can only speak about it in the abstract. That means the discussion may be broader than just what one person contributes to it. If this offends you or the OP, that was not the intent, but in the spirit of constructive discussion, if you find my response so unhelpful, it is better to disregard it and move along than to repeat the same point over and over again.

                        • By randomdata 2024-08-1314:46

                          > It is easy to generate mock implementation code

                          Not in any reusable way. Take a look at mockgen and testify: All they do is provide a mechanism to push implementation into being defined at runtime by user code. So, if they, or something like it, is in use the implementation is still necessarily there for all to see.

                          > Also, the original author of the function that consumed the apparently unnecessary interface type may have intended to test it

                          Okay, sure, but this is exactly what the commenter replied to was talking about initially. What is a repetition of what he said meant to convey?

                          > That means the discussion may be broader than just what one person contributes to it.

                          Hence why we're asking where the relevance is. There very well may be something broader to consider here, but what that is remains unclear. Mocking in and of itself is not in any way interesting. Especially when you could say all the very same things about stubs, spies, fakes, etc. yet nobody is talking about those, and for good reason.

                          > If this offends you

                          For what logical reason would an internet comment offend?

          • By ukoki 2024-08-1310:08

            Agreed. It's not as 'traditional Go' but I find there is way less interface boilerplate if you just pass functions around.

            ie instead of

            ```

            type ThingDoer interface { DoThing() }

            func someFunction(thingDoer ThingDoer) { ThingDoer.doThing() }

            ```

            just have

            ```

            func someFunction(doThing func()) { doThing() }

            ```

            Then when testing you can just pass a test implementation of the 'doThing' function that just verifies it was called with the expected arguments.

          • By neonsunset 2024-08-1222:382 reply

            Can't Go compiler statically prove that such single implementation interfaces are indeed that and devirtualize the callsites referring to them?

            Either way, the problem seems to happen in most languages of today, if they (or their community) ever happen to accidentally encourage passing an opaque type abstraction over a concrete one.

            • By nasretdinov 2024-08-1310:121 reply

              I think it actually does that, but in local contexts, where this analysis is somewhat easy.

              I also believe you don't actually have to prove it statically: PGO can collect enough data to e.g. add a check that a certain type is usually X, and follow a slow path otherwise

              • By neonsunset 2024-08-1312:36

                I understand that it does so when the exact type is observed - a direct call on a concrete type. But I was wondering if it performs whole-program-view optimization for interface calls. E.g. given a simple AOT-compiled C# program:

                    using System.Runtime.CompilerServices;
                
                    var bar = new Bar();
                    var number = CallFoo(bar);
                
                    Console.WriteLine(number);
                
                    // Do not inline to prevent observing exact type
                    [MethodImpl(MethodImplOptions.NoInlining)]
                    static int CallFoo(Foo foo) {
                        return foo.Number();
                    }
                
                    interface Foo {
                        int Number();
                    }
                
                    class Bar: Foo {
                        public int Number() => 42;
                    }
                
                On x86_64, 'CallFoo' compiles to:

                    CMP byte ptr [RDI],DIL ;; null-check foo[0]
                    MOV EAX,0x2a ;; set 42 to return value register
                    RET
                
                There is no interface call. In the above case, the linker reasons that throughout whole program only `Bar` implements `Foo` therefore all calls on `Foo` can be replaced with direct calls on `Bar`, which are then subject to other optimizations like inlining.

                In fact, if we add and reference a second implementation of `Foo` - `Baz` which returns 8, `CallFoo` becomes

                    ;; calculate the addr. of Bar's methodtable pointer
                    LEA    RAX,[DevirtExample_Bar::vtable]
                    MOV    ECX,0x8 ;; set ECX to 8
                    MOV    EDX,0x2a ;; set EDX to 42
                    ;; compare methodtable pointer of foo instance with Bar's
                    CMP    qword ptr [RDI],RAX
                    ;; set return register EAX to value of EDX, containing 42
                    MOV    EAX,EDX
                    ;; if comparison is false, set EAX to value of ECX containing 8 instead
                    CMOVNZ EAX,ECX
                    RET
                
                Which is effectively 'return foo is Bar ? 42 : 8;'.

                Despite my criticism of Go's capabilities, I am interested in how its implementation is evolving. I know it has the feature to manually gather a static PGO profile and then apply it to compilation which will insert guarded devirtualization fast-paths on interface calls, like what OpenJDK's HotSpot and .NET's JIT do automatically. But I was wondering whether it was doing any whole-program view or inter-procedural optimizations that can be very effective with "frozen world single static module" which both Go and .NET AOT compilations are.

                EDIT: To answer my own question, I verified the same for Go. Given simple Go program:

                    package main
                
                    import (
                        "fmt"
                    )
                
                    func main() {
                        bar := &Bar{}
                        num1 := callFoo(bar)
                
                        fmt.Println(num1)
                    }
                
                    //go:noinline
                    func callFoo(foo Foo) int {
                        return foo.Number()
                    }
                
                    type Foo interface {
                        Number() int
                    }
                
                    type Bar struct{}
                
                    func (b *Bar) Number() int {
                        return 42
                    }
                
                'callFoo' compiles to

                    CMP        RSP,qword ptr [R14 + 0x10]
                    JBE        LAB_0108ca68
                    PUSH       RBP
                    MOV        RBP,RSP
                    SUB        RSP,0x8
                    MOV        qword ptr [RSP + foo_spill.tab],RAX
                    MOV        qword ptr [RSP + foo_spill.data],RBX
                    MOV        RCX,qword ptr [RAX + 0x18] ;; load vtable slot?
                    MOV        RAX,RBX
                    NOP
                    CALL       RCX ;; call the address loaded from the vtable?
                    ADD        RSP,0x8
                    POP        RBP
                    RET
                    LAB_0108ca68                                    XREF[1]:
                    MOV        qword ptr [RSP + foo_spill.tab],RAX
                    MOV        qword ptr [RSP + foo_spill.data],RBX
                    CALL       runtime.morestack_noctxt                 
                    MOV        RAX,qword ptr [RSP + foo_spill.tab]
                    MOV        RBX,qword ptr [RSP + foo_spill.data]
                    JMP        main.callFoo
                
                It appears that no devirtualization takes place of this kind. Writing about this, it makes for an interesting thought experiment what it would take to introduce a CIL back-end for Go (including proper export of types, and what about structurally matched interfaces?) and AOT compile it with .NET.

                [0]: VMs like OpenJDK and .NET make hardware exception-based null-checks. That is, a SIGSEGV handler is registered and then pointers that need to throw NRE or NPE either do so via induced loads from memory like above or just by virtue of dereferencing a field out of an object reference. If a pointer is null, this causes SIGSEGV, where then a handler looks if the address of the invalid pointer is within first, say, 64KiB of address space. If it is, the VM logic kicks in that recovers the execution state and performs managed exception handling such as running `finally` blocks and resuming the execution from the corresponding `catch` handler.

        • By nasretdinov 2024-08-1220:52

          Yeah too much concurrency and too many channels definitely hit home hard...

      • By mapcars 2024-08-1215:418 reply

        I do programming interviews and I found candidates struggling a lot in doing http request and parsing response json in Go while in Python its a breeze, what makes it particularly hard, is it lack of generics or dict data type?

        • By conor- 2024-08-1215:53

          I think it depends on what kind of data you're dealing with. If you know the shape of your data, it's pretty trivial to create a struct with json tags and serialize/deserialize into that struct. But if you're dealing with data of an unknown shape it can be tricky to work with that. In Python because of dynamic typing and dicts it's a little easier to deserialize arbitrary data.

          Go's net/http is also slightly lower level. You have to concern yourself directly with some of the plumbing and complexity of making an http request and how to handle failures that can occur. Whereas in Python you can use the requests lib and fire off a request and a lot of that extra plumbing just happens for free and you don't have to deal with any of the extra complexity if you don't want to.

          I find Go to be bad for interviewing in a lot of cases because you get bogged down with minutiae instead of working directly towards solving the exact problem presented in the interview. But that minutiae is also what makes Go nice to work with on real projects because you're often forced into writing safer code

        • By nrr 2024-08-1222:513 reply

          It comes down to how the standard library makes you do things. I don't think there's any reason why a more stringly-typed way of handling JSON (or, indeed, a more high-level way of using HTTP) is outside of the realm of possibility for Go. It's just that the standard library authors saw fit not to pursue that avenue.

          This variability is honestly one of the reasons why I dislike interviews that require me to synthesize solutions to toy problems in a very tightly constrained window of time, particularly if the recruiter makes me commit at the outset to one language over another as part of the overall interview process. It's frustrating for me, and, having been on the other side, it's noisy for the interviewer.

          (In fact, my favorite interview loop of all time required that I use gdb to dig into a diabolical system that was crashing, along with some serious code spelunking. The rationale was that, if I'm good with a debugger and adept at reading the source that's in front of me, the final third of synthesizing code solutions to problems and conforming to institutional practice can be dealt with once I'm in the door.)

          • By marcus_holmes 2024-08-132:021 reply

            My favourite tech interview (so far) was similar: "here's the FOSS code base we're working on. This issue looks like about the size we can tackle in the time we have. Let's work on this together and solve it".

            I got to show how I could grok a code base and work out where the problem was quickly, and work out a solution to the problem, and how I understood how to contribute a PR. Way better than random Leetcode bullshit, and actually useful: the issue was actually solved and the PR accepted.

            • By mapcars 2024-08-181:45

              I'm not a fan of this approach because candidates may see it as a "cheap" way to do actual work without being payed.

          • By throwaway2037 2024-08-131:121 reply

            I like your story about debugging during an interview. I can say from experience, you always have one teammate that can just debug any problem. I am always impressed to watch and learn new techniques from them.

            • By nrr 2024-08-131:49

              This has also been my experience, yeah. My interviewers were very interested in watching me rifle through that core dump. (:

              Ultimately, it feels to me like selecting for people who both can navigate existing code and interrogate a running system (or, minimally, one that had gone off the rails and left clues as to why) is the right way to go. It has interesting knock-on effects throughout the organization (in, like, say, product support and quality assurance) that are direly understated.

          • By mapcars 2024-08-181:50

            In our case we give some high-level description beforehand (which mentions working with REST apis) and allow candidates to use any language of their choice.

            Also in our case the API has typing in form of generated documentation and example responses. I even saw one Go-candidate copying a response into some web tool to generate Go code to parse that form of json.

            I can also say that people who chose Java usually have even more problems, they start by creating 3-4 classes just to follow Spring patterns.

        • By badrequest 2024-08-1220:162 reply

          I think other languages cause folks to understand JSON responses as a big bag of keys and values, which have many convenient ways of being represented in those languages. When you get to Go and you want to parse a JSON response, it has to be a well-defined thing that you understand ahead of time, but I also think you adapt when doing this more than once in Go.

          • By ljm 2024-08-1220:582 reply

            If I had one complaint, it’s the use of ‘tags’ to configure how json is handled on a struct, such that it basically becomes part of the struct’s type. It can lead to a fair bit of duplication of structs whose only difference is the json handling, or otherwise a lot of boilerplate code with custom marshal/unmarshal methods. In some cases the advice is even to do parse the json into a map, do the conversion, and then serialise it again!

            The case I ran into is where one API returned camelCase json but we wanted snake_case instead. Had to basically create another struct type with different json tags, rather than having something like decoders and encoders that can configure the output.

            I like Go and a lot of the decisions it makes, but it has its fair share of pain points because of early decisions made in its design that results in repetitive and overly imperative code, and while that does help create code that is clear and comprehensible (mostly), it can distract attention away from the intended behaviour of the code.

          • By kloop 2024-08-1221:141 reply

                var res map[string]any
                err := json.Unmarshal(&res)

            • By scubbo 2024-08-131:543 reply

              Uh huh....and what comes next?

              Trying to descend more than a couple of layers into a GoLang JSON object is a mess of casts.

              • By lenkite 2024-08-138:021 reply

                Well, one used to have https://github.com/mitchellh/mapstructure which assisted here, but the lib then got abandoned.

              • By kloop 2024-08-1411:311 reply

                The same things that happens in JS or python.

                If you get a key wrong it throws (panics in go)

                • By scubbo 2024-08-1518:23

                  I wasn't talking about getting the keys wrong, but rather the insane verbosity of GoLang - `myVariable := retrievedObject.(map[string]interface{})["firstLevelKey"].(map[string]interface{})["secondLevelKey"].(string)` vs. `myVariable = retrievedObject["firstLevelKey"]["secondLevelKey"]`

                  "Oh, but that's just how it is in strongly-typed languages" - that may well be true, but we're comparing "JS or python" with GoLang here.

              • By Yoric 2024-08-136:48

                Especially when you're not certain of the type used for numbers.

        • By randmeerkat 2024-08-1222:052 reply

          > I do programming interviews and I found candidates struggling a lot in doing http request and parsing response json in Go while in Python its a breeze, what makes it particularly hard, is it lack of generics or dict data type?

          Have you considered that your interview process is actually the problem? Focus on the candidate’s projects, or their past work experience, rather than forcing them to jump through arbitrary leet code challenges.

          • By travisd 2024-08-1222:521 reply

            Making an HTTP request and dealing with JSON data is a weed-out question at best. Not sure if you are interpreting the grandparent comment as actually having them write a JSON parser, but I don't think that's what they meant.

            • By mjevans 2024-08-1223:13

              I either had that come up in an interview recently myself, OR it wasn't clear to me that I was allowed to use encodings/json to parse the json and then deal with that. I happened to bomb that part of the interview spectacularly because I haven't written a complex structure parser in years given every language I've used for such tasks ships with proper and optimized libraries to do that.

          • By mapcars 2024-08-181:56

            Well these are not arbitrary, we work with a number of json apis on a weekly basis, supporting the ones we have and integrating new ones as well. This is a basic skill we are looking for, and I don't see it as a "leet code challenge".

            Candidates might have great deal of experience debugging assembly code or generating 3d models, but we just don't have tasks like that.

        • By Yoric 2024-08-136:461 reply

          There is a dict-equivalent data type in Go for JSON (it's `map[string]any`), it's just rather counter-intuitive.

          However, as a Go developer, I'm one of the people who consider that JSON support in Go should be burnt down and rebuilt from scratch. It's both limited, annoying, full of nasty surprises, hard to debug and slow.

        • By sethammons 2024-08-1310:58

          I've done, literally, hundreds and hundreds of coding interviews and an http api was a part of lots of them. Exported vs non-exported fields and json tags are about the only issues I've seen candidates hit in Go and I would just help in those kinds of cases. Python is marginally easier for many.

          The problem was java devs. Out of dozens upon dozens of java devs asked to hit a json api concurrently and combine results, nearly ZERO java devs, including former google employees, could do this. Json handling and error handling especially confounded them.

        • By jay_kyburz 2024-08-1220:526 reply

          Hold on, did you just say Go doesn't have a Dictionary data type?

          I'm a Javascript, Lua, Python, and C# guy and Dict is my whole world.

          • By AmpsterMan 2024-08-1221:01

            It does. https://go.dev/blog/maps

            What the poster was alluding to is that you usually prefer to deseialize to a struct rather than a record/dict/map

          • By zhengyi13 2024-08-1221:063 reply

            Not a programmer, so this is every programmer's chance to hammer me on correctness.

            No, Go doesn't have a type named Dict, or Hash (my Perl is leaking), or whatever.

            It does have a map type[1], where you can define your keys as one type, and your values of another type, and that pretty closely approximates Dicts in other languages, I think.

            [1]: https://go.dev/blog/maps

            • By tialaramex 2024-08-1223:451 reply

              So, these types (and many more) are hash tables.

              https://en.wikipedia.org/wiki/Hash_table

              They're a very common and useful idea from Computer Science so you will find them in pretty much any modern language, there are a lot of variations on this idea, but the core idea recurs everywhere.

              • By simiones 2024-08-1310:13

                I have a quibble here. A hash table, the basic CS data structure, is not a two-dimensional data structure like a map, it is a one-dimensional data structure like a list. You can use a hash table to implement a Map/Dictionary, and essentially everyone does that. Sets are also often implemented using a hash table.

                The basic operations of a hash table are adding a new item to the hash table, and checking if an item is present (potentially removing it as well). A hash table doesn't naturally have a `V get(key K)` function, it only naturally has a `bool isPresent(K item)` function.

                This is all relevant because not all maps use hash tables (e.g. Java has TreeMap as well, which uses a red-black tree to store the keys). And there are uses of hash tables besides maps, such as a HashSet.

                Edit: the term "table" in the name refers to the internal two-dimensional structure: it stores a hash, and for each hash, a (list of) key(s) corresponding to that hash. Storing a value alongside the key is a third "dimension".

            • By mjevans 2024-08-1223:181 reply

              I think I'd want to try to decode into map[string]interface{} (offhand), since string keys can be coerced to that in any event (they're strings in the stream, quoted or otherwise), and a key can hold any valid json scalar, array, or object (another json sub-string).

              • By simiones 2024-08-1310:36

                That of course works, but the problem is then using this. Take a simple JSON like `{"list": [{"field": 8}]}`. To retrieve that value of 8, your Go code will look sort of like this:

                  var v map[string]any
                  json.Unmarshal(myjson, &v)
                  lst := v["list"].([]any)
                  firstItem := lst[0].(map[string]any)
                  field := firstItem["field"].(float64)
                
                And this is without any error checking (this code will panic if myjson isn't a json byte array, if the keys and types don't match, or if the list is empty). If you want to add error checking to avoid panics, it gets much longer [0].

                Here is the equivalent Python with full error checking:

                  try :
                    v = json.loads(myjson)
                    field = v["list"][0]["list"]
                  except Exception as e:
                    print(f"Failed parsing json: {e}")
                
                [0] https://go.dev/play/p/xkspENB80JZ

            • By duskwuff 2024-08-1221:241 reply

              And, if you hate strong typing, there's always map[string]any.

              • By dwattttt 2024-08-1222:291 reply

                Really, the mismatch is at the JSON side; arbitrary JSON is the opposite of strongly typed. How a language lets you handle the (easily fallible) process of "JSON -> arbitrarily typed -> the actual type you wanted" is what matters.

                • By throwaway2037 2024-08-131:17

                      > arbitrary JSON is the opposite of strongly typed
                  
                  On the surface, I agree. In practice, many big enterprise systems use highly dynamic JSON payloads where new fields are added and changed all the time.

          • By bitwize 2024-08-1220:59

            Go has had a dict-like data type from the jump; they're called "maps" in Go.

            Some of early Go's design decisions were kinda stupid, but they didn't screw that one up.

          • By absoflutely 2024-08-1221:121 reply

            Go has maps, json parsing and http built in. I'm not exactly sure what this person is referring to. Perhaps they are mostly interviewing beginners?

            • By marcus_holmes 2024-08-132:092 reply

              Go maps have a defined type (like map[string]string), so you can only put values of that type in them. A JSON object with (e.g) numbers in it will fail if you try and parse that into a map of strings.

              As others have said, the issue with Go parsing JSON is that Go doesn't handle unstructured data at all well, and most other languages consider JSON to be unstructured data. Go expects the JSON to be strongly typed and rigidly defined, mirroring a struct in the Go code that it can use as a receiver for the values.

              There are techniques for handling this, but they're not obvious and usually learned by painful experience. This is not all Go's fault - there are too many endpoints out there that return wildly variable JSON depending on context.

              • By claytongulick 2024-08-133:181 reply

                I feel like good JSON handling is sort of table stakes for any language for me these days.

                The pain of dealing with JSON in Go is one of the primary reasons I stick mostly with nodejs for my api servers.

                • By lelanthran 2024-08-136:031 reply

                  > The pain of dealing with JSON in Go is one of the primary reasons I stick mostly with nodejs for my api servers.

                  Unless you're dealing with JSON input that has missing fields, or unexpected fields, there is no pain. Go can natively turn a JSON payload into a struct as long as the payload's fields recursively match the struct's fields!

                  If, in any language, you're consuming or generating JSON that doesn't match a specific predetermined structure, you're yolo'ing it and all bets are off. Go makes this particualr footgun hard to do, while JS, Python, etc makes it the default.

                  Default footguns are a bad idea, not a good idea.

                  • By marcus_holmes 2024-08-136:181 reply

                    This.

                    In $other_language you'll parse the JSON fine, but then smack into problems when the field you're expecting to be there isn't, or is in the wrong format, or the wrong type, etc.

                    In Go, as always, this is up front and explicit. You hit that problem when you parse the JSON, not later when you try to use the resulting data.

                    • By simiones 2024-08-1310:501 reply

                      Go's JSON decoder only cares if the fields that match have the expected JSON type (as in, list, object, floating point number, integer, or string). Anything else is ignored, and you'll just get bizarre data when you work with it later.

                      For example, this will parse just fine [0]:

                        type myvalue struct {
                          First int `json:"first"`
                        }
                      
                        type myobj struct {
                          List []myvalue `json:"list"`
                        }
                        js := "{\"list\": [{\"second\": \"cde\"}]}"
                        var obj myobj
                        err := json.Unmarshal([]byte(js), &obj)
                        if err != nil {
                          return fmt.Errorf("Error unmarshalling: %+v", err)
                        }
                        fmt.Printf("The expected value was %+v", obj) //prints {List:[{First:0}]}
                      
                      This is arguably worse than what you'd get in Python if you tried to access the key "first".

                      [0] https://go.dev/play/p/m0J2wVyMRkd

                      • By marcus_holmes 2024-08-140:592 reply

                        It totally makes sense from a Go perspective: You created a struct, tried (but failed) to populate it with some json data, and ended up with a value initialised to its zero-value. This is fine :)

                        One of the techniques for dealing with JSON in Go is to not try to parse the entire JSON in one go, but to parse it using smaller structs that only partially match the JSON. e.g. if you endpoint returns either an int or a string, depending on the result, a single struct won't match. But two structs, one with an int and one with a string - that will parse the value and then you can work out which one it was.

                        • By lelanthran 2024-08-147:351 reply

                          > It totally makes sense from a Go perspective: You created a struct, tried (but failed) to populate it with some json data, and ended up with a value initialised to its zero-value. This is fine :)

                          To me it looks like a footgun: if the parsing failed then an error should have been signalled. In this case, there is no error and you silently get the wrong value.

                          • By marcus_holmes 2024-08-150:43

                            Yeah, I presented that wrong. It's not actually a failure as such.

                        • By simiones 2024-08-149:491 reply

                          > It totally makes sense from a Go perspective: You created a struct, tried (but failed) to populate it with some json data, and ended up with a value initialised to its zero-value. This is fine :)

                          I do agree that there are good reasons on why this behaves the way it does, but I don't think the reason you cite is good. The implementation detail of generating a 0 value is not a good reason for why you'd implement JSON decoding like this.

                          Instead, the reason this is not a completely inane choice is that it is sometimes useful to simply not include keys that are meant to have a default value. This is a common practice in web APIs, to avoid excessive verbosity; and it is explicitly encoded in standards like OpenAPI (where you can specify whether a field of an object is required or not).

                          On the implementation side, I can then get away with always decoding to a single struct, I don't have to define specific structs for each field or combination of fields.

                          Ideally, this would have been an optional feature, where you could specify in the struct definition whether a fields is required or not (e.g. something like `json:"fieldName;required"` or `json:"fieldName;optional"`). Parsing would fail if any required field was not present in the JSON. However, this would have been more work on the Go team, and they generally prefer to implement something that works and be done with it, rather than working for all important cases.

                          Separately, ignoring extra fields in the JSON that don't match any fields in the struct is pretty useful for maintaining backwards compatibility. Adding extra fields should not generally break backwards compatibility.

                          > One of the techniques for dealing with JSON in Go is to not try to parse the entire JSON in one go, but to parse it using smaller structs that only partially match the JSON. e.g. if you endpoint returns either an int or a string, depending on the result, a single struct won't match. But two structs, one with an int and one with a string - that will parse the value and then you can work out which one it was.

                          I have no idea what you mean here. json.Unmarshal() is an all-or-nothing operation. Are you saying it's common practice to use json.Decoder instead?

                          • By marcus_holmes 2024-08-150:48

                            > I have no idea what you mean here. json.Unmarshal() is an all-or-nothing operation. Are you saying it's common practice to use json.Decoder instead?

                            No, I mean you create a struct that deals with only a part of the JSON, and do multiple calls to Unmarshal. Each struct gets either populated or left at its zero-value depending on what the json looks like. It's useful for parsing json data that has a variable schema depending on what the result was.

              • By lenkite 2024-08-138:041 reply

                Umm, you can unmarshall into a map[string]any, you know ?

                    dataMap := make(map[string]any)
                    err = json.Unmarshal(bytes, &dataMap)

                • By simiones 2024-08-1310:391 reply

                  You can, but then it's a lot of work to actually traverse that map, especially if you want error handling. Here is how it looks like for a pretty basic JSON string: https://go.dev/play/p/xkspENB80JZ. It's ~50 lines of code to access a single key in a three-layer-deep JSON.

                  • By lenkite 2024-08-1312:201 reply

                    Its more like 30 lines of code without the prints. However, one generally should code generic functions for this. The k8s apimachinery module has helper functions which is useful for this sort of stuff. Ex: `NestedFieldNoCopy` and its wrapper functions.

                    https://github.com/kubernetes/apimachinery/blob/95b78024e3fe...

                    Ideally, such `nested` map helper functions should be part of the stdlib.

                    • By simiones 2024-08-1312:50

                      Sure, in production you'd definitely want something like that, but the context was an interview exercise, I don't think you should go coding generic wrappers in that context.

          • By mseepgood 2024-08-134:22

            It does (it's called a map) and Go does have generics, the previous poster clearly doesn't know what they're talking about.

      • By cdelsolar 2024-08-1312:411 reply

        Go has ruined all other languages for me. I really fell in love with Gleam recently and was trying to implement a fun side project in it. The problem is I really don’t have enough time to learn the intricacies of it, with a startup, two kids, etc. As soon as I have to look at some syntax and really _think_ about what it’s doing every time I look at it, I lose interest. I kept trying and eventually implemented it in Go much faster. And while doing it in Go I kept wishing I could just use actors and whatever to make it simpler but, is it really simpler?

        • By conor- 2024-08-1314:51

          I haven't looked too deeply into it but I came across https://github.com/ergo-services/ergo not too long ago and thought it could be pretty interesting to try using OTP in Golang

          Packaging a Go service in Docker and dumping it into k8s is probably the easier/better understood path but also deploying Go services onto an Erlang node just sounds more fun

      • By justinsaccount 2024-08-1215:191 reply

        Yep.. say you wanted to make a simple http service that needs to

        * request a json.gz file from another HTTP service * decompress it * deserialize the json, transform it a bit

        That's net/http (and maybe crypto/tls), compress/gzip, encoding/json. I need to make zero decisions to get the thing off the ground. Are they the best libraries in the world for those things? no.. but will they work just fine for almost every use case.

        • By chubot 2024-08-1220:331 reply

          Sounds like

              curl ... | jq ...
          
          to me!

          Not saying you shouldn't use Go for that problem, in a particular context, but it does drive home how much of programming is glue ... there is combinatorial amounts of glue, which is why JSON, HTTP, compression, etc. end up being part of so many problems

          • By dartos 2024-08-1222:15

            There’s a big difference between building something on curl and jq and building something using a language’s standard library.

            Everything is just bits at the end of the day. Just about anything can do anything.

      • By cookiengineer 2024-08-130:17

        I feel the same.

        Especially in the UI/UX world when you want to just start building a demo you're paralized by dozens of build toolchains in between.

        Wanna get started with a starter template? Tough luck, it hasn't been updated for a year, so it takes even longer.

        In go, everything is opinionater and unified upstream. Conventions matter, because they allow efficiency and reuse of patterns and architectures.

    • By sanderjd 2024-08-1312:452 reply

      It has not been my experience that Go is good for almost everything. On the contrary, it seems good at a couple very specific (though very common) niches: network services and cli utilities. But for most of what I do right now - data heavy work - it has not turned out to be very good (IMO). It really is just not better in any way to have to constantly write manual loops to do anything.

      • By ratorx 2024-08-140:441 reply

        I think Go is pretty OK as a language for building data pipelines (I’m assuming you meant statistical ones, but the same argument applies to more data transform-y ones). What it is not good for is doing exploratory analysis (which is where Python shines).

        Manual loops are pretty annoying when the focus is on figuring out which loops to write (exploratory phase). However, they are pretty nice once you’ve figured it out and need write a durable bit of code where your prioritise readability over conciseness (productionisation).

        Going from Python to <any language> between the exploratory phase and the productionised pipeline is going to be a pain, I don’t think Go is particularly worse than others. At that point it’s all about the classic software tradeoffs (performance vs velocity vs maintainability) and I didn’t think Go is a good choice in many situations.

        • By sanderjd 2024-08-1415:21

          Well I totally disagree that writing manual loops is ever "pretty nice", but I agree that it's not as big an issue in final-version code as it is in exploration.

          And I'm also in strong agreement that making any language transition between exploration and implementation is problematic. I do think go is worse than most, because I just think it has a mostly cultural allergy to manipulating collections of data as collections rather than element-by-element, but I agree that this is mostly lost in the noise of doing any re-write into a new language.

          But this is why Python is best in this space. It simply has the best promotion path from experimentation to production. It is better than other "real" languages like go, because it thrives in the exploratory phase, and it is better than purpose-specific languages, like R, because it is also a great general-purpose language.

          The other contender I see is Julia, which comes more from the experimentation-focused side, while trying to become a good general purpose language, but unfortunately I think it still needs to mature a lot on that side, and it's not clear that it has the community to push it far enough fast enough in that direction (IMO).

          Even very performance-critical use cases work with python, because the iteration process can follow experimentation -> productionization -> performance analysis -> fixing low-hanging bottlenecks by offloading to existing native extensions -> writing custom native extensions for the real bottlenecks.

      • By lnenad 2024-08-1313:251 reply

        I agree with this, there is a reason Python is the king of data.

        • By sanderjd 2024-08-1318:15

          Yeah, but Go is also worse (in my experience) than most, if not all, of the other general-purpose languages I've used, for this niche.

          For instance, Rust is actually pretty great in this space, despite being very ... not-python, and Java also has decent libraries. Then C++ (and Fortran!) are good for a very different reason than python, "on the metal" performance, which Go also isn't a great fit for.

    • By lenkite 2024-08-137:522 reply

      Go is not good for data science and ML. It doesn't even have a proper, maintained dataframe library for data-science. R and Python beat it hands on. Rust also beats it now thanks to polars. And mobile ? gomobile is not maintained. Fyne is amateur level on mobile.

      AFAIK Go has no maintained 3d game engines.

      Go has its well-established niche for middle-ware services and CLI tools. And that's about it. If your domain is outside this, its best to choose another language.

      • By Lyngbakr 2024-08-139:583 reply

        Is the reason for the absence of a well-maintained dataframe library lack of demand? It looks like Gota and Dataframe-go are abandoned, while Gonum isn't particularly active. Did these wither on the vine because no one used them?

        • By aragilar 2024-08-1312:491 reply

          It's likely because it's a pain to call Fortran or C (I'd be suspicious of anyone trying to reimplement openblas in go).

          • By Something1234 2024-08-1315:272 reply

            Why would you be suspicious?

            • By sanderjd 2024-08-1412:41

              There is no good reason to pursue a project like this (or if there is, it would be very surprising to me), so it would reek suspiciously of "not invented here".

            • By aragilar 2024-08-1412:30

              Because more likely than not, they're going to screw it up, and the choice to use go was not made for sound engineering reasons (go is not the only language this would apply to, but because of the lack of good FFI, its more likely to happen). The exception would be if said person had a solid background in numerical computing and was up-to-date with the state of the field, but that's pretty easy to find out.

        • By sanderjd 2024-08-1411:47

          I think it's lack of demand, yeah, but I think that's downstream of a real culture clash. Exploratory data analysis is just really not a good fit, culturally, for go. You don't want to be explicit about everything and check every single error case, etc.

          But then it's natural to evolve production systems out of exploratory analyses, rather than re-writing everything from scratch, unless there is a very compelling reason to do that. The compelling reason is usually to get more speed, but that's not go's strong suit either.

      • By merlindru 2024-08-139:12

        Also anything that requires interop/FFI, syscalls, and lower level stuff. It's extremely hard to record your screen on Go, for example. On Rust this is much more doable and there even are crates for it

        It's not impossible with Go, though. There's an amazing windows impl here: https://github.com/kirides/go-d3d

        ...but if you look at the code, it's clear that you have to work against the language in some capacity

    • By drdaeman 2024-08-133:181 reply

      I can totally agree that Go is good enough for most projects, to the extent it's a go-to choice for many. However, it's not always the best tool - frequently it wins just because it allows to prototype quicker. YMMV, but for me, it's not a love relationship, it's a love-hate relationship.

      > The language is extremely simple.

      Simplicity is a double-edged sword. It didn't even have basic generics for a long while, and it's painful to remember how bad it was without them. Still doesn't have a lot of stuff anyone would expect from a modern language.

      > If you know 20% of C you are already a Go expert.

      Strong disagree. Knowing $language means knowing the patterns and caveats and (sorry for a cliche term, I'm not fond of it, but I don't have a better term so I hope you get the meaning) "best practices". Those are drastically different between C and Go. Especially when it comes to concurrency and parallelism.

      > The toolchain and the core libraries alone can do 90% of what most people will ever need.

      This is provably false. Virtually every serious project out there pulls a ton of dependencies, like database drivers, configuration toolkits, tracing and logging libraries and so on. Heck, I think a lot of shops have project templates for this reason - to standardize on the layout and pull the company-preferred set of libraries for common stuff. Core libraries are slowly getting there but so far they don't have very basic stuff like MapSet[T] or JSON codecs for nullable types like sql.NullString, so you gotta pull third-party stuff.

      • By AnimalMuppet 2024-08-1312:351 reply

        "Good enough" plus "I already know it" almost always beats the perfect tool that I don't already know.

        • By sanderjd 2024-08-1412:471 reply

          There is obviously truth in this, but I think it is more often the case than many people think it is that it is more efficient to learn the better tool "good enough" than to use the worse "I already know it" tool.

          For instance, I've seen lots of people not want to learn sql and instead write complicated imperative implementations of relational primitives in the "I already know it" programming languages that are clearly "good enough" (because they are turing complete after all). But sql is usually a much better way to do this, and isn't hard to learn "good enough" to do most things.

          • By AnimalMuppet 2024-08-1414:191 reply

            Well, there's a balance. I'm not saying "never learn anything new, assembly's good enough and I know how to use it". Learn newer and better tools.

            At the same time, don't learn every newer and better tool. There are too many. You don't have enough time, even if you never do anything but learn.

            SQL is enough better to be worth learning. The web framework of the week? Not so much.

            And what's "worth learning" depends on what you're trying to do. For a home project, I'll use what I know, unless the goal of the project is "learn how to use X". For work, the question is whether it brings enough to the table to be worth the learning time. Sometimes it is; often it isn't.

            • By sanderjd 2024-08-1415:10

              Yeah what I'd say is: Seek out and be open to advice. There's a "don't know what you don't know" problem here, as always. But this is also part of the point of reading sites like HN! People here are saying "actually there are tools that are net positive to learn a bit because they are much better choices for particular niches". That is advice! It's fine and all to say "nah, I'm good", but in many cases that's doing yourself a disservice. I really do see people writing tedious for loops in go because it is what they're comfortable with, when they would be much better served writing sql and using a language with dataframes.

              Most of the time people aren't just on a kick about selling some hot new thing (and I'm old enough that go was the hot new thing for me at one point!), they actually have relevant experience and are giving useful advice.

    • By danpalmer 2024-08-1223:421 reply

      > When people argue about the validity of these claims, I simply point them to this talk https://go.dev/talks/2012/concurrency.slide#42

      There's nothing impressive in these slides. This may have been pretty good in 2012, but this code looks very much like the equivalent Swift or Rust.

      • By sapiogram 2024-08-139:17

        It's better in Swift or Rust. Go channels are the worst thing about the language, even trivial use cases are difficult to get right.

    • By eweise 2024-08-131:392 reply

      Personally I don't find it great for prototyping. There's just too much boilerplate, which is why I use other languages for fun/personal projects.

      • By sanderjd 2024-08-1412:48

        Absolutely, it's my least favorite prototyping environment. It forces me to think about a bunch of stuff I don't want to think about up front.

      • By pants2 2024-08-135:232 reply

        Is there much boilerplate aside from err checks and JSON tags? Even then, your IDE / copilot should automatically insert those along with imports and package names.

        • By eweise 2024-08-1313:51

          Err checks are a big one. I don't want to worry about error handling when prototyping. There are little things like having to prefix methods with the struct name and type, and bigger things like no default arguments and by name parameters, which makes setting up test fixtures cumbersome. Also, functions don't compose well because there can be more than one return value, so you end up just writing more intermediate values.

        • By lifthrasiir 2024-08-136:45

          The requirement to define at least one function by itself is a boilerplate. Also IDE doesn't fully solve the inability to compile and run a partially written program (in fact, Go compiler is even more pedantic than rustc in some aspect), which happens a lot when working with dynamically typed languages and their strongest use cases.

    • By actionfromafar 2024-08-1222:032 reply

      My personal Python threshold is 10k lines. After that I tend to loose track of what I am doing and I start to miss static typing and nowadays, an IDE to navigate it. Maybe future Python IDEs can AI scan the codebase and compensate.

      • By sanderjd 2024-08-1414:45

        Type annotations plus modern IDEs have (in my experience) made this mostly a non-issue. It does require a bit more setup to get passable static analysis set up, which comes "for free" in languages with good compilers (like go!), but it's at least possible now to get to a (IMO) good point.

        (FWIW, I moved away from python and ruby over a decade ago because of exactly this frustration, but I'm finding modern python to be pretty pleasant.)

      • By JodieBenitez 2024-08-135:401 reply

        Wait... in what year was this comment written ? ;)

        There's type hinting and IDEs do navigate it.

        • By The_Colonel 2024-08-1310:53

          optional hints

          Both make them much less useful than in a typical statically typed language. Besides that, the tooling sucked hard not too long ago.

    • By ternaryoperator 2024-08-136:42

      Agreed on all points except #3 re the core libraries. Coming from the Java ecosystem, it was a bit of a shock to see how small the standard libraries are. For example, the minuscule collections library, among others.

    • By api 2024-08-1221:581 reply

      Go has very little story when it comes to desktop or mobile GUI apps, which is too bad because it would be a very productive language for that kind of thing.

      • By anta40 2024-08-1313:43

        This makes me wonder: why there's less effort on mobile side? For web, well there are plenty of stuffs.

        Perhaps Go devs are not interested in native solution (like Kotlin/Swift etc) and prefer web stack (JS, React etc)?

    • By noisy_boy 2024-08-145:12

      > Once your project exceeds 100 lines

      100 lines in which language? 100 lines in Go will probably have 50 lines of if err != nil :P

    • By Yoric 2024-08-136:431 reply

      > The author lists multiple reasons for this, but for me the biggest one is the first one: Go is good for almost everything.

      I'd nuance that claim.

      I haven't found Go to be _particularly good_ at any specific task I've undertaken, but Go was _good enough_ for many of these tasks. Which makes Go a reasonable general programming language.

      • By sanderjd 2024-08-1414:491 reply

        This is true, but there are a number of _good enough_ languages, and personally I don't think go is top-tier at this use case of being the go-to swiss army knife. I do think it is top-tier at being a good choice for tools in its niche. But not as "default language I reach for when I don't want to waste time thinking about it".

        • By Yoric 2024-08-1621:51

          Agreed. For my personal sensibilities, Python or TypeScript are better default languages. Of course, I'm a bit obsessive about quality/safety, so I'm probably going to use Rust for most tasks :)

    • By jayd16 2024-08-1220:464 reply

      I'm asking this earnestly but is Go suitable for native GUI apps (not web)? 3D graphics/Games? Audio processing?

      • By ssebs 2024-08-1315:23

        I've written a GUI in Fyne, it's decent but go is not a first class GUI language. Fyne is... fine but building code is annoying.

        See https://GitHub.com/ssebs/go-mmp

      • By xyproto 2024-08-1221:061 reply

        Yes, because one can either turn off the GC, allocate memory up front (arena style), or call C or Assembly from Go.

        The performance is likely to be good enough with GC turned on, though, because it is unusually fast.

        • By jayd16 2024-08-1222:532 reply

          Turning off the GC isn't the blocker. Plenty of GC languages manage.

          I'm more interested in how Go handles graphics APIs and binding a render thread or working with GUI threads considering how goroutines are done. Does one need to write non-idiomatic go, avoiding goroutines or is there some trick?

          For example, GTK isn't thread safe so you can't just use that library without considering that.

          • By irq-1 2024-08-1317:57

            No, go is not for native gui apps. I recently made some rough go bindings to minifb and while easy to do, it wasn't productive at all. Errors are hard to follow; are they go or minifb? Callbacks work until I have too many calls then the app might freeze altogether.

            Go is great for image/draw and things like passing pixels: (C.uint)buffer.Pix

            It comes down to Google wanting Go to use the web as the interface, which in practice means not doing dynamic linking (except in Windows.)

            To your question, go gui apps will have 'runtime.LockOsThread()' near the start so it's all green/light/go threads and only 1 OS thread.

          • By mjevans 2024-08-1223:20

            Offhand, I think the general pattern is to bind related external libraries (E.G. a gui stack) into one agent thread and use channels to send it messages.

      • By nasretdinov 2024-08-1220:511 reply

        For games potentially Ebiten, I believe it has some audio processing support too.

        • By runeblaze 2024-08-1221:10

          Probably not Ebiten for 3d games. To be fair at this point when you are doing somewhat specialist things Go starts to lose its edge. I remember trying to replicate some Numpy code in Go and that was a pain. However that's just because Python is too good at scientific things.

      • By 0cf8612b2e1e 2024-08-1221:521 reply

        People write games and GUI apps in Python, so why not?

        • By jayd16 2024-08-1222:501 reply

          Is there some particular reason Python is similar to go with regards to GUIs or are you saying everything can do everything?

          • By 0cf8612b2e1e 2024-08-1222:58

            Python is a slow interpreted language (I write Python every day). Go is compiled and an order of magnitude faster for most everything.

            So, I guess, yeah, everything can do everything. Computers are fast where language is rarely going to be the deciding factor.

    • By mjw_byrne 2024-08-1312:581 reply

      4. Stability and backwards-compatibility. I have never seen a Go version upgrade break anything. Meanwhile I have colleagues who do a thousand-yard-stare if you so much as mention upgrading the version of Python we're using.

      • By sanderjd 2024-08-1414:50

        Python upgrades are fine, actually. The python library ecosystem is a bit of a mess in general, which does affect this, but the tools have actually improved to make this more manageable lately.

    • By cube2222 2024-08-1214:351 reply

      Yeah, this. It's just good enough for 95% of use-cases, while being very productive.

      Personally, one of the biggest selling points for me is that imo modelling concurrency and asynchronicity via fibers (goroutines), rather than async/await, is just a ton easier and faster to work with. Again, there are use-cases for the alternative (mainly performance, or if you like to express everything in your type-system) but it's just great for 95% of use-cases.

      • By sanderjd 2024-08-1414:53

        I always find it odd when people refer to it as being "very productive". I find all the boilerplate very un-productive to work with. Every time I pick go back up I'm shocked how many manual loops and conditionals I have to write.

    • By kaba0 2024-08-1214:325 reply

      How is it different than, say, java for this generalist purpose?

      • By lantry 2024-08-1215:003 reply

        IME, there are two main differences between go and java:

        1) go is more "batteries included". Modules, linting, testing, and much more are all part of the standard cli. Also, the go stdlib has a ton of stuff; in java, there is almost always a well-built third party library, but that requires you to find and learn more things instead of just reaching for stdlib every time.

        2) golang is "newer" and "more refined". this is pretty subjective, but golang seems to have fewer features and the features are more well-planned. It's a more "compact" or "ergonomic" language. Whereas java has built up a lot of different features and not all of them are great. You can always ignore the java features you don't like ofc, but this is still a bit of cognitive overhead and increased learning time.

        • By nerdponx 2024-08-1220:24

          There are surprisingly few languages in this category, especially if you limit consideration to statically typed. Go, C#, Swift? Nim, Crystal? F#? Kotlin?

          Go is not my favorite language, but it really is exceptional in terms of its effective utilitarian design.

        • By hughesjj 2024-08-1220:372 reply

          Eh, imo the go libraries still aren't up to par with out of the box java libraries. Like there's still no Set class, nor the equivalent of Map.keys. yeah they're easy to write but that's still not an included battery.

          Also, while the cli to add stuff is useful, there's still nothing to the level of maven or gradle for dependency management, and I usually find myself doing some fun stuff with `find -execdir` for module management.

          Different strokes for different folks though. Java (really, kotlin) still makes a ton of sense for backend to me given how the jvm is architecture independent and you don't have to make tradeoffs/switch to graal if it's a long lived service.

          Golang is nice, love it, but it's still got a bit to grow. I'm just happy they added modules and generics. I don't think it's a matter of being 'well thought out' as much as it's a simple language that cares a lot about simplicity and backwards compatibility and has iterated ever since. For my the killer app isn't go routines so much as you can produce a binary that's resilient even to shared/dynamically linked libraries in all platforms, which is awesome for portability independent of environment. No more gcc vs clang vs msvc headaches, no more incompatible shared libraries, no more wrong version of jvm or a bad python modules path etc.

          Oh, also java had like a 15 year headstart on golang, and it wasn't until java 8 that many of my biggest complaints were addressed. And yeah stuff like apache commons +log4j+mockito/junit are pretty much required dependencies, and maven/gradle aren't language native.

          The best STL is probably python imo but even that doesn't support a proper heap/priority queue inplementation. For data structures specifically I think java/kotlin has the best STL. All of this ignoring .NET or apple platforms.

          • By onionisafruit 2024-08-133:381 reply

            maps.Keys is coming in a few days with 1.23.

            I generally agree with what you are saying. Although I wouldn’t hold out Maven as a paragon. I used to make my living untangling pom.xml files. I don’t think anybody is feeding their family helping people with go.mod messes — although I wish somebody would do that for kubernetes.

            • By hughesjj 2024-08-1317:17

              > maps.Keys is coming in a few days with 1.23.

              Yup! Lots of good stuff in the go/x libraries, happy map.Keys is graduating.

              Also totally agreed on maven not being an ideal, or even lovable dependency management system, but trust me that there are absolutely people spending too much time wrangling go.mod files, especially in open source where go.mod files cross repo boundaries (and thus consolidated test automation).

              I typically do a bunch of git reset --hard's as I iterate on my find -exec to run the golang cli commands for module upgrades (for whenever rennovate fails). I like how it's much easier to experiment with over maven (maybe I just didn't know maven well enough), but it's still definitely a headache.

          • By duskwuff 2024-08-1221:292 reply

            > Like there's still no Set class

            map[T]struct{} ?

        • By sanderjd 2024-08-1414:58

          FWIW, I think versions of java from the last 5ish years feel both "newer" and "more refined" than go.

          But I do think go and java are very comparable languages. To me, go's advantage over java is more about use-case; go is the clear choice for little cli tools, because it's pretty far off the beaten path to coax java to start up quickly enough for this. This is the sweet spot for go, IMO.

      • By conor- 2024-08-1215:043 reply

        As someone who isn't super proficient in Java I usually find Java daunting to get started with full of buckets of "meta" issues like in my other comment.

        What JVM do I use? Does it matter?

        Does it matter what version I install, what if I have to install/manage multiple versions?

        If I want to write a web service can I use vanilla Java stdlib or do I have to use Spring or some framework? If I use Spring, do I have to get into the weeds of dependency injection and other complexity to actually get my app off the ground?

        With Go, none of those questions exist. I install the latest Go, create a main.go file, I use net/http and I'm off to the races.

        • By neonsunset 2024-08-1219:44

          I think it's good that Go, among other well-made toolchains, brought attention to importance of good CLI UX.

          But it's not something that is unique to Go:

              dotnet new web
              dotnet run
              curl localhost:5050
          
          also 'dotnet watch' for hot-reload.

        • By simon_void 2024-08-1311:01

          > What JVM do I use?

          latest LTS (longterm-support) version, so currently 21, next 25 (late 2025).

          > Does it matter?

          new is always better

          > what if I have to install/manage multiple versions?

          https://sdkman.io/

          > If I want to write a web service can I use vanilla Java stdlib or do I have to use Spring or some framework?

          Spring

          > If I use Spring, do I have to get into the weeds of dependency injection

          yes, but it's not hard (or I've done it long enough)

          > other complexity

          that might just be inherent complexity when have to deal with a webservice. personally I dislike have to configure security

          So basically go to Spring Initializr (https://start.spring.io/) pick the latest LTS Java&Maven (or Kotlin&Gradle-Kotlin) and You're off to the races.

        • By pjmlp 2024-08-138:05

          Besides what neonsunset points out for .NET world, where alongside C#, we get the pleasure to enjoy F#, VB and C++/CLI, it is relatively easy for Java.

          When one doesn't know, just like with Go, one picks up the reference implementation => OpenJDK.

          For basic stuff, the standard library also has you covered in the jdk.httpserver module.

          By the way, where is the Swing equivalent on Go's standard library?

      • By colonelpopcorn 2024-08-1214:561 reply

        Just from an ergonomic standpoint it's a million times easier to deploy a go binary instead of a whole jvm and a jar file.

        • By pjmlp 2024-08-138:06

          When it is a pure Go application, only as CLI or UNIX server.

      • By Dansvidania 2024-08-1214:583 reply

        java on its own is not a competitor to go, IMO, due to the batteries included "culture" in the go ecosystem.

        I would need to compare it with, for example, Java + Spring(Boot).

        I find Go to be simpler and more pleasant to use.

        • By matwood 2024-08-1222:39

          I agree with Go being simpler, but modern Java and Spring Boot is also fine. Backend programmers are spoiled with riches nowadays with all the completely workable options we have.

        • By vips7L 2024-08-1315:03

          Batteries included but there’s not even a Set[T] in the standard library..

        • By pjmlp 2024-08-138:06

          Where is the GUI battery in Go?

      • By aaomidi 2024-08-1215:03

        Dependencies in Java are a pain in the ass.

        Publishing libraries for your own consumption is even harder.

        Publishing libraries for world consumption means now you also need to be a PKI expert.

    • By createaccount99 2024-08-1310:23

      Builtins well thought out? Have you seen the bignumber api?

    • By riku_iki 2024-08-1320:26

      > Go is good for almost everything.

      Java is good in more areas than go

HackerNews