The cost has always been the sum of:

1. The time spent to think and iteratively understand what you want to build 2. The time spent to spell out how you want to build it

The cost for #2 is nearly zero now. The cost for #1 too is slashed substantially because instead of thinking in abstract terms or writing tests you can build a version of the thing and then ground your reasoning in that implementation and iterate until you attain the right functionality.

However, once that thing is complex enough you still need to burn time on identifying the boundaries of the various components and their interplay. There is no gain from building "a browser" and then iterating on the whole thing until it becomes "the browser". You'll be up against combinatorial complexity. You can perhaps deal with that complexity if you have a way to validate every tiny detail, which some are doing very well in porting software for example.

There could be many plausible explanations.

1. The model's default world model and priors diverge from ours. It may assume that you have another car at the wash and that's why you ask the question to begin with.

2. Language models do not really understand how space, time and other concepts from the real-world work

3. LLM's attention mechanism is also prone to getting tricked as in humans

I haven't used it in a while but RedHat used to feel quite a bit like Windows.

Similar questions trick humans all the time. The information is incomplete (where is the car?) and the question seems mundane, so we're tempted to answer it without a second thought. On the other hand, this could be the "no real world model" chasm that some suggest agents cannot cross.

I use Obsidian to record decisions, plan every day and take detailed notes. Very handy for recalling the nitty gritty for future reference be it performance reviews, writing blogs or updating my resume.