By playing this game I'm helping to train AI how to be less detectable?

>Who benefits from AI is smaller businesses who could not afford custom application development at previous development costs.

Of course, as AI reduces the cost to operate in niches, those small businesses who just gained the ability to build an app are also more likely than before to see a bigger player drink their milkshake.

Not to mention that small businesses will have a harder time absorbing the inevitable price hike that will come once everyone has made themselves completely dependent on AI to get any work done.

I agree with you that it's about tradeoffs.

The cost ($$$, opportunity cost, and mental toll) of maintenance is very real. It can be hugely advantageous to outsource that effort to a professional, PROVIDED the professional is trustworthy and competent. To ensure that most professionals are trustworthy and competent two things need to be present:

1. A very high degree of transparency, so that it's very difficult for a service provider to act contrary to their user's interests without the user knowing about it.

2. Very low switching costs, so that if the service provider ever does act against their users' interests, they will be likely to lose their users.

As long as our laws encourage providers to operate in black-box fashion, and to engineer artificially high switching costs into their products, I believe there will continue to be a case for self-hosting among a minority of the population. And because they are a minority, they will be forced to also make use of centralized services in order to connect to the people who are held hostage by those high switching costs.

Somewhere in the multiverse, there's a world in which interoperability and accountability have been enshrined as bedrock principles and enforced since the beginning of the internet. It would be very interesting to compare that world with the one we inhabit.

Alcohol is harmful, and you want to prevent minors from obtaining it without parental supervision. Do you pass a law requiring every car to log the age of every occupant in case the driver drives to an establishment that sells alcohol? No, that's stupid. You require the person providing the alcohol to check age only when they are about to hand over the alcohol. Until someone actually attempt to access alcohol, they should not be asked their age.

Now exchange "car" for "OS" and "alcohol" for "age-sensitive content"

A security camera, on its own, doesn't tell the grocery store who you are. There was a time when CCTV didn't even exist and yet we still had commerce.

"What we've got" isn't "the best we can do". There absolutely are better possibilities that would protect consumers. The best way to ensure we never get to experience those better systems is to shrug our shoulders and passively accept whatever treatment we receive.