OP has his retirement prepared. That might increase their perception of the upsides and negate some of the downsides of adopting AI.

At this point theists often try to smuggle God back in as the source of morality through culture.

But I agree, empirically religion and moral behavior seem at best uncorrelated.

Transcendental moral principles can still be secular.

One that I find compelling is that Rawls' veil of ignorance lets us imagine that we might be on either side of a conflict, and that therefore moral actions are equitable to both sides. This gives us a secular morality that doesn't come with the baggage of religious outgroup dynamics.

No it's just stochastic like everything about LLMs. The md file will bias results towards a certain set of outcomes.

That's just you being unable to let go of your biases? In languages with pointers giving primacy to offsets makes sense. In every other context aligning with natural language and math (first, second, 1..n inclusive) is perfectly logical.