AI diminishes most humans...

 there's actually some utopic SF out there - i claim the society described in John Brunner's Shockwave Rider that hides behind the self-replciating worm and lives in houses grown out of trees with addresses like on least mean square, off of mean free path is like that, or the new territories in the Neal Stephenson;s Diamond Age with people handcrafting paper, or the folks in colelctives in Cory Doctorow's Walk Away (and even many people on many planets in Iain ? Banks Culture series) have a fine old time.

however, they are (almost all, almost always) creatives, participants,, engaged ("concerned"). But most people are counch potatoes most of the time. Most people have neither the innate ability or time to learn the skills & knowledge to be so wonderful. Most people will rot.  To quote Billy Strayhorn

"AI is mush

Stifling those who strive
I'll live a lush life
In some small dive

And there I'll be
While I rot with the rest
Of those whose lives are lonely, too"

AI diminishes Humans

 The more I see people talk about the benefits of AI, the more I see it as a tool for reducing humanity. 

It is very much the false idol, indeed the goal of AGI is simply Deep Fake Humanity, and this not just crossing the uncanny valley. All the tasks AI does are things humans might delight in - we are not talking about better robots for driving EV taxis or industrial production lines- we're talking about things that make people people. By definition, AI does not give humans agency, it takes it away. 

The areas I am fine with "AI" is where we use it to accelerate things like physics models (e.g. weather prediction). But that's really just neural operators as a fast approximator for PDEs, and also Bayes and causal inference where we get an explanation of why X probably makes Y happen.

I really think we should stop other kinds of AI, as they are a crime against humanity waiting to happen.

When we talk about AI as an existential threat, most of the time we're referreing to AI linked to weapons (nukes, bio-weapons etc) but in Speculative Fiction (e.g. Childhood's End or other great classic stories) when human's encouncter super-smart, often benificient or completely benign, but sometimes super helpful aliens, the usual result is a rapid diminutation of the human spirit. A collapse into couch-potatoe status for the whole of planet earth. and the complete loss of ambition to do anything (e.g. explore space, or even just our selves).

I'm wondering if Adrian Tchaikovsky will write a sequel to the very excellent Shroud and where that will go?

AI for science, for whom, exactly?

Science, from OED, is "knowledge, understanding, secular knowledge, knowledge derived from experience, study, or reflection, acquired skill or ability, (...as granted by God)".

Excluding the last point in brackets, it seems that the key point is omitted, as unsurprisingly, no-one considered what happens if we advance knowledge, but in absentia humans. Imagine for a minute that we wrote out the knowledge in a book and hid it in the British Library amongst 18M books, or wrote it down in a language no-one knew and would take more than a lifetime to learn.

A similar argument could be made against the validity of automated proofs - a proof is "evidence or argument establishing a fact or the truth of anything", where the elephant in the (court) room is the target for whom the fact or truth is established.

So yes, an AI can advance science and can proove facts, in principle, without violating these defniitions, but I suspect that if we went back in time to when the notions were first being firmed up, we might find some resistance to the idea that a mechanical discovery or proof that was never witnessed or understood knowingly by a living being might be contrary to the intentions.

 Intention being the operative term - conscious people of free will, might want to take actions based on the knowledge or evidence, but why should they trust it if it isn't vouchsafed by other people? Sounds like "do this because I know better" or proof-by-authority, which is a well known logical fallacy, e.g. see here for why.

monstering ahoy

 it is unbrearably common to hear people mix up the master and marguerita,

or the villain and hero - in these hysterical final days, for example,

the existential threat from AI is almost always couched in terms of arnie

forgetting that he actualy saved people, and it was skynet that was bad - there

are lots more example, see below (spot the deliberate mistakes..)

frankenstein & monster

terminator and skynet

Wopr and Lightman

Colossus and Forbin

Marvin and Deep Thought

Robbie and the Monsters of the Id

Herbie and Susan Calvin

Wintermute and the Matrix

instead imagine we named the Mad Scientist after a type of rice

and their poor maligned AI robot creation after some kind of pasta--

here are some modest proposals - feel free to use them in any scribblings

you might undertake...

Professor Bomba and her loyal Stringozzi

Dr Glutinous' Vermicelli

President Arborio's long lost Linguine

Sushi's secret Capellini

Master Matta's FEvered fusilli

Baron Basmati of the ridiculous Fettuccine

The Lady Jasmine's zealous  Ziti

General Arrack's tragic Trenette

Police Constable Patna's terminally trivial Tripoline