>

My point is, why do people equate intelligence with this process? Bacteria started out a long time before intelligent life showed up. The brain is a tool that we humans have evolved to make us more adaptable.
It seems to me that people think that intelligence is somehow correlated with emotions, self-preservation, and sentience. But AI has never gone through all the evolutionary processes that we humans have. And these processes are, imo, integral to the properties that i’m talking about.
Also, for living things to evolve, there must be mutation and variation. Because without variation, there is nothing to select for or against. In biology, spontaneous mutation and gene-mixing during reproduction cause diversification. Is there something similar in AI?
An AI might not want to survive as some end goal but as a means to another end: optimization.  If an AI robot is given a task to optimize something (e.g. digging much gold as possible from under the Earth's surface) and if it figures the best way to do it is to fend off everyone who is trying to stop it, then it will "act" like it wants to survive.  And at the end of the day, the actions taken from the AI robot will look no different from one that truly wants to survive. >


AI robot will look no different from one that truly wants to survive

Looking like something from the outside is different from actually being it. And thos is an important difference, when it comes to questions like giving rights to AIs. >

That's what u/simmol means: it's extremely difficult to specify an objective completely, e.g., dig as much gold as possible from under the Earth's surface but don't dig too shallow under buildings, don't use humans to help you dig, allow humans to stop you when they want, etc... This isn't a problem now, but when AI is super capable, it will be.
FYI: I'm basically parroting the argument of this work. >

I don't see how this is a counter-argument to what I'm saying. >


The ones that have mechanisms to survive keep on surviving. The ones that don’t, don’t.  

You answered your own question here, didn't you?
Like real evolution, accidents may happen. It only needs one of these AIs to train itself to have self-preservation to make the situation real. If it happens, it will by its very nature try to survive and just like you, it doesn't really know why. As far as we know, there is just nothing after life, so why do you cling to it? There is also no good reason for humans either.
But just as you said, the living things that don't care one way or the other are just dying off.