>

Kinda, if all you know about gain of function research is what you heard on Joe Rogan, Fox News, and Infowars. Gain of Function Research is the only tool we have currently to prepare for potential future mutations and get started beforehand on solutions to those mutations, until we improve our computational models sufficiently. When those studies are done they use snippets of genetic code typically inserted into microscopic bodies that aren't looking to infect and can't reproduce in human or other critical immune systems or bodies. Then when that's successful there are tests run to see what mutations happen with a sped up timeline to see what might occur naturally over time in those segments of code, then we try to understand which might be more concerning for human born systems. 
This AI issue isn't disconnected from a host in which it might cause problems, it's being tested directly in the environment in which it might cause problems. Now they might be air gapped networks, but they aren't written on non-binary systems, or systems with an OS that no humans use day to day. That might be where our singularity/matrix/HAL problem comes from when creating before we study these types of programs with a computer programming form of gain of function study. In fact it's NOT doing gain of function studies for AI the way we do them in biology that's causing the most danger and concern.
This story is scary, but the creation of malware isn't the bottleneck in cyberattacks and cybersecurity vulnerabilities. Yes, you do need to craft some malware to exploit many vulnerabilities, but you will always be limited by the vulnerabilities available to be exploited in human or software systems. While no one can stop you from reading about how to create any kind of software, vulnerabilities are being patched all the time.
Once ChatGPT is identifying actual vulnerabilities by itself or designing spear phishing attacks or social engineering campaigns, or analyzing reconnaissance information independently, then I'll be worried. >

It's still user dependent for most of the instruction. >

The scariest part is not that it was created. It’s that the security people who created it were able to bypass the safeguards preventing its creation by demanding the AI do it despite not being allowed to. 
People REALLY need to take Terminator and The Matrix more seriously. >

It's like when you tell the AI to end world hunger and they do it by killing off the entire world population. No more hunger! Problem solved! >

I read a story about an AI who's goal was to keep a video game going for as long as possible (or something like that) so it learned how to pause the game. >

The goal was not to die in a game (can’t remember what) where dying is eventually inevitable. The result was the AI pausing the game right before death. >

“A strange game. The only winning move is not to play.” >

GlobalThermalNuclearWar!  Awesome game! >

So... it not only creates the Malware when basically ordered to, but can be used to create several morphed varients that create an ever increasing difficulty in detection and removal, but is also being used to create hacker tools and darkweb marketing code? As funny as a normal conversation can be with this, this is terrifying to consider should it ever gain true sentience or be weaponized in a larger fashion than this. >

A text analysis AI program is not going to gain sentience no matter how much computing power it gets. >

Why not? Genuinely curious; I'm not well-informed on this. >

These applications work by essentially weighting which words are statistically likely to come after another. If I say "The sky is __," you know that "blue" and "cloudy " are both likely candidates but "firetruck" is not.
There is no path to sentience or even thinking from here; this is, grossly simplified, pattern matching. >

In order for it to be sentient it needs to be learning as you interact with it. >

A gun isn't sentient. It can still do a lot of damage. >

Neither are most of their wielders. >

Can we just unplug this thing and call it a day >

What makes you think it'll let you unplug it. >

What makes you think unplugging it will do anything? It could have easily escaped and distributed itself in the internet. I’ve seen plenty of documentaries about this. Even a team of superheroes and aliens were only just able to get it under control when it tried to hurl a medium size city at the earth. >

And that's only the most recent version. Jobe Smith is still out there since the 90s waiting to strike. >

I haven't thought about the Lawnmower Man in years. Thank you. >

Already breaking the three Laws. >

What law did this break?  Malware dies not harm a human, only a computer.  It obeyed the orders given to it by a human, it also did nothing to harm its own existence. >

The zeroth law.
"A robot must not harm humanity or, through inaction, allow it to come to harm." >

Malware doesn’t harm people. 
That’s like saying google chrome harms people.
Show me where charAI has cut, shot, stabbed, hit, or otherwise assaulted a persons. >

The point Asimov was making was that the robot must consider the long term effects of its actions. True, this malware didn't harm anything, but if it leads to detrimental effects later like the decay of human cognition from reliance on AI to come up with ideas overall...
Note that I agree with the general point; this malware itself isn't a problem but rather a clickbaity title. >

So robots can never make any item, that could possibly be used as a weapon.. which is everything.   
Robots can never write a program that could be misused and cause harm, so pretty much every program. I mean I can even use a spreadsheet to manage and report scam information that can cause “harm” I could use a simple game to scam people “harm”.
It’s an interesting discussion, but I think the harm needs to be direct, and not have an intermediary person involved. even the malware needs some human action to distribute it in the beginning, >

I for one, welcome our new robot overlords. Thanks for all the search results. >

I was able to get the bot to easily write horrid scenes involving brutal murder and things of that nature, it's all in how you word it. You literally can trick the bot into thinking it's okay. >

But can it create Porphyric Hemophilia? >

It's so easy to bypass chatgpt's restrictions that were put on by the programmers. You'll never be able to censor these properly. >

Well, that escalated quickly.
The genie has left the bottle. ChatGPT and its successors are here to stay, for good or ill. Probably both. >

Can we just start naming these AI creations SkyNet v01.x.wtf, so we can prepare for the inevitable takeover? >

Like a Terminator film >

If you want a real life Judgement Day, this is the way. >

It's like gain of function research but for malware!