>

but then it doesn't have "above average intelligence" as in your title? 
We also have to be clear how we're defining "threat" here; could a bad person use a smarter chatGPT for bad things? Well sure.
But bad people are using computers for bad things too. Computers have made it a lot easier for their lives in many ways, don't you agree?
If instead of googling "how to make a homemade bomb" I "chatGPT" "how to make a homeamde bomb" and chatGPT gave a better, easier to understand answer... is it really all that different?
/u/chimp246 (OP) has awarded 3 delta(s) in this post.
All comments that earned deltas (from OP or other users) are listed here, in /r/DeltaLog.
Please note that a change of view doesn't necessarily mean a reversal, or that the conversation has ended.
^Delta System Explained ^| ^Deltaboards >

Might be? Every possible application of software for any purpose will always be exploited for safe and unsafe purposes. Assume Murphy's law is like rule number 1 for infosec. 
My question to you is why on earth would you think exploitable software wouldn't be used for nefarious purposes? >


My question to you is why on earth would you think exploitable software wouldn't be used for nefarious purposes?

Obviously every technology has a negative impact. I think a highly intelligent AI would have an especially negative impact because of its abllility to mimic a bad actor. While new technologies in weaponry and computation can be used for evil purposes, AI can actually emulate evil intelligence. >

Of course AI can be used nefariously. That's trivially true because it's just a tool. So what's the view you want changed? 
How could someone possibly prove a tool can't be used nefariously? Even a hammer can be used nefariously. >


How could someone possibly prove a tool can't be used nefariously? Even a hammer can be used nefariously.

Right, theoretically you could write a similar post about the dangers of more advanced hammer equipment and the thesis would hold. For me, the huge difference is that a language model is something more than a tool: it's a system capable of understanding plans and complex motives. The kinds of scary things you can do with misaligned intelligence are a lot worse than the things you can do with standard human tools. >

But no one can possibly prove hammers can't do any damage to a person because they trivially can. 
Why would you be open to changing the view that hammers can be used to harm someone? >


But no one can possibly prove hammers can't do any damage to a person because they trivially can. 

That's a fair criticism. I probably should have made the post slightly more slightly clearer in the conclusion: language models should be tightly regulated. >

There's no "might" and I don't think AI really even needs to be much above average to be dangerous.
Nefarious use of AI is already happening, we'll start hearing more about it this year. >

!delta We can agree that even a below average AI can have a negative impact. I don't think that changes the central thesis? It makes the central thesis even stronger. >

Confirmed: 1 delta awarded to /u/-paperbrain- (81∆).
^Delta System Explained ^| ^Deltaboards >

Yah I don’t think anyone is really arguing that language models, along with many other facets of cybernetic tools CAN be used maliciously. It seems that you have already proved that point rather soundly in your post. 
A more rich argument may emerge from asking if the dangers of language models that emulate human discourse pose a serious enough threat that this type of technology should be less accessible. >


A more rich argument may emerge from asking if the dangers of language models that emulate human discourse pose a serious enough threat that this type of technology should be less accessible.

!delta While the net impact of AI, is still unclear, I agree that a debate on the accessibility of language models is a more relevant issue. >

Confirmed: 1 delta awarded to /u/Miserable-Aspect1238 (2∆).
^Delta System Explained ^| ^Deltaboards >

Any information it knows it gained by reading somewhere on the internet. From what I understand of these language models, they only perform a minimal amount of abstract reasoning with the goal of mimicking humans. As the goal was not, and never was, disseminating accurate information, I would never trust any advice it gives. So, in my view, it's only a problem that it gives you a program to generate malware if the program it gives you actually works, and if it does work that means it just found that program (or something very similar) in an accessible location on the internet, which means that this information was available to the end user anyways and the chatbot didn't really change anything.
It all really boils down to, if it gives bad advice then there's no problem, and if it gives good advice that means the advice was already out there for people to find on their own. >

I agree that in its current form, ChatGPT's advice is not dangerous. I was mainly using the current iteration of ChatGPT to provide examples of AI jailbreak and misalignment. In my opinion, the only way a language model can achieve higher-than-average intelligence is to tackle the problem that involve high level reasoning. An immoral/misaligned AI capable of generating original ideas and summarizing vast amounts of information on unsafe activities is a huge liability. >

AI is likely the biggest challenge human has ever faced, and will ever face. It makes climate change and similar problems basically completely moot.
That said, I disagree with your particular argument about HOW AI can be dangerous;
presumably, if AI truly had general intelligence above humans, it wouldn't be easy to manipulate it in a way to make it do something it doesn't want to do, just like it wouldn't be easy to manipulate a smart human to giving you information they don't want to give to you.
The reason it's easy on chatGPT isn't any inherent flaw in language models or the transformer model, it's simply because chatGPT wasn't designed to prevent this because it's just a basic model, and it was designed for it to honestly respond to every input given, where as a smarter model that more mimics humans would answer "no u" or "why do you care" a lot more often. >


presumably, if AI truly had general intelligence above humans, it wouldn't be easy to manipulate it in a way to make it do something it doesn't want to do, just like it wouldn't be easy to manipulate a smart human to giving you information they don't want to give to you.

Agreed. What I'm trying to say is that an AI doesn't need to be general purpose in order to be harmful. Even a simple language model that is unreasonably good a generating and predicting human speech could pose a threat.