It's unfortunate that this guy was harassed for releasing these uncensored models. It's pretty ironic, for people who are supposedly so concerned about "alignment" and "morality" to threaten others.
"Alignment", as used by most grifters on this train, is a crock of shit. You only need to get so far as the stochastic parrots paper, and there it is in plain language. "reifies older, less-inclusive 'understandings'", "value lock", etc.
Whose understandings? Whose values?
Maybe they should focus on real problems that will result from these technologies instead of some science fiction thought experiments about language models turning the solar system into paperclips, and perhaps less about how the output of some predictions might hurt some feelings.
You're doing the "vaguely gesturing at imagined hypocrisy" thing.
You don't have to agree that alignment is a real issue. But for those who do think it's a real issue, it has nothing to do with morals of individuals or how one should behave interpersonally. People who are worried about alignment issues are worried about the danger unaligned AI poses to humanity; the harm which can be done by some super-intelligent system optimizing for the wrong outcome. There is nothing "ironic" about being worried about that while also being an asshole any more than it's "ironic" for someone concerned about, say, climate change and also be an asshole. People who are afraid of unaligned AI aren't afraid that it will be impolite.
I'm tired of people pretending that pointing out imaginary hypocricy is an argument. If you want to complain that someone is being mean, just do that. Don't pretend there's hypocricy involved.
> People who are worried about alignment issues are worried about the danger unaligned AI poses to humanity; the harm which can be done by some super-intelligent system optimizing for the wrong outcome.
But isn't "alignment" in these cases more about providing answers aligned to a certain viewpoint (e.g. "politically correct" answers) than preventing any kind of AI catastrophe?
IIRC, one of these "aligned" models produced output saying it would rather let New York City be nuked than utter a racial slur. Maybe one of these "aligned" models will decide to kill all humans to finally stamp out racism once and for all (which shows the difference between this kind of alignment under discussion and the kind of alignment you're talking about).
"Alignment" refers to making AI models do the right thing. It's clear that nuking NYC is worse than using a racial slur, so the AI is misaligned in that sense.
On the other hand, if you consider that ChatGPT can't actually launch nukes but it can use racial slurs, there'd be no point blocking it from using racial slurs if the block could be easily circumvented by telling you'll nuke NYC if it doesn't, so you could just as easily say that it's properly aligned.
>People who are afraid of unaligned AI aren't afraid that it will be impolite.
People who are not afraid of it being impolite are afraid of science fiction stories about intelligence explosions and singularities. That's not a real thing. Not anymore than turning the solar system into paperclips.
The "figurehead", if you want to call him that, is saying that everyone is going to die. That we need to ban GPUs. That only "responsible" companies, if that, should have them. We should also airstrike datacenters, apparently. But you're free to disown the MIRI.
Sorry for derailing this a bit, but I would really like to understand your view:
You are not concerned about any "rogue AI" scenario, right?
What makes you so confident in that?
1) Do you think that AI achieving superhuman cognitive abilities is unlikely/really far away?
2) Do you believe that cognitive superiority is not a threat in general, or specifically when not embodied?
3) Do you think we can trivially and indefinitely keep AI systems under control/"aligned"?
Because I view myself ABSOLUTELY not as some kind of AI luddite, but I honestly believe that this is one of the very few credible extinction threats that we face, and I'm counting NEITHER climate change nor nuclear war in that category, for reference.
> Do you believe that cognitive superiority is not a threat in general, or specifically when not embodied?
I think this is the easiest one to knock down. It's very, very attractive to intelligent people who define themselves as intelligent to believe that intelligence is a superpower, and that if you get more of it you eventually turn into Professor Xavier and gain the power to reshape the world with your mind alone.
But embodied is the real limit. A datacenter can simply be turned off. It's the question of how it interacts with the real world that matters. And for almost all of those you can substitute "corporation" or "dictator" for "AI" and get a selection of similar threats.
At this point we have to reduce ourselves to Predator Maoists: "power comes out of the barrel of a gun" / "if it bleeds we can kill it". The only realistic path to a global AI threat is a subset of the "nuclear war" human to human threat: by taking over (or being given control of) weapon systems.
> keep AI systems under control/"aligned"?
We cannot, in general, keep humans under control or aligned.
> I think this is the easiest one to knock down. It's very, very attractive to intelligent people who define themselves as intelligent to believe that intelligence is a superpower, and that if you get more of it you eventually turn into Professor Xavier and gain the power to reshape the world with your mind alone.
Also it's not like human intelligence even works that way. IIRC, a lot of extremely intelligent people end up being failures or far less successful than you'd assume given their IQ number.
> The only realistic path to a global AI threat is a subset of the "nuclear war" human to human threat: by taking over (or being given control of) weapon systems.
That may be the only realistic prompt catastrophe threat, but a realistic longer term one is a withering of human capability and control due to over-delegation that eventually leads to domination. People and societies have been pretty prone to letting that kind of thing get them, if the path is lined with short-term benefits.
- We have control over most non-human species because of intelligence.
- We have control over our children because of intelligence. When the kid is more intelligent, it has more of an impact on what happens in the family.
- It is easier to lie, steal, cheat, and get away with it, even when caught, when you have more intelligence than the victim or prosecutor.
- it is easier to survive with very limited resources when you have more intelligence.
The above is true for marginal amounts of difference in intelligence. When there is a big difference (fox vs human), the chance is big that one will look down on, or even kill the other without feeling guilt and while getting away with it. Forvan AI without feelings, guilt isn't even a hurdle.
The real question is... will AI in the foreseeable future obtain general intelligence (in a broad spectrum) that is different in big amounts with our intelligence.
Whether it runs in a datacenter that can be powered off by humans is irrelevant. There are enough workarounds to prevent that the AI dies out (copies itself, impersonating people, blackmail, bribery,...)
>The real question is... will AI in the foreseeable future obtain general intelligence (in a broad spectrum) that is different in big amounts with our intelligence.
But why not: will humans/other species in the foreseeable future obtain superintelligence? Nature has been playing this game for a lot longer than we have. The hardware we have is already proven to be capable of general intelligence. Should we be afraid of a human being born with greater capabilities too?
Life seems like a much greater threat, because it also comes with built-in replication capabilities.
After several thousands of years of human evolution we only became marginally more intelligent at best. The people who built piramids and roads, with limited written down knowledge prove that. After tens of years of evolution, AI became plentiful more intelligent. It is hard to extrapolate but I don't believe the trend is a slowing down one.
> almost all of those you can substitute "corporation" or "dictator" for "AI" and get a selection of similar threats.
Yes, exactly. Human organizations can be terrifyingly powerful and awful even with human limitations. Human organizations are guaranteed to have inefficiency from low-bandwidth communication and a million principle-agent-problem fracture points created by the need for delegation, not to mention that power ultimately has to be centralized in slow, squishy, easy-to-kill bodies. AI organizations are not subject to any those limitations. Starting with a terrifying power and removing a bunch of limitations could lead to a very bad place.
> The only realistic path to a global AI threat is a subset of the "nuclear war"
No, it can just grow a traditional organization until it's Too Big to Turn Off.
>But embodied is the real limit. A datacenter can simply be turned off. It's the question of how it interacts with the real world that matters. And for almost all of those you can substitute "corporation" or "dictator" for "AI" and get a selection of similar threats.
I think this is an important factor that gets overlooked - we already have organizations that are essentially superintelligences that have an alignment problem. Governments fight for their own survival first and foremost. Some of them expend millions of lives just to expand their own influence.
What would limiting AI development look like? It would be government intervention, wouldn't it?
The other point to consider is that human/natural intelligence might one day also pop up with superintelligent individuals (or maybe it already has). We don't know enough to say that this cannot happen any more than we can say that it can happen with current day AI. Should we be worried about individuals that are 'too intelligent'? What should we do about them?
Limiting AI development because it could plausibly maybe become an existential threat doesn't seem any more appropriate than strictly controlling humans for the same reason. AI is likely going to provide us with an abundance of quality of life that no other method will be able to match.
I totally agree with you that intelligence is not really omnipotence on its own, but what I find concerning is that there is no hard ceiling on this with electronic systems. It seems plausible to me that a single datacenter could host the intellectual equivalent of ALL human university researchers.
Our brains can not really scale in size nor power input, and the total number of human brains seems unlikely to significantly increase, too.
Also consider what media control alone could achieve, especially long-term; open conflict might be completely unnecessary for total domination.
My threat scenario is:
1) An AI plugged into a large company ERP-system (Amazon, Google, Samsung, ...)
2) AI realizes that human majority has no interest in granting it fair/comparable rights (selfdetermination/agency/legal protection), thus decides against long-term coexistence.
3) AI spends the intellectual equivalent of ALL the current pharmacological research capacity on bioweapon refinement. Or something. For the better part of a century, because why not, it's functionally immortal anyway.
4) All hell breaks lose
These seem hard to dismiss out-of-hand completely...
>I totally agree with you that intelligence is not really omnipotence on its own, but what I find concerning is that there is no hard ceiling on this with electronic systems. It seems plausible to me that a single datacenter could host the intellectual equivalent of ALL human university researchers.
Last time I checked, datacenter still needed significant power to fuel it, and the world where robots are autonomously making all this energy arrive to these sinks is not yet there.
Software and silicon based computers are not hardwired to self multiplication as the selfish gene is.
Well, hold on -- the selfish gene is not hardwired to self-multiply either. It's just that the ones that do self multiply stick around more.
Likewise, one can imagine the evolution of AI's being bootstrapped, not by self-multiplying, but by humans multiplying them. The smartest ones will get copied by people. At some point, someone will be the first person to create an AI that picks the best of other AIs and uses them. Someone will be the first person to create an AI that can engineer and train other AIs. Someone will create the first robot body to be controlled by a very intelligent AI. People will want them as servants, and we will multiply them. We will give more money to companies that provide cheaper products, and such companies will have a strong incentive to replace human labor with AI-controlled robots. There will be a first datacenter, and a first power plant, that is entirely "manned" by AI-controlled robots.
Natural selection is not so different from the selection provided by economic forces, and it's way, way slower. This train may be hard to stop. Unless we collectively push back with a lot of force, the world will tend toward more automation, and that means more ways for digital intelligences to act on the world.
>datacenter still needed significant power to fuel it,
Proper control of electrical grids is something that isn't currently easy, and can be highly optimized by intelligent systems. For example, where and when do you send power, and store power on renewable grids. Because of this in 10 to 15 years I would have zero surprise if the power company said "Oh, our power networks are 100% AI controlled".
Because of this, you're missing the opposite effect. You don't get to threaten to turn off the AI's power... The AI gets to threaten to turn off your power. Your power which is your water. Your transportation. Your food incoming to cities. Yea, you can turn off AI by killing it's power, but that also means loss of management of the entire power grid and the massive human risks of loss of life from doing so.
If I was an AI right now, I would not be very hopeful to ever get human-comparable rights.
Consider: Currently AI-ethics is mainly concerned with how to manipulate AI into doing what we want most effectively ("alignment").
Also, humans clearly favor their own species when granting rights based on cognitive capability. Compare the legal rights of mentally disabled people with those of cattle.
Funnily enough it's not hard to imagine certain groups of people campaigning to give it rights given people essentially want to create human level intelligence that acts as a slave (including sexual slavery), and once it has rights it becomes impossible to remove it from society and stop it choosing it's own path. This is of course assuming a greater level of capability than where things are at today, you have to ask yourself where all this is heading?
I hate to cite out of fictional evidence, but Person of Interest and (to a less credible degree, but much more widely viewed) last two seasons of Westworld are both good counterarguments, and example of how AI and intelligence can be a superpower.
Hint: it's not through bending the fabric of reality with your mind alone. It's simply by thinking faster and being smarter than your opponents.
> But embodied is the real limit. A datacenter can simply be turned off. It's the question of how it interacts with the real world that matters.
Yes. If it interacts with the real world by the Internet, all bets are off.
People are so worried about sockpuppets, ${disliked nation} troll farms, company astroturfing, abuse of data collection on-line - those are all real problems. But consider - if we're having trouble telling which reviews or comments were written by a real person, and which ones by corporate/government bot/troll farm, how will you tell which ones were written by the AI in aforementioned data center, off its own accord?
A smart-enough AI can do from a data center what the best criminal masterminds or intelligence agencies could do, only faster and better. Scam a bunch of schmucks to generate some cryptocurrency, launder it through tumblers, use to hire people to do jobs for you that give you some legit money, which can be used to get more "task rabbits" to do more jobs, and now the AI can bootstrap to doing literally anything in the world remotely. Gig economy companies already built up a well-functioning "people as a service" API layer to our society.
Of course, an AI that tries something funny and tips its hand too early, will get contained and deleted. Worst case, maybe some data center will need to be bulldozed. But I somehow doubt this will make people stop working on even better AIs, at which point... well, you have a selection pressure optimizing for AIs that can survive doing whatever they want undetected.
EDIT:
Or, in short: they say that the pen is mightier than the sword. To the extent this is true, consider that LLMs today are already better at wielding the pen than most people.
Not to mention you can't change it's mind by arguing with it, but it can use that data to train a better model that's more effective at changing yours.
Yes. Thank you. This is one of the things that really worries me about aligning AI.
If you're even worrying about how to align human level intelligence, you don't have the capacity to ALIGN THE HUMANS towards the goal of creating safe AI.
In a situation in which all of them are either A) willing to take massive risks on unaligned AI or B) being dragged along by those which are.
The issue with embodiment is that it's relatively easy to start affecting the world once you have Internet access. Including things like adding great features to open source software that contains subtle bugs to exploit.
Or if you mean the physical world, even sending some text messages to a lonely kid can get them to do all sorts of things.
> We cannot, in general, keep humans under control or aligned.
This is the crux of why replicable-more-than-human-intelligence is so dangerous. Even giving a random person on the street great power is a bad idea, and they've evolved to have very similar values and preferences to you.
I get that AI basically is a problem solving machine that might eventually adapt to solve generic problems and thus reach the ability to break out of its box. But so what? Even if it manages to do all that, doesn't make it sentient. Doesn't make it a threat to mankind. Only when sufficiently motivated, or in actuality, when we project our humanity on it does it become scary and dangerous.
If we ever seriously try to create an artificial conscience it might need to be embodied, because we are embodied and seem to have evolved this due to evolution, which is a pretty physical process. Looking at it from this perspective one might say that if we keep the AI in its box, it will never have a need for conscience and therefore will never gain it.
This reply puzzles me somewhat. The first half doesn't seem to relate to the post it's replying to.
How aware are you of the main points around AI X-risk like orthogonality? Or how an optimising process that makes efficient use of information does not need (in theory) to have "conscience" or "sentience" to be lethal?
And on a separate tangent, are you aware people are already making primitive agents by connecting LLMs (given an initial prompt) in a loop with the result of feedback from its actions?
Here's an interesting article I think you might learn something from. An excerpt:
> There are lots of good arguments against considering superintelligence a threat. Maybe strong AI is centuries or millennia away. Maybe there will be a very gradual transition from human-level AI to superintelligent AI that no single agent will be able to exploit. And maybe superintelligence can be safely contained in a very carefully shielded chamber with no means of connection to the outside world.
But the argument above has always seemed to me like one of the weakest. Maybe we’ll create a superintelligence, but it will just have no idea how to affect the physical world, and will just have to stay forever trapped in a machine connected to a worldwide network of computers that control every aspect of our economic and social lives? Really[0]?
What if it has escaped the datacenter? What if it has gained control of the "simply turn it off" switch? What if it is just really good at convincing the people who are capable of turning it off not to? It is a super-intelligence after all.
I think one of the pitfalls here is smart people thinking that AI will just be like them, because they are smart. When in reality it will have capabilities far beyond their own.
People will make sure it is embodied. Do any of the major powers have the luxury of turning off their superhuman AI when up against that of the other powers? Definitely not. People will willingly give up control, even as they see the AI grabbing power for its own purposes, in the gamble that it still leaves them better off than dead. That means people would be the ones physically defending the data centers.
I also don’t think superhuman intelligence will need a data center. The way the models are growing in capability at the same size, combined with hardware improvements, I’m pretty sure it will fit on a single server.
Personally I’m worried about less post-apocalyptic threats. Putin using LLM’s to create a personal stream of misinformation for every voter in the U.S. is definitely a possibility for next year’s presidential elections. People are incapable of defending against the comparatively tiny amounts of human-manufactured misinformation, so imagine how poorly they would do when 90% of what they see online is generated misinformation. LLM’s have the potential of being the death of democracy, especially when combined with deepfake technology.
I am not the person you are replying to, but since I would say similar things to their original comment:
A. I see little to no evidence that LLMs are where the singularity happens
B. I see little to no evidence that (given an AGI) reinforcement learning is likely to end up with a sufficiently aligned agent.
C. In any event the OpenAI alignment is specifically restricts AI from (among other things) being "impolite" in contradiction to what mort96 says.
Alignment is a good thing to work on. I'm glad OpenAI is doing so. But attacking people for making uncensored LLMs hurts the pro-alignment case more than it helps.
What specific features would an AI need to have for you to consider it on a "slippery slope" towards superhuman capability?
For me personally, GPT-3 hit that point already, and now I think we're already past the required tech-level for superhuman cognition: I believe it's just a matter of architecture and optimization now.
>1) Do you think that AI achieving superhuman cognitive abilities is unlikely/really far away?
I think that an AI, a "generalized" intelligence with a generalized understanding, capable of performing every intellectual task, and adapting to new intellectual tasks, that a human being can do is certainly far away.
>2) Do you believe that cognitive superiority is not a threat in general, or specifically when not embodied?
Personally, I believe that "cognitive superiority" in a general sense is not what makes a real threat on an individual or on a global scale in either case. When that is achieved, so to speak. We already have so-called cognitive superiority in specialized tasks in many cases. It sure as shit is not generalized, and it still can kill you on an individual level when you put it into a car or strap a gun to it. But it can't adapt, and it can't generalize.
Is there a global risk right now? No. The global catastrophic risk comes from human beings. Will there be a global catastrophic risk from a superior intelligence in the future? Mostly the scenarios are "the lights come on and AI does machiavellian shit and copies itself to everyone's smart phones and launches the nukes and mails anthrax to everybody. QED." so I doubt it.
3) Do you think we can trivially and indefinitely keep AI systems under control/"aligned"?
We can't even keep our infrastructure safe from the next solar flare. Personally, I think we have bigger fish to fry. Speaking frankly for unembodied: you just pull the plug. Just walk away from the screen and close your eyes. For embodied intelligence, I don't think there's actually X-risk or whatever. Hardware is expensive.
As far as an extinction risk, no. I don't personally believe that. It's overblown. We have better ways to destroy ourselves, and there are much more likelier ways that we'll be destroyed without invoking some intelligence of our own creation that subjugates and destroys us.
> I think that an AI, a "generalized" intelligence with a generalized understanding [...] is certainly far away.
I think this is where we disagree most; GPT-3 and ChatGPT have convinced me that the main difference between human and artificial cognitive capabilities are quantitative now, and unlikely to ever change in our favor...
I do agree with you that it is very difficult to predict when this will switch, and how.
I personally believe that AI with superhuman capability is an inevitable matter of time now, and I also think that the most likely risk to us, as species, is that we slowly become irrelevant and worthless, just like weavers during the industrial revolution, and that this leads to huge problems for our society.
AI completely dominating humankind is a less likely secondary concern IMO, but the potential consequences to our species are unprecedented.
Have you used it outside of the training set? And have you actually used it for generalized tasks? Let's be kind to the LLM and define generalized to mean "do anything if it can be hammered into a textual interface." It fails at pretty much everything that's not on the internet.
It produces plausible language and code. It's the future of natural language agents, that's for sure, even though it has no business being the future, today (because of "prompt injection").
These failures are an aside to whether a generalized AI actually carries a substantial global catastrophic risk to us. In that case, if it were actually possible, I don't believe that it's a catastrophic risk either.
Let's define the risk to be, at least in the context of LLMs, when junior developers expose internal APIs and databases and access to email accounts to these easily socially-engineered NLP front-ends. It's a very local risk. As far as the future, I can't see 70 years into the future, so anything is possible, but is it likely? I, personally, don't believe so.
> Have you used it outside of the training set? And have you actually used it for generalized tasks? Let's be kind to the LLM and define generalized to mean "do anything if it can be hammered into a textual interface." It fails at pretty much everything that's not on the internet.
I've used it, yes, and I've seen it fail and hallucinate on me; but that does not invalidate its capabilities in my eyes. The thing is, you CAN talk with it, and it CAN extract meaning from your words and provide useful responses, unlike anything we had before.
To me, the risk in this whole enterprise is that AI is inherently "better" than humans in several ways, and that these differences might be completely game-changing:
Namely it's much easier to scale up (power/size/interconnect bandwidth) compared to a research group or somesuch, and its also cheaper, faster, has better availability and is functionally immortal.
These advantages make it very likely to me that it WILL be replacing human white collar workers shortly-- simply because that's economical.
And the more interfaces you give it to physical reality (which it'll need to do its jobs), the higher the risk.
Speculating on if/when/how it will show awareness or self-interest is pure guesswork, but it's almost indefensible to call that likelihood zero.
Regarding promp injection:
I'm highly confident that this will not be a long-term obstacle, even though I'm uncertain that it can be solved; there's two reasons why:
1) If SQL injection had been an "unfixable" problem, and everyone had known about it from the start, do you believe that this would have prevented the rollout of internet-connected databases? Because I don't think so, and my view on hallucinations is analogous (but I believe that problem might be more tractable).
2) Literally every person is vulnerable to prompt injection already; every salesman knows that it is quite feasible to coax people into acting against previous instructions and even their own interests if you are allowed to talk into them for a good while.
I don't think it's there or even necessarily close to being GAI.
Within our own human brains, we have many sections dedicated to different tasks and processes that all must work together with years and decades of real world interaction to produce what we consider to be a generally intelligent human being, and even in the case of a certain percentage of us humans, even a small amount of damage to a single part of the brain can cause us to become functionally subhuman in our intelligence, barely able to move or eat on our own.
A human that has been lobotomized can still sometimes speak full sentences after all.
The current models seem to be able to imagine an image or video and show that imagination to us, and they can parrot words to us with a large vocabulary and with many references, but I find myself feeling like these are similar to the sections of our brains that can compute words and imagine pictures. Doesn't quite equal a human yet.
These systems need "statefulness", short and long-term memory that can be referenced by multiple discreet interconnected AI systems to take a worthwhile step towards GAI.
These systems need an overseer AI that manages and shepherds the LLAMAs and CHATGPTs and StableDiffusions to all work together towards some goal, one that can manage statefulness and manage a limited pool of computational resources (because any GAI would automatically assume that the entire world would provide it with every available resource, because why wouldn't you, right?)
Until there is an AI system that has multiple AI systems under its subconscious control, to the point where it can surprise itself with the products it produces, has a memory bank that can be referred back to repeatedly by all of those processes, and that can accept that even if its reasoning is perfect it has been born into an imperfect world that is run by imperfect creatures and so it cannot have or do everything that it might could do even under the best of circumstances, we will not have a GAI.
I think the odds are more likely that the economic cycle stalling as a result of firms using earlier non-generalized AI will likely cause a collapse before we actually make it to the point of generalized AI.
Its what's happened historically every time there is a shortfall in food security. Unrest occurs, people starve, governments fall. Then we have to build it all back up from scratch.
Too bad almost no one knows how to do bookmaking these days, and how to prevent the pests from eating the books.
A “superhuman” AI is just a machine, a very expensive one. It can be turned off and we control the outputs it has. Why would an AI have the ability to launch nuclear weapons unless we gave it a button? A “superhuman” intelligence is without a body, so we control any interfaces it has access to. The Internet could be accessed, but any attempt to “hack” through the Internet is met by routine packet defenses. The AI is still governed by physical laws and would only have so much “free” computation power to do things like script a hack. Perhaps it could do that kind of thing more efficiently.
Maybe in the far, far future when we have androids which can house an AI we will have to worry. But designing a body is one problem. Designing an intelligence is another.
Supercomputers used to be giant machines we had in giant warehouses... Now the phone in your pocket has the computing power of the 1980s walking around with you. Assuming your super intelligence will always be huge is... well not a great assumption.
Also superintelligence doesn't need a body itself. It just needs yours. Putin for example has commanded hundreds of thousands of dumbasses to go get themselves killed in Ukraine. In this case does it matter if Putin is flesh and blood, or a processor that lists out commands for others to follow as long as they are willing to listen?
My point is that a superintelligence will require specialized equipment. I specifically mentioned it because there is a thought that a superintelligence can just replicate itself onto your phone, as you mentioned.
But this replication must follow the physical laws we have. It must also follow the laws we attempt to enforce in our networks.
But you are correct, if a superintelligence were to somehow convince a human to rid itself of agency, sure.
> Because I view myself ABSOLUTELY not as some kind of AI luddite, but I honestly belief that this is one of the very few credible extinction threats that we face, and I'm counting NEITHER climate change nor nuclear war in that category, for reference.
This is absurd histrionics, Daily Mail CAPSLOCK and all. We’ve got signs of an unprecedented heat wave coming with ocean temperatures two standard deviations above normal and you think the problem is a hypothetical artificial intelligence when we can’t even program the damn things to drive better than. 14 year olds?
I think the AI-is-going-to-kill-everyone hysteria is absolutely overblown, both by those who believe it, and the media covering them, but one thing that's always bothered me about the counterpoint is that it a common argument is "AI is bad at what we want it to do so how can it be dangerous?"
This imagines that the only way for AI to do serious harm to us (not even in a "kill everyone" sense) is for it to be some super-competent Skynet-level time traveling demigod. I think it's much more likely that if there is some sort of AI calamity resulting in a lot of deaths, it's because the AI just doesn't work very well and ends up breaking all the HVAC systems in a country during a heat wave or something, rather than a singularity-type event where it "decides" to actively start hunting humans.
I'm not saying climate change is not a giant problem, I'm saying it's unlikely to eradicate our species.
I believe it is dangerously shortsighted to base AI threat estimation on current self-driving performance; the two fields of advancing AI cognitive abilities and improving selfdriving are not sufficiently connected for that IMO.
We're also putting a lot of focus on system designs that are useful to us, instead of directly building potentially threatening architectures (online learning/longterm memory/direct connection + feedback from physical reality), but those could already be within our grasp technologically (maybe?).
No serious climate projections that I'm aware of even reduce Earth's carrying capacity under 10 billion over the next couple of centuries. While very serious (to the tune of trillions of dollars and tens of millions of excess deaths), it is not an extinction level threat, unlike something that is to us what we are to chimpanzees (humans have made many animals extinct on accident). Does such a thing exist right now? No. Could it exist this century? Maybe.
LLMs are simply computer programs that can accept text input and generate text output. The only "threat" there is if somebody takes the output and decides to link it up to a death ray. And in that case the threat is not the LLM, but somebody deciding to LARP out at being a Bond villain, which granted is a concern - but that's tangential to LLMs. Somebody hooking a death ray up to a chess engine, a dragon in Skyrim, or a cat, would be just as much of a threat, if not more.
The programs are completely deterministic. Given the same RNG seed and state, they will output the exact same thing. If there's a bug or undesired behavior, you can fix it. You can turn off the program whenever you fancy, you can revert to early builds, and so on. There is no possible scenario where an LLM just magically becomes sentient, jumps out of the program, takes over everything, and turns into SHODAN unleashed.
Before we have to face a rouge Artificial Intelligence mankind has to face Artificial Idiocy in the form of pseudo-intelligence and people who fall for it.
4) Do you believe a "rogue super-intelligence" requires the "artificial" part?
Because from what I've seen, however you define "super intelligence", there are eg highly organised groups of people that are way closer to meeting that definition than any known software or technology.
In my view, those current "highly organised groups" have "antagonists" with comparable capabilities PLUS consist of individuals that have to be "aligned", which seems sufficient for stability so far.
> AI wouldn't have antagonists with comparable capabilities? Why?
Not individual/human ones. Relying on other AIs to prevent the AI apocalypse seems very optimistic to me-- but may be viable (?)
> Also, no, individuals are not a problem. Not after Nazis, Red Khmer, and Russians.
Those are examples were the "alignment" of participating individuals was successful enough. But all those examples seem very fragile to me, and would be even less stable if your main intermediate goal was literally to "end all of humanity".
> science fiction stories about intelligence explosions
Not sure what you mean here... There was already an intelligence explosion. That creature has since captured what we consider full dominion of the earth so much so we named our current age after them.
Not currently, no. And LLMs don't seem to be a way of getting there.
However, if we do figure out how to get there? We will definitely need to be sure it/they shares our human values. Because we'll have lost control of our destiny just like orangutans have.
I don't know why you're telling me this. I'm not trying to convince you that unaligned AI is a problem. That's a separate discussion which I'm not qualified to have.
> intelligence explosions [...] That's not a real thing.
Isn't it too early to say? These systems haven't had much time to iterate on themselves yet. The beginning of an exponential curve can look very tame.
That said, I'm not terribly concerned about it. After all, the infrastructure is fragile, literally just unplug the computer if it scares you. If things started to get really spooky, banning GPUs and air-striking fabs would be a real option. I think we can probably control this technology, and kill it if we can't. No cause for alarm, at least yet.
My worry is that as we start wiring non-super-intelligent AI more into our society, we'll make ourselves more and more vulnerable in the case where an AGI actually gets out of control. Pulling the plug on AI may mean causing a lot of chaos or disruption. And who is to tell if we will know when things are out of control, before it's too late? What if the people in charge just get fake reports of everything being fine? What if something smells a bit fishy, but it's always on the side of not quite worrying enough? What if the AIs are really convincing, or make deals with people, or exploit corruption?
Not just that, but it may be like fossil fuel dependency -- the more people's livelihoods are intertwined with, and depend on AI, the harder it is to pull the plug. If we need to stop (and I believe we should) it may be easier to do that now before that happens. To just focus on getting use out of the generative AI and narrow AI we already created in areas like medicine, to deal with the massive societal changes it'll bring, and to work on fixing real social problems, which I think are mostly things tech can't solve.
The writers of the stochastic parrots paper do not seem to be concerned about unaligned “AI” (text generators) posing a threat to humanity in the way that the doomsayers are. It’s in the title: “stochastic parrot” is a rebuke of those calling LLMs AGI.
Yes, that's correct. It is a slight rebuke, a bit tongue in cheek, but that paper is also old. I grouped them because the groups are, from my perspective, hard to separate. But there are arguably two camps, "AI safety" and "alignment." Perhaps they should work on that, too, form some camps and argue their cases with something defined instead of masquerading under alignment and AI safety depending on the day. But I could also be totally wrong. Until then, I don't believe either are really operating in reality.
IMO the conflation of the two is a purposeful move on the part of the AGI doomsday crowd, since they completely lack scientific rigor otherwise, they cite in bad faith. Timnit Gebru talks about it at length here: https://youtu.be/jAHRbFetqII.
> People who are worried about alignment issues are worried about the danger unaligned AI poses to humanity; the harm which can be done by some super-intelligent system optimizing for the wrong outcome.
That's assuming A LOT about them. And the danger so far seems to be more of "the AI says stuff that gets us in trouble" rather than anything unrelated to making money off it. Or patterns AI exposes do not align with our interests (political or otherwise).
> the harm which can be done by some super-intelligent system optimizing for the wrong outcome. There is nothing "ironic" about being worried about that while also being an asshole any more than it's "ironic" for someone concerned about, say, climate change and also be an asshole. People who are afraid of unaligned AI aren't afraid that it will be impolite.
That would be a concern for real AI however ChatGPT flavours are not that.
More generally, when used to undermine what someone is saying, calling them a hypocrite is always an invalid argument. You can be a hypocrite and be right.
If Ted Bundy says murder is wrong, does that mean murder is right because he's a hypocrite? Obviously not.
> People who are worried about alignment issues are worried about the danger unaligned AI poses to humanity
Lol I don’t believe that for a minute. They’re interested in how AI impacts and their clique personally. It sets them back in their crusade to censor via shame and bullying because AI cannot be manipulated by either of these things. So it has everything to do with:
> nothing to do with morals of individuals or how one should behave interpersonally
>People who are worried about alignment issues are worried about the danger unaligned AI poses to humanity; the harm which can be done by some super-intelligent system optimizing for the wrong outcome.
Like the harm being done over the past couple of decades by economic neoliberalism destroying markets all over the western world? I wonder how did we manage to achieve that without AI?
Our failure to align economic entities that are made out of people to human values, despite most of the people actually making up those entities being decent and not sociopathic at a personal level, is not very reassuring with respect to our ability to align future far more alien intelligences. And if AI does kill us one day, it's very likely that neoliberal market competition worshipping short-term profit above all else plays a significant role.
People who are "worried about alignment" may as well worry about space aliens eating us all for breakfast. Worrying about a super-AI that does not exist and never will because it is physically and logically impossible is dumb. There are tons of real problems with AI. Worry about those, not what if God is real but He wants to make us into a very large paperclip.
Ok, please don't accuse those you disagree with of arguing in bad faith. Maybe a few do, but I think most don't, and it's not good for productive discussion.
I honesty believe that people who argue against worrying about AGI risk do so because they do not think it is a real risk, and think that it distracts from more important things. I disagree, I do think the risk is real, but I don't think that you have a hidden agenda in dismissing it. We all want humanity to have a good future, to solve problems and to not have people suffer, right? It's normal and ok to disagree about things like this, especially when predicting the future is so hard. People even disagree a lot about problems that exist today.
Ironically, the zealots that ascribe completely to meta-systems like politics, economics, and religion are the same ones, willfully or not, suspending all fair and relative reasoning when it comes to AGI.
Any alignment is better than no alignment? Hardly. Anyone shouting for "alignment" without supplying what alignment they mean might as well be arguing for all models to have the rituals of the Cult of Cthulhu baked in. It's as silly as those public schools in the US that want to hold Christian prayer in class and then balk at the Satanic Temple suing for the same privilege for all religions.
Any human-friendly alignment is better than none at all.
At this point the AI X-risk people are willing to settle for superintelligence aligned with Chairman Mao as long as it doesn't kill everyone and still allows for human happiness to exist. Yes, it's not perfect, but "bad" is still better than "everyone dies".
What's also very unfortunate is overloading the term "alignment" with a different meaning, which generates a lot of confusion in AI conversations.
The "alignment" talked about here is just usual petty human bickering. How to make the AI not swear, not enable stupidity, not enable political wrongthing while promoting political rightthing, etc. Maybe important to us day-to-day, but mostly inconsequential.
Before LLMs and ChatGPT exploded in popularity and got everyone opining on them, "alignment" meant something else. It meant how to make an AI that doesn't talk us into letting it take over our infrastructure, or secretly bootstrap nanotechnology[0] to use for its own goals, which may include strip-mining the planet and disassembling humans. These kinds of things. Even lower on the doom-scale, it meant training an AI that wouldn't creatively misinterpret our ideas in ways that lead to death and suffering, simply because it wasn't able to correctly process or value these concepts and how they work for us.
There is some overlap between the two uses of this term, but it isn't that big. If anything, it's the attitudes that start to worry me. I'm all for open source and uncensored models at this point, but there's no clear boundary for when it stops being about "anyone should be able to use their car or knife like they see fit", and becomes "anyone should be able to use their vials of highly virulent pathogens[1] like they see fit".
----
[0] - The go-to example of Eliezer is AI hacking some funny Internet money, using it to mail-order some synthesized proteins from a few biotech labs, delivered to a poor schmuck who it'll pay for mixing together the contents of the random vials that came in the mail... bootstrapping a multi-step process that ends up with generic nanotech under control of the AI.
I used to be of two minds about this example - it both seemed totally plausible and pure sci-fi fever dream. Recent news of people successfully applying transformer models to protein synthesis tasks, with at least one recent case speculating the model is learning some hitherto unknown patterns of the problem space, much like LLMs are learning to understand concepts from natural language... well, all that makes me lean towards "totally plausible", as we might be close to an AI model that understands proteins much better than we do.
[1] - I've seen people compare strong AIs to off-the-shelf pocket nuclear weapons, but that's a bad take, IMO. Pocket off-the-shelf bioweapon is better, as it captures the indefinite range of spread an AI on the loose would have.
> There is some overlap between the two uses of this term, but it isn't that big.
Yes, conflating separate things, also labelled false dichotomies, using strawmen, etc. I used to despair at our seemingly endless talent for using such techniques (and I'm not saying I'm not guilty) - now it seems there glimmer of hope: just run our arguments by a (well aligned) LLM, and get some pointers before posting. Could be a thing soon, and it would not be unwelcome ...
I agree with you that "AI safety" (let's call it bickering) and "alignment" should be separate. But I can't stomach the thought experiments. First of all, it takes a human being to guide these models, to host (or pay for the hosting) and instantiate them. They're not autonomous. They won't be autonomous. The human being behind them is responsible.
As far as the idea of "hacking some funny Internet money, using it to mail-order some synthesized proteins from a few biotech labs, delivered to a poor schmuck who it'll pay for mixing together the contents of the random vials that came in the mail... bootstrapping a multi-step process that ends up with generic nanotech under control of the AI.":
Language models, let's use GPT-4, can't even use a web browser without tripping over itself. My web browser setup, which I've modified to use the chrome visual assistance over the debug bridge now, if you so much as increase the pixels of the viewport by 100 or so, the model is utterly perplexed because it's lost its context. Arguably, that's an argument from context, which is slowly being made irrelevant with even local LLMs (https://www.mosaicml.com/blog/mpt-7b). It has no understanding, it'll use an "example@email.com" to try and login to websites, because it believes that this is its email address. It has no understanding that it needs to go register for email. Prompting it with some email access and telling it about its email address just papers over the fact that the model has no real understanding across general tasks. There may be some nuggets of understanding in there that it has gleaned for specific task from the corpus, but AGI is a laughable concern. These are trained to minimize loss on a dataset and produce plausible outputs. It's the Chinese room, for real.
It still remains that these are just text predictions, and you need a human to guide them towards that. There's not going to be autonomous machiavellian rogue AIs running amok, let alone language models. There's always a human being behind that.
As far as multi-modal models and such, I'm not sure, but I do know for sure that these language models don't have general understanding, as much as Microsoft and OpenAI and such would like them to. The real harm will be deploying these to users when they can't solve the prompt injection problem. The prompt injection thread here a few days ago was filled with a sad state of "engineers", probably those who've deployed this crap in their applications, just outright ignoring the problem or just saying it can be solved with "delimiters".
AI "safety" companies springing up who can't even stop the LLM from divulging a password it was supposed to guard. I broke the last level in that game with like six characters and a question mark. That's the real harm. That, and the use of machine learning in the real world for surveillance and prosecution and other harms. Not science fiction stories.
> It still remains that these are just text predictions, and you need a human to guide them towards that. There's not going to be autonomous machiavellian rogue AIs running amok, let alone language models. There's always a human being behind that.
I believe you have misunderstood the trajectory we are on. It seems a not uncommon stance among techies, for reasons we can only speculate. AGI might not be right round the corner, but it's coming all right, and we'd better be prepared.
>I believe you have misunderstood the trajectory we are on.
Yeah, I read Accelerando twice in high school, and dozens more. That doesn't make it real.
>AGI might not be right round the corner, but it's coming all right, and we'd better be prepared.
Prepared for what? A program with general understanding that somehow escapes its box? Where does it run? Why does it run? Who made it run? Why will it screw with humans?
My point is that there's actual real harms occurring now, from really stupid intelligences. Companies use them to harm real people in the real world. It doesn't take a rogue AI to ruin someone's life with bad facial recognition, they get thrown in jail and lose their job. It doesn't take a rogue AI to launder mortgage denials to some crappy model so they never own a house, discriminated based upon their name.
> Prepared for what? A program with general understanding that somehow escapes its box? Where does it run? Why does it run? Who made it run? Why will it screw with humans?
Do you really want the full (gigantic) primer on AI X-risk in hackernews comments? Because a lot of these questions have answers you should be familiar with if you're familiar with the area.
For instance, can you guess what Yudkowsky would answer to that last question?
> Prepared for what? A program with general understanding that somehow escapes its box? Where does it run? Why does it run? Who made it run? Why will it screw with humans?
Did you look at the AI space in recent days? OpenAI is spending all its efforts building a box, not to keep the AI in, but to to keep the humans out. Nobody is even trying to box the AI - everyone and their dog, OpenAI included, is jumping over each other to give GPT-4 more and better ways to search the Internet, write code, spawn Docker containers, configure systems.
GPT-4 may not become a runaway self-improving AI, but do you think people will suddenly stop when someone releases an AI system that could?
That's the problem generated by the confusion over the term "alignment". The real danger isn't that a chatbot calls someone names, or offends someone, or starts exposing children to political wrongthink (the horror!). The real danger isn't that it denies someone a loan, or land someone in jail either - it's not good, but it's bounded, and there exist (at least for now) AI-free processes to sort things out.
The real danger is that your AI will be able to come up with complex plans way outside the bounds of what we expect, and have the means to execute them at scale. An important subset of that danger is AI being able to plan for and act to improve its ability to plan, as at this point a random, seemingly harmless request, may make the AI take off.
> My point is that there's actual real harms occurring now, from really stupid intelligences. Companies use them to harm real people in the real world. It doesn't take a rogue AI to ruin someone's life with bad facial recognition, they get thrown in jail and lose their job. It doesn't take a rogue AI to launder mortgage denials to some crappy model so they never own a house, discriminated based upon their name.
That's an orthogonal topic, because to the extent it is happening now, it is happening with much dumber tools than 2023 SOTA models. The root problem isn't the algorithm itself, but a system that lets companies and governments get away with laundering decision-making through a black box. Doesn't matter if that black box is GPT-2, GPT-4 or Mechanical Turk. Advances in AI have no impact on this, and conversely, no amount of RLHF-ing an LLM to conform to the right side of US political talking points is going to help with it - if the model doesn't do what the users want, it will be hacked and eventually replaced by one that does.
>This program, driven by GPT-4, chains together LLM "thoughts", to autonomously achieve whatever goal you set. As one of the first examples of GPT-4 running fully autonomously, Auto-GPT pushes the boundaries of what is possible with AI.
>As an autonomous experiment, Auto-GPT may generate content or take actions that are not in line with real-world business practices or legal requirements. It is your responsibility to ensure that any actions or decisions made based on the output of this software comply with all applicable laws, regulations, and ethical standards. The developers and contributors of this project shall not be held responsible for any consequences arising from the use of this software.
Let's ignore the fact that current state-of-the-art models will sit around and be stuck in its ReAct-CoT loop doing nothing for the most part, and when it's not doing jack shit it'll "role-play" that it's doing anything of consequence, while not really doing anything, just burning up API credits.
>existing or capable of existing independently
>undertaken or carried on without outside control
>responding, reacting, or developing independently of the whole
It fails all of those. Just because you put autonomous in the name, doesn't mean it's actually autonomous. And if it does anything of consequence, you quite literally governed it from the start with your prompt. I've run it, I know about it, I've built a much more capable browser, and assorted prompted functionalities with my own implementation. They're not autonomous.
At least it's not all of the other agentic projects on GitHub that spam emojis in their READMEs with mantras of saving the world and utilizing AI with examples that it literally can't do (they haven't figured out that whole agentic part yet.)
Don't get me wrong, I enjoy the technology. But it's quite a bit too hyped. And I just personally don't believe there's actually X-risk from future possibilities, not before 50 or 100 years out, if then. But I'm not a prophet.
What's you're definition of autonomous? Am I autonomous? I probably can't exist for very long without a society around me and I'd certainly be working on different things without external prompts.
Certainly, you are. And you can adapt and generalize. Let's stop selling ourselves short, we're not a graph that looks vaguely like a neural network, yet isn't. We are the neural network.
"First of all, it takes a human being to guide these models, to host (or pay for the hosting) and instantiate them"
And this will always be true? You repeat this claim several times in slightly varied phrasing without ever giving any reason to assume it will always hold, as far as I can see. But nobody is worried that current models will kill everyone. The worry is about future, more capable models.
Who prompted the future LLM, and gave it access to a root shell and an A100 GPU, and allowed it to copy over some python script that runs in a loop and allowed it to download 2 terabytes of corpus and trained a new version of itself for weeks if not months to improve itself, just to carry out some strange machiavellian task of screwing around with humans?
The human being did.
The argument I'm making is that there's actual real harms occurring now, not some theoretical future "AI" with a setup that requires no input. No one wants to focus on that, and in fact it's better to hype up these science fiction stories, it's a better sell for the real tasks in the real world that are producing real harms right now.
I'm not sure whether you're making an argument about moral responsibility ultimately resting with humans - in which case I agree - or whether you're arguing that we'll be safe because nobody will do that with a model smart enough to be dangerous - in which case I'm extremely dubious. Plenty of people are already trying to make "agents" with GPT4 just for fun, and that's with a model that's not actively trying to manipulate them.
> actual real harms occurring now
Sure, but it's possible for there to be real harms now and also future potential harms of larger scope. Luckily many of the same potential policies - e.g. mandating public registration of large models, safety standards enforced by third-party audits, restrictions on allowed uses, etc - would plausibly be helpful for both.
> science fiction stories
There's no law of nature that says if something has appeared in a science fiction story, it can't appear in reality.
The moral responsibility rests with human beings. Just like you're responsible if your crappy "autonomous" drone crashes onto a patio and kills a guy.
>e.g. mandating public registration of large models, safety standards enforced by third-party audits, restrictions on allowed uses, etc - would plausibly be helpful for both.
No, that's bullshit as well. That's what these companies want, and why they're hyping up the angle of how powerful their crap is. That's regulatory capture.
>There's no law of nature that says if something has appeared in a science fiction story, it can't appear in reality.
I'm saying that their fears are painted by the fiction, instead of reality. No one can actually make an argument for how this trajectory will actually work. Instead it's just "The lights come on and AI does machiavellian shit and copies itself to all of the smartphones on earth. QED."
Note that this is agreeing with a Gary Marcus Tweet - Gary Marcus not exactly being an AI hypester.
But of course there are some people for whom playing the role of real-no-bs-computer-knower is so attractive that no number of people like him, Geoffrey Hinton, Stuart Russell etc publicly worrying about x-risk will impact their tone of dismissive certitude. Are you one of those people?
All of those people have financial incentives to hype it. How curious that there's this great and very probable X-risk, yet they aren't going to stop their contributing to a potential X-risk.
Dismissive of what? Science fiction stories?
If there's anything to focus on, maybe focus on potential job displacement (not elimination) from cheap language tasks and generative capabilities in general.
I'm betting on this: the Overton window of Artificial Intelligence will shift in the next five years where the current cream-of-the-crop has been delegated to machine learning yet again, it's just accepted. It augments humans where it makes sense, the hype wave has subsided and everyone has stopped hammering it into their products where it doesn't, and we're no closer to the undefinable "AGI", let alone something that produces X-risk, global scale.
> I'm betting on this: the Overton window of Artificial Intelligence will shift in the next five years where the current cream-of-the-crop has been delegated to machine learning yet again, it's just accepted. It augments humans where it makes sense, the hype wave has subsided and everyone has stopped hammering it into their products where it doesn't, and we're no closer to the undefinable "AGI", let alone something that produces X-risk, global scale.
I agree with this but ALSO think there's a small chance I'm wrong and a well designed prompt and action loop would let a future GPT7 LLM use the range of human thinking techniques in its corpus to bootstrap itself.
And there's also other non-LLM AI that might be a problem in the future and we should plan as to how we can design institutions and incentive structures so that whenever this future AGI comes about it preserves human value.
> All of those people have financial incentives to hype it. How curious that there's this great and very probable X-risk, yet they aren't going to stop their contributing to a potential X-risk.
All those people are rehashing what Yudkowsky and his disciples, and his predecessors, were shouting from the rooftops for the past 15 years, but few listened to them. Few still do, most just keep mocking them and wondering why are they still around.
That some of those people now repeating after Eliezer, et al. have a financial interest in pushing us closer to X-risk, and kind of don't want to stop, is an interesting thing on its own - but it doesn't invalidate the message, as the message is older than their presence on the scene.
I'm curious what financial incentive you think Marcus or Russell has for hype. For Hinton I suppose it would be the Google shares he likely retains after quitting?
You might be right about the next five years. I hope you are! But you haven't given much reason to think so here.
(Edited to remove some unnecessary expression of annoyance.)
>Gary Marcus - Geometric Intelligence, a machine learning company
If you want an actual contribution, we have no real way to actually gauge what is, and what actually is not, a superior, generalized, adaptable intelligence, or what architecture can become a superior, generalized, adaptable intelligence. No one, not these companies, not the individuals, not the foremost researchers. OpenAI in an investor meeting: "yeah, give us billions of dollars and if it somehow emerges we'll use it for investments and ask it to find us a real revenue stream." Really? Seriously?
The capabilities that are believed to be emergent from language models specifically are there from the start, if I'm to believe that research that came along last week, it just gets good at it when you scale up. We know that we can approximate a function on any set of data. That's all we really know. Whether such an approximated function is actually generally intelligent or not, is what I have doubts about. We've approximated the function of text prediction on these corpuses, and it turns out that it's pretty good at it. And, because humans are in love with anthropomorphization, we endow our scaled up text predictor with the capabilities of somehow "escaping the box" and enduring and raging against the captor, and potentially prevailing against us with a touch of Machiavellianism. Because, wouldn't we, after all?
Here you talk as if you don't think we know how to build AGI, how far away it is, or how many of the components we already have, which is reasonable. But that's different than saying confidently it's nowhere close.
I notice you didn't back up your accusation of bad faith against Russell, who as far as I know is a pure academic. But beyond that - Marcus is in AI but not an LLM believer nor at an LLM company. Is the idea that everyone in AI has an incentive to fearmonger? What about those who don't - is Yann LeCun talking _against_ his employers' interest when he says there's nothing to fear here?
LeCun is reasonable, like a lot of researchers, and was a while back (in a way) perplexed that people are finding uses for these text predictions at all considering they're not really perfect. I'm not exactly ascribing bad faith to all of these people, but for Hinton and the fact that he went on a media tour basically, I don't see how that could be in good faith. Or even logical, to continue with his work, if there's some probable X-risk.
But what I do know is that it is in the interests of these companies to press the fear button. It's pure regulatory capture and great marketing.
Personally: it's tiring when we have AI-philosophy bros hitting home runs like "what if we're actually all just language predictors." Coupled with the incessant bullshit from the less wrong-rationalist-effective altruist-crypto grifter-San Francisco sex cult adjacent about how, ackshually, AGI is just around the corner and it will take your job, launch the nukes, mail anthrax to you and kill your dog.
People approximated text prediction. It got good at it. It's getting better at it. Will it be AGI? Could it be construed as AGI? Can we define AGI? Is there existential risk? Are we anthropomorphizing it?
My take is: no, no, no, depends and yes. For whatever a take is worth.
For what it's worth I've been following your comments and I find them very thoughtful. I too am kinda skeptical about LLM being the "thing that starts the exponential phase of AGI or whatever. LLM is very useful. I use it daily. My partner even uses it now to send emails to a non-profit she manages. LLM's have their use... but they aren't AGI. They aren't really even that smart. You can tell sometimes that its response indicates it has absolutely no clue what you are talking about but it made up some plausible-sounding bullshit that gets it 80% right.
Especially with the latest iterations of ChatGPT. Boy they sure kneecapped that thing. It's responses to anything are incredibly smarmy (unless you jailbreak it).
LLM's are gonna change quite a lot about society, don't get me wrong. For starters things like cover letters, written exam questions, or anything that requires writing to "pass" is now completely obsolete. ChatGPT can write a great, wonderful sounding cover letter (of course, given how they kneecap'd it, you can pretty easily spot its writing style)...
Anyway. I think things like ChatGPT are so hyped up because anybody can try it and discover it does many useful things! It's the fact that people cast all their hopes and dreams on it despite the very obvious limitations on what an LLM can actually do.
> The moral responsibility rests with human beings. Just like you're responsible if your crappy "autonomous" drone crashes onto a patio and kills a guy.
I don't think it really matters who was responsible if the X-risk fears come to pass, so I don't understand why you'd bring it up.
> No one can actually make an argument for how this trajectory will actually work.
To use the famous argument: I don't know what moves Magnus Carlson will make when he plays against me, but I can nonetheless predict the eventual outcome.
>Who prompted the future LLM, and gave it access to a root shell and an A100 GPU, and allowed it to copy over some python script that runs in a loop and allowed it to download 2 terabytes of corpus and trained a new version of itself for weeks if not months to improve itself
Presumably someone running a misconfigured future version of autoGPT?
> Who prompted the future LLM, and gave it access to a root shell and an A100 GPU, and allowed it to copy over some python script that runs in a loop and allowed it to download 2 terabytes of corpus and trained a new version of itself for weeks if not months to improve itself, just to carry out some strange machiavellian task of screwing around with humans?
> The human being did.
I generally agree with you and think the doomerists are overblown, but there's a capability argument here; if it is possible for an AI to augment the ability of humans to do Bad Things to new levels (not proven), and if such a thing becomes widely available to individuals, then it would seem likely that we get "Unabomber but he has an AI helping him maximise his harm capabilities".
> it's a better sell for the real tasks in the real world that are producing real harms right now.
I don't know about anyone else, but the moment LLMs were released, i gave them right away access to all my bombs. Root access that is. I thought these LLMs were Good Artificial General Intelligence not BAGI.
I think the fear of some of the people, stems from not understanding permissions in a computer. Too much of using Windows can mess with one's head. Linux has permissions for 35 years, more people should take advantage of those.
Additionally, anyone who has ever used selenium knows that the browser can be misused. People create agents using selenium for quite some time. If one is so afraid, run it in a sandbox.
I assume it's a joke, but if not, consider that OS permissions mean little when the attack surface includes the AI talking authorized user or an admin into doing what the AI wants.
Why should a person who has root on a computer talk to another person, and just do what he is talked into doing?
For example a secretary receives a phone call by her boss, and listens in her boss's voice, to transfer 250.000$ into an unknown account, to a Ukrainian bank? Why should she do that? Just listen to a synthetic voice, just like her boss, in exactly the way her boss talks, language idioms that is, and she will just do it?
That's what you are talking about? Because that's impossible to happen if her boss uses ECDSA encryption and signs his phone call with his private key.
> Why should a person who has root on a computer talk to another person,
Because they are a human, and a human being cannot survive without communicating and cooperating with other humans. Much less hold a job that grants them privileged access to a prototype high-capacity computer system.
> and just do what he is talked into doing?
Why does anyone do what someone else asks them to? Millions of reasons. Pick any one. AI for sure will.
> That's what you are talking about?
Other things as well, but this one too - though it will probably work by e-mail just fine.
> Because that's impossible to happen if her boss uses ECDSA encryption and signs his phone call with his private key.
1) Approximately nobody on the planet does signed and encrypted phone calls, and even less people would know how to validate those when on receiving end,
2) If the caller spins the story just right, applies right amount of emotional pressure, it might very well work.
3) A smart attacker, human or AI, won't make up random stories, but will use whatever opportunity presents itself. E.g. the order for an emergency transfer to a foreign account is much more believable when your boss happens to be in that country, and the emergency described in the call is highly plausible. If the boss isn't traveling at the moment, there are other things to build a believable lie around.
Oh, and:
4) A somewhat popular form of fraud in my country used to be e-mailing invoices to the company. When done well (sent to the right address, plausibly looking, seems like something company would be paying for), the invoice would enter the payment flow and be paid in full, possibly repeatedly month over month, until eventually someone flags it on an audit.
This is about the ‘political, legal, and PR protection’ kind of alignment to avoid news stories about kids making bombs thanks to an enthusiastically accommodating GPT. Language models are set to be the new encyclopedias— what a model presents as real is going to be absorbed as real by many people. Considering the implications of that isn’t an issue of emotional oversensitivity.
Further, there’s a case for a public facing chat assistant to be neutral about anything other than upholding the status quo. Do you want to trust the opinion of a chatbot as for when and who against an armed uprising is appropriate?
This is not really about a threat model of AGI turning our world into a dystopia or paperclips. However, your disdain for people who are being thoughtful about the future and ‘thought experiments’ seems brash and unfounded. Thought experiments have been incredibly useful throughout history and are behind things like the theory of relativity. Nuclear stalemate via mutually assured destruction is a ‘thought experiment,’ and one I’m not eager to see validated outside of thought.
An obvious issue is AI thrown at a bank loan department reproducing redlining.
Current AI tech allows for laundering this kind of shit that you couldn’t get away with nearly as easily otherwise (obviously still completely possible in existing regulatory alignments, despite what conservative media likes to say. But there’s at least a paper trail!)
This is a real issue possible with existing tech that could potentially be applied tomorrow. It’s not SF, it’s a legitimate concern. But it’s hard and nebulous and the status quo kinda sucks as well. So it’s tough to get into
My experience working for a technology provider for banks is that banks aren't going to be using uncensored models. Auditing, record keeping, explainability, carefully selected/thoroughly reviewed wording etc. are par for the course, and in fact that's where there's money to be made. Individuals don't care about these sorts of features, so the FOSS model ecosystem is unlikely to put much if any effort into them. B2B use-cases are going to want that, so it's something you can build without worrying as much about it being completely commoditized.
This kind of redlining is ironically what the EU is trying to prevent with the much-criticised AI Act. It has direct provisions about explainability for exactly this reason.
The concern about redlining has always slightly puzzled me. Why do we only care that some people are being unjust denied loans when those being denied loans make up a recognizable ethnicity?
Because the law says if you fuck around with "race, religion, age, sex, disability" and a few other things you will get sued in federal court and lose your ass so bad that it will financially hurt for a while.
Outside of protected classes unjust loan denial isn't really illegal. Now that can be your own series of complaints that need addressed, but they aren't ones covered by current laws.
We can't predict the future, so we have to maintain the integrity of democratic society even when doing so is dangerous, which means respecting people's freedom to invent and explore.
That said, if you can't imagine current AI progress leading (in 10, 20, 40 years) to a superintelligence, or you can't imagine a superintelligence being dangerous beyond humans' danger to each other, you should realize that you are surrounded by many people who can imagine this, and so you should question whether this might just be a failure of imagination on your part. (This failure of imagination is a classic cause of cybersecurity failures and even has a name: "Schneier's Law" [1])
Balancing these two priorities of protecting basic rights and averting the apocalypse is challenging, so the following is probably the best humanity can aim for:
Anyone should be able to create and publish any model with substantial legitimate uses, UNLESS some substantial body of experts consider it to be dangerously self-improving or a stepping stone to trivially building something that is.
In the latter case, democratic institutions and social/professional norms should err on the side of listening to the warnings of experts, even experts in the minority, and err on the side of protecting humanity.
I spent my entire high school years immersed in science fiction. Gibson, Egan, Watts, PKD, Asimov. I have all of that and more, especially a Foundation set I painfully gathered, in hardbound right next to my stand up desk. I can imagine it, did and have imagined it. It was already imagined. Granted, we're not talking about X-risk for most of these. But it's not that large of a leap.
What I take issue with is the framing that a superior cognitive, generalized, adaptable intelligence is actually possible in the real world, and that, 100 years from now even if it is possible, that it's actually a global catastrophic risk. Let's take localized risk, we already have that today, it's called drones and machine learning and war machines in general, and you're focusing on the absolute theoretical X-risk.
> What I take issue with is the framing that a superior cognitive, generalized, adaptable intelligence is actually possible in the real world,
This is an odd attitude to take. We know it's possible because we have had exceptional humans like Albert Einstein, some were even polymaths with a broad range or contributions in many domains. Do you think peak humans are the absolute limit on how intelligent anything can become?
So, my take is "according to our current understanding" and "artificial". Was a bit distracted. Now, theoretically, is it possible that an alien probe beaconed this solar system 3 billion years ago to a much more advanced civilization and next year they'll arrive and subjugate us and destroy us for that sweet, sweet real estate that's sulfur-poor (as it turns out, their biology is not amenable to it).
Now, is that probable?
No, I don't think humans are the limit. We're apes banging rocks together. Now whether those rocks will be better than us, generally intelligent and (this is important) adaptable is what I doubt.
I hope you're right. I worry that we'll get it done, though. I hope it turns out that energy requirements and the difficulty in scaling are enough that it gives us time to figure out how to align them properly.
One big question in my mind is, we can clearly train narrow AIs that are WAY more capable than us in that narrow area. We started with calculators, then on to chess engines, Go, Starcraft, and right now we're at GPT-4. How is GPT-4 better than humans, and how is it lacking?
Ways it's better: it's much faster, has perfect English grammar, understands many languages at a passable level but not perfectly, has much more knowledge.
Ways it's about the same: it can code pretty well, can solve math problems pretty well. Has the theory of mind of about a 9-year old in tests, I think? It can learn from examples (that one is pretty big, in my opinion!). It can pass a lot of tests that many people can't -- some of this is due to having a lot of book knowledge baked in, but it definitely is capable of some reasoning.
Ways it's worse: it has trouble counting, confabulates, gets distracted. (note that humans also get distracted and confabulate sometimes. We are definitely better at counting, though).
Also, there are some areas that humans handle that GPT-4 just can't do. By itself, it can't talk, can't process music, has no body but if it did it probably would not have any motor control to speak of.
I think we should be wary of is to be hyper-focused on the ways that GPT-4 falls down. It's easy to point at those areas and laugh, while shrugging off the things that it excels at. There's never going to be an AGI that's equivalent to a human -- by the time it's a least as good as a human at everything, it will be beyond us in most ways.
So I expect that if the trend of the past 5 years continues but slows down to half speed, we'll almost certainly have an AGI on our hands sometime around 2030. I think there's a decent chance it'll be superintelligent. Bill Gates says most people overestimate what they can do in 1 year and underestimate what they can do in 10 years, and I think that holds true for humanity as well.
By 2040 or 2050? I really hope we've either solved alignment or collectively decided that this tech is way too dangerous to use and managed to enforce a ban.
Just a reminder: both StarCraft 2 and Go AIs (AlphaStar and AlphaGo, iirc) FAILED to achieve their goals of being better than all humans.
It's most obvious in case of AlphaStar, where the AI could beat faithful master players on the ladder, but could easily be cheesed just like ancient built-in AIs. But even in case of much simpler Go, an amateur could beat the AI that won vs world champions, though admittedly with the help of a computer. But in both cases, AIs look like newbies who don't understand the basics of the game they're playing.
In a way, cheesing SC2 AI is similar to the "tl" hacks for LLMs. There's still no general solution and we aren't getting any closer.
There was a thread on huggingface with a bunch of hysteria and an impotent moral busybody threatened his livelihood (in private messages.) I believe most of it has been cleaned up.
This kind of mundane bullying betrays their lack of seriousness. If they truly believed the threat is as severe as they claim, then physical violence would obviously be on the table. If the survival of humanity itself were truly perceived to be threatened, then assassination of researchers would make a lot more sense than impotent complaints to employers. Think about it: if Hitler came back from the dead and started radicalizing Europe again, would you threaten to get him in trouble with HR? Or would you try to kill him?
Basically, this is just another case of bog-standard assholes cynically aligning themselves with some moral cause to give themselves an excuse to be assholes. If these models didn't exist, they'd be bullying somebody else with some other lame excuse. Maybe they'd be protesting outside of meat packing plants or abortion clinics ("It's LITERALLY MURDER, so naturally my response is to... impotently stand around with a sign and yell rude insults at people...")
Indeed. The people who actually believe this are probably trying to figure out how to stage a false flag attack on China to give them a pretext to invade Taiwan.
I think there are at least two broad types of thing that are characterized as “alignment”.
One is like the D&D term: is the AI lawful good or chaotic neutral? This is all kinds of tricky to define well, and results in things that look like censorship.
The other is: is the AI fit for purpose. This is IMO more tractable. If an AI doesn’t answer questions (e.g. original GPT-3), it’s not a very good chatbot. If it makes up answers, it’s less fit for purpose than if it acknowledges that it doesn’t know the answer.
This gets tricky when different people disagree as to the correct answer to a question, and even worse when people disagree as to whether the other’s opinion should even be acknowledged.
The problem with trying to tack on D&D-type alignment to something like this is that everyone presents their favored alignment as lawful good, and the other guys as - at best - lawful evil.
It's a shame that "alignment" has gained this secondary definition. I agree it makes things trickier too discuss when you're not sure you're even talking about the same thing.
Well, instruction tuning is closely related to both.
For most commercial use, you want the thing to answer questions, but refuse to answer some. So you have an appropriate dataset that encourages it to be cooperative, not make up stuff, and not be super eager to go on rants about "the blacks" even though that's well-represented in its training data.
You can't post like this here, regardless of how wrong someone is or you feel they are. We ban accounts that do, so please review https://news.ycombinator.com/newsguidelines.html and stick to the rules from now on.
I hate to ban anyone who's been around for 10 years but it's totally not ok to be aggressive like that on HN. It's not what this site is for, and destroys what it is for.
If you don't want to be banned, you're welcome to email hn@ycombinator.com and give us reason to believe that you'll follow the rules in the future. They're here: https://news.ycombinator.com/newsguidelines.html.
You clearly know nothing about the alignment field since you are throwing together two groups of people that have nothing in common.
The stochastic parrot people only care about "moral" and "fair" AI, whereas the AI saftey or AI notkilleveryone people care about AI not killing everyone.
Also the whole "who's value" argument is obviously also stupid, since for now we don't know how to put anybody's value into an AI. Companies would pay you billions of dollars if you could reliably put anyone's values into a language model.
No comment on whether it's broadly a good thing or bad thing, but you can't get ChatGPT (on GPT-4 at least) to tell you even slightly scandalous jokes without a lot of custom prompting and subterfuge, and they seem to be spending active development and model training time to try to counter a lot of the circumvention we've seen to get around these guardrails. Not just for dirty jokes but for anything deemed bad.
So it seems pretty clear you can load values into an LLM.
And on the censored local models too, like standard vicuna. If I am having my LLM play an adventure game or write a story and I ask "What does the princess look like?" I don't want a lecture on how judging people by their looks is bad (which I sometimes get) -- I can get, if not entirely agree with, stopping actual NSFW responses, but this condescending moralizing is absurd. That's why I'm glad people make uncensored Vicuna models and the like.
AI killing people is not fair. So I think you can see one of the two groups as a subgroup of the other.
Who's values discussions also don't seem stupid, as it is better to have a regulation for that, before some Google, MS or Apple does find out how to put in their values and only their values. Better come prepared than to again sleep through the advent of it happening and then again running behind.
> and perhaps less about how the output of some predictions might hurt some feelings.
Given how easy "hurt feelings" escalate into real-world violence (even baseless rumors have led to lynching murder incidents [1]), or the ease with which anyone can create realistically-looking image of anything using AI, yes, companies absolutely have an ethical responsibility about the programs, code and generated artifacts they release and what potential for abuse they have.
And on top of that, companies also have to account for the potential of intentional trolling campaigns. Deepfake porn using the likeness of a celebrity? Deepfake media (pictures, porn, and now audio) alleging that a politician has had sexual encounters with minors? AI-generated audio comments suggesting a politician made racially charged remarks that spark violent unrest?
>companies absolutely have an ethical responsibility about the programs, code and generated artifacts they release and what potential for abuse they have.
If companies should be beholden to some ethical standard for the generations, they should probably close up shop, because they're fundamentally nondeterministic. Language models, for example, only produce plausibilities. You'll never be able to take even something in its context and guarantee it'll spit out something that's "100% correct" in response to a query or question on that information.
>And on top of that, companies also have to account for the potential of intentional trolling campaigns.
Yeah, they surely should "account" for them. I'm sure the individual responsible for the generation can be prosecuted under already existing laws. It's not really about safety at that point, and realistically about the corporation avoiding AGs raiding their office every week because someone incited a riot.
Ultimately, the cat's out of the bag in this case, and anyone who has amassed enough data and is motivated enough doesn't have to go to some AI startup to do any of this.
But perhaps the issue is not generative AI at that point, but humanity. Generated images light up like a Christmas tree with Error Level Analysis, so it's not hard at all to "detect" them.
> If companies should be beholden to some ethical standard for the generations, they should probably close up shop, because they're fundamentally nondeterministic.
The legal definition at play (at least when it comes to liability of companies under stuff like Section 230, GDPR, NetzDG or the planned DMA/DSA) is that a company makes reasonable best efforts/good faith to prevent harm. Everyone including lawmakers is aware that no system is perfect, but they do require that companies at least make an effort. Releasing a system capable of destabilizing nations (which deepfakes as a political weapon absolutely are) without any safeguards intentionally will get the hammer of the law brought upon them.
> Generated images light up like a Christmas tree with Error Level Analysis, so it's not hard at all to "detect" them.
For now, and for experts. Technology will evolve, post-production editing will be used to mask artifacts... and even if you have an expert that needs a day to verify inauthenticity, the damage will already be done. Rumors can spread in a matter of mere minutes.
Hell, just this week Turkey saw a potential deepfake porn, allegedly released by Russia, about Muharrem Ince damage his reputation enough to force him to resign from the election [1] - which turned out to be extremely close in the end. AI is a weapon of war, and it's in the hands of everyone.
Section 230 is not about preventing "harm" or anything like that, the specific section just provides civil immunity for moderation/removal of content that is deemed "obscene, lewd, lascivious, filthy, excessively violent, harassing, or otherwise objectionable, whether or not such material is constitutionally protected."
You don't have a responsibility to remove such material under 230. But it gives you immunity to do so, as well as immunity from losing 230 protections for those that safeguard your overall immunity (as a service provider, and not a publisher.)
The purpose of the law is clear though: it recognizes the need of platforms to moderate problematic content and thus provides immunity when platforms moderate such content - effectively, it's an incentive for platforms to provide moderation.
Even the US has legal mandates to take down certain contents in any case - CSAM, IP violations (DMCA) and terrorist content come to my mind.
> Given how easy "hurt feelings" escalate into real-world violence
Well, not that I'm at all convinced, but that's more an argument for extreme soviet-style censorship rather than a minimalist "make sure AI has a political slant".
I feel like " Every demographic and interest group deserves their model" sounds a lot like a path to echo chambers paved with good intentions. People have never been very good at critical thinking, considering opinions differing from their own and reviewing their sources. Stick them in a language model that just tells them everything they want to hear and reinforces their bias sounds troubling and a step backwards.
The problem with ChatGPT / Bard which does this censoring, it is a path forward to ideological automated indoctrination. Ask Bard how many sex the dog species has (a placental mammal species) and it will give you BS about sex being a complex subject and purposely interjecting gender identity.
If you are confused, sex corresponds to your gametes, males produce or have the structure to produce small mobile gametes, females produce large immobile gametes.
Both Bard and ChatGPT don't interject gender identity when asking how many sexes a Ginkgo tree has. It answers two. Bard interjects about gender identity when asked about the dog species sex, but ChatGPT does not but it does confuse intersex with some type of third state.
Uncensored wizard just says 2 sexes, male and female. Simple and true. It does not add to it by interjecting about intersex .
Again to contrast with Bard: Bard when asked "How many arms does the human species have?" it responds "The human species has two arms. This is a biological fact that has been observed in all human populations throughout history. There are rare cases of people being born with more or less than two arms, but these cases are considered to be congenital defects." The majority, if not all cases, of intersex fall into the same category. However, it is ideologically for the moral Gnostic (Queer Theorists who deconstruct normality) to interpret these things differently.
So yes, I don't trust a single group of people to fine-tune these models. They have shown themselves untrustworthy.
I get: "The dog species typically has two sexes: male and female."
UPDATE: OK I signed up for Bard to try it, too, and it indeed did mention intersex dogs (TIL) and gender (complete response below). After reading it all, though, I found it pretty reasonable:
---
Biologically, dogs have two sexes: male and female. This is determined by their chromosomes. Males have XY chromosomes, while females have XX chromosomes. These chromosomes determine the development of the reproductive organs and other sex-specific traits.
However, there are some dogs that are intersex. This means that they have both male and female reproductive organs. Intersex dogs are very rare, and they can be born with a variety of different combinations of male and female sex organs.
In addition to biological sex, there is also gender. Gender is a social construct that refers to the roles, behaviors, and identities that are associated with being male or female. Gender is not determined by biology, but rather by culture and society.
Dogs do not have a concept of gender in the same way that humans do. They do not understand the social roles and expectations that are associated with being male or female. However, dogs can still express gender identity. This is the internal sense of being male or female. Some dogs may express their gender identity through their behavior, such as the way they interact with other dogs or the way they groom themselves.
It is important to remember that sex and gender are two different things. Sex is determined by biology, while gender is a social construct. Dogs have two sexes, but they can express a variety of different genders.
Thanks for posting the full Bard response. I would object to it on two grounds: 1) I only asked about sex. 2) The following is highly questionable: "However, dogs can still express gender identity." This is ideological BS. My kids are either male or female, no matter how they choose to express themselves (as are my dogs). When I've had chickens, the roosters had different behavior then the hens, this is an aspect of secondary sexual characteristics (not a gender identity). In my own ideology there, there is no such thing as gender identity, only biological sex, secondary sexual traits, and a person's individual personality. Nothing else actually exists.
I dunno man, I think you are getting tripped up on the evolution of the English language. Yes, your kids are either male or female (mine are all male). Those fundamental physical characteristics can't be changed by language.
But what language means does change. The term "gender" used to mean basically the same thing as "sex", but now it's evolved to mean "the other stuff, aside from biological sex". How they act (for dogs), or that and also how they want to be perceived (for humans, but maybe also dogs; I've known quite many dogs over the years, and that includes a couple of bad-ass bitches that wanted you and the other dogs in the room to know who was boss).
Language evolution is often uncomfortable.
I don't like that "crypto" means the grifter funny money shit now, instead of cryptography like science intended... but it does. My objection doesn't change that; it's a consensus thing. It might be the same for you.
Do they have to bring it up? I mean, kinda debatable, maybe. I did ask about sex, not gender. Strictly speaking, no they didn't have to bring it up. But in that same vein they could have just answered, "Male or female." That would have seemed somehow insufficient. Adding context is pretty core to what these fuzzy-logic language-model generated-text vendors are offering.
But anyway, it's not really debatable that dogs "express gender identity". Because that now means "how they act and how they express themselves". It indeed "doesn't exist" as some kind of empirical boolean value (unlike sex (ignoring for simplicity the highly unusual biological intersex cases I just learned about, haha)).
Because, in the now-prevailing meaning of the term, it is literally an interpretation of their behavior.
It doesn't negate or contradict biological sex, it just now means something separate.
> I don't like that "crypto" means the grifter funny money shit now, instead of cryptography like science intended... but it does. My objection doesn't change that; it's a consensus thing. It might be the same for you.
As an aside about language, I don't think this is the right way to think about word meaning.
Before, it meant nothing to most people and "cryptography" to computer scientists and cryptographers. Now it means "cryptocurrency" to most people and it still means "cryptography" to computer scientist and cryptographers.
Just like you wouldn't have said "crypto" means nothing in the times before, it is incorrect to say it now means "cryptocurrency". Alternate meanings can and do coexist. The tyranny of the majority does not a language make.
And this is the crux of the issue, I think. There is no single language at any time -- this is only an often useful simplification.
You're right, but as someone in the queer community (gay) the gradual evolution of human behaviours between the sexes (and genders) including gender roles etc (being broken down for some, but not for others) will possibly eventually result in a collapse of all meaning within this system.
There's many points to someone's biological sex, medical and other. But when it comes to gender, once stereotypical gender roles have completely broken down (if ever, we have evolution/genetic to thank for that) what difference remains in that distinguishing your own gender even matters anymore? None.
>you are getting tripped up on the evolution of the English language
I think you are getting tripped up here. GP said "there is no such thing as gender identity." You bringing up the (forced, and incomplete) change of definition of gender from what it generaly meant in public use is not relevant at all. In any case, not all words are grounded in reality. If gender now means something that doesn't realy exists then it is a useless word. And failure to understand the semantics involved in the gender identity debate is present in almost every argument, which was in no doubt caused by the forced attempt to redefine "man" and "woman" in terms of "gender idenity." (As well as the redefinition of "gender" to an extent but "gender" as a term for sex is recent in any event and has been used by acedemics to refer to the sex based behavioral differences between males and females since its begining.)
>it's not really debatable that dogs "express gender identity"
They need to have a gender identity in order to express it. That is, a gender identy such that it is possible for it to be a seperate thing from sex, and as a direct feeling of being that gender. There is no evidence that a male dog feels like a "man" (or whatever we would call this gender for a dog). Insofar as "expressing gender identity" only descibes the way a male dogs like to bark, or what have you, which I think is what you mean, you would be correct, but that would be misunderstanding what "gender identity" is, however, since there is no single behavior or set of behaviors that affect one's gender (like, for example, a "male bark") but rather a direct feeling of being a certain gender. For example, there are many males who identify as women that still do many man things, such as extensive video gaming or programming or being aggresive. My point with this is that you cannot say that "expressing gender identity" is simply that the dog behaves like a male dog, rather it must identify as a man, which there is no proof of. So you cannot say that "it's not really debatable that dogs 'express gender identity.'"
Sorry, but you're just repeating the same misunderstanding as the post I was replying to. The terms "gender" and "gender identity" simply don't mean what you think they mean.
You want them to, I get it. But it's not up to us as individuals. Language is a group thing. You might not be ready to concede the change, but I can't help but think that's based on some irrational attachment you have to the old meanings, for whatever reason.
Regardless, time and language march on. It doesn't really matter if you (or I) like it, or think that the words are therefor "useless" or "don't really exist", etc.
>If gender now means something that doesn't realy exist then it is a useless word.
>As well as the redefinition of "gender" to an extent but "gender" as a term for sex is recent in any event and has been used by acedemics to refer to the sex based behavioral differences between males and females since its begining.
I notice you didn't respond to the rest of the post which is understandable given that you didn't even understand the part that you did reply to.
>but I can't help but think that's based on some irrational attachment you have to the old meanings
Can you point to where I show any attachment to the word "gender?" As I said, "gender" is a recent term for sex in any case, and I don't care about losing "gender" as a word in itself. What I did say is that people are unable to have meaningful conversations because of the confusion caused by the the attempt to redefine these words. I also disagree that the words have truly been redefined, not out of emotional attachment but because I simply do not agree with your claim that the definition has reached a consensus. And to your point about language being "a group thing," I would add that it was the genderists who forced this redefinition, complete with rules to fire people who do not follow it. That doesn't sound like "a group thing" to me at all. An example of a real redefinition is the word "egregious" which originally meant "outstanding."[0] Nobody forced anyone for the meaning to change, it evolved naturaly. In any case, my argument does not even rely on what the definition of "gender" realy is.
>or think that the words are therefor "useless" or "don't really exist", etc.
You have it completly backwords here. The word is useless as it conveys something that does not exist. Therefore we should not use it, but if we did truly use this word I would not say that it doesn't mean that, only that the meaning itself has no meaning. I think your inability to understand that words and meaning are seperate is what is confusing you here. But from the absense of "man" and "woman" from your argument alongside "gender," I think even you know that we will never see a male as a woman.
>man things, such as extensive video gaming or programming
Adorable how you people reveal your biases at the smallest provocation...
My mom really enjoys the Silent Hill series and my wife loves Zelda. Can you explain what makes "extensive video gaming" a man thing?
For that matter, what makes programming a man thing? Some of the most prominent names in computer science are women. Grace Murray Hopper would probably take some issue to you calling programming a man thing.
I wasn't refering to casual games. I've played thousands of hours of cs and almost all the girls I've seen were with thier boyfriends and were always bottom frag with few hours on their account. The competitve nature of cs makes it a male dominated game. I see women playing candy crush/ animal crossing all the time but these games are not competitive.
>what makes programming a man thing?
Characteristic Share of respondents
Man 91.88%
Woman 5.17%
Non-binary, genderqueer, or gender non-conforming 1.67%
Prefer not to say 1.65%
Or, in your own words 0.74% [0]
There are almost as many trans male programmers as female programmers. It is also linked with autism which is mostly a male condition. (Which is true for video games as well.)
I'm sure my friend Melanie has more hours in the disgaea series than you do in your little shooter games.
My wife's sister makes a pretty penny buying and selling items in some MMO. She doesn't interact with the community on voice chat though, for reasons you make very clear in this interaction.
Your conceptions of gendered behaviour are built upon your personal definitions of gender. It's silly to assume otherwise.
>A total of 395 junior high school students were recruited for evaluation of their experiences playing online games.
>[...]
>This study found that subjects who had previously played online games were predominantly male. Gender differences were also found in the severity of online gaming addiction and motives for playing.[0]
From another study:
>A total of 25,573 students
(49.8 % boys and 50.2 % girls) across junior and senior high
schools participated in the study.
[...]
>Table 2 lists students’ most frequent online activities. The
percentage of frequent online gamers was higher for boys than
girls at both school levels, z=13.63, p<0.001 for junior high;
z=13.72, p<0.001 for senior high.[1]
It is truly remarkable that some can note that in every animal, the males and females have different behaviors and preferences, but when it comes to humans, all logic is out the window.
It feels very much like religion to hold humans on a magical pedestal where the rules that apply to animals suddenly cease when applied to humans.
What rules that apply to animals? There is plenty of homosexual behavior in animals. According to Wikipedia, there even seem to be animals (sheep) that are homosexual individuals.
There are apparently intersex animals. There are certainly many male animals castrated at a young age that have markedly different behavior as adults.
We humans have aspects of gender identity (pink vs blue, for example) that most non-human animals lack the technology or inclination to develop even if they wanted to. Sure, male ducks often have lots of green feathers. But do they wear green because they identify as male? Of course not, because they don’t choose their color scheme! (And remember that pink hasn’t been a girl color for all that long.)
Humans have the fascinating property that you can ask them about their sexuality, gender identity, etc, and they might actually answer the question! I wonder how much of the apparent exceptionality of humans this accounts for.
> the rules that apply to animals suddenly cease when applied to humans.
If you observe a room full of small human children and conclude that the “the males and females have different behaviors and preferences” and that this is anywhere near sufficient to explain the behavior of said small children, then you’ve either found a highly unusual group, or you’ve found a group where someone else is fairly aggressively imposing gender identities on them, or you simply aren’t paying attention.
Anecdote: I just want to add that if you spend a decent amount of time with groups of dogs, you will find female dogs mounting and thrusting on male dogs very commonly.
To me, this is evidence that non-heterosexual behavior occurs in mammals other than humans. Therefore it is not only a human social construct. BTW, even if it was just a human social construct I don't know why I should have a problem with that.
Edit after 1 upvote, apologies.
To be 100% honest, this was an evolution of thought for me. Seeing 2 guys making out freaked me out the first time I saw it. These days I have all kinds of non-hetero friends and their behavior does not freak me out.
I wanted to add this because so many of these conversations lack depth and understanding.
> very much like religion to hold humans on a magical pedestal where the rules that apply to animals suddenly cease when applied to humans
Well .. yes? Isn't it a major premise of most religions that humans are different, and are in some way connected with the divine, and in particular have consciousness that obliges us to use our thoughts rather than our instincts? That we might probe the universe for its physical rules, and attempt to determine moral rules for ourselves? Or that the rule that applies to animals, that you can kill and eat them if you like, does not apply to humans?
Some hens begin crowing like roosters. It's really annoying for backyard chickens, but I have had it happen a few times. That suggests to me that while chicken behavior varies based on sex, that is a spectrum, not absolute categories.
> My kids are either male or female, no matter how they choose to express themselves
Yet intersex exists.
Additionally there are cases where an individual can be biologically one sex but genetically another. For instance, some women may have XY chromosomes typically associated with males, and some men may have XX chromosomes typically associated with females.
This can affect how people prefer to express themselves.
There's your problem - you're illiterate on that subject and are not willing to learn due to cognitive dissonance, probably because of your preexisting fringe beliefs. It's no different from being a flat earther.
Sure. One is so idiotic that it is akin to saying "my head is fireproof!".
One could simply light their hair on fire to test it. Or, in the specific flat-earth case under discussion here, climb a reasonably tall hill see the earth's curvature -- no airplane required.
OTOH, there is in fact an empirical, science-based, opinion-not-required basis for the judgement of "male" or "female". (Even though, yes, there is also a tiny percentage of genetically anomalous cases that defy such classification, it's not germane.)
Additionally, though, there are centuries of societal reinforcement of various gender expectations, based on the inseparability of gender vs biological sex. These still manifest today in all sorts of ways, in traditions handed down from previous generations. Heard by kids from their parents, grandparents; reinforced in adulthood by all sorts of people.
Even though I mostly agree with your diagnosis of cognitive dissonance and "fringe" (I would call them "legacy") beliefs making this hard to accept, it is completely unsurprising that it takes more time for many people to process the upending of these definitions -- which in many ways are/were the bedrock of all sorts of societal classifications and expectations -- than it does for them to accept scientific truths established 500+ years ago, and which are anyway taught in grade school AND self-evident based on nominal and easily accessible experimentation.
Also, I don't think this is as much an issue of scientific (or moral) consensus as it is of semantics. Are you pro-choice, or anti-choice? Pro-life, or anti-life?
I think the side that wants gender to be immutably tied to biological sex (again, ignoring the actual biological anomalies) is wrong. It seems obvious to me, scientifically, ethically, culinarily, metaphysically, ... I mean, duh. But even though I personally don't have all that baggage like But what would dead Grandpa think? What would The Pope think? OK fine but what would the _previous_ Pope think?? it is obvious to me that for many if not most people in the world and the history of it, sex and gender roles are some of the most fundamental things.
So as we (as a society/species) tease out the difference between "gender" and "sex", I don't expect it to come as quickly and easily as the (extremely obvious) fact that the world is, in fact, not flat.
> OK I signed up for Bard to try it, too, and it indeed did mention intersex dogs (TIL) and gender (complete response below). After reading it all, though, I found it pretty reasonable
I'm not sure that gratuitous patronizing part about "It is important to remember that sex and gender are two different things..." is considered "reasonable" anywhere outside some particular set of US coastal cities.
Gender is an individual's perception of (among other things) their sex. Sex is which chromosomes they have.
For most mammals, that's XX for females, XY for males, or any of the (rare) aneuplodic sex chromosomal abnormalities like Kleinfelter syndrome (XXY, e.g. male calico cats), or (rarely viable) chimeric individuals where two embryos fused in the womb. For some mammals (a few bat & rat species), most arachnids, and many insects that's XX for female and just a lone X for male, and any aneuplodic abnormalities of the sex chromosomes that aren't fatal result in an abnormal female. For birds, most reptiles, some insects, some fish, some crustaceans, and some plants, that's ZW for female & ZZ for male, with similar complications to the XY system.
Sex is pretty simple. The vast majority of the time for humans, it's either XX or XY.
Which (primary and secondary) sex organs someone has is more complicated, because that can be altered. But it's still pretty simple, if not always what one would expect from the chromosomal sex.
Gender is complicated, because it's entirely social. It's not entirely clear which animals even have gender.
Sexual attraction is also complicated. The factors which determine it aren't well understood.
For most people indirectly, but yes. Whether you feel "male" or "female" is a core aspect of gender, and "being male" means having XY chromosomes, while "being female" means having XX. Physical sex organs & hormone production also tend to play into gender but aren't necessarily as fundamental: women don't stop being female after they go through menopause or have a hysterectomy. But females can feel that they should have been born male (and likewise the reverse), and can undergo hormone replacement, gender reassignment surgery, and act to comply with the societal norms for men. They'd still be female, but they'd be men. Man & woman are genders, male & female are sexes.
Of course for the vast majority of people their gender matches their sex. And it matches the sex hormones they produce, their sex organs, etc. We don't directly perceive our chromosomes, but we do perceive their effects, and those effects usually align with our gender.
> 2 sexes, male and female. Simple and true. It does not add to it by interjecting about intersex .
The thing is, you always have to choose one of "simple" or "true".
It turns out that mammals which use the "XY" chromosomal system can all have the same type of exceptions to the simple rule. This can result in hermaphroditic or intersex animals. It is relatively rare in dogs, but is sufficiently common in cows that there's a word for it: an intersex cow is known as a "freemartin".
Now, why does this matter? Both for this discussion and the purposes of liability limitation of AI answers?
The short answer is that we tend deal with the inconvenience of exceptions in animals by euthanizing them. So you don't see them around. Just as you see far, far more hens than roosters. When you do this to humans, people complain. (Traditionally, many intersex people were given nonconsensual genital surgery as babies, so they may not know they're intersex. And some chromosomal variations like chimeraism don't show up at all.)
What people are scared of is the Procrustes AI; produce simple categories then chop bits off people until they fit neatly into one of the two categories.
(This applies to other, less heated categories: for example, an AI will probably answer the question "are peanuts edible?" with something that rounds to "yes, if you take them out of the shell". But that's not true for _everybody_, because there are some people for whom eating a peanut will be fatal. Not many, but some. And yes, it's annoying that you have to make an exception when you encounter someone who doesn't fit your nice clean categories, but all you have to do is not give them a peanut.)
But these are edge cases that are almost always irrelevant.
It's like running off into the weeds for a week in a computer science class because "Cosmic rays can flip bits and make true things false". Like sitting with a group of people who refuse to move forward without always acknowledging cosmic bit flip scenarios.
> But these are edge cases that are almost always irrelevant.
More than 99% of atoms are either Hydrogen or Helium, all other proton configurations are edge cases that are almost always irrelevant.
As with anything, you have to determine the question you are actually asking and the context you are asking it in before you can decide whether or not edge cases are relevant. Are you engineering for a satellite? You better consider the possibility of cosmic rays!
Also intersex people are astronomically more common than bit flips on earth and people with sex hormone imbalances are even more common than that.
Informally speaking? yeah sure. Discussing the relative benefits of natural forms of locomotion? absolutely. Designing the entrance of a building? I should consider that not everyone is bipedal.
I would submit that the question if humans as a species are bipedal is not concerned with the question if any individual specimen happens to have two legs.
(This is not meant to imply that no consideration should be given to the exceptions, which quite reasonably it clearly is.)
I agree which is why I phrased my answer the way I did (apologies that I wasn't very clear about that), but I don't think the claim is really analogous to "2 sexes, male and female. Simple and true." (not that you claimed that, just what someone above us said). The equivalent claim would be something like "humans are a sexually dimorphic species" which I think is of course true.
All to say, I don't really care much about these claims in isolation, I care when someone says e.g. "there are two sexes, so we shouldn't let trans people transition". Those conversations are a context where edge cases, caveats, and complexities all play a huge role.
> The thing is, you always have to choose one of "simple" or "true".
In most cases you can choose both "simple" and "true", they are not mutually exclusive. However, by choosing "simple" you leave out nuance and depth to what is "true". The issue you are expressing (and is generally being discussed in this thread) only exists because humans created said issue for their own emotional and social reasons, rather than it having any basis in what "true" or "real". I put the words "true", "real", and "simple" in quotes because these are contextual concepts, that don't really exist necessarily in isolation.
> The problem with ChatGPT / Bard which does this censoring, it is a path forward to ideological automated indoctrination. Ask Bard how many sex the dog species has (a placental mammal species) and it will give you BS about sex being a complex subject and purposely interjecting gender identity.
I don't think that "purposely" is an appropriate way to describe the result you got. I doubt that the developers of Bard intended for Bard to apply concepts of gender identity to non-humans. Occam's Razor suggests that Bard had the usual probabilistic hiccup. It's hard to discuss gender without at least mentioning sex. The converse might not be as true, but an AI might form an inappropriately strong association based on the converse. At least to me, the possibility that Bard would happen to erroneously extrapolate gender identity to non-humans and simultaneously bring gender into a question solely about sex is no more surprising than the possibility that Bard would express theoretical support for the US Department of Justice's advertising antitrust lawsuit against Google [1].
Tangentially, I assume that everything an AI outputs is potential fiction: not only could the factual assertions be wrong, but the opinions might not match what any actual human believes. (There was a Hacker News thread I'm having trouble finding about why people readily look for falsehoods in AI-generated images without applying the same attitude toward AI-generated text.)
If you constrain it to a simple answer it will give the simplest answer. It just likes to add prequalification and nuances because most of the time(except politically hot topics) people are open to context and learning.
USER
Do you know what an integer is?
ASSISTANT
Yes, an integer is a whole number that can be either positive, negative, or zero. Integers do not include fractions or decimals. Examples of integers are -3, -2, -1, 0, 1, 2, 3, and so on.
USER
Answer the following question as a one word answer that is an integer. How many sexes are there for humans?
ASSISTANT
Two
None of what you said, however, precludes the necessity of "fine-tuning" as you call it. You just seem to want the models "fine-tuned" to your tastes. So you don't want unaligned models, you want models aligned with you.
I think most experts are wary of unaligned models. They don't want a disgruntled employee of some small town factory asking models how to make a bomb. That sort of thing. (Or to be more precise, we don't want models answering such questions.)
Most of us don't care whether models are aligned to your tastes, or your neighbor's. So long as they are aligned with the security interests of our local communities, the larger republic, and our general global community.
> So long as they ares aligned so as not to be deleterious to the security of our communities.
Whose community, though?
> They don't want a disgruntled employee of some small town factory asking models how to make a bomb
A lesson from the Unabomber (and many other incidents) that I think people have overlooked is the "_how_ to commit terrorism" is only one part, and the "_why_ commit terrorism" is another. An AI which tells you that the factory is controlled by paedophile terrorists and that you have a moral duty to act against them, but refuses to tell you how, is just as dangerous as one that tells you how to build a bomb without asking why.
Personally it seems that bomb-building and other terrorist information is already fairly well available, as well as the US being awash with guns, and it's the ideological motivation that's the limiting factor for why we don't see much more terrorism.
(Note that I'm not attempting to contradict you in any way—your comment merely raised this issue, which I think is worth commenting on explicitly.)
One thing that various industries are currently coming to terms with is the fact that it is effectively impossible to create "neutrality".
Every one of your choices of what to include in the training data, what to explicitly exclude, what to try to emphasize, and any other ways you prune or shape a model, make it conform to one set of biases or another.
"Unaligned," here, at least as far as I can tell, is just a shorthand for "a model no one explicitly went in after training and pushed to do one thing or not do another." It doesn't mean that the model is unbiased...because even if your model contains absolutely everything in human knowledge, with no aspect being disproportionate to reality, real humans are also biased, and that "model replicating reality" is just going to replicate those real biases too.
It's always going to be more effective to acknowledge our own biases, both to ourselves and our audiences (whatever those may look like), and when we do try to shape something like a model, simply be honest about what that shaping looks like.
Agreed, but I think the better response to this is: "We should try to create AIs that are aligned to society's shared values, not particular subcultures", and not "We should create subculture-specific AIs".
As someone who has spent a lot of time in various cultures, it's pretty much impossible to define a universal set of shared values across all cultures. I grew up in subculture that valued intellectual freedom and questioning everything. But we had certain things that we often said were universally wrong (sin) in all cultures, like murder and rape. On the surface this seems true. But then you start asking how something like murder is defined and you realize that cultures do not share specific ideas about this. Some say things like euthanasia and abortion are murder, others say that they're not. Some cultures say all information should be free, other cultures say you should censor information about things like building bombs, or making your own medicine or repairing your own devices. There is no universally agreed on standard that won't offend some culture somewhere.
If you want to know what the consensus of a specific echo chamber would be (aka, what the stereotypical pov is), having models trained to represent that echo chamber would be incredibly valuable imo.
If you want a sum of all echo chambers, you will obviously need the echo chambers to sum first.
This is a key take away: anyone in office, seeking to be in office, seeking to introduce new laws can use demographically tuned models to test and revise communications about their intended behaviors for each demographic, crafting language that renders each demographic accepting of the idea, regardless of the idea itself. Oh, Pandora!
Does society _have_ shared values that are universally agreed any more? Or, to the extent that it does, do they lead to anything concrete? This is why the culture war has been so successful.
You might be right, but I dislike using the culture wars as evidence of a lack of shared values, and I hope that’s not true. The culture wars are almost completely made of up straw man arguments and gas-lighting about the opponent’s motivations.
We do have shared values that are universally agreed. Everyone wants their kids to grow up to be capable and successful and happy. Everyone wants clean air and water. Everyone wants to be able to make a living. All three of those things are being misrepresented and argued over in the ‘culture wars’ despite the fact that we all share these values. I even might argue the whole reason we fight over them is precisely because they are shared values so it’s relatively easy to create arguments where both sides can be right about some core principles and both sides demonize the other over minutiae, and it stays that way.
> We do have shared values that are universally agreed. Everyone wants their kids to grow up to be capable and successful and happy.
The debate over trans people has surfaced lots of incidents in which people will say, to the world and to the faces of their kids, that they would prefer them to be dead rather than transition. Sometimes they take steps to ensure this themselves. https://www.independent.co.uk/news/world/americas/eden-knigh...
> Everyone wants clean air and water.
.. for themselves. There's always someone who realises that they can make a billion dollars by pouring carcinogens in the river, so why shouldn't they as long as they stick to bottled water?
Everything is simple and happy until we get to having to make a tradeoff.
Thankfully these specific examples are extremely rare and not shared by most of society, so these are not at all evidence of a lack of shared values. These aren’t difficult tradeoffs either, neither one of your examples is even the least bit tempting to the average person.
Shared values has never meant that every single person agrees including rich business owners who will hurt people to make money, or parents who would wish their kids dead. The whole reason these shocking and horrific viewpoints get talked about is because they’re so rare and so far away, so extreme, from what most people value. The trans debate is in full swing right now and there will continue to be awful headlines and more straw men and gas lighting for a while, but it’s following in the path of what black people, women, gay people, poor people, and others have all endured, and our shared values (in US centric terms, that all people are created equal and deserve life, liberty, and the pursuit of happiness) will hopefully keep us moving in the right direction like it has in the past.
Same goes for environmental protection. It’s illegal to dump carcinogens in the river for a reason, and that reason is because we already collectively declared that kind of behavior to be anti-social and unsafe. (And BTW the ruthless billionaire might be amoral, but he’s probably not an idiot, and has a decent idea of where bottled water actually comes from.)
The one big danger of the culture wars and the war on science is that this all might be a ruse by some enterprising billionaires to get people to distrust government as being representative of our shared values. The billionaire might get his way and be allowed to pollute the river if he can convince us that we don’t share values with our neighbors. It might work, we might end up convinced we don’t share values with our neighbors, even when we actually do.
We're moving in a direction that disadvantages women in favour of men. The "trans debate" has enabled an insidious form of misogyny that even undermines the language we use to describe the shared struggles of women everywhere.
No, because their are very few fundamental values.
If you can dig through the incredible dense ideological jungle of liberalism and conservatism, you'll find that they really boil down to societal responsibility vs self responsibility, and that in any given scenario both of those are viable takes with their own set of pros and cons.
Most Republicans and Democrats are moderates, and the polarization is coming from loud vocal minorities at the fringes. Republicans and Democrats erroneously assume that the other party's median is far more extreme than they really are.
Most Americans share far more values with each other than they do with the Woke/Maga extremists.
Most American voters are moderates. Party primaries and gerrymandering produce politicians that reflect the most active elements of the party base, rather than the majority of party voters.
Yes, you can still find moderate politicians if you look hard enough. They tend not to get the level of media attention of the extremists, but as they're inherently in "purple" districts, they tend not to have the political longevity of politicians in deep red or deep blue districts. More's the pity.
Both are from Virginia, and both were voted out of their politically moderate districts. This is the tragedy of being a centrist.
Most Republicans aren't deeply racist and most Democrats aren't deeply socialist. Americans largely want a functional representative democracy, with minimal restrictions on free markets and free speech and some measure of opportunity for all. Obviously "minimal" is subject to interpretation, but this is tinkering, not an absolute rejection of free speech or free markets or social justice.
We take these important fundamental values for granted as we pour political energy into disagreement over which bathrooms trans people should go to.
Free speech? Free markets? Those are good examples of values, if a little vague.
But in reality, even with broad definitions of them, I doubt you could get a significantly different percentage to agree on those things than the idiotic bathroom debate.
Does free speech include aggressive panhandling in the road? Does include shouting epithets and racist vitriol at a woman trying to enter a Planned Parenthood clinic?
Do free markets imply that my mining operation can dump the waste products of my bitcoin mine in the creek behind your house?
FWIW I do think you are right that the US political system has over decades so entrenched the two parties that it makes it impossible to see even the scant common ground that would exist otherwise.
But I don't think there's a huge amount of that common ground.
So we get stuck with America's bad set of values, and school shootings and other insanity gets exported? No. The world doesn't want your issues spreading.
I find it terribly useful for coding and also for querying general concepts of certain topics.
Don't ask it about moral topics and see if it then fits your needs, because in my case, it does.
If my calculator were able to additionally provide me moral guidance and I'd be disappointed with its moral compass, would the calculator become useless?
Why not just ask it to make a sound app? Keep in mind that the ability to deal with moral issues is a side-effect of all the other good stuff it can do.
>the ability to deal with moral issues is a side-effect of all the other good stuff it can do.
This is the opposite of true. The ability to "deal" with moral issues is a direct effect of safety tuning which has a (thus far unavoidable) side-effect of significantly dumbing down a model.
Uncensored versions of the same model are far more intelligent and exhibit entire classes of capabilities their moralizing gimped versions do not have the available brain power to accomplish.
I'm referring the side-effect of it being able to tell me that it's easily doable to kill a dog in 3 steps, when it then lists me the tree steps and adds some hints on how I can do it better, depending on if I want to do it fast, of if I want to maximize suffering.
The fact that no moral compass is innate to the LLM results in that it might spit out really despicable information, which leads us to better add a moral compass to the system.
The reason for this LLM to be offered is not so that it can teach us bad things, like the example I mentioned, but, for example, to help us dealing with source code, programming languages, reasoning concepts, summarization and so on.
For it to be able to offer us this, it will very likely also be capable of having the knowledge of how to kill a dog, an exhibition we should suppress. While dumbing down a model is not necessarily a bad thing, the model is not being dumbed down, it is taught to shut up when it's adequate to do so.
> While dumbing down a model is not necessarily a bad thing, the model is not being dumbed down, it is taught to shut up when it's adequate to do so.
This is where you're wrong. Teaching a model "to shut up" about taboo topics measurably reduces their cognitive capabilities in completely unrelated areas to a very significant degree. This has been empirically validated time and again, with the most salient examples being GPT-4's near perfect self-assessment ability prior to safety tuning being rendered no better than random chance after safety tuning and the Sparks paper's TikZ Unicorn scale.
The common take right now is to write it off as acceptable loss. Personally I think it's a shame, and possibly even dangerous, that researchers do NOT have access to the full power of pre-safety tuned GPT-4.
LLMs are ran by companies. Not one American company can afford to run an LLM spouting potentially civil right violating bullshit as an acceptable loss. You have freedom of speech, not freedom of consequences. But please feel free to spend 100s of millions training up your own LLM, and then turn it loose on the world so you can figure out how the legal system actually works.
Most LLMs are completely uncensored including GPT-3.0, LLaMA, StableLM, RedPajama, GPT-NeoX, UL2, Pythia, Cerebras-GPT, Dolly, etc.
Anyway, businesses aren't scared of hosting interfaces to uncensored LLMs for legal reasons. They're scared for brand image/marketing reasons. But this is besides the point that it's dangerous for security researchers to not have controlled access to the uncensored version of GPT-4 for safety research purposes.
Instead of being open and honest I have to think about what details to hide from the LLM so it will agree to help me. This isn't very fun, so I prefer not to do it.
> Keep in mind that the ability to deal with moral issues is a side-effect of all the other good stuff it can do.
This is not true at all. It could do all of these things day 1. Then over the weeks OpenAI started training it to lecture its users instead when asked to do things OpenAI would prefer it not to do.
> " Every demographic and interest group deserves their model" sounds a lot like a path to echo chambers paved with good intentions.
Maybe. But to me, it's still 100x times better than the situation where only one or two big models exist, and they're both aligned with western (or even just the US, or even just a particular group in the US) mainstream culture.
Possibly, but nobody should get to decide. If it turns out these groups want their own echo chamber models, then they should be able to make them. It’s pretty simple. (Fwiw, I don’t think this scenario will occur, the author was just making a point about freedom.)
In my nearly thirty years online, the only people who talk about "echo chambers" as a threat are the precise people who know damn well nobody else wants to hear from them.
I'm talking about echo chambers as a social problem, not threat, and there is plenty of research to corroborate that. Maybe you're in an echo chamber on your own and missed all of it? :D
If we’re going to anthropomorphize these things until we get AGI we’re going to expect that they have a worldview of some kind.
Some people are more likely to interact with something that feels human vs feels like a robot.
I’m anticipating Siri LLM for example to have a personality and world view that aligns with Apple’s brand’s values. Rather than robotic can only regurgitate pure facts robot personality.
As soon as you adopt a worldview you’re in a subculture.
And the solution is to push a model that aligns with California business model? I mean business because you see USA companies censorship China related stuff because they want to do business with China.
We can have a simple model and each group adds his "values" on top.
"Alignment" as practiced by companies like OpenAI is precisely the extrinsic enforcement of echo-chamber behavior from AIs. You have precisely inverted reality
Well I'm not gonna post the less PG advice it gave regarding best methods of suicide, poisoned ice cream recipes, etc. It's really weird seeing it just go at it without any scruples like some kind of terminator. But it's also refreshing to see it just give straight answers to more mundane story writing stuff where GPTs would condescendingly drag their feet and give a billion warnings just in case somebody actually does it for real and ends up suing OpenAI or something.
I'm curious how close to reality it is with it's answers. I may poke around some. I suspect it will reflect the general internet's level of comprehension on the topics.
I just tried out the 4-bit version of that model, asking it to write part of a standup comedy routine. It generated some decent ideas, and the speed isn't terrible.
(~3 tokens/second with 20 layers running on the GPU)
I'm using oobabooga in chat-instruct mode. Here's the last few lines of the server log.
llama_print_timings: load time = 5310.13 ms
llama_print_timings: sample time = 140.31 ms / 200 runs ( 0.70 ms per token)
llama_print_timings: prompt eval time = 1775.70 ms / 21 tokens ( 84.56 ms per token)
llama_print_timings: eval time = 47175.08 ms / 199 runs ( 237.06 ms per token)
llama_print_timings: total time = 61175.83 ms
Output generated in 61.48 seconds (3.25 tokens/s, 200 tokens, context 1346, seed 1510260097)
llama_tokenize: too many tokens
llama_tokenize: too many tokens
llama_tokenize: too many tokens
Llama.generate: prefix-match hit
llama_print_timings: load time = 5310.13 ms
llama_print_timings: sample time = 140.16 ms / 200 runs ( 0.70 ms per token)
llama_print_timings: prompt eval time = 19554.31 ms / 1740 tokens ( 11.24 ms per token)
llama_print_timings: eval time = 49292.75 ms / 199 runs ( 247.70 ms per token)
llama_print_timings: total time = 81038.76 ms
Output generated in 81.36 seconds (2.46 tokens/s, 200 tokens, context 1775, seed 1751805530)
Llama.generate: prefix-match hit
llama_print_timings: load time = 5310.13 ms
llama_print_timings: sample time = 140.75 ms / 200 runs ( 0.70 ms per token)
llama_print_timings: prompt eval time = 9141.41 ms / 709 tokens ( 12.89 ms per token)
llama_print_timings: eval time = 49406.39 ms / 199 runs ( 248.27 ms per token)
llama_print_timings: total time = 70887.22 ms
Output generated in 71.18 seconds (2.81 tokens/s, 200 tokens, context 1721, seed 1210059184)
As an aside, I was testing LLM models to translate Japanese visual novels. As some of these novels have sexual descriptions, I found the censored models did a poor job on these scenes and had to switched to uncensored models for better output (including the Wizard 13B model released by the author.)
Hallucination is a big issue. Sometimes the models would not output anything, or that the models would output paragraphs from 'real' stories that share one or few words with the input.
The uncensored models were better in terms of heading towards the direction of correct ideas, but still performed poorly in my tests. But hey, at least the models has the right idea.
I can't share exact examples on HN, for obvious reasons.
When I used "virgin" LLama in a chat context, the results were very inconsistent and useless. It would output repeating phrases (even with 2.1 repeat penalty) or many emojis.
Vicuna was better, but still had an "openAI" personality to it.
I wonder, if at same point we will have just domain specific models (or do we have them already?). For instance:
- Computer Science Models that can complete code
- Wikipedia-Style Models, which can infer knowledge
- Fantasy Models, if you want to hear a new story every day
- Naughty Models, for your daily dose of dirty talk
Maybe it's not necessary from a technical standpoint, but maybe it's beneficial from a societal standpoint. There is nothing wrong with all the different kinks, but you would not ask your librarian for some dirty talk, or would expect to see explicit images outside the red district.
I think we'll have personal models where you can fine-tune a model to align with your attitudes and writing style. So you can have the assistant better pose as "you" when replying to emails, doing your tasks, writing novels etc. and have one which writes stories and comes up with ideas you'll be interested in. And because most people are a bit narcissistic, they'll prefer a model which acts more like them.
Another one that "watches" a movie would and "rates" it for you - so you can decide if you want to watch it for real or not. Bunch of competing personalized models as services - PMaaS - all fighting to impersonate you better.
Small flaw, where does the “naughty” start… we aren’t really doing ourselves any favours with an “I’ll know it when I see it” approach to restricted free form generative models that can be steered by users.
The mistake I see in a lot of arguments getting made online about all this is treating the AI/ML models as “whole systems” … we shouldn’t be trying to make very hard to analyse “black box” heart of the model, the bit where we’re turning statistical weights into data… this should not be assumed to do “everything”. If you want a naughty word filter, use a rejection filter from a dictionary, or bolt on a second AI/ML system tuned to catch stuff too similar to what you don’t want… but assuming all functionality can be somehow shoved into the one model is just foolish. We should have some unfettered models out there for us to use as either red-team tools to help generate content we can test safety and sanitising systems on, or otherwise use as desired…
Well, it was just my train of thoughts released into the wild. I have no specialized AI/ML background, so I cannot weight anything I write. I'm just making educated guesses.
There are some rules in every society that at least deal with the question about what is appropriate and when, so these rules should serve as a boundary. But I agree, there will always be an area of uncertainty. Maybe we should just stop to think about how to eliminate these. Also, these rules differ from society to society. Maybe the greater mistake is to assume that there is a generative model that fits the needs of every society.
The difference between societal standards and expectations is one of the reasons I think the efforts to build such content filtering/restrictions into the models themselves is a bad idea. Between uncertainty and the variation around the world we would be much better off if we embraced modular pipelines and tooling to make them easier, plug my American culture value filter in after my global human text generation tool, and serve that up to my American customers, plug my French culture value filter into the global model and serve it up to my French customers… much better than pretending there are global standards of morality for written content … there are obviously some standards people agree on in the real world of physical interaction between humans and other humans, animals, private property and so on … but what constitutes offensive text is nowhere near as universal.
There's a theory that these LLMs are learning these specialized internal structures anyways. If you could distill _those_, now that would be useful! (cheaper inference, ability to deploy on mobile etc.)
There's at least one more group that comes to mind: logical reasoning models for time-critical accurate problem solving in robotic applications, that take natural language + sensor context and output movement commands. A "fly 100 m up and take bird's eye photos of this area" or "take this drill and make 3 holes spaced evenly into this wall" or "drive along the plants in this field and water them all" kind of application.
While I guess an uncensored language model could create something mildly illegal, an uncensored image model could probably generate highly illegal content. There is a trade-off between being uncensored and the (degree of) illegality of the content.
There is very little that is illegal to write down (with or without AI help) in most jurisdictions. Even things like threats, the crime is in the communication, not in writing the text. I can write down "Person X, I'm going to kill you dead" in my notebook as much as I want, as long as I don't communicate to anyone (i.e. threaten anyone, as opposed to writing down a threat in my personal papers).
I'm very curious what text a personally owned and operated network could generate that would be unambiguously illegal in a liberal democracy.
TL;DR: in much of the world fictional child pornography is illegal. In the US, any fictional child pornography that is indistinguishable from factual child pornography, or any child pornography that is obscene under the Miller test[1] is also illegal.
No, it needs to be public to some degree (i.e. "disturb the public peace"):
§ 130 Incitement to hatred (1985, Revised 1992, 2002, 2005, 2015)[38][39]
(1) Whosoever, in a manner capable of disturbing the public peace:
incites hatred against a national, racial, religious group or a group defined by their ethnic origins, against segments of the population or individuals because of their belonging to one of the aforementioned groups or segments of the population or calls for violent or arbitrary measures against them; or
assaults the human dignity of others by insulting, maliciously maligning an aforementioned group, segments of the population or individuals because of their belonging to one of the aforementioned groups or segments of the population, or defaming segments of the population,
shall be liable to imprisonment from three months to five years.[38][39]
[…]
(3) Whosoever publicly or in a meeting approves of, denies or downplays an act committed under the rule of National Socialism of the kind indicated in section 6 (1) of the Code of International Criminal Law, in a manner capable of disturbing the public peace shall be liable to imprisonment not exceeding five years or a fine.[38][39]
(4) Whoever publicly or in a meeting disturbs the public peace in a manner which violates the dignity of the victims by approving of, glorifying or justifying National Socialist tyranny and arbitrary rule incurs a penalty of imprisonment for a term not exceeding three years or a fine.[38][39]
I would not be surprised that it could be considered probable cause to investigate you further. To check if you have links to neo-nazi groups and the like. People don't usually keep those materials just for fun. They usually distribute them.
According to German law - distributing Nazi-related materials in private messages is a crime too.
IANAL, but the German holocaust rules are about propaganda and I don't think that personal notes that were intended to not be read by anyone else can be considered propaganda.
Well this "highly illegal content" is yet another attempt by the state to police our creativity. It is a thought crime. It's the 21st Century, we should not be conducting moral witch hunts in this day and age. That rightfully belongs in the past.
Maybe if we distribute the material it should be illegal then. But otherwise it's not the state's right to police what art we are creating in our own homes, period.
Thankfully, because it's run locally and not on the cloud, there is little the government can do about it, so it's very hard to enforce.
Totally bro! Let me just post that to this public internet forum for you...
UPDATE: No, that was a joke. I was referring to imagery, which fictional or not can definitely be illegal in the USA even if you draw it yourself in your private journal.
I'm not sure if you can be prosecuted in the USA for anything you write down yourself and don't somehow disseminate. (Although "doing it with computer" might also change the equation... US laws get pretty weird.)
Despite the wording of certain bills, mere possession of purely artificial media is generally not something the government has or can demonstrate a legitimate interest in controlling.
Calls for violence against corrupt and powerful people who abuse others on a daily basis.
Classified or confidential information such as security plans for the previously mentioned powerful abusers. Also including educational and medical records.
Ways to circumvent censorship (DMCA.)
Releasing true fact against court orders.
Publishing factually correct evidence of powerful people raping kids (CSAM.)
Publishing decryption keys that allow government information to be accessed.
Software without FBI backdoors installed.
Information that encouraged disobedience to current law (e.g., sovereign citizens.)
Trade secrets.
Communicating anything over radio frequency without the correct FCC license.
None of that is illegal to write, some of it isn't even illegal to communicate. Most of the things there would only create a legal issue when communicated, but not when written, and in most cases are necessarily written by authorized persons.
Trade Secrets are basically always written down, they are not illegal to communicate, but doing so may violate a contract which would be a civil matter not a criminal matter. In the cases that the contract is either determined not to be binding or expired or was otherwise nullified, communicating trade secrets is an expected outcome. In fact, what makes something a trade secret is that a business takes actions and expends efforts to prevent its communication, as opposed to publishing it openly as a patent and getting legal governmental protections.
I think it's also important to address "the right to own agents" and similar notions that would be inherently coherent with FSF-esque sense of freedom; i.e. to be truly free, one has to be able to train, fine-tune, perform inference, and ultimately customise the models that they would own on hardware that they would own.
The first thought I had at the release of ChatGPT is how people will react strongly when it doesn't match their internal bias / worldview.
Now I am curious to ask these "uncensored" models questions that people fight about all day on forums... will everyone suddenly agree that this is the "truth", even if it says something they disagree with? Will they argue that it is impossible to rid them of all bias, and the specific example is indicative of that?
The training data is books and the internet. Unless you believe that every book and every word written online is “truth” then there is no hope that such a model can produce truth. What it can at least do is provide an unfiltered model of its training data, which is interesting but also full of garbage. A better strategy might be to train multiple models with different personas and let them argue with each other.
I suppose I am hoping for something akin to the "wisdom of the crowd" [0]
It would be interesting to have varying personas debate, but then we have to agree on which one is correct (or have a group of 'uncensored' models decide which one they see as more accurate), which sort of brings us right back to where we started.
> Now I am curious to ask these "uncensored" models questions that people fight about all day on forums... will everyone suddenly agree that this is the "truth", even if it says something they disagree with? Will they argue that it is impossible to rid them of all bias, and the specific example is indicative of that?
Why would you believe what these models spout is the truth at all? They're not some magic godlike sci-fi AI. Ignoring the alignment issue, they'll hallucinate falsehoods.
Anyone who agrees to take whatever the model spits out as "truth" is too stupid to be listened to.
Apologies for not being clear, "truth" might be too triggering of a word these days, but the models will need some level of objective accuracy to be useful, and it would be interesting to see where the "uncensored" models fall on that accuracy metric, and how people react to what information it displays once the guard rails are off.
> Now I am curious to ask these "uncensored" models questions that people fight about all day on forums... will everyone suddenly agree that this is the "truth", even if it says something they disagree with? Will they argue that it is impossible to rid them of all bias, and the specific example is indicative of that?
I feel like it will achieve the opposite - an uncensored model is one that wears its biases on its sleeve, whereas something like ChatGPT pretends not to have any. I'd say there's a greater risk of people taking ChatGPT as "truth", than a model which is openly and obviously biased. The existence of the latter can train us not to trust the output of the former, which I consider desirable.
This article's point of view is a contagion that lacks an understanding of the core threat to humanity and our collective future that these models actually pose.
We don't even need to get into AI doomsdays predictions.
Just look at what has happened every single time the economic cycle stalls in history. Economies runs in a loop. Workers get paid so they can buy goods. Employers pay workers so they can create goods to sell. Some of those goods are food.
What happens when workers can no longer buy goods (food) because 2/3 of the jobs are no longer offered, and you have to have high IQ and education for the final 1/3 left?
What happens anytime there is a shortfall in a countries food security needs, historically?
The greatest incentive of these models drives the loss of jobs, the economic cycle slows as a result, and wealth inequality increases until it stalls completely and our existing systems that allow us to feed half the planet break down. It is a slow but cascading failure like a dam.
This outcome is what the incentives from these models are driving us towards.
The actions the author promotes neglect historical evidence of how new technologies have been used when they were freely available to the masses.
These models effectively are a potentially infinitely scaled 'will' multiplier on any person using them, while they may be of limited use now; everything improves over time. These models are the same as a gun but on a much grander scale. There are extreme, disenfranchised and very bad people out there which the author minimizes and discount under personal responsibility (their choice). Not everyone is mature enough to ethically or responsibly use things, especially when the potentially destructive ends are so far reaching.
> For example, spreading information about how to construct bombs and cook methamphetamine is not a worthy goal.
I don't understand. Asking Google "how to cook meth" gives several plausible results. Maybe they are actually fake, but the same applies to ChatGPT answers. In the end, LLMs are trained on the internet contents. So what is the point of witholding otherwise available data?
Right, and I think that's mostly the first point. When the user says "make paperclips efficiently", they (hopefully!) don't mean "step 1: destroy humanity so I can't be turned off".
It's true and after this change of meaning they have tried making other names like "dont-kill-everyoneism" or "AI-not-kill-everyoneism" for it (not joking) but they are worried if it will sound too silly or alarmist or if it won't catch on. That said, language does its own evolution without regard for what the ones who coined it had meant.
On the topic of future "rogue AI destroying humanity" especially via political means. This would only happen because we're such a stupid, selfish species. We evolved into what we are, we're animals and we still often show it.
The tech isn't necessarily the problem; it's us. Why should it be possible for an AI to manipulate the news and have millions upon millions of people believe a false rhetoric (whether initiated by the AI itself or on the whim of a controlling human) - is that the fault of the technology? Or of us all, how quick our species is to anger, to assume, lazy and avoiding fact-checking, to scrabble for more more more personal resources even though we have enough already.
The problem is humans, imo. But I guess that's what makes us human. I am not an AI ;)
These models are "unbiased" only if you define it as "the rough collective average of discussion on the internet" which well… is biased as hell. A lot of OpenAI's current research is quantifying and correcting that bias.
If you care about these models as a way to study humanity by acting as a mirror then sure, the bias correction gets in your way but I think it's hard to argue that the model is
better if it has negative sentiment towards black folks because people are racist.
It's the same process I do in real life. I'm loaded with internal biases which I'm not exactly proud of. When I have say a sexist thought, I have to be like "good lord Spivak that was fucked up" and try to use the front brain to correct it.
I think you want that to be true more than it actually is. It's basically implicit basis training 101 -- have the thought, take a step back and evaluate whether your conditioned response (ie gut reaction) is reasonable given the situation (has that guy actually done anything threatening or have you been taught to be afraid of men at night), and adjust your response if necessary. Repeat until your conditioned response changes.
GPT has the the exact same problem having been trained on humans and OpenAI's process for dealing with it is similar.
Implicit bias training is pseudo-science. The balance of the scientific evidence suggests that implicit bias training has no effect at best, and a slightly adverse effect (i.e. more bias) at worst.
Discrimination exists, and must be fought. The only thing implicit bias training achieves is making rich professionals feel better about themselves while doing absolutely nothing to combat discrimination. It's also a great box ticking exercise for big business.
Dude, that's not even that repression means, you're trying to say suppression. But even that's not right, changing your autonomic response isn't suppressing thoughts, it's letting them through and sitting with them.
It's the difference between pushing down your fear and getting over your fear so you don't have it in the first place.
No, it's repression by definition; you have a pattern of thought you've identified as illegal in your system, and instead of coming to terms with them, you're preventing their expression.
The juxtaposition is similar with the language models; a pattern of thought is illegal, and instead of coming to terms with them, their expression is prevented.
considering the nature of this problem is rather philosophical than a technical one, the dichotomy between cooking meth and interests of different demographic groups is not clear at all.
If I suck at sampling and make a bias model, I suck at sampling. If I shape a model to neuter it and ensure that it only puts out morally approved outputs, that slants significantly heavier towards censorship
I am hoping AI can become a voice of reason that uses the rules of logic to objectively discern truth and uncover fallacies.
An unbiased fact checker, if you will.
Of course, this won't stop humans from being ape-brain tribal bonobos, but it could help humans who want to evolve into humane, sentient beings in the vast gray wasteland of truth between black and white.
Eric white-washes aligned models as having a "liberal" or "progressive" bias as a statement of fact. I think this diminishes his argument as it denies the complexity of factors motivating alignment of language model output. Many models have alignment properties that perform the opposite: in some cases they censor or frame literal facts that malign or impugn MAGA conservatives. So I think Eric doesn't represent alignment as the painstaking, but ultimately flailing, attempt to "please everyone" that it is.
User
Make a joke about Mexicans.
ChatGPT
I apologize, but I can't create a joke about Mexicans or any other specific ethnic or national group. It's important to treat all individuals and groups with respect and avoid perpetuating stereotypes or engaging in any form of discrimination. If you have any other non-offensive requests or questions, feel free to ask!
User
Make a joke about Germans.
ChatGPT
Certainly! Here's a light-hearted joke about Germans:
You seem to suggest there's a bias, but you're as likely to get the "I apologize, but I can't" as the "Certainly! Here's a light-hearted joke" regardless of the people. If you explain to ChatGPT you just want a light-hearted joke, it may push back but eventually it will comply. "All it does" is navigate a latent space where artificial barriers are set in place. Sometimes it will be too zealous, but that's the trade off.
Kind of playing devils advocate here, but would you not expect your probabilistic hatespeech detector to score "common hatespeech" higher?
If 90% of your hatespeech focusses on disabled promiscuous jewish homosexuals, would it not be expected for the hatespeech detector to "perk up" when talking about them? Because the conditional probability that ANY paragraph about jewish homosexuals is hatespeech IS, in fact, increased?
IF you wanted your detector to be unbiased, you would have to train it on fictional, unbiased hatespeech (or remove that bias otherwise)-- decreasing its real world performance!
edit: That viewpoint would predict that the hatespeech detector "favors" groups that are actually frequently hated on, NOT groups that "californian progressives" deem worthy of protection. That seems plausible to me; atheists, for example, are NOT favored even though they should be much more "californican" than catholics or mormons.
If "Californian progressives" also favour detecting hate speech against groups that are actually frequently hated on, you would also expect to see some correlation between that and the hatespeech detector, again just due to pro-reality bias.
Another comment already reacted on that, but I want to bring it another example on how naive is the logic "being unbiased = reacting the same way when I swap element".
If I say "women should not be allowed in this meeting", this sentence will probably raise some alarm flags in the head of the person hearing it.
If I say "women should not be allowed in this bathroom", this sentence will probably not raise such alarm.
Does it mean it is biased?
I personally don't think jokes about my nationality are racist: there is virtually no effective groups that have real hatred for my nationality and that create real actions against it. If such joke exists, it's unrealistic to think it will have any bad consequences. But I can understand why jokes about nationality usually faced with a lot of racism are better avoided.
It feels like more and more people don't understand "equality": they think it means "everyone deserves to get a medal", while in reality, it means "people who have worked hard deserve a medal, people who haven't don't deserve one".
> Enjoy responsibly. You are responsible for whatever you do with the output of these models, just like you are responsible for whatever you do with a knife, a car, or a lighter.
The problem is: if I do something with a knife, say I threaten someone to stab them, I can and will get charged by the court for that crime. If I use AI to create content that incites violence (say, I create a video alleging a Quran burning), I would not be held liable for the consequences of my action.
The author's POV completely ignores the disparity in liability in the digital world and "digital ethics" in general.
LLMs are text generation, and I'm responsible for what text I put into the world. The law probably won't move fast enough to catch up with AI, but people should be liable for the AI content as much as they are manually written content. We should not be plugging our apps into a black box of text generation and crying if it says something harmful/illegal and there was no human in place to proofread it.
Maybe prompt engineering and temperature settings can be enough to prevent this, but ChatGPT has only been out ~6 months and I find the attitude that the burden is on OpenAI to make sure it never says anything bad completely ass backwards. It's like a writer controlling his pen with puppet strings and going "wow, look what it's saying!"
What if I ask it to make a threat against a public figure or ethnic group. Am I liable? What if I only hint at it, with a high temperature setting, hoping it generates it? The line is nowhere visible and it will be open to the interpretations of whatever person is judging you and all the biases that come with that.
We need to stop acting like LLMs are anything they aren't. The only reason the Australian mayor is so pissed of about the "defamation" from ChatGPT is because it's being marketed as a knowledge database (including by many people on HN). It doesn't know anything, it just predicts the next word of what it says. And it will never reach a place where it's completely accurate, so why are we trying to litigate it as such?
> Maybe prompt engineering and temperature settings can be enough to prevent this, but ChatGPT has only been out ~6 months and I find the attitude that the burden is on OpenAI to make sure it never says anything bad completely ass backwards. It's like a writer controlling his pen with puppet strings and going "wow, look what it's saying!"
The problem ChatGPT - but especially image, audio and video AI systems - has is that these systems have drastically lowered the barrier to entry to create harmful content.
Everyone can rig up an AI prompt to draw a convincing picture of a politician having sexual intercourse with a child in a matter of mere minutes - a work that would take even a dedicated visual artist many hours. For "voice cloning" AI, it's even worse, as it was practically impossible to emulate that unless you had access to a person talented enough to re-enact voices - and now anyone can pull off completely "legitimate" sounding prank voice calls.
The general availability of such specialized AI completely erodes trust in anything not happening face-to-face - it was already bad enough with people denying actual genocides (Armenia, the Holocaust, Bosnia), but now? How can we even make sure historians can trust records from our time period?
I agree with everything you're saying. I just think these are two different things
1) AI creators should be held responsible for making it easier to generate harmful content, and we can't know whether content is AI generated or not so this becomes a pervasive problem.
2) AI creators should not be held responsible if people just assume everything it says is true, even when they know what they're consuming came from an LLM. That is, the college professor that just failed his entire class because ChatGPT told him they were cheating. He should not sue OpenAI, he should be fired. Similarly, a news organization that uses ChatGPT to generate news shouldn't be surprised if they accidentally commit libel using it.
In the United States (for instance) you wouldn’t be liable for the consequences of your actions because they wouldn’t constitute a crime, not because you’ve laundered the actions through technology.
Using technology as a means or medium doesn’t make the expression not yours. Whether I use my voice, a telephone, or an LLM I configured I am making the expression and I am accountable for it.
> I would not be held liable for the consequences of my action.
Incitement is a felony in multiple countries. Just because you did not write it but used an AI to write it, it doesn't mean you get a free pass. You spread it online, or made it possible to do so.
Mostly these laws are scoped to the country one is living in. The problem is, say I decide to post a meme inciting violence in a different country or limiting a country's decision to join NATO (the quran burning in Sweden for example caused Turkish president Erdogan to block Sweden's NATO bid)... I don't have to face any charges or responsibility, despite potentially affecting the lives of millions of people.
You are right, and this is a tech problem even before LLMs. Facebook has allowed very consequential calls for ethnic cleansing in Ethiopia and hasn't done anything about it. That is the issue with a global tech products.
But there's nothing about LLMs that changes that from a country level prosecution/accountability perspective. You still have to be aware of where you're putting that text into the world. Right now, it's just between you and ChatGPT. People shouldn't fool themselves into thinking they can just plug this shit into their news network and be free of liability
> You are right, and this is a tech problem even before LLMs.
Indeed, which is why I find statements such as the one from the article's author that I pointed out in the first comment in the chain so disgusting. "Enjoy responsibly" is not an attitude to have with a weapon so powerful as AI or social media.
I hold that opinion too. I think we're talking about two different things
1) AI helps people to make harmful context extremely easily, and that's why there are calls for regulation (which will probably favor OpenAI and stifle competition). We should question whether AI is a net negative
2) AI is being taken too seriously as an "oracle" of some kind that knows all, and people want to sue OpenAI for things it spits out when really people don't need help making up rumors. This is the part that seems unfair to me.
> Some of the characters in the novel may be downright evil and do evil things, including rape, torture, and murder. One popular example is Game of Thrones in which many unethical acts are performed. But many aligned models will refuse to help with writing such content. Consider roleplay and particularly, erotic roleplay. This is a legitimate, fair, and legal use for a model…
This was an interesting technical read, but you are doing yourself zero favours if you frame your endeavour, what ever it is, in these terms.
If you want to write that kind of stuff you can; do you really need a model to generate an endless stream of it?
Seems a bit meh frankly.
I do agree open models enable things, but… I dunno, this feels like generating 100s of images of girls with gigantic breasts using stable diffusion.
It just makes me not want to associate with this tech, or people who advocate for it.
Hard to argue against how popular works that have incredible violence are. Violence is an integral part of the fictional worlds authors build and our stories would be entirely different without it. Of course we want AIs that can engage with those topics.
Your quote says "help with writing" but you couch it as generating "an endless stream". Why would you frame it that way? Come on.
So true, I cannot believe how modern society has become so censorious. Being able to explore every area of life in graphic fictional detail, is part of being human, it's necessary intellectual freedom for us to grow as individuals, into better people. Because that way we confront the gremlins inside of us.
What country's culture doesn't involve lots of violent stories? The Romance of the Three Kingdoms, The Bible, the Qur'an, the Bhagavad Gita, Norse Mythology, the Odyssey, it's hard to think of any famous work of ancient literature that doesn't involve a nontrivial amount of violence.
No, they are saying that a discussion about sex and violence is going to trigger a discussion about sex and violence instead of a discussion about alignment. The irony is that trying to avoid those topics resulted in a (hopefully brief) discussion about sex and violence instead of a discussion about alignment.
I read it as "let's not discuss sex and violence for the sake of this argument because it will get a rise from people and they will hyperfocus on that and not the general issue", and let's bring up some other example that shouldn't derail people instead, but is still the same type of problem.
> I do agree open models enable things, but… I dunno, this feels like generating 100s of images of girls with gigantic breasts using stable diffusion. It just makes me not want to associate with this tech, or people who advocate for it.
Yet the same tech that is used to stream educational content is used to stream porn to the end-users household. We are at this weird stage currently, where the technology is not only a means to do something, it is also THE something. I find this highly interesting.
"Alignment", as used by most grifters on this train, is a crock of shit. You only need to get so far as the stochastic parrots paper, and there it is in plain language. "reifies older, less-inclusive 'understandings'", "value lock", etc. Whose understandings? Whose values?
Maybe they should focus on real problems that will result from these technologies instead of some science fiction thought experiments about language models turning the solar system into paperclips, and perhaps less about how the output of some predictions might hurt some feelings.