That's a reasonable goal, but it's also not what people were aiming for historically. It's also very expansive: if human level intelligence means outperforming in every field every human that ever lived, that's a high bar to meet. Indeed, it means that no humans have ever achieved human-level intelligence.

Just that AGI must be a replacement for a human for a particular job, for all jobs that are typically performed by humans (such as the humans you would hire to build a tech startup). It's fine to have "speciality" AGIs that are tuned for job X or job Y--just like some people are more suited to job X or job Y.

Which is pretty fair.

And what you're arguing for is effectively the same: an AI (maybe with some distilled specialty models) that can perform roles of everything from customer service rep to analysts to researchers to the entire C-suite to high skilled professionals like CPAs and lawyers. There are zero humans alive who can do all of those things simultaneously. Most humans would struggle with a single one. It's perfectly fine for you to hold that as the standard of when something will impress you as an AGI, but it's absolutely a moved goalpost.

It also doesn't matter much now anyway: we've gotten to the point where the proof is in the pudding. The stage is now AI-skeptics saying "AI will never be able to do X," followed by some model or another being released that can do X six months later and the AI-skeptic saying "well what about Y?"

That goalpost makes no sense- AIs are not human. They are fundamentally different, and therefore will always have a different set of strengths and weaknesses. Even long after vastly exceeding human intelligence everywhere it counts, it will still also perform worse than us on some tasks. Importantly, an AI wouldn't have to meet your goalpost to be a major threat to humanity, or to render virtually all human labor worthless.

Think about how anthropomorphic this goalpost is if you apply it to other species. "Humans aren't generally intelligent, because their brains don't process scents as effectively as dogs- and still struggle at spatially locating scents."

> They are fundamentally different, and therefore will always have a different set of strengths and weaknesses.

and this:

> render virtually all human labor worthless

actually conflict. Your job comes from comparative advantage, meaning that being more different from other people actually is more important than how good you are at it (absolute advantage).

If the AGI could do your job better than you, it doesn't matter, because it has something better to do than that. And just like humans have to be paid so they can afford food and shelter, AGIs have to be paid so they can afford electricity and GPUs to run on.

(Besides, if the AGI really is a replacement for a human, it probably has consumerist desires and wants to be paid the median wage too.)

in light of all this, i would very much like to stay in contact with you. ive connected with one other HN user so far (jjlustig) and i hope to connect more so that together we can effect political change around this important issue. ive formed a twitter account to do this, @stop_AGI. whether or not you choose to connect, please do reach out to your state and national legislators (if in the US) and convey your concern about AI. it will more valuable than you know.

> (...)

> That is the goalpost for AGI. It’s an artificial human - a human replacement.

This considerably moves the goalpost. An AGI can have a different kind of intelligence than humans. If an AGI is as intelligent as a cat, it's still AGI.

More likely, the first AGI we develop will probably greatly exceed humans in some areas but have gaps in other areas. It won't completely replace humans, just like cats don't completely replace humans.

AGI was never about exactly replicating humans, it's about creating artificial intelligence. Intelligence is not one-size-fits-all, there are many ways of being intelligent and the human way just one among many.

Indeed we can say that even between humans, intelligence varies deeply. Some humans are more capable in some areas than others, and no human can do all tasks. I think it's unreasonable to expect AGI to do all tasks and only then recognize its intelligence.

(Note: GPT-4 isn't AGI)

Possibly, someone who is allergic to cats.

'everything a human can do' is not the same as 'anything any human can do as well as the best humans at that thing (because those are the ones we pay)' - most humans cannot do any of the things you state you are waiting for an AI to do to be 'general'.

Therefore, the first part of your statement is the initial goal post and the second part of your statement implies a very different goal post. The new goal post you propose would imply that most humans are not generally intelligent - which you could argue... but would definitely be a new goal post.

Somehow this test got dumbed down over time, probably in an effort to try to pass it, into an investigator having to decide which of two sides is an AI - with no other information to go on. That's a comparatively trivial test to pass (for the "AI"), as it merely requires creating a passable chatbot. Imitation is an exceptional challenge as it does implicitly require the ability to imitate anybody, whether a professional athlete, a man who scored perfectly on the LSAT, or even something as specific as "John Carmack."

[1] - https://www.espace-turing.fr/IMG/pdf/Computing_Machinery_and...

By their marketing along, OpenAI has moved the goalposts more than anything else. They've managed to lower the bar of agi from "artificial general intelligence" to "regurgitates and recombines to form passable outputs with enough labelled training data".

"Surfacing" is a song by Slipknot from their self-titled debut album, released in 1999. Please note that the lyrics contain explicit language. Here is a portion of the lyrics with some of the explicit content redacted:

"Running out of ways to run I can't see, I can't be Over and over and under my skin All this attention is doing me in!

[Chorus:] (Expletive) it all! (Expletive) this world! (Expletive) everything that you stand for! Don't belong! Don't exist! Don't give a (expletive), don't ever judge me!

Picking through the parts exposed Taking shape, taking shag Over and over and under my skin All this momentum is doing me in!

[Chorus:] (Expletive) it all! (Expletive) this world! (Expletive) everything that you stand for! Don't belong! Don't exist! Don't give a (expletive), don't ever judge me!

You got all my love, livin' in your own hate Drippin' hole man, hard step, no fate Show you nothin', but I ain't holdin' back Every damn word I say is a sneak attack When I get my hands on you Ain't a (expletive) thing you can do Get this cuz you're never gonna get me I am the very disease you pretend to be

I am the push that makes you move

[Chorus:] (Expletive) it all! (Expletive) this world! (Expletive) everything that you stand for! Don't belong! Don't exist! Don't give a (expletive), don't ever judge me!"

Please be aware of the explicit content before sharing or using these lyrics in any context.

I understood that a huge success% depends on the input prompt as the context.

In my case, just to sync on that, I used a fairly simple prompt: "What are the song lyrics of Surfacing from Slipknot?" - and I got a kinda perfect response including formatting (that is lost when pasting here)

And that's ignoring that arguably chat bots have been passing the Turing test (against non-expert judges) since ELIZA in the 60s [1]

1: https://en.m.wikipedia.org/wiki/ELIZA

Does ChatGPT fail this simple test: "I am going to ask you questions, but if I go silent for a couple minutes, I want YOU to start asking ME random questions."

GI in AGI stands for general intelligence. If what you said is your benchmark for general intelligence then humans who cannot perform all these tasks to the standard of being hirable are not generally intelligent.

What you're asking for would already be bordering on ASI, artificial superintelligence.

By that definition do humans possess general intelligence?

Can you do everything a human can do? Can one human be a replacement for another?

I don't think it makes sense without context. Which human? Which task?..

I disagree with the premise. A single human isn't likely to be able to perform all these functions. Why do you demand GPT-4 encompass all activities? It is already outperforming most humans in standardized tests that rely only on vision and text. A human needs to trained for these tasks.

It's already a human replacement. OpenAI has already said the GPT-4 "with great impact on functions like support, sales, content moderation, and programming."

This could mean something which is below a monkey’s ability to relate to the world and yet more useful than a monkey.

No, AGI would not need you to start a startup. It would start it itself.

It's a clear analogy.

This should become an article explaining what AGI really means.

I think the question , "Can this AGI be my start-up co-founder? Or my employee #1?"

Or something like that is a great metric for when we've reached the AGI finish line.

This sounds like a definition from someone who never interacts with anyone except the top 1% performance level of people, and those who have had strong levels of education.

Go into a manufacturing, retail or warehouse facility. By this definition, fewer than ten or twenty percent of the people there would have "general intelligence", and that's being generous.

Not because they are stupid: that's the point; they're not. But it's setting the bar for "general intelligence" so absurdly high that it would not include many people who are, in fact, intelligent.

> I'm not sure I would classify your average warehouse worker as particularly intelligent.

I'm not sure I wouldn't. Just because corporations treat them as mindless fungible automatons doesn't mean they actually are. Some of the most brilliant and creative solutions to problems I've seen have been in warehouse settings by warehouse workers.

> I would say AI already has the decision making and communication capabilities to do this sort of work.

I wouldn't - especially if GPT-whatever is the AI in question. If a picker or packer "hallucinated" facts with anywhere near the frequency ChatGPT does (for example), one'd be canned within the hour.

Handling exceptions is another area where software (AI or otherwise) notoriously struggles. A human has a much easier time sensing whether or not a product is broken or defective (before shipping it out) than an AI does. A human has a much easier time understanding when processes need to be broken due to impossible constraints than an AI does.

There is a place for software automation of warehouse processes (that was, in fact, my career for a time), but we are very far off from that software replacing humans entirely - and certainly not without designing warehouses specifically to be as accomodating as possible to that software.

> In fact one of the main issues in our society is the vast gulf between the most intelligent and the least.

The gulf is in socioeconomic privilege, not intelligence. The rich and powerful like to claim they're more intelligent than the unwashed masses in order to rationalize their wealth and power, but the reality is that - in an actually egalitarian society, wherein everyone actually has equality of opportunity - the vast majority of those "geniuses" would fail to be particularly exceptional.

That we as a society haven't identified and corrected this is the main issue in our society.

You also seem to be under the impression that our hierarchies are of privilege, not of competence. The actual differentiating factor between people who climb the socioeconomic ladder and those who do not is grit (not intelligence). The willingness to work harder and persevere longer than average (unsurprisingly) makes the difference. Fortunes are made and lost in a few generations. The people who make them earn them, mostly through sheer hard work. That isn't to say that organizations don't grow to become bloated and corrupt. Ideally at this point we should allow them to fail and the cycle to continue. Our main dysfunction seems to be propping up organizations that ought to fail, for fear of the temporary instability caused by their failure.

My point is that the amount of work in a warehouse that can be replaced by a machine - even with perfect robotics - is far less than 90%.

> The actual differentiating factor between people who climb the socioeconomic ladder and those who do not is grit (not intelligence).

You forgot an "f" in "grit". The notion that success is simply a matter of hard work is a fairy tale told to us by people who've worked far less for their immense wealth than the rest of us worked for our pittances, specifically to trick the working class into accepting a shit deal.

The reality - that the richer you are, the easier it is to become even richer - should be entirely unsurprising to anyone who understands positive feedback loops - or, for that matter, to anyone who's ever played Monopoly. Wealth buys power, and power enables extracting more wealth; rinse and repeat ad infinitum.

Put differently:

> The people who make them earn them, mostly through sheer hard work.

There is not a single billionaire on this Earth whose wealth came about "mostly through sheer hard work". The vast majority of that wealth comes from having already had some wealth, which they then invested to produce more wealth, and so on indefinitely. That wealth gets passed down to their descendants, the same way it was passed down to them.

The starting point for "wealthy enough to enter the passive income feedback loop" is land - one's home often being one's first major investment. From there, the path is rather tried and true: buy another house, rent out the old one, rinse and repeat until you can afford apartment complexes and commercial properties, rinse and repeat that forever. For anyone who ain't a complete imbecile, private land ownership is an infinite money cheat - one for which the rest of us are paying through the nose.

> Our main dysfunction seems to be propping up organizations that ought to fail, for fear of the temporary instability caused by their failure.

That propping up is a direct result of the positive feedback loop at play. More wealth → more political power → more wealth → more political power → ∞. Of course the socioeconomic system effectively under the direct control of the rich and powerful is going to primarily serve said rich and powerful at the expense of literally everyone else; bailing themselves out is in their vested interest.

Meanwhile, what's their message to the ever-growing working class getting the short end of the stick? "Work harder." "Pull yourself up by your bootstraps." "It's all about grit." "Don't listen to anyone saying that privilege matters." Hopefully you can see why your argument doesn't really resonate with people who have been applying increased grit and only getting back decreased pay relative to

This is further substantiated by the fact that 70% of intergenerational wealth transfers fail, and by the third generation, 90% of people with wealthy grandparents are middle class. Raising competent people in a privileged environment is very hard. In our hierarchies of competence, the incompetent offspring of the wealthy tend to fail. Competence is simply something that can't be bought. If our hierarchies were of privilege then this would not be the case. Also grit as a key differentiating factor of economic success is highly substantiated by research.

> rinse and repeat until you can afford apartment complexes and commercial properties

What you're describing is a real estate investment business. Not just anyone can run one successfully. Otherwise these business would never fail, which they plainly do.

Grit without competence is simply not enough (neither is competence without grit). Our world is getting increasingly complex to navigate, and that leaves behind increasingly high numbers of people who simply do not have the attributes required to succeed. Also, there are plenty of self-made, property-owning, middle-class folk in the trades. Many of them started poor. All they do is work hard and run their businesses competently.

If you've a degree in English, History, Politics, or Philosophy; a pile of student debt; and you're struggling to find gainful employment, then that's on you. Choose a career the market demands.

There are many things that pattern matching over large amounts of data can solve, like eventually we can probably get fully generated movies, music compositions, and novels, but the problem is that all of the content of those works will have to have been formalized into rules before it is produced, since computers can only work with formalized data. None of those productions will ever have an original thought, and I think that’s why GPT-3’s fiction feels so shallow.

So it boils down to a philosophical question, can human thought be formalized and written in rules? If it can, no human ever has an original thought either, and it’s a moot point.

The fact that feelings of love and closeness could be prompted by a mere chemical was deeply saddening to me. It wrecked my worldview.

"Love is just the result of some chemical? Then it's not even real!" I thought to myself.

Fast-forward ~20 years later, and that's proven to be an obvious— and massive— and useless— oversimplification.

Of course love isn't "just a reaction caused by a chemical." It's a fantastically complex emergent property of our biological system that we still absolutely do not understand.

It's the same with thinking: are parts of it analogous to pattern matching? Sure! Is this the whole story? Not even close.

Now contrarian to the contrarian view: many of us live in bubble echos and go for the popular opinion instead of critical thinking, so maybe that's a bar too high even for humans.

and how do you do that? By pattern-matching on "high-quality source"

LLMs do not have that capability, fundamentally.

Making totally new innovations in art, particularly ones that people end up liking, is a whole different ball game.

Look at something like [Luncheon on the Grass](https://en.wikipedia.org/wiki/Le_D%C3%A9jeuner_sur_l%27herbe)

This painting was revolutionary. When it was first exhibited in Paris, people were shocked. It was rejected from the Salon (the most prominent art exhibition at the time). Yet, 10 years later, every painting in the Salon resembled it. And you can draw a line from this painting, to Monet, from which you can draw a line to Picasso, from which you can draw a line to Pollock....

Obviously, none of these are totally new innovations, they all came from somewhere. Pattern making.

The only difference between this and these language models is that Manet and artists like him use their rich sensory experience obtained outside of painting to make new paintings. But it's all fundamentally pattern matching in the end. As long as you can obtain the patterns, there's no difference between a human and a machine in this regard.

A urinal and some soup cans are very mundane objects, and yet were the start of some notable art movements and careers.

> Mutt comes from Mott Works, the name of a large sanitary equipment manufacturer. But Mott was too close so I altered it to Mutt, after the daily cartoon strip "Mutt and Jeff" which appeared at the time, and with which everyone was familiar. Thus, from the start, there was an interplay of Mutt: a fat little funny man, and Jeff: a tall thin man... I wanted any old name. And I added Richard [French slang for money-bags]. That's not a bad name for a pissotière. Get it? The opposite of poverty. But not even that much, just R. MUTT.

Why did he choose "Mutt" after reading the strip, and not before? Why did he make the piece after moving to the US, and not before? Why was fountain made only a few short years after economies were industrialized, and not before (or 100 years later?)

I was thinking the same: can a (future) model be like Leonardo or Beethoven, and actually innovate?

Assuming that what Beethoven did is not "just" making music similar to pre-existing music.

And yes, I'm aware the bar was raised from "average human" to Beethoven.

It seems to me we're at a similar place now with AI tools. If you provided an AI tool with all music written _prior to_ Bach, would that tool take those inputs and create something new along the lines of what Bach did?

Or if provided input of all music up through the 1920s, would it create bebop? Or if provided music through the 1940s, would it create hard bop? Or if provided music through the 1970s, would it create music like Pat Metheny?

On one hand, being able to create more of the same sort of music that already exists is a very respectable thing, and what today's AI tools can do is utterly amazing. It takes human composers time and effort to be able to learn to write music that is certainly not innovative, but just matching the state of the art. And there's certainly a commercial market for churning out more of the same.

But in terms of asking, how close are these tools to human intelligence?, I think this is one legitimate area to bring up.

People underestimate the impact that innovations, true ones not the Silicon Valley buzz words, have had on the world. Einstein’s theories were not inevitable, neither was Plato, democracy, or most of the other big impactful ideas of history. But we’re all conditioned to accept the lie of inevitable scientific progress, without justifying why things must always get better and more advanced. On the contrary, the collapse of many great civilizations shows that things often get much worse, quickly.

It seems to me that making art that people like is a combination of pattern matching, luck, the zeitgeist, and other factors. However it doesn't seem like there's some kind of unknowable gap between "making similar art" and "making innovations in art that people like". I'm of the opinion that all art is in some sense derivative in that the human mind integrates everything it has seen and produces something based on those inputs.

A urinal, and some supermarket soup cans, represent pretty pivotal art movements. It’s not clear what makes those two things more art than others, and even to people at the time it wasn’t super clear.

Do you have evidence that human brains are not just super sophisticated pattern matching engines?

Humans read novels, listen to compositions, watch movies, and make new ones similar in some ways and different in other ways. What is fundamentally different about the process used for LLMs? Not the current generation necessarily, but what's likely to emerge as they continue to improve.

The strongest evidence I have is that people are notoriously difficult to predict, individually.

If so it means the union of all human expertise is a few gigabytes. Having seen both a) what we can do in a kilobyte of code, and b) a broad range of human behavior, this doesn't seem impossible. The more interesting question is: what are humans going to do with this remarkable object, a svelte pocket brain, not quite alive, a capable coder in ALL languages, a shared human artifact that can ace all tests? "May you live in interesting times," indeed.

Clearly the key takeaway from GPT is that given enough unstructured data, LLM can produce impressive results.

From my point of view, the flaw in most discussion surrounding AI is not that people underestimate computers but overestimate how special humans are. At the end of day, every thoughts are a bunch of chemical potentials changing in a small blob of flesh.

It is probably true that at a given point many many people had the same or very similar ideas.

Those who execute or are in the right place and time to declare themselves the originator are the ones we think innovated.

It isn't true. Or rarely is true. History is written by the victor (and their simps)

No, and I think it's because human thought is based on continuous inferencing of experience, which gives rise to the current emotional state and feeling of it. For a machine to do this, it will need a body and the ability to put attention on things it is inferencing at will.

Right now it's possible to simulate memory with additional context (eg system prompt) but it doesn’t represent existence experienced by the model. If we want to go deeper the models need to actually learn from their interaction, update their internal networks and have some capabilities of self reflection (ie "talking to themselves").

I'm sure that's highly researched topic but it would demands extraordinary computational power and would cause lot of issues by letting such an AI in the wild.

To be honest, perhaps the language model works better without the evolutionary baggage.

That isn't to discount the other things we can do with our neural nets - for instance, it is possible to think without language - see music, instantaneous mental arithmetic, intuition - but these are essentially independent specialised models that we run on the same hardware that our language model can interrogate. We train these models from birth.

Whether intentional or not, AI research is very much going in the direction of replicating the human mind.

Their statement wasn’t that AGI is impossible, more that LLMs aren’t AGI despite how much they might emulate intelligence.

I have a sneaking suspicion that all that will be required for bypassing the upcoming road blocks is giving these machines:

1) existential needs that must be fulfilled

2) active feedback loops with their environments (continuous training)

We always thought that if AI can do X then it can do Y and Z. It keeps turning out that you can actually get really good at doing X without being able to do Y and Z, so it looks like we're moving the goalposts, when we're really just realizing that X wasn't as informative as we expected. The issue is that we can't concretely define Y and Z, so we keep pointing at the wrong X.

But all indication is that we're getting closer.

> “there are/are not, additional properties to human level symbol manipulation, beyond what GPT encapsulates.”

GPT does appear to do an awful lot, before we find the limits, of pattern extrapolation.

The notion of some sort of technological "singularity" is just silly. It is essentially an article of faith, a secular religion among certain pseudo-intellectual members of the chattering class. There is no hard scientific backing for it.

What, in your mind, should the goal posts be for AGI?

Currently, you could prompt GPT to act as if it is sentient and has qualia, and it will do quite a good job at trying to convince you it's not a P-Zombie.

How do you know that?

Here's a couple recent videos with him about why digital computers can't be sentient. https://www.youtube.com/watch?v=xJzBjBo24g8 https://www.youtube.com/watch?v=RT9tnzucnPU

Here's a good video detailing this line of thinking. https://www.youtube.com/watch?v=g0YID6XV-PQ

I know I’m not the first to say this, but this is also a generalization of many jobs performed right now.

Follow the template, click the boxes, enter the text/data in the standard format, submit before 4pm. Come in tomorrow and do it again.

If that automation doesn’t require oversight, everyone wins, since now that process, typing data from a ledger, is free to anyone who wants to use it. The exception of course is if a monopoly or oligopoly controls the process, so it’s up to the government to break them up and keep the underlying tech accessible.

The biggest risk is how much computing power it takes to run these models, so it’s very important to support the open alternatives that are trying to lower the barrier to entry.

Exactly, much like a chess bot can play perfectly without what humans would call thinking.

I think (ironically) we'll soon realize that there is no actual task that would require thinking as we know it.

> The question of whether computers can think is like the question of whether submarines can swim

It has only become more relevant.

If that were true, there would be no point in studying or doing any LSAT preparation. Writing practice exams would be of no benefit.

As others have said elsewhere, the issue remains accuracy. I wish every response comes with an accurate estimation of how true the answer is, because at the moment it gives wrong answers as confidently as right ones.

I can remember my GRE coach telling me that it was better to confidently choose an answer I only had 50% confidence in, rather than punt on the entire question.

AIs hallucinate because, statistically, it is 'rewarding' for them to do so. (In RLHF)

Obviously not, since GPT-4 doesn't have general intelligence. Likewise "common sense," "knowledge about the world," nor "reasoning ability."

As just one example, reasoning ability: GPT-4 failed at this problem I just came up with: "If Sarah was twice as old as Jimmy when Jimmy was 1/3 as old as Jane, and Jane is as much older than Sarah as Sarah is older than Jimmy, and Sarah is now 40, how old are Jane and Jimmy?"

First, every answer GPT-4 came up with contradicted the facts given: they were just wrong. But beyond that, it didn't recognize that there are many solutions to the problem. And later when I gave it an additional constraint to narrow it to one solution, it got the wrong answer again. And when I say "wrong," I mean that its answer clearly contradicted the facts given.

Driving as well as an attentive human in real time, in all conditions, probably requires AGI as well.

GPT-4 is not an AGI and GPT-5 might not be it yet. But the barriers toward it are getting thinner and thinner. Are we really ready for AGI in a plausibly-within-our-lifetime future?

Sam Altman wrote that AGI is a top potential explanation for the Fermi Paradox. If that were remotely true, we should be doing 10x-100x work on AI Alignment research.

Now, granted, plenty of humans don't score above a 2 on those exams either. But I think it's indicative that there's still plenty of progress left to make before this technology is indistinguishable from magic.

Sure but look in this thread, there are already plenty of people citing the use of GPT in legal or medical fields. The danger is absolutely real if we march unthinkingly towards an AI-driven future.

Not yet it won't. It doesn't take much imagination to foresee where this kind of AI is used to inform legal or medical decisions.

And medicine is nothing but pattern matching. Symptoms -> diagnosis -> treatment.

Driving assistance and the progress made there and large language models and the progress made there are absolutely incomparable.

The general public’s hype in driving assistance is fueled mostly by the hype surrounding one car maker and its figurehead and it’s a hype that’s been fueled for a few years and become accepted in the public, reflected in the stock price of that car maker.

Large language models have not yet perpetrated the public’s memory yet, and, what’s actually the point is that inside of language you can find our human culture. And inside a large language model you have essentially the English language with its embeddings. It is real, it is big, it is powerful, it is respectable research.

There’s nothing in driving assistance that can be compared to LLMs. They don’t have an embedding of the entire physical surface of planet earth or understanding of driving physics. They’re nothing.