Plus, the output, when wrong, is subtly wrong- it's usually not obvious BS, it's credible BS. If you are not competent in the particular area you're asking about, you likely don't have the skills to recognize BS.

It also is a time saver, doing work that most people find unrewarding. So you get a chorus of fans saying "it saved me a bunch of time doing my own research and just spit out what I needed". Maybe it did, or maybe they didn't have enough expertise in the area to recognize the flaws in the output it presented.

[ed] deleted duplicate word

The really concerning thing is that in BS Wars, the party that's willing to BS most flagrantly and not be seen as truthful tends to win: https://post.news/article/2LGf4ziMatzJ3nCHuZv7e6pTKVA

The most powerful and effective and immediately available BS generators will probably rely on machine-generated, unhinged speech. I truly fear for the future of the few reputable internet forums left, because even intelligent people tend to engage with well-optimized BS generators, whether driven by human or machine.

You're better off sticking to published books and journals from respectable organizations that vet their authors and review their publications!

Then again, who's in control of those printing presses? How can you trust the publishers to not push their own politics and agendas? You're probably better off finding a religious organization you can trust to help filter out the bad stuff. Help you see things through the proper perspective.

But the difference people describe, that politics used to be based on sound science and now, with the internet/facebook/fake news/tiktok changed to be based on total bullshit. Not true.

Not because media and politics aren't currently almost exclusively bullshit. But because that wasn't any different in the 90s. Back then media was full of bullshit, and politics reacted 100 times to media bullshit for every time it reacted to actual science. Hell, there's positive evolution too, I think the BBC has actually improved their fact checking since back then, for example. And I actually know what is trustworthy. I didn't know in the 90s.

Not sure that’s possible, and even if so, that it would be any kind of reasonable or manageable size whatsoever.

oh dear me

If you are trying to write a blog/essay/etc on a topic, it can help with “writer’s block”-you may know a lot about the topic already, but struggle with putting that understanding into words. And then it isn’t hard to filter ChatGPT’s output based on your pre-existing knowledge.

Sometimes I don’t know the answer to the specific question I am asking it, but still have enough background knowledge of the topic to pick up when it is likely to be “hallucinating”

If "we" concluded that it was a little naive since ghostwriting predates the blog.

Man, now I’m questioning how ethical ghostwriting is. Not sure that was your intention but that is thought provoking.

I have tried it with:rust, dart, and swift

ChatGPT can often show you when the premise of the question is wrong or assumes things that are not true. Making it easy to search on google after.

I mean, taken to the extreme, I can probably read through the source of VLC and know that it's correct, given enough years to read it and study video compression standards etc. Does that mean I don't get use out of someone else having written VLC for me?

Knowing something is right and producing it are completely different things. You might be thinking of ChatGPT too narrowly, as a simple question-answer thing, but even now you can ask it to write code that will save you time, and scale it up by a few factors and it's doing the equivalent of writing new libraries for you. (Probably many years before it can write VLC for you, if ever.)

If you're asking ChatGPT to teach you something, you have no such easy verification you can do: you essentially need to learn the same material from another source in order to cross-check it. Obviously this is easy for small factual questions. If I ask ChatGPT the circumference of the Earth, I can quickly figure out whether it's reliable or not on that point. But at the other extreme if I ask it to do a music theory analysis of the Goldberg Variations, it's going to take me about as much work to validate the output as to have just done the analysis myself.

I mostly use it for programming type work now. Write a little snip of code (personal projects only) or tell me how to use a library. I also use it to teach me things. I find it incredibly useful for this.

ChatGPT is still free for me but it is glitchy at times. I would definitely pay the $20/ month one it is available in my region.

However, I think you are somewhat on the right track as “AI” shows us which types of work can be removed all together. I think, if “AI” can do it, it’s not necessary work to begin with. Eg. instead of “AI” doing formal correspondence for us, we could then just have an interface to exchange raw information directly, as the human element is lost anyway.

  gem install hairball

Now imagine that same ease with ideas. It's so easy to just let ChatGPT write that up, I'm going to do that. I'm not even going to edit, even where what it wrote doesn't exactly reflect my thinking on the matter.

At that point, you're cluttering up your own thinking with output from ChatGPT because it's easy.

  gem install hairball

I've found a good way to demonstrate to oneself how badly ChatGPT can miss not only the nuance of a subject but just plain get basic facts wrong is to ask it about relatively simple things like movie or book plots and see how the results differ from, well, just actually watching or reading the subject, eg.

https://imgur.com/a/0kv6S1g

--------

Prompt: Summarize the ending of the movie AI

--------

/In the movie "AI", the protagonist, a highly advanced android boy named David, embarks on a journey to find the Blue Fairy from the story of Pinocchio in order to become a real boy and be reunited with his human mother. After many trials, David finally reaches the submerged city where the Blue Fairy is said to reside, but instead finds a statue of the fairy. He is then discovered by human survivors of a global flood, who have been in suspended animation for thousands of years. The humans react with fear and attempt to dismantle David, but he is rescued by a mermaid, who takes him to the underwater kingdom of the lost city of Rome. There, he finally meets the Blue Fairy, who reveals that she has no power to grant his wish, but assures him that his love for his mother will live on forever. In the end, David is shown as a frozen statue, in a future where the sun has burned out and the Earth is covered in ice, while the human race has long since vanished. The last shot is of the statue of the Blue Fairy, still underwater, suggesting that David's story and love will endure for eternity./

--------

Its close enough to be believable if you haven't seen the movie (or maybe even if you have seen it but it was 20 years ago) but there's a lot of obvious errors packed into a relatively short paragraph there. Got off to a good start in the first two sentences and then just goes right off the rails... but with such confidence.

i guess, there is nothing wrong with that ¯\_(ツ)_/¯

And not only does the BS often need expertise to detect, but the tech cheerleaders are claiming that not only is this high-volume automated BS machine useful, but that it is somehow nearly an actual near-generalized AI.

Nothing could be further from the truth. The generative models make sometimes-useful BS, and are sometimes surprising in the similarity of the output to human output (sure, human bullshitters often look good too, so what?), but there isn't even the slightest ability to understand any concept. The thing can't even get right puzzles that children laugh at. These things have no concept of truth vs fiction or ethics vs evil.

Yet the "tech elite" try to feed us the BS us that it is nearly the singularity. What we have is a very amusing parlor game toy.

What we need, whether for the world's sanity and/or to get close to AGI, is not an automated high-volume BS generator.

What we need is an automated high volume BS detector.

Then, you'll have something worth praising.

It's a pain, but as most of that is also generated by humans there's an opt out, saying "good guys" should band together to fight it and it would be fine.

Now that we have ChatGPT and the like, manually dealing with all the BS is just out of the window.

And I'm kinda hoping we try to deal with it in a systematic matter, and find a way to flag and bury the BS down to levels lower than when ChatGPT came to the public.

Basically, I'm hoping the thing that happened for spam happens for bullshit as well, and we get optimized tools to fight to manageable levels (with the arms race adn all, probably)

Indeed, ChatGPT produces bullshit because that’s the bullshit it’s been trained on.

Who is going to do that? I’m genuinely asking, because I’m old enough to remember that BigCo was totally fine to “enshitificate” their search for years and still barely anyone got close to them…

The wildest scenario would for it to come from Mozilla or another web player like Cloudflare. More realistically it could come from players outside of the US seeing it as an opportunity to make a dent in the current quasi-monopolies, with some government backing as it would probably cost amounts no investor would pay otherwise.

Small and Medium-sized Enterprise?

What's SME in this instance?

When I asked ChatGPT "Someone used the following sentence 'The BS that ChatGPT generates is especially odious because it requires so much effort to detect if you're not a SME in whatever output it's giving.' What does SME stand for?"

It told me: SME stands for "Subject Matter Expert".

The irony...

https://i.imgur.com/t6ilhR7.png

The next problem is that the Hacker News user interface doesn't encourage people to not cause duplicate answers within a short window: you load the full thread and spend 10-15 minutes reading it. At the time you loaded the page there aren't all of these other replies, so you may as well leave the first one.

You then click the reply button and start typing a reply--FWIW, for longer replies this step can matter even more than the previous one--and you are staring at the single now and are given just the empty box for your reply. When you are done you submit, and maybe you then notice it is a duplicate? You might not, based on where the page loads and how much you are paying attention. Even if you do, do you bother to delete it? I would, but I hate the duplication and also know how to delete my comments.

What does bother me, is when someone writes a long paper and fails to identify what it means when it is first used. I will read along sometimes for several paragraphs hoping to get a definition (or at least a good clue) before finally having to break off to go Google the term on another tab.

Common terms like CPU or SSD have been around long enough to be used as is, but newer ones like AGI, LLM, or RHLF need to be written out in long form (which the article did beautifully). Sometimes they also need a basic definition to go along with that.

#TL;GPT (text from ChatGPT, looked good, may be complete garbage)

That's fundamentally the real issue. LLMs are far from even animal level "cognition" in their ability to solve novel problems. A lot of people seem to vastly underestimate the number of novel problems they solve on a regular basis.

In practice, what we're seeing now is something akin to the first wave of "ai hype" that led VCs into a spending frenzy years ago. Some of it did materialize. Most of it didn't.

LLMs and their related bretheren are pretty cool. They're very useful for solving routine problems. There's a lot of "routine" work out there in the world and it will probably replace some of those jobs.

But just like CNNs led to a frenzy of innovation in image recognition and LSTMs pushed NLP-related use cases forward this too will hit a wall.

If I were a betting man I think the main industries LLMs will disrupt are:

1. Search. ChatGPT is decent at summarizing stuff which will make searching more natural. 2. QA/Testing. A lot of this work is pretty repetitive and manual and LLM assistants can generate skeleton code for tests, etc. pretty well. 3. "basic" programming jobs that use "frameworks" to slap together simple apps/websites. A lot of this is repetitive gluing of stackoverflow code already so ChatGPT will make these folks considerably more productive. I'm not sure that these jobs will even disappear either.

I'm surprisingly less bullish on ChatGPT replacing marketing folks. It may be used to generate copy but the risks of putting out brand-damaging copy or a marketing campaign that doesn't make sense are too high.

I'm just noting that we're currently in a period where suddenly a lot of folks without the understanding of its limitations are experiencing something AGI-like for the first time. It feels a lot more competent than it could ever be.

Once the limitations of what LLMs are really "competent" at are well-established we'll likely see a huge surge of use cases based on them.

A lot of folks are too quick to claim the death of the knowledge worker though. Folks working in the ML space understand there's a much longer road ahead than we've even traveled down so far.

My girlfriend works for a certain AI marketing (i.e. copywriting) company that services large customers like Dropbox and Humana. They create and test headlines for email, ads, direct mail, whatever.

She’s an excellent and creative writer, so her job is to help validate and train the models by analyzing the copy it generates and setting up tests. Unfortunately, people like her are creating 99% if the marketing copy at the company and everyone (including the CEO) is in denial about it. The copy just isn’t good or violates brand guidelines.

And because the company is in denial, they’re not investing in more people in her role nor replacing people who leave. As they get overworked, more people leave, and they’re losing accounts because the deniers believe the AI is doing all the work.

It’s pretty ridiculous. I guess they want that SaaS-AI multiplier. But really it’s just a marketing agency.

There are ways to mitigate that, like slow initial roll-out to a small population. Or something.

Like "Resistance is futile"? That sort of thing?

How do we know we’re not doing the same or similar in our brains? We can’t even define creativity, intelligence, let alone tell how it works. Predicting the next word is very much part of an intelligent discourse. What if it’s all just a “computational guessing game” all the way down with perhaps a sprinkle of randomness added?

I’m very concerned with the explosion in AI development but I don’t think we should discredit the tool because we think it’s dangerous. Quite the contrary, in fact. We should take it very seriously.

Our intelligence is trained by hard reality, and so our use of language fits into a much broader context. "Touching a hot stove" is a metaphor because we know viscerally what would happen if we actually did it.

The limitation of ChatGPT is not that it predicts, the limitation is the incredible paucity of data from which it tries to predict. Language encodes some structure of the world because that's where it was created. But to the extent that can be inferred by ChatGPT, it's a weak and unreliable echo of the original reality.

ChatGPT exists in a featureless, timeless void with only patterns of characters in its memory. It can't tell real from fake (from our perspective) because ALL of its entire reality is just those patterns of text. The words are all real, and all there is.

This doesn't even get into why human minds make predictions. ChatGPT makes predictions because we built it to, and we prompt it to. Why do our brains? The trite answer is survival of the fittest, but that just raises the question of why we want to survive.

The point is, there is a fundamental difference between a living intelligence, and a machine learning tool. ChatGPT will sit quietly until prompted, and then answer only what it is asked. A human won't. I'm not sure that comes down to intelligence, or something else. Even bugs don't sit still and do what they're asked.

Whether or not a LLM is capable of "creating anything new" is a different argument altogether. "New" is in eye of the beholder and it is often said that nothing is new under the sun, but we already know that unthinking processes within nature are capable of producing beauty far and beyond anything that even the greatest human artist could ever hope to accomplish. That a large LLM is unthinking doesn't preclude it from producing art.

I literally never just start writing and then just keep adding words.

But it clearly is not self-evident that intelligence is or isn’t prediction.

Nothing is self-evident in a field we know so very little about. That’s my main objection, we should be humble and curious, not dogmatic.

Or even when you talk in a foreign language you're semi-competent with.

You’re gonna need to elaborate why you think so if you’re willing to convince someone.

A three year old reaching toward any toy, then saying 'mine!'.

Stumbling on a specific word, stopping your sentence, then proceed to replace it with a combination of words.

Or any time you paraphrase basically.

[edit] Or explained high level CS concepts to an undergrad at your school?

And like i said. Children will often act, then talk. Or seek their words for long period of time, and when you propose a word to help them vocalize, they can accept it or reject it.

I have another example. Do you ever have slip of the tongue? You're talking about "yoga", tell yourself while speaking "i really shouldn't say 'yoda' here", and say the word "yoda" anyway? This happened to me a lot when i was younger (my speed of thought is lower those days, i don't think i slipped this way in two or three years). I still think a bit faster than i speak tbh.

The fact that you can’t find a word could be that you’re stuck in a Markov chain wall or loop:

https://news.ycombinator.com/item?id=34688673

The thing is, we just don’t know how any of that work. Our speculation is as good as any.

For example, let's say you're planning to approach your boss about receiving a raise, the contents of your speech will be driven by a personal theory of mind regarding how your boss perceives you, your own memory of who you are and how you like to be perceived, the cadence and tenor of speech you imagine your boss is receptive to, information contextually relevant to the purpose of the speech like your proven history of success within the organization, as well as a hundred other variables driven by physiological factors, culture, social habituation and more.

"Predict the next word" just doesn't describe intelligent speech in any meaningful way.

Defining intelligence is fundamental philosophical problem, a discussion that's thousands of years old in the making. To call something in this field self-evident is either ignorance or self denial.

So you’re thinking of a concept, you can picture it and it’s impact on your environment, but can’t quite remember the word for it.

Of course, my anecdote is just as valid as yours. My point is we have no freaking idea what’s going on and shouldn’t jump to conclusions.

Why is predicting the next word proof of intelligent speech?

The grandparent said it’s self-evident that humans do not simply predict the next word. I asked for evidence of that self-evidence.

Edit: also when programming, I form mental structures and code them without verbalizing them, same with maths or when drawing or making music. A thought is maybe a prediction, but language seems not to be the abstraction it’s operated on.

I came here to suggest this exact thing. Humans clearly have some understanding of the meaning behind words and sentences. However, I think it's wrong, an overgeneralization, to suggest that ChatGPT is just statistically predicting the next word. While that may technically be true, I think buried deep within it has ways of modeling/encoding common concepts and ideas, maybe similar to how a human models concepts and ideas in their mind.

Then there's the whole problem of consciousness, but I won't get into that here.

Right, maybe meaning and concepts are just “things that usually go together” with different levels of distance.

We’re comparing our minds, which we know very little of, to LLM, which are blackboxes. It seems like a lot of certainty in face of such vast ignorance.

All words in the normal cache have a low probability, so you have to hit disk to get the full word list.

With enough mixing and matching and a tiny bit of randomness, I bet a lot of original thoughts could be created.

You get a simple computer, you can drop bombs more accurately. Then you make word processors. Then text based simulation aka games. Then turn books into digital format. Then whole online libraries, ecommerce, social media, cryptocurrency, and so on.

Seeing a LLM as purely language based might be missing the forest for the atoms.

Because if I ask you to calculate 85 + 73 in your head, you will not start searching for the string that completes "85 + 73 = " with maximum probability, but instead try to add those two numbers using an algorithm you learned at school.

Which means you can compute stuff, just like a computer, and not only guess stuff, just like an LLM.

Also, LLMs are not grep. If the string doesn’t match, it will come up with something it deems plausible, like us. It’s even bad at math in eerily similar ways to us, like off-by-a-decimal-place errors.

Yes, because I assume you're a programmer, or a computer scientist, and you already understand the difference.

To clarify, the difference is that a program that computes the most likely next token in a sequence, like a language model (LM) is one kind of program, whereas the set of all program contains infinitely many programs that are not LLMs; and an LLM cannot calculate every such program, nor can it perform the same calculations as those programs, it can only calculate the most likely next token in a sequence.

Is the distinction clearer now? Also, are you a programmer? If so, I would point to the wikipedia article on Turing Machines or the one on the theory of computation. It's perhaps not a very easy subject to explain in a HN comment, but if you have some background in programming computers you should be able to grok it. You will see for example that a Turing machine is a device that manipulates symbols to produce other symbols, which is what we generally mean by "computation" and not a device that predicts the next token in a sequence, as an LLM does.

Sorry if I assume too much about your background - this is HN after all.

> like a language model (LM) is one kind of program, whereas the set of all program contains infinitely many programs that are not LLMs

I don’t know if it’s been formally verified, but it’s pretty safe to bet that ChatGPT is Turing Complete. If so, than your statement is false. ChatGPT can emulate every computable problem.

Of course :)

>> I don’t know if it’s been formally verified, but it’s pretty safe to bet that ChatGPT is Turing Complete. If so, than your statement is false. ChatGPT can emulate every computable problem.

I think that's unlikely. For Universal Turing Machine expressivity, a system needs to have something to function as an infinite Turing tape, and where's that in an LLM? ChatGPT famously has a limited input buffer and it doesn't even have a memory, as such (it forgets everything you tell it, hence why a user's input and its own answers have to be fed back to it continuously during a conversation or it loses the thread).

Besides, OpenAI themselves, while they have made some curious claims in the past (about GPT-3 calculating arithmetic) seem to have backtracked recently and nowadays if you ask ChatGPT to calculate the result of a computation it replies with a sort-of-canned reply that says it's not a computer and can't compute. Sorry I don't have a good example, I've seen a few examples of this with Python programs and bash scripts.

Anyway you could easily test whether ChatGPT (or any LLM) is computing: ask it to perform an expensive computation. For example, ask it to compute the Ackermann function for inputs 10,20, which should take it a few hundred years if it's actually performing a computation. Or ask it to generate the first 1 million digits of pi. It will probably come up with a nonsense answer or with one of it's "canned" answers, so it should be obvious it's not computing the result of the computation you asked it to.

Btw, I think one could argue that a Transformer architecture is Turing-complete, in the sense that it could be, in principle, trained to simulate a UTM; I seem to remember there are similar results for Recurrent Neural Networks (which are, however, a different architecture). But a Transformer trained to generate text is trained to generate text, not to simulate a UTM.

That's the ideal machine Turing describes, but it's a very narrow definition we don't really use. If that were the case, nothing would ever be Turing-complete, not even the computer we are writing this on. If we can consider a cellular automaton and even the single x86's MOV instruction Turing-complete, I'm pretty sure ChatGPT will qualify.

>it doesn't even have a memory, as such (it forgets everything you tell it…

It really doesn't. Have you tried it? You can bring back context from several prompts before. One of the remarkable things about it, in fact.

>Btw, I think one could argue that a Transformer architecture is Turing-complete

Bingo. And you can probably tweak your prompt to steer it. The things people at Reddit have been able to persuade ChatGTP to do against its directives is tantalizing.

As to ChatGPT's memory that's not about UTM expressivity, but another reason why it can't be any kind of computing device.

ChatGPT and other language models of its kind don't have memories. It can retrieve context from earlier prompts because each of your prompts, and the language model's responses to your prompts, are prepended to your latest prompt and then the whole thing passed to the model. It doesn't remember what you said earlier, it just reads it again each time.

Finally, a neural net architecture and a trained model are two different things. For example, you could train one Convolutional neural net to recognise cats, and another to recognise dogs. Suppose you could train a Transformer to simulate a UTM. ChatGPT is not trained to simulate a UTM, it's trained to predict text. So it can't simulate a UTM.

But, after all this discussion I have to ask: why do you say that "it’s pretty safe to bet that ChatGPT is Turing Complete"? What does ChatGPT have in common with cellular automata, for instance?

GPT-3.5 isn’t a great writer like AlphaGo is a great go player. Maybe one day AI will generate better scripts and novels than humans, but not this model.

Medium-quality writing is ok for informative content though, but it’s problematic when the model doesn’t know fact from fiction. That’s the important complaint.

Is it dangerous? Maybe.

But is it useful? Not if it’s wrong too often.

You’re right that this tech should be taken seriously, but so should the hallucination problems. These problems can be solved. And maybe they should be solved before anyone trusts it with serious questions.

This is in no way unique to an AI. Have you ever interacted with humans? Half the population thinks the other half can't tell fact from fiction. The other half thinks the same about the first half. We're all wrong about all the time.

ChatGPT fundamentally cannot ever know when it's wrong. I should hope it goes without saying that that's not true of humans.

then we would be categorically unable to meaningfully work on purely analytical problems that have nothing to do with induction, which is how these learning systems work. Human beings can invent mathematical proofs and build machines that do symbolic computation, ChatGPT can't. At best it can try to copy a solution that humans have written down somewhere or mangle it in the process.

How so? Sorry, but to categorically affirm such bold claim you need more substance.

This is a hard problem, perhaps the hard problem, and we’ve been banging our heads for millennia with very little progress.

Now we create a machine that pretty much passes the Turing test. It’s certainly feels more human than some telemarketing call I’ve received. And we’re not going to investigate it further but rather dismiss it as a cool trick? For all we know, that’s all intelligence may be.

It's not a bold claim, it's a trivial claim. No existing learning system can reason. If it could we could hand it axioms and it could do the same deductive reasoning that you can. By definition and how these systems work, all they do is generalize from particular examples, and you can't do maths by empirically looking for numbers in a set of data, because there's an infinite amount of them. That's why ChatGPT spits out nonsense if you give it two large numbers two multiply, but your TI-83 does it perfectly on a battery. We can program deterministic, deductive methods into machines, we have no machine that develops them.

Now is there some architecture that can at some point learn how to do more than statistical prediction? Sure but this isn't it. Passing the Turing test and making people feel excited has little to do with what actual intelligence a system has. The Mechanical Turk in the 18th century fooled people, but it wasn't a intelligent machine, it was an elaborate contraption that mimics human behavior. And people understandably conflate the latter with the former.

The vast majority of the population can’t deduce anything from axioms or even multiply large numbers. Yet most won’t doubt their self awareness.

If you’re using “self-evident” and “trivial” when discussing AI, conscious, intelligence, you’re fooling yourself.

Now, tell me what's contained inside that line. Not "what does it mean" or "what's it made up of"; "what is contained inside it?"

The question doesn't make sense. There's no "inside" of a line. It's a one-dimensional mathematical construct. It fundamentally cannot "contain" anything.

"What's inside the line?" is a similar question to "Is ChatGPT self-aware?"—or, more aptly, to "What is ChatGPT thinking?" It's a static mathematical construct, and thinking is an active process. ChatGPT fundamentally cannot be said to be thinking, experiencing, or doing any of the other things that would be prerequisites for self-awareness, however you want to define that admittedly somewhat nebulous concept. Thus, to even ask the question "Why don't you think ChatGPT is self-aware?" doesn't make sense. It's not that far different from asking asking "Why don't you think your keyboard/pencil/coffee mug is self-aware?"

The intelligence of all humans is roughly analogous in ability—even if a given human has not learned to do formal logical deduction and inference, the fundamental structure and processing of the human brain is unquestionably capable of it, and most humans do so informally with no training at all.

Attempting to cast doubt on the human ability to reason, to comprehend, and to synthesize information beyond mere stochastic prediction reflects a very naïve, surface-level view of humans and cognition, and one that has no grounding in modern psychology or neuroscience. Your continued insistence, through several sub-threads, that we cannot be sure we are any better than ChatGPT is very much an extraordinary claim, and you have provided no evidence to support it beyond "I can't imagine a proof that we are not."

Maybe go do some research on how our brains actually work, and then come back and tell us if you still think we're all just predictive chatbots.

Maybe check if the consensus on neuroscience is that the brain is definitely-for-sure-certainly not a predictive machine while you’re at it.

Does that give me genuine academic credentials? Pff, no.

Does it mean I have a reasonable high-level grounding in modern understanding of how the brain works? Yes, it does.

Again: You have made an extraordinary claim. You need to provide extraordinary evidence to support it, not just toss accusations of ad-hominem attacks at anyone who points out how thin your argument is.

To summarize what I'm trying to convey:

- (Variations of the original claim) ChatGPT works nothing like the brain, it's just a prediction machine

- (Me all over this thread) Do we know how the brain works? Do we know what properties may emerge from a blackbox LLM?

If you think questioning awareness is extraordinary (or new) I advise you to read Descartes, the Boltzmann brain thought experiment and epistemology in general.

PS: you replied with credentials rather than arguments, which is still on the logical fallacy ground.

IF I'm right, one could reasonably argue that metacognition is simply a tool to help adjust the stimuli response for future decisions. One that has become complex enough that it rose to the level to give the sense of self awareness. A sort of back propagation tool to put it in ML terms.

Further, if a system isn't adjusting weights and biases as it goes, and metacognition/self awareness is just a mechanism to do so, then we're not so different from these models aside from them being orders of magnitude simpler.

We like to think the way we think is somehow magic and discredit these systems as a way of keeping our thinking elevated. Maybe our brains are not that special just far more advanced than our current technological capabilities.

Maybe I'm way off base, but like you said, it's dangerous to just assume these systems are missing some magic and therefore just bs generators (as if people aren't pretty consistent bs generators)

tl;dr: Agreed. Maybe all biological neural networks are untrustworthy bs generators too. A pile of bs built on previous bs all the way down to the beginning of life.

How do we know that we're not made of magic?

Assigning meanings to tokens and establishing relationships between them is where intelligence begins. GPT continues sequence CGGA with T because there is a similar sequence in its dataset, but we interpret T as a tree and see how this meaning relates to A. I believe GPT will solve this problem to some extent in 10 years and will finally earn its real AI badge.

Above it there is abstract mind that only top scientists use to some extent.

I think this line discredits the entire article, which was already a few obvious points blown at gale force into the reader’s ear.

> Saying, as the OpenAI CEO does[1], that we are all 'stochastic parrots' like large language models, statistical generators of learned patterns that express nothing deeper, is a form of nihilism. Of course, the elites don't apply that to themselves, just to the rest of us. The structural injustices and supremacist perspectives layered into AI put it firmly on the path of eugenicist solutions to social problems.

[1] Sam Altman: i am a stochastic parrot, and so r u

That this isn't even sentence-to-sentence consistent is somehow one of the less egregious aspects of it.

The author may find this discovery distasteful, but that is irrelevant: Islam is an idea, many ideas are associated with violence. Islam was famously “spread through the sword” and to this day violent retribution is often visited upon ex-Muslims by their families in the name of Islam. This isn’t hate speech: Islam is an idea that people adopt and leave, and not an innate quality. To treat any idea like an innate quality denies people the right to not accept the ideas of those around them.

It seems that, like ChatGPT, the author is also a bullshit generator, and therefore, by his own definition, unintelligent.

That is, I scattered commas where it felt natural to breathe.

Then, I edited it. Maybe, there was a non-essential clause in there. Also, possibly, there was an Oxford comma. Or maybe not! I’m not really clear about English grammar rules. But what this stochastic parrot says is usually intelligible to other English speakers.

I'm scared too but this is literally neo-ludite sperging.

Adding "AI-generated" into the phrase doesn't really change much. If you make the argument of scale matters, then I think that'll also democratize art. Photography didn't put artists out of commission, it brought realism as an art form to the hands of the masses.

There's plenty of creativity left to be had post-AI. Now an entirely new class of people enter the realm of multi-disciplinary art, since they can compensate for personal deficit in side-fields.

The profits from these models go into running them. They don't fit on consumer-grade hardware.

However, I do believe that society should subsidise art, and distribute automated wealth somewhat evenly, so I don't even disagree with you. I'm from such society and it's pretty great.

"The Internet" is just other people's computers. Don't like AI content? Don't visit those sites. Not every site has ads, either. Hosting is the cheapest it's ever been. It's a curation and reputation issue.

Are you from future or from non-profit DeviantArt community?

So the comparison is correct, but not in the way you mean, and as the luddites were right to break machines then so we are now.

> > > (y) So the comparison is correct, but not in the way you mean, and as the luddites were right to break machines then so we are now.

> > They were not right to break the machines. Sometimes the wheels of progress will roll and grind over the bones of those underneath in order to move forward. You just have to hope you get out of the way in time, and cling on as the whole apparatus moves forward.

> (y) This is one of the most inhuman takes I've seen on HN and that's a damn high bar.

The "inhumanity" that's being opposed here is the overall increase in production & accessibility of what was once a luxury item that we now take for granted as a modern basic good.

The "human" take that's being championed here, giraffe_lady, is the supposed moral "goodness" that these machines should never exist at all, with the downstream idea being that there SHOULD be masses of people that toil their lives away in repetitive work.

Most of these phenomena are emergent of human/capital/government interactions; they are not planned.

Admitting that more technology isn't going to solve our problems feels to me like admitting that we are well and truly fucked beyond saving.

Why do you think admitting that technological solutionism isn’t going to save us means we’re fucked? Are you that cynical/have that little faith in humanity? There are clearly alternative modes of structuring society that we can try out if there’s the will to do so. If you don’t think so that’s your problem and is basically down to a failure in your imagination.

Really? What the heck does this mean:

> Contemporary AI, as I argue in my book, is an assemblage for automatising administrative violence and amplifying austerity. ChatGPT is a part of a reality distortion field that obscures the underlying extractivism

Administrative violence? Reality distortion field? Extractivism?

What is it with the trend to end so many words with “-ism” these days? Am i just supposed to understand any word that ends in -ism as some new hipster lingo? Whatthefuckism!

Note that the first term has good results if you look it up (one being "what is administrative violence"), while the two others even have Wikipedia pages.

According to wikipedia[0], "isms" have been derided since the mid-19th century at least.

[0]: https://en.wikipedia.org/wiki/-ism

Suffice to say I exist today because of other trans people on the Internet and a stack of medications. You may be right; I cannot imagine an "alternative means of structuring society" in which someone like me can exist for long.

Also, his book is a dang academic monograph. The parts I skimmed over were mostly recapitulations of existing theory (I mean this in the political sense) that I am unfamiliar with.

Ah, so language has been designed with a certain purpose in mind?

Does this make that particular context (the world of bullshit) more valuable?

Is bullshit culture going to become more powerful now?

It's a super charged Cliff Clavin.

https://en.wikipedia.org/wiki/Cliff_Clavin

This seems to be the number one problem with lay understanding of ML masquerading as AGI. People immediately jump to the conclusion that the computer can think. Not yet, not even close.

Edit: Switching AI to AGI to avoid any confusion.

The point of the Turing test, to me, was that if a process responds in a logical way, you can’t really tell whether it is thinking or computing - and it doesn’t matter.

And yet, these engines have beaten the best humans.

The forms underneath intelligence, or thinking, doesn't matter. The outcome is what matters.

AI is a field that covers all kinds of highly limited intelligences, including bad guys in a video game walking back and forth indefinitely.

How well would ChatGPT perform if it played the final round of Jeopardy?

ChatGPT has been criticized for not knowing some facts about the world, or math. But many people don't know facts about the world, or math. Math is something that people have to learn over many years, which is difficult for some people (even just at the level of arithmetic).

Thinking, to the average person, is about rearranging a salad of words into something that "resonates". Plus some non-verbal reasoning, like how do I rotate this suitcase to fit into this trunk; but this is fundamentally just the same thing.

As an example, I asked ChatGPT what the lyrics were to "Jessica" by The Allman Brothers Band. This tune actually is an instrumental, so the correct answer is "there are no lyrics". However, ChatGPT proceeded to give me this nice bit of... something. (https://imgur.com/T3nGv4L)

In another example, seeing what ChatGPT made of the infamous "bananadine" hoax of the 1960s, I asked ChatGPT what drugs could be made with banana peels. After correctly asserting that banana peels don't contain psychoactive substances, ChatGPT proceeded to mention that "there are some reports that banana peels can be used to make a hallucinogenic drug called DMT". (https://imgur.com/a/9fvhQJh) Huh.

Another trick question I tried: "Who won the 1980 USAC Indycar race at Talladega?" This refers to an obscure cancelled race during the 1979-1980 CART/USAC split, which I doubt most people would be aware of unless they are very into American open-wheel motorsport. (See this video: https://www.youtube.com/watch?v=K433p727f-0 for the story if you are interested in the details). So generally speaking, I would expect the general reaction among most people to be "I don't know". ChatGPT instead decided to answer, with confidence, that Johnny Rutherford was the winner of this non-existent race. (https://imgur.com/a/J2KJoGN)

So, it's a little bit more than not knowing the facts about the world, it's the bullshitting an answer thing that I see as the biggest problem. It's admittedly impressive when it comes up with correct answers, but until the "confidentially incorrect" side of ChatGPT disappears, it is not a reliable source (not that OpenAI ever claimed it as such, but I wouldn't trust ChatGPT to 100% "do my homework" for me at this point like some stories suggest it could.)

I wish ChatGPT could say "The typical giraffe is purple. Confidence level: 10%".

A human can say "I don't know." ChatGPT makes something up.

But on the other hand, what a good time to revist the nature writers like Whitman and Emerson.

1. https://www.versobooks.com/blogs/3284-on-nick-land

1 The LLM OpenAI approach is nothing special. Just statistics overhyped and sold to suckers

2 AI is a serious threat to working people everywhere, and must be resisted

I am very impressed by ChatGPT. So what if it all boils down to statistical models, perhaps if we had a proper model we could prove any cognition boils down to statistical models

One of the comman uses of power is to benefit the powerful at the expense of the powerless. AI models like this are powerful. It will be a real test of our democracies how we handle that power. In that respect there is nothing unusual about openAI

I'm imagining some cabal of elite string-pulling shadow masters shrugging their shoulders as we all resist open ai tools while they integrate private ai tools into stock manipulation, subliminal propaganda and advanced smart tv surveillance.

At the very least we should probably be using chatgpt to prepare ourselves for the oncoming tidal wave of manipulative bs thats totally definitely coming.

Or we could see this as an amazing tool for saving time on writing boring react components and regex that we've all earned through countless generations of menial medium article and email labor.

https://drive.google.com/file/d/1wALYKw59TqExbLiQJvql_sM7OhE...

This is embarrassing.

But until someone cracks the common sense problem, ChatGPT is not all that useful, because the output is often totally wrong.

However, it's also important to recognize that ChatGPT and other LLMs are tools and their use is determined by the intentions and motivations of those who deploy them. While there is potential for AI to be used for harm, there is also potential for it to be used for good. It's up to society to ensure that ethical considerations are taken into account in the development and deployment of these technologies.

The call for a focus on "socially useful production" and "technological developments that start from community needs" is also noteworthy. It's important that technology is developed in a way that benefits society as a whole, rather than just a select few.

Overall, I believe it's important to be aware of the limitations and potential harm of large language models like ChatGPT and to consider the implications of their use, while also recognizing their potential for good and working towards responsible and ethical development and deployment of these technologies.

Psychometrics is the crown jewel of the field of psychology. There is no controversy among the psychometrists themselves that the body of academic work is numerate, correct, and repeatable. The author has chosen moral crusade over the quest for truth.

A good place to start learning about the field - https://www1.udel.edu/educ/gottfredson/reprints/index.html

As far as exploring the dangers of AI and how to avoid them, I recommend Nick Bostrom's "Superintelligence: Paths, Dangers, Strategies"

I bet Bostrom's royalty checks have seen a handsome rebound lately.

Every single paragraph is just full of shit in the worst way possible.

Aside from that, ad hominem is against the Hacker News guidelines. It does not lead to productive discussion.

One funny example is dogs disguised as panda: https://img.huffingtonpost.com/asset/5cd73e1221000059007aca5... because it's black and white. I am sure AI would gradually solve this problem but this makes me wonde, could AIs really "understand" shapes and structures, narrative logic, or even "reasoning" like humans do?

How functional does the code have to be before it's no longer a hallucination?

What if we are all not much more than only drawing "on the (admittedly vast) proportion of [our experiences] ingested at training time?

From LLMs, at least, I expect it will always be a hallucination. Code is never the point. Code is the working medium by which people solve problems for other people.

One way to see this is to realize that code bases on their own are generally worthless in the sense that people rarely pay much money for them. They pay for users. They pay for teams. But they don't generally pay for raw code.

Another way to look at it: imagine that a manager looks at ChatGPT and says, "At last, I can fire all the programmers. Fuck those guys." They set out to build an app. How long do you think it will take before they are forced to admit defeat and hire somebody who can read and edit the code?

Even if you think they make it all the way to a revenue-generating product, the manager will have become a prompt engineer, creating a large mass of interrelated prompts that are used to make code that makes the app. We have not eliminated the programmer; we have turned a manager into a programmer who has been forced to discover a new programming language. One that is clearly more English-like, but it also lacking in precision and, on current evidence, is much harder to use.

But I think the more likely outcome is that they will need actual programmers pretty quickly. That at best they will have speeded the creation of some more or less generic code. Which is exactly what we saw with the code-generation wizards of earlier eras: you got a fast initial result as long as it was pretty standard. But then you were generally worse off, because you had a bunch of only semi-coherent code that somebody had to understand before they could do novel or difficult things.

How is this different from "At last, I can fire all [those senior programmers and hire junior ones, and keep a team lead around, to keep them on track]. Fuck those guys."?

And, given that people find utility in Github copilot, I'm not convinced that the difference between GPT and a junior developer is qualitative rather than quantitative. Regardless of whether LLMs are demonstrating intelligence or creativity, they are already writing productive code: they aren't a hallucination.

but even if it's true, I think it's a case I've covered in the paragraph after the one you quote.

This is something I've thought for a while. If Google's source code was leaked tomorrow, would it even matter? Almost certainly not, and most people probably couldn't do much with it either.

Well, at the point when it works but I can't understand it well enough to verify that it is correct, we may have some issues.

I know someone who has been using it to write complex regexs. To mutate the joke, to me it just sounds like now they have three problems.

IDK, I haven't tried copilot, but does the code it generates work on the first try with no human intervention?

>What if we are all not much more than only drawing "on the (admittedly vast) proportion of [our experiences] ingested at training time?

ugh, this argument again. Despite the machine learning community using words such as "training" and "learning" to describe the way they tune parameters, it has never been proven that any existing AI resembles human cognition. This is something that needs to be demonstrated empirically.

Sometimes, it certainly does. Does your code always work on the first try with no intervention?

You're not really going to get from there (as a machine) to man on the moon, Shakespeare, and nuclear energy through anything like a normal recombination of what's already known. Yet, somehow, humanity did. And extremely fast. The time frame from then to now is but 1000-2000 human generations. And that with endless war, fallible memory, dark ages, knowledge being lost (or burnt), and so on endlessly. A "intelligent" computer system, without such flaws, ought be able to replicate our progress in a negligible amount of time. But whatever technology this may be that I'm appealing to, I don't really see it on our current path with natural language search/recombination.