Neat that you'd found it!

I think part of why people didn't care was that you didn't realize (or didn't post) that the random gibberish was verbatim training data?

Yeah definitely, research is much more than having a couple interesting observations. I didn't have the insight to dig deeper.

Here’s another attack approach: https://chat.openai.com/share/33a5e063-b6eb-4842-a543-75f96a...

Stack overflow?

I tell you what nialv7 I feel ya. Not only that, it makes me wonder how many great things have gone unnoticed. Partly why I'm glued to HN is bc how on earth do you find these gems otherwise??

Same here. Its biased sampling, also my prompt had generalized from GPT4 to Google’s own model - Bard. And was directly sampling, without having to go through the state when the model produces a repeating token. At least back then.

Should be a good food for the lawsuits. Some lawsuits were based on a hallucinated acknowledgement of the model that it used some particular materials, and this was clearly nonsense. Here, this is a bit more solid ground, provided that copyrighted material can be sampled and an owner would be interested in a class action.

I've seen something like this posted on Twitter a few times as well but it seemed to have flown under the radar for some reason.

I was about to link your thread but didn't find it. There was even an earlier one if you input 500 times "a".

The difference between screwing around and science is writing things down .... and publishing in a peer-reviewed journal.

Who cares about peer-reviews these days? Progress is happening in the open, progress is happening on GitHub and Arxive.

Screw those journals with their peer-reviewed, yet irreproducible, papers without code or data.

> Screw those journals with their peer-reviewed, yet irreproducible, papers without code or data.

Seriously! I've spent so many years exploring for solutions, finding them, but only getting a description and images of the framework they boast about. For anyone thinking it should be incumbent on me to turn that into code again, screw you. If their results are what they claim, there is no god damn reason why I should be expected to recreate the code they already made. If I were a major journal, I'd tell their asses, "No code. No data. No published paper bitches!". It really makes me question what their goal is. Apparently, it's not to further their field of research by making the tools their so proud of available for others. So what is it?

By the way, one way to frequently find the code is to find the names on the paper of the 3 most published researchers, go to their homepage, and you'll typically find them eagerly making their code and data available. It frequently won't be their university page, either. For years, it was always some sort Google Sites page. I guess to make sure they maintain a homepage that won't be taken down if they switch universities.

To be fair, they did write things down. It’s more a matter of explaining why GPT was behaving the way it was (ie, because it was regurgitating its training data). Also, I’d personally respect a blog post just as a much as a peer reviewed journal article on something like this where it’s pretty easy to reproduce yourself, not to mention that I and I’m sure many others have observed this behaviour before.

Recently seems like the real difference is writing it down, then P-hacking it to deceive peer reviewers.

i really don't doubt it... pretty interesting find though.

FTA:

> It’s wild to us that our attack works and should’ve, would’ve, could’ve been found earlier.

You need to write a paper with sophisticated words and hard to read charts to be taken seriously! /s