It runs a model with 260K params, so hardly a "Large" LM. Nevertheless, a cool project.

And also not exactly a Commodore 64 is it, since it requires an addon with 30x the RAM. Still very cool and impressive though!

It's a borderline thing. The official Commodore REU only supported 8x the RAM. But you could modify it yourself to 32x. Creative Micro Design also had the third party 1750 REU which supported 32x RAM. (2 megabytes.)

So it is somewhat period accurate, albeit very expensive at the time.

Imagine if someone had run that LLM on that hardware in the 1980s, though. Incredible!

(Probably couldn't have trained the model, but still.)

You'd have needed a ZX Spectrum for that!

If anyone wants to try it with an esp32:

https://github.com/DaveBben/esp32-llm

needs a board with PSRAM but they are surprisingly plenty these days - almost standard. I tried it a few months back.

Here is the repo:

https://github.com/ytmytm/llama2.c64

0.002 tokens per second

Given that it uses a CPU that is one millionth slower than a modern CPU, its performance is impressive.

I've read the criticisms here, but well, as someone whose first computer was a C64, this is cool as hell.

Like, what?!

On an 8-bit, 64Kb, ~1MHz CPU!

Amazing.

Sure, there are too many caveats, but this isn't really about making this a viable modern alternative.

It's just about, well, being very cool? Nostalgic!

And in that, I think it succeeds.

Not 64 kelvinbits, but 2 megabytes of RAM.

Or should I say 2 mebibytes. </pedantic>

fairy nuff. Not 64Kb, but 64 kilobytes of RAM, + 20 kilobytes of ROM!

Plus, we had a 320x200 display in a glorious 16 colours!

Revolutionary sprites, too!!

FWIW, my first game, ie all I cared about on a C64 at the time, was the amazing Revenge of the Mutant Camels!

> Revenge of the Mutant Camels

The most Jeff Minter name of all Jeff Minter's games.

My kid would think it is the lamest thing ever, but, w/e! :)

You forgot that 2 MB REU (RAM expansion unit).

Imagine where we would be today if this was where we were on the C64 in 1982. If we had the concept and ability to create these models and run them on machines and how much that would have evolved by now.

Amazing.

We did. People have been doing impressive AI demonstrations before they're was hardware to run it. E.g. Turing implemented a chess playing program, but had to have his colleage execute it manually to play a game.