I don't have the hardware to run the 60B model to test this at the moment -
How does it perform with programming, for example making a basic python script to scrape a website, or a bash script, etc?
I've managed to run the 13B* at 8bit with decent performance on a 4090 - but it's only 24GB of VMRAM so I've been struggling to run the 30B at anything more then a snails pace.
How does it perform with programming, for example making a basic python script to scrape a website, or a bash script, etc?
I've managed to run the 13B* at 8bit with decent performance on a 4090 - but it's only 24GB of VMRAM so I've been struggling to run the 30B at anything more then a snails pace.