I'm a generalist with a focus on backend and infrastructure -- in my career that's meant a lot of diving into logs, probing systems, and solving distributed bugs. In my own time I am working on an object-oriented Javascript framework and a custom autoencoder for visualizing latent states of LLMs. Looking for the opportunity to build something great and solve interesting problems.
From talking to people, the average user relies on memories and chat history, which is not easy to migrate. I imagine that's the part of the strategy to keep people from hopping model providers.
I got a Sovol SV06 ACE a few months ago as it seemed to have most of the nice features of the Bambu (like auto bed leveling) without the closedness. The printer runs Klipper and you can ssh into it. So far there's been one issue where I had to replace a fan but otherwise it's been great. Much cheaper than a new Prusa too.
Firefox has made Mozilla billions over its lifetime by selling the default search engine rights to Yahoo and Google. Chrome, having a much greater user base, would demand a correspondingly higher fee (probably around $10b a year). Now, the other problem is there is no other search engine to compete with Google at that level, but that might change with independence of Chrome.
If you spend 1000W on 1 gigaflops, but could have gotten 1 teraflops instead on newer hardware, you are mostly just throwing money away. Unless fab capacity is severely limited in the future or energy becomes too cheap to meter, the opportunity cost is just too great for your statement to be true.
Been working on an autoencoder that converts the hidden states of transformer models into a spatial representation that can be visualized. Started more on the toy scale but now I'm trying to scale it beyond my humble 3060. Using LLMs to help with torch and such but they are limited in the details of tensor twiddling.
Gemini is probably using ring attention. But scaling to that size requires more engineering effort in terms of interlink that goes beyond the purpose of this release from Mistral.
According to the paper, the dataset goes up to 1978 because that's when copyright law was updated to automatically apply to newswires. It's unfortunate that we got into the situation where academia has to play by the rules wrt copyright while big private labs flaunt it.
This is, unfortunately, how our (US) justice system works.
The entire concept of an "NDA" has been bastardized in this manner, if you think about it. Conceptually you may think of an NDA as protecting sensitive data from disclosure, a sort of intellectual property right. However, it's been co-opted by folks to do nothing more than cover up inconvenient truths because they realize most people cannot afford to either (a) give up money they are promised in the future or (b) bankrupt themselves in their own defense.
So it's basically a game where whomever has the most money can ensure their narrative wins out in the end, because competing narratives can simply be "bought out".
Remote: Sure
Willing to relocate: No
Technologies: Javascript (Node/Web/Bun), Linux, Python, SQL (SQL Server/MySQL/Postgres/Sqlite), Azure, k8s/docker, React, Typescript, and many more
Resume: https://rileystew.art/files/resume.pdf
Email: me @ the above domain
Website: https://rileystew.art/
I'm a generalist with a focus on backend and infrastructure -- in my career that's meant a lot of diving into logs, probing systems, and solving distributed bugs. In my own time I am working on an object-oriented Javascript framework and a custom autoencoder for visualizing latent states of LLMs. Looking for the opportunity to build something great and solve interesting problems.