The last-modified headers indicate that the front page and docs have not been updated since May 2018. This is not a viable platform and the analysis of Twitter's technical shortcomings is facile. The original 140 character limit, now doubled to 280 characters, was originally constrained by the SMS interface Twitter supported but ultimately imposes a preferred content style that distinguishes the microblogging experience.
The scaling problem is what AWS's James Hamilton use to call the "Hairball Problem" in which there is no single database partitioning scheme that performs well under all workloads. The Subscriptions [1] section of the documentation demonstrates this problem:
> I immediately came up with the naïve solution: keep a list of users to which users are subscribed, then when you want to get updates, iterate over the list and find the last entries for each user. And that would work, but it's kind of costly in terms of disk I/O. I have to visit each user in the list, retrieve their last few entries, and store them somewhere else to be sorted later. And worse, that computation has to be done every time a user checks their feed. As the number of users and subscriptions grows, that will become a problem.
> So instead, I thought about it the other way around. Instead of doing all the work when the request is received, Blërg tries to do as much as possible by "pushing" updates to subscribed users.
This approach works well except when some users have hundreds of thousands of followers which results in an explosion of "pushing" updates. This doesn't even take into account geography related latencies of a global user base. Furthermore, naive attempts at databases without fault tolerance assume that losing recent data is not a big deal. It isn't, the big problem is failures that corrupt data and can't be restored to a known good state.
> The last-modified headers indicate that the front page and docs have not been updated since May 2018. This is not a viable platform and the analysis of Twitter's technical shortcomings is facile. The original 140 character limit, now doubled to 280 characters, was originally constrained by the SMS interface Twitter supported but ultimately imposes a preferred content style that distinguishes the microblogging experience.
Your analysis of Blerg's analysis of Twitter's technical shortcomings is facile. It's a system that the author put together while intoxicated to poke fun at Twitter. Reading any more into it than that is at best highly unadvised.
> It's a system that the author put together while intoxicated to poke fun at Twitter.
The HN title is "a microblogging platform", the landing page has a section titled "But what's wrong with Twitter?", and the documentation has a "Design" [1] section. If the author intended to poke fun at Twitter as a stand-in for all microblogging platforms then leave out the engineering aspects and if the author intended to poke fun at Twitter engineering then land some technical blows. If the point of the exercise is to demonstrate how much code and documentation can be pumped out during a drinking binge then the author should state that upfront; that context is useful.
Despite my ex-roommate's well-intentioned defense, I was probably sober for most of it. It was also a decade ago and I'm somewhat wiser now. As mentioned elsewhere, I learned a lot about making such bold claims to the Choir of Ultimate Pedants. But it was something I had fun with, and I think the world needs more of that. :)
When I see someone writing a database from scratch I search for the rationale and the core problem being addressed. The HN title is misleading and needs a date context which is obviously not your fault but comments attempting to provide the missing context should not be dismissed as pedantry either.
If the context of a fun learning experiment was made clear on the landing-page and/or documentation then this thread could have been about architecture and programming which I think was what you wanted originally. I suspect the broken git repository didn't help matters. Overall a missed opportunity.
The scaling problem is what AWS's James Hamilton use to call the "Hairball Problem" in which there is no single database partitioning scheme that performs well under all workloads. The Subscriptions [1] section of the documentation demonstrates this problem:
> I immediately came up with the naïve solution: keep a list of users to which users are subscribed, then when you want to get updates, iterate over the list and find the last entries for each user. And that would work, but it's kind of costly in terms of disk I/O. I have to visit each user in the list, retrieve their last few entries, and store them somewhere else to be sorted later. And worse, that computation has to be done every time a user checks their feed. As the number of users and subscriptions grows, that will become a problem.
> So instead, I thought about it the other way around. Instead of doing all the work when the request is received, Blërg tries to do as much as possible by "pushing" updates to subscribed users.
This approach works well except when some users have hundreds of thousands of followers which results in an explosion of "pushing" updates. This doesn't even take into account geography related latencies of a global user base. Furthermore, naive attempts at databases without fault tolerance assume that losing recent data is not a big deal. It isn't, the big problem is failures that corrupt data and can't be restored to a known good state.
[1] https://blerg.cc/doc/#subscriptions