I also desperately want something like this to succeed. I want to be able to see others' reactions on all NYTimes articles, for example, not just the handful each day they enable comments on, or the occasional one submitted to HN.
And I want it to have all of the standard functionality -- hierarchical comments (like HN and Reddit), upvoting and downvoting, auto-collapsing tangential threads with less votes (like Reddit), giving more weight to users who are consistently upvoted, etc.
But also, I'm not sure if URL's are exactly the right pieces of content to associate with. Not only do you have the problem of determining canonical URL's when sites themselves don't provide them as metadata, but there's a lot of content out there that isn't a webpage: TV episodes, TV series, movies, podcast episodes, podcasts, music tracks, music albums, and so forth. Right now the closest thing there is for some of those (some TV episodes, some podcast episodes) are official posts in dedicated subreddits.
It almost makes me wonder if it would need some kind of dedicated URL scheme that could be managed by Wikipedia or MediaWiki or something, like tv://startrek-strangenewworlds.us.2022/s02/e03#tomorrow_and_tomorrow_and_tomorrow.
Funny, I've always hated scrolling down and accidentally seeing comments on news articles, which are almost uniformly garbage. I think it's a terrible way to present any kind of serious story, it makes for a worse experience.
If I want to discuss an article, there's Twitter, Reddit, etc. The NYT used to have a good public editor (Margaret Sullivan) you could complain to, but she's long gone.
I will say that The Verge has a good approach, where there's a comment section on every story but it's hidden behind a button at the bottom you have to actively click. That's a fine compromise.
Funnily, this is an idea that keeps recurring. I talked with a would-be founder circa 1999 who wanted a discussion sidebar for every website. He too was excited about wresting control of the experience of a site from the site's owners. My memory is kinda fuzzy here, but I think this is what the idea turned into:
It looks like it went from being a global community (which raises lots of questions, both the ones you mention and things like who moderates it) to something with a much clearer revenue model, one where companies pay for add-on forums. And then blew up at the same time Bubble 1.0 did.
The fact that people keep trying this but that we still don't have it makes me suspect there are fundamental flaws, but I've been wrong about that before.
Your url comment reminds me of the concept of the semantic web [0] Whereby we can have a structured machine readable and pure URL structure backed by ontology and linked data. There's a project that is working on this for Wikipedia called Dbpedia. [1]
[0] Unfortunately this concept has been completely bastardized by random research groups shoehorning the technology for EU grants, from my experience working in one such group.
That reminds me of Wikidata[0], which aims to be a general purpose semantic knowledge graph, which runs on Wikibase[1], a MediaWiki extension set that you can run yourself for any collection of data.
I desperately do not want this to succeed. Who moderates it? No moderation means a fractal of spam, hate, propaganda, and low-value posts. You cannot have a system like this that's resistant to Sybil attacks without centralization or federation (or at least not without forcing users to get into crypto to use it... No thanks).
You either accept that it's centralized, and it becomes siloed (see: reddit, Twitter) or you federate it and it's an echo chamber or nearly empty.
Yup it sounds awful, at least with Reddit and to a much lesser extent Twitter the conversation is kept somewhat in control by people having to subscribe to a Subreddit or follow accounts that interest them.
Imagine the signal to noise ratio on some random NY Times article, or any other moderately popular website. I expect most people are thankful the NY Times only enables comments on selected articles.
> Imagine the signal to noise ratio on some random NY Times article
Yes, it's pretty fine on the articles they enable it for. They're sorted by upvotes so the first few comments are usually pretty decent. There's no reason to think it would be different on the articles they don't enable it for, since they're enabled on a wide range of types of articles (straight news, divisive opinion, etc.).
> I expect most people are thankful the NY Times only enables comments on selected articles.
I think it's the exact complete opposite -- nobody is thankful for that. Comments don't appear unless you click to open the comments panel, so anybody who never wants to see comments, never does. But of course people who do want to see comments, are prevented from doing so if they haven't been enabled.
OK, let's spitball. Maybe too much for an HN comment, but here we go:
- Define an open protocol
- There are multiple viewers (browser plug-ins), so different users can choose a different viewing/comment ranking interface experience
- There are multiple "channels" of comments for any piece of content, but usually channels will be topic-wide, sitewide, or even internet-wide. All content for a channel comes from a single server. If channels were pulling from existing sites, an episode of TV might have a channel of Reddit comments, a channel of Vulture comments, a channel of AV Club comments, and anyone can create a channel. (Maybe Reddit and Vulture don't participate at all)
- The person/group who runs a channel does/delegates all of the moderation for that channel. They do this out of the good of their hearts, much like Reddit moderators
- Each plug-in gives you a default, or a choice of, approved/blocked channel lists according to your political/cultural/etc. preferences. These lists are maintained similar to adblock lists, each of which are curated by groups with those preferences. So you can avoid unmoderated toxic channels or view them exclusively
- And so each viewer will combine all of the comments from all approved channels into a single comment list. Each top-level comment can come from a different channel, but all subcomments always belong to that same channel. When you leave a top comment, you can choose which channel to leave it in (or have it default to whichever channel is most popular, similar to picking the most popular subreddit on a certain topic)
- User accounts are managed by separate servers, but user servers don't hold comments/upvotes/etc. They merely provide signing to prove comments came from the same user, index comments left by that user, sum the user's received votes (which can be independently verified by the channels), etc. These servers are similarly moderated to kick out abusive users. Channels don't allow comments from users from user servers that don't kick out abusive users. Ultimately your identity is proved by a private key you can always take to a new server. Probably, the groups behind the most popular channels and/or most popular lists will provide these user servers as well
The details are immaterial. Who stores the data at rest? If it's distributed, you have a cesspool of spam and nonsense. If it's controlled by a central authority, it's subject to the whims of that authority. If you choose a federated hub (a la Mastodon), you're stuck in your echo chamber or an empty room.
The protocol doesn't matter. Right now, those are the only three options (minus crypto) given our current technology. Without going into the technical feasibility of your solution of federated identity with distributed storage or deciding who becomes a moderator or how to avoid a bad actor spinning up thousands of servers to create millions of fake users to spam or any of the other problems, the issue is social: people choose to be in the place where the moderation is most agreeable to them, which will become the natural partition for how the users divide themselves. Whoever those moderators are responsible to (how do you get "fired"? who decides?) ultimately answers the question of centralized, federated, or distributed, and decides the implications and downsides of that approach.
Each channel stores all the comments left on it. (They're not copied anywhere else, except for caching clients might choose to implement.)
> If it's distributed, you have a cesspool of spam and nonsense.
Channels that moderate will not, and most people won't be viewing channels that don't moderate.
> people choose to be in the place where the moderation is most agreeable to them
Yes, those would be the lists of approved channels that clients use, like I said maintained analagously to adblock lists. There would probably be one or two popular "mainstream" lists, and then various "progressive-only", "right wing-only", "lulz" etc. ones.
Think of it basically like Reddit, except each subreddit is its own server (channel) that moderates according to its own policies, and whenever you view a NYTimes article you pull up combined comments from the well-moderated subreddits and not from the badly/un-moderated ones.
> Yes, those would be the lists of approved channels that clients use
Who moderates the list of channels? Who decides which curated lists of channels are available?
> most people won't be viewing channels that don't moderate.
Who decides the moderation policies? Who enforces that the moderators are doing their jobs, nominates new ones, and fires ineffective ones?
> Does that make sense?
Nothing you described is technically infeasible, it's just you're trying to describe a federated system while avoiding storing data on federated servers.
The problem is not in storing or distributing the bytes, it's about policy and governance. What you're describing is a federated system, with power being put in the hands of the people who maintain the lists of channels and the people who decide which lists to publish to their users (assuming they're different people). Someone is still in charge, just not a singular centralized authority. That's a federated system, and they exist already, and the social and political problems with running them are well known.
Like I said, groups of people create lists the way adblock lists are currently created.
> Who decides which curated lists of channels are available?
Viewers will come with default popular lists (like adblockers do), but anyone can add any list or channels directly.
> Who decides the moderation policies? Who enforces that
The owner(s) of each channel.
> it's just you're trying to describe a federated system while avoiding storing data on federated servers.
Exactly, since there are a lot of problems we know are associated with that.
> What you're describing is a federated system, with power being put in the hands of the people who maintain the lists of channels and the people who decide which lists to publish to their users (assuming they're different people)
Exactly. That's really the most crucial part here.
> the social and political problems with running them are well known.
Are they? Because I'm not aware of any federated system set up in this particular way. Subreddit moderation works pretty well, and adblock lists that adblockers use work pretty well, so I'd love to know if there's an obvious social/political gotcha I'm missing here.
I agree with your idea - I've been working on a similar one for a while.
However, what you write above could almost completely describe USENET news groups. Learning what worked for USENET news and what did not will give you a starting place for this idea.
For the most part, USENET discussions worked just as well as Reddit discussions according to the skill and motivation of the moderators, although that was the very early net and most of those involved were educated white men, not teenagers or nations engaging in information warfare.
I'm continuing to work on a design for a federated discussion system just because I think the world needs discussion boards that aren't bound by national law or corporate desire any more than necessary.
However, I think the original intent of the sidewiki stuff was slightly different. That said, they could be a combined system, with one presentation of the various groups/subs/channels being via a reddit like website, and another being a side bar attached to a web browser window. It depends on what is being discussed.
I thought about this sort of thing as a reddit replacement. Browser extension that creates a forum on any arbitrary page. Backend is just nntp so pick your own server.
Can't work on it because of corporate ip ownership and all that nonsense. It's a pretty trivial idea though, pretty much anyone could implement it if you don't bother writing a bespoke news client.
>Can't work on it because of corporate ip ownership and all that nonsense.
Is this an American thing? I'd understand that being an issue if you were using company provided equipment, doing it on company time, or creating a competitor.
But if you're just doing a side project in your own time on your own computer, how can your employer assert any sort of ownership over that?
I really don't think that is the case, but I'm interested to hear why you think it is. In my understanding the community is not controlled by a central authority.
The rules for how Wikipedia is edited are ultimately set by the people who run Wikimedia. They have the final say. It's a centralized service (there is a single Wikipedia at a singular domain, being the sole source of truth).
In the context of this discussion, there is no such notion of a federation of wikis on various hosts contributing to the content. There's obviously no decentralization of Wikipedia.
>And I want it to have all of the standard functionality -- hierarchical comments (like HN and Reddit), upvoting and downvoting, auto-collapsing tangential threads with less votes (like Reddit), giving more weight to users who are consistently upvoted, etc.
This is what I most certainly would not want to see. The prioritisation of "upvoted" content and comments is the number one source of all kinds of perverse problems with modern social media (echo chambers, manipulation, incentivisation of simplistic or short-form content, promoting base anger and outrage, etc)
> The prioritisation of "upvoted" content and comments is the number one source of all kinds of perverse problems with modern social media
I (unsurprisingly) like the HN method… the public can’t see the exact number of votes for a comment, but the comments are still ordered by number of votes. And comments with negative votes are styled differently, but you don’t know exactly what the vote totals are.
Yes, you can still have echo chambers and manipulation, but it’s more difficult to effectively manipulate. (At least without a lot of external coordination, in which case the site is already fighting a losing battle).
Yep, the main problem is that whatever.com could be a completely different page for user A and B, and if they both commented on it, they'd be commenting about different things, then user C goes to view it, and is confused.
Which could be helped by storing the page source or something (which still has a ton of issues associated with it) but then you have a problem of discovery, and how do users tell the pages apart with the same URL? Simple version numbers?
Another problem is you might have 2 URLs that are basically the same page, but with some query parameters that subtlety distinguish them - they are different URLs though. Should there really be two discussion sections for a page that's only differentiated by a referral link or a stylistic preference? Probably not, but there is not an objective metric here we can use to make the call.
What if we use just the base path, e.g. facebook.com? It could be useful for small sides but for large sides you'll run into issues with scoping topics.
There are browser extensions like Reddit Check or What Hacker News Says which will show Reddit and/or HN comments on any website you visit. So it's very similar in functionality. Sometimes it can be useful.
I desperately want some annotation service/standard to succeed. The idea goes as at least far back as 1993! (https://en.wikipedia.org/wiki/Web_annotation) Discussion is too important to be subject to the editorial whims to the host of the content it is discussing. I'd love to take the insight of the Hacker News community (for example) everywhere I go on the web.
I don't think it's for lack of trying. From my understanding, bidirectional links / annotations are the big thing the web is missing compared to Project Xanadu. It seems to be an idea that lots of people get bitten by, never quite goes anywhere in the general sphere, and they end up pivoting to a narrow implementation wholly contained on one website.
In effect it's not that much different from Reddit, being a link aggregator where people can comment on the links. The main difference would be that it works on any website and it doesn't have to be posted to reddit to spark a thread.
I've read people's comments on here but I still think this is a great idea. As long as it's properly executed, meaning there should be no centralized ownership and control.
And yes I know that's a catch 22 because there would be no moderation, but let's put that technical detail aside for a moment and focus on the fact that this would allow users to help other users with any website. Whether that be criticism, warnings, or praise, depends on the website!
I know that technically being a link aggregator is reddit's main thing, but almost all my posting and reading has been more like a discussion forum, with most posts not being links but text posts (which may or may not have one or more links).
While one type of discussion can of course be the discussion about exactly 1 link, I've still always found it slightly weird that reddit makes this distinction between a link and a text post, rather than just have text post (in which you can put links anyway).
> And yes I know that's a catch 22 because there would be no moderation, but let's put that technical detail aside for a moment
You cannot put that aside while considering this idea, because without moderation this idea is dead in the water at best, and becomes a harassment tool at worst.
"I know my new pollution-free power plants cause urine and feces to emerge unbidden from people's ceiling fans at night, but let's put that technical detail aside for a moment."
I was thinking of doing something like this few days back. I planned out everything like writing a sidebar extension for Chrome & Firefox. The whole time I was wondering why anyone else did not think of this.
Then I realize this will not work on Mobile and almost gave up on idea immediately.
It is rather strange Chrome has no extensions on mobile. I wonder if it has more to do with iOS or an insidious desire to control more of the web experience.
A couple years ago reddit switched from a link aggregator to full blown content stealer and rehoster service. Including bans for linking to the original picture/video.
It's more of a challenge than centralized moderation, but spam and other junk comments could be managed through personalized reputation/trust systems, or subscription/collaborative filtering systems.
I remember this, I always found the concept fascinating.
But looking back, looking at what the web has become, there was no chance this could have ever worked. At best, it being centralized, it would have become essentially a Google controled social-media-ish space where all the "inconvenient" comments would be removed anyway. Probably Google would even disable the comments on some pages, like disabling sidewiki comments on the pages of some big newspaper that demanded Google to do it because people were criticizing that giving media outlet. Let alone the sheer problem of dealing with spam, which, specially now with LLMs will get much harder to filter.
All these things could have been solved and dealt with it one way or the other, if we REALLY wanted. But ultimately, I think there was no interest on the people who maintain the internet, the people who create the tools that most use to access the internet: there is no reason why they would keep a tool that would allow people to voice their opinions.
On the contrary, what we have been seeing on the internet is precisely the oppose happening. They removed the dislikes, they removed comments in many videos. Hell, Netflix used to have an option where people could leave reviews on the movies they watched. The internet of 2020 is clean and "family-friendly" and "politically correct" and it fits the narrative and the current thing, and they knee before the big corporations and anything that could hurt their brand must be eliminated.
In the 2020s internet there is no space for a service like Google Sidewiki, and even if someone build this as an open standard, and even if the spam issue wasn't a problem... Google, Apple and Microsoft would never implement and built this feature on their browsers, and most people using the internet are too noobs to sideload an extension themselves that enables this feature to a point of it gaining enough of a userbase to matter and actually be useful.
In short, this whole product is a child of a more naive internet where people still believe that services could actually be better. Funnily enough, Gab launched a similar extension called "Dissenter" some years later and it got banned from Chrome and Mozilla extension stores.
> Hell, Netflix used to have an option where people could leave reviews on the movies they watched
There's a very specific reason Netflix removed reviews / star ratings in favour of a simple thumbs up & down system: those were mostly used to recommend content, and they found that ratings are uninformative (something true for most recommender system applications) and even harmful because giving a precise score keeps many people from giving feedback -> lowering the quality of recommendations. See this article [1].
The same train of thought would go for something like Google Sidewiki: it likely just didn't get enough traction (vs digg, reddit, etc.)
Just raising this since people might not know about it, but this is a Web standard now https://www.w3.org/annotation/. There are a couple of companies which have created browser plugins for this.
That is super interesting, but looking at the diagram[1] my first gripe is with the nature of their decentralization.
It's not a bad idea, it's a good first step, but they're proposing that any group or private person can host an annotation service.
This will cause first of all competition, because which service has the most annotations? Using my annotation service hosted on my minecraft server might be useless because it only has my own annotations.
So obviously users will want to use a larger service, and here comes all the standard issues of a user-driven internet, trust, donations, groups forming and such. Also you might end up in a bubble, seeing annotations only from one particular side of the political spectrum.
Of course this model does not have the same issues as the fediverse where annotation services can de-federate from each other, the user is free to pick and choose whichever they want. There might even be a helpful counter in the UI that shows how many annotations a particular service has for this website.
The only real solution to all this is some sort of global database, like IPFS perhaps, where we can store the annotations. And then we can all individually host gateway servers to this database that the end user connects through.
But that has its own problems of course, you can't just magically make a distributed database without heavy bandwidth consumption and its own hosting requirements.
Web Annotations spec is a great starting place & I hope we can some day see some real breakout wins from it. I'd love to see some cross integration with ActivityPub, as a syndication/transport!
What I really want is no site to win; true success doesn't come from centralized solutions. Each user should have their own annotation feed!
What I'd love to see is something like the return of blogrolls, an annotater's list of people they follow. Users promoting users. A good extension could let us do a N-degrees exploration, let us see comments of people we follow, people they follow, people those people follow... Expanding the network & implicitly suggesting to us other people we might want to follow.
I personally really really loved the social aspect of del.icio.us. Finding other people who were searching deep for interesting content was something I spent time on & it rewarded me handsomely, back in the day. I hope for similar thing here we're not just using this to have annotations, we're also using it as a content discovery tool, seeing what content there is from people we follow.
I'd try to suggest sites should have something like pingbacks, to make it so the site can keep track of annotations. But that would let them filter anotations which I don't like, and more problematically, it's an opt in mechanism. Having centralized search systems seems obvious. Ideally maybe some kind of kademlia hash might offer a P2p alternative. It's quite possible maybe bittorrent pex's P2p layer could be used/abused for this.
There's also a competing W3C standard named Webmention [1] based on WordPress' pingback/linkback protocols [2] which in turn were based on XML-RPC and exploited for DDoS attacks, like most things WordPress (it's one of the reasons your http access logs are chock full of 404s for xmlrpc.php). AFAICS, pingback remains the most used method though, or the only one that ever went mainstream before web commenting consolidated onto a couple news aggregators, HN and reddit among them.
The Web annotations protocol has been published as W3C spec in 2016 already (so not "now"), and, as a child of its time, uses god-awful JSON-LD, just like previous W3C specs chased XML whether it was a good fit or not when exchange of text data is one of the actual use cases for markup languages.
Is there an English word for always getting it wrong and blindly promoting formats? In German, there's the term Schlaglochsuchmaschine (pot hole search engine) as a metaphor borrowed from the automotive domain.
Hi, offtopic from the above but, I am trying to use your sgml npm package to parse some OFX files, I wonder if you could give some guidance on a problem i am having? I am trying to move away from a wasm compiled OpenSP.
I'm using am example from your website to try and convert from ofx to xml.
"content must start with document element when document type isn't specified" is the error I get.
Ask on Stack Overflow and include the term "sgml". If you're not on SO, temporarily add a personal mail address to your profile (in the about field) so I can get in touch.
Problem with this idea is moderation - Google is not going to moderate a discussion that is happening in the side bar of your site.
What is love to see, and tried to make once, is something like this but just shows discussions about a URL from Hacker News/Reddit. Sites that are moderated and people have actual quality discussions (usually).
I tried this a long time ago (2014) with a extension project: http://www.3delement.com/?p=394
Searches for discussions and backlinks to your current URL on used. I gave the project up pretty quick but it's still something I'd love to see as a Chrome extension (though might not actually be possible now after the Reddit API changes)
I use Hypothesis a lot, and it's underrated, but it's pretty buggy, though. The experience is often not smooth and requires technical understanding to work around roadblocks. E.g. without installing the extension (which is Chrome-only and not available for Firefox) try annotating HN comments.
Back when The Screen Savers was on ZDTV--and ZDTV existed for that matter--they used to host a chat room for the show that would appear at the bottom of the screen during live broadcasts, kindof like a ticker tape. You accessed the chat room by using a web browser that would turn every URL into a chat room. I don't remember if it was standalone browser or a plugin. I highly doubt it was a plugin but my memory fails me here.
Didn't a version of this exist where you could collaboratively draw over any website using its contents as the background? (You can also guess the end result.)
The biggest reason why I built it was to be able to get insight from a community on every site. Most web sites don't have comments and some filter a lot so I wanted to have a platform where the website owner can't modify or change the comments from people.
I didn't work on finding users though and it has about 10 people using it.
This is an example of how Google valued shipping new products but put little emphasis on keeping those products active. Someone would get a project like this approved, push through a delivery, collect their bonus and/or promotion and then move on to a new product. My understanding is that these services shutdown because the team behind them has also moved on.
> Although, we basically have the same thing when people share links and people comment on them.
True, but having the comments appear naturally in context makes all the difference. Imagine seeing comments/corrections on a New York Times article that aren't moderated by the paper, or comments on a real estate listing that list all the problems that the listing omits, or reviews of a show on Netflix, etc.
Both! I see a lot of real estate listings where I'm sure the average browser is unaware that a large construction project is about to begin on an adjacent property and ruin their views. For rental listings, prospective tenants might not know about excessive noise from a nearby road.
I use Hypothesis a lot. It works on arbitrary sites by just invoking their javascript bookmarklet which I keep in my bookmarks. It is also very easy to integrate with static sites which is nice.
I was just thinking about how I wanted something like this the other day - like a lot of other PM-type people at my work, quite a bit of my output is "docs and decks", which I have found carry around a strong "work in progress" vibe. Whether you're sharing one out on your screen in a video call or sending it to someone so they can open it themselves, it's impossible to make a piece of work feel like a finished product, it always feels like a workspace.
I'd love to start publishing stuff in the form of static websites or another similar medium, but Word's ability to add annotations to specific runs of text and allow for conversations on them is a must-have.
Too late to edit, but wanted to ask the question directly: can anyone recommend a static site generator or CMS plugin that adds this functionality to a content-driven site? Anything/any generator or CMS is fine, I'm in the "poking around" phase of getting ideas. It would be for small-scale usage, on the order of a team annotating and discussing simple web content with basic text.
Maybe there should be a proper wiki for this. A sort of wikipedia but not for abstract knowledge but for url's (that fit a certain profile: blog posts and the like).
A browser extension could pull the wiki data when visiting a url that has been commented on.
The possibilities for abuse and gaming would be there of-course. As with wikipedia and any multi-user online platforms it would be down to policies, moderator tools and enforcing accountability to try to keep negative patterns spiral out of control.
But fundamentally, once a url is permanent published resource it makes sense to think how we document public reactions to it.
I’ve wanted a general annotation tool for websites (and email messages) that would be private. It would be nice to be able to attach a note to a message in my inbox about why I’m holding onto it, or to be able to have persistent highlights and annotations on webpages. I guess I’ll just keep dreaming.
Didn't one of the most recent versions of Explorer integrate something like this natively? I remember thinking it's neat, but it's hard to reach critical mass, and without reaching it, it's not very useful.
And I want it to have all of the standard functionality -- hierarchical comments (like HN and Reddit), upvoting and downvoting, auto-collapsing tangential threads with less votes (like Reddit), giving more weight to users who are consistently upvoted, etc.
But also, I'm not sure if URL's are exactly the right pieces of content to associate with. Not only do you have the problem of determining canonical URL's when sites themselves don't provide them as metadata, but there's a lot of content out there that isn't a webpage: TV episodes, TV series, movies, podcast episodes, podcasts, music tracks, music albums, and so forth. Right now the closest thing there is for some of those (some TV episodes, some podcast episodes) are official posts in dedicated subreddits.
It almost makes me wonder if it would need some kind of dedicated URL scheme that could be managed by Wikipedia or MediaWiki or something, like tv://startrek-strangenewworlds.us.2022/s02/e03#tomorrow_and_tomorrow_and_tomorrow.