This is the same problem in the US. Legislation (that protects the environment, minorities, the ability to compete in the market, etc) that took years, even decades to get signed into law, is getting repealed today by the current administration via executive order or simple majority vote. Because sabotage is much easier than building something.
Unfortunately the only answer that I know of is eternal vigilance, which is the price of liberty.
I decided to look up who that saying is attributed to, and apparently it's John Philpot Curran, not Thomas Jefferson. But I like Orwell's saying better, because it shows why all of you are just as ineffectual at steering government policy as I am:
I'm sure you just linked the first google result you found, and it's not like the internet wasn't full of crappy 'quote' websites in the halcyon days of 2021, but it's incredibly depressing to click that link and get drowned in paragraphs of worthless AI blathering.
After a quick search - and ignoring Google's helpful clanker who tries to point you to the _wrong_ Orwell text - it's not hard to find a clean source:
So every area of our lives that feels like it doesn't work like it used to - cost of living, healthcare, education, antitrust enforcement, journalism, accountability at the highest levels - represents a segment of the economy which has been corrupted.
Through this lens, socioeconomic policies start to make sense. For example, if your goal is to skim a fraction of the income from everyone in an economy and redirect those funds to specific goals/organizations/individuals, you could put tariffs on common goods and pass the funds collected on to companies granted large government contracts. Then the largest companies like GM and Ford see their profits reduced or even show a loss, while Grok and Palantir have all the money they need for mass surveillance.
Explanations for regulatory capture aren't normally this reductive, but wealth inequality has reached such monumental proportions that the simplest answer tends to be the right one when the needs of the few outweigh the needs of the many.
So through that lens how do you explain the healthcare situation before the start of tariffs, or is your entire worldview shaped based only on the news you read in the last three days?
The alternative would have been some kind of single-payer option where US tax dollars would fun Medicare and/or Medicaid for all. Here is an explanation I found:
A public option likely would be 20-50% less expensive than private due to the 80/20 rule and the fact the European healthcare is about 1/2 the cost of US healthcare, so a current $400/month plan might be $200-320/month:
Unfortunately Obamacare (Romneycare) forced everyone to get insurance or face a tax penalty mandate, which was lifted by Trump in the Tax Cuts and Jobs Act of 2017:
Knowing these facts, what's really going on? Two things:
* Obamacare did little to address the natural monopoly aspect of healthcare, so costs exploded
* Republican concerns about government overreach into our private lives (knowing our info and determining who receives care) and negative impacts on private health industries have not been heard
Natural monopolies are things that everyone needs, such as water, sewer, trash, electricity, education, healthcare, etc:
Meaning that eventually running a capitalist system results in 1 or 2 companies controlling the entire market and charging whatever they wish, since high cost of entry prevents competition. Without competition, there is no supply and demand curve to counteract price increases. In other words, whoever controls the water can sell it at any price. Or insulin, or Epipens, or Hepatitis C treatments.
The only way to bring costs down on a natural monopoly is through regulation. Society chooses which firm(s) will supply the good or service, and how much it will cost. Overages are paid through subsidies, which can be high, but are at least under public review, unlike when private industry controls a market.
These are the reasons why healthcare got so expensive. But the meaning behind it, that's more nuanced.
The US is very into individual responsibility. Our belief is that a strong citizenry ensures a strong nation. Because we defeated not just fascism in WWII, but the rise of socialism and communism during the Cold War. Our privately funded athletes beat state-funded athletes in the Olympics. Our private industry runs more efficiently than (for example) national construction projects in Russia and China which built cities that nobody lives in. We have our own pork barrel projects, but they tend to be limited by public scrutiny, unlike in communist nations.
The US is also very into privacy. We have medical privacy laws like HIPAA which may not exist with state-funded medical care. Now, this is a half-truth, because Europe has arguably the same or better privacy than we do. Because there is little incentive to sell medical information there, unlike here.
What it really comes down to is that people who are used to paying through the nose for US health insurance after a lifetime of hard work aren't ready to see others receive it for free through the government. They don't want someone determining how long they have to wait for care, or if they receive it at all. They perceive government red tape as making health providers even more expensive or putting them out of business. As in, why would doctors go to medical school merely to be paid little more than teachers and other public servants? Yes, privacy may be impacted with single-payer. But in the US, the answer is usually about $$$.
Sadly, this debate has resulted in the lose-lose we see today: rising costs with no backstop, and threatened privacy due to regulatory capture by health companies which have misaligned incentives and are too big to fail.
-
My solution to this political impasse would be a gradual transition to single-payer healthcare (probably Medicare for All) on an opt-in basis, either at the state or individual level. Knowing that taxpayers who opt in may pay a higher rate for a time to supplement those on private insurance. Until enough people are in the single-payer system that it becomes self-evidently better and the majority switch to it. Similar to private school vouchers, except going the other direction: public healthcare vouchers.
People could still buy private insurance to supplement Medicare for All and go to the front of the line. I don't like it, because I believe that healthcare is a human right that shouldn't depend on money, but this is America. If someone has the money to pay doctors overtime, then it probably makes little economic sense to stop them.
Medical research should go back to the previous university/publicly funded model. So grants would be available for companies pursuing billion dollar cures for cancer and other diseases which diminish quality of life or its duration. Then medications would be sold at close to wholesale price. This eliminates the current problems where pharmaceutical companies sell treatments instead of cures, that tend to start out very expensive.
-
Sorry this got so long. I believe that you'd receive a similar summary from an LLM, and that without this context, a debate would fall into dogmatic attacks that lead nowhere.
"Nurses should be at the very top of our social hierarchy but we live in a world where it’s just the opposite," she says. "This film is a love letter to the profession."
I moved furniture for 3 years in the early 2000s to support my shareware business. Something like 500 moves, sometimes 9+ hours per day, occasionally 6 days per week. It made me who I am today, but also broke me in countless ways.
I witnessed single mothers giving us $20 tips each (around $40 today) while wealthy people didn't even offer us water. I saw the best and worst of the human condition, sexist pay policies, how workers are exploited by not being provided a schedule for the next few days so they have to call in each morning, how truck fuel costs more than workers' pay, how Right to Work states allow businesses to throw employees away on a whim, how tax brackets at the bottom create the impression that any additional pay gets skimmed by the man, among a great many other injustices, and how all of those conspire to keep the working class down so that a handful of individuals can become fabulously wealthy.
With every improvement in tech, I see the gulf widening. We can talk about how poor people now have cell phones and flatscreen TVs, while conveniently ignoring how people with a net worth over $10 million who couldn't spend that money in a lifetime are now buying politicians to shred the social safety net, among other dubious endeavors.
I say with complete confidence that the arrival of AGI will bring about ultimate wealth inequality. I foresee a world where 10 billion people work performatively to survive long after robots can do the work better, while less than 1 million people live like gods, free even of senescence. Assuming that we stay on this timeline and don't shift to a more equitable one.
I went into computers in the late 1980s to eventually build an android like Data. I didn't know that Turing test-passing AI would arrive 20 years early, or that I would spend the first quarter of the 21st century hustling to make rent due to unfortunate geopolitical realities driven by unmitigated greed and regulatory capture.
Now I'm not so sure that I even want to stay in tech anymore. It has been anything but kind to me. Every time I level up, so does the world, and expectations upon me just grow for the same pay. My people-pleasing has cost me my health on a number of fronts. I know that someday, I'll have to choose computers or my life.
After all of that, one might think that I'm all doom and gloom. But I'm not. I've come to treasure my time at the warehouse as a teaching tool after a great deal of shadow work. I admire my foreman for being the provider that I can only hope to be someday. I look in awe upon the borderline homeless vets, deadbeat dads and ex-cons who showed me what it is to give without expectation of reward. I see them in all of us, even the people I disagree with, and that gives me hope that maybe we can come together and avoid the iceberg that's about to sink this ship.
I'm reminded of this scene from Jaws which always stuck with me:
I see a world divided into two camps: one that does much for little, and one that does little for much.
It makes me wonder where we all sit in relation to this. What kind of impact we could have on the future, and how that might help everyone to self-actualize.
I don't pretend to have the answers. Some of my best years happened when things were at their worst, and vice versa.
But I do know that everything is upside-down right now, and always has been, since the dawn of civilization. I feel that tech won't really be tech until it addresses and undoes that injustice.
That's why I write this malarky when I should be working. For all the other working people who don't have time to say what needs to be said.
Although I am younger than you, I feel somewhat similar. The never ending treadmill of technology seemingly makes things worse rather than better. I’m working in robotics, and I think is this actually the right thing to be doing? - but if I dont, then I wont eat and afford my future.
Really feels like society is burning the candle at both ends.
After writing a response about using this for games below, it occurred to me that most tunneling solutions have one or more fatal flaws that prevent them from being "the one true" tunnel. There are enough footguns that maybe we need a checklist similar to the "Why your anti-spam idea won’t work" checklist:
Your solution..
( ) Can't punch through NAT
( ) Isn't fully cross-platform
( ) Must be installed at the OS level and can't be used standalone by an executable
( ) Only provides reliable or best-effort streams but not both
( ) Can't handle when the host or peer IP address changes
( ) Doesn't checksum data
( ) Doesn't automatically use encryption or default to using it
( ) Doesn't allow multiple connections to the same peer for channels or load balancing
( ) Doesn't contain window logic to emulate best-effort datagrams over about 1500 bytes
( ) Uses a restrictive license like GPL instead of MIT
Please add more and/or list solutions that pass the whole checklist!
You could just use streams - they are extremely lightweight. But those would then be reliable datagrams, which comes with some overhead you might not want.
So how hard would it be to implement window logic on top of RFC9221 datagrams?
I'm not sure I fully understand this window logic question. QUIC does MTU discovery, so if the link supports bigger datagrams the MTU will go up. Unreliable datagrams using RFC9221 can be sent up to the MTU size minus the QUIC packet overhead. So if your link supports >1500 bytes then you should be able to send datagrams >1500 bytes using iroh.
Fragmenting datagrams (or IP packets) is generally not a good idea. All protocol designs have been moving away from this the past few decades. If you want unreliable messages of larger than the MTU maybe taking some inspiration from Media-over-QUIC is a good idea. They use one uni-directional QUIC stream per message and include some metadata at the start of each stream to explain how old it is. If a stream takes too long to read to end-of-stream and you already have a newer message in a new uni-directional stream you can cancel the previous streams (using something like SendStream::reset or RecvStream::stop in Quinn API terms, depending on which side detects the message is no longer needed earlier). Doing this will stop QUIC from retransmitting the lost data from the message that's being slow to receive.
Right, I should have been more clear about that. Window logic was perhaps the wrong term, since I don't care about resends.
The use case I have in mind is for realtime data synchronization. Say we want to share a state larger than 1500 bytes, then we have to come up with a clever scheme to compress the state or do partial state transfer, which could require knowledge of atomic updates or even database concepts like ACID, which feels over-engineered.
I'd prefer it if the protocol batched datagrams for me. For example, if we send a state of 3000 bytes, that's 2 datagrams at an MTU of 1500. Maybe 1 of those 2 fails so the message gets dropped. When we send a state again, for example in a game that sends updates 10 times per second, maybe the next 2 datagrams make it. So we get the most recent state in 3 datagrams instead of 4, and that's fine.
I'm thinking that a large unreliable message protocol should add a monotonically increasing message number and index id to each datagram. So sending 3000 bytes twice might look like [0][0],[0][1] and [1][0],[1][1]. For each complete message, the receiver could inspect the message number metadata and ignore any previous ones, even if they happen to arrive later.
Looks like UDP datagram loss on the internet is generally less than 1%:
So I think this scheme would generally "just work" and hiccup every 5 seconds or so when sending 10 messages per second at 2 datagrams each and a 99% success rate, and the outage would only last 100 ms.
We might need more checklist items:
( ) Doesn't provide a way to get the last known Maximum Transmission Unit (MTU)
And optionally:
( ) Doesn't provide a way to get large unreliable message number metadata
Iroh will do hole punching through NATs. It will even work in many cases when there are NATs on both sides.
There are some limitations regarding some double NATs or very strictly configured corporate firewalls. This is why there is always the relay path as a fallback.
If you have a specific situation in mind and want to know if hole punching works, we got a tool iroh-doctor to measure connection speed and connection status (relay, direct, mixed):
There might be some confusion here, holepunching is a core functionality of iroh. There are still some firewall configurations that iroh can not yet holepunch and that can still be improved, but in general the holepunching works rather well.
It would be nice if dumbpipe revealed the local and remote IP and UDP port numbers via something like STDERR or a signal so that apps could send UDP datagrams on them with ordinary socket calls. I'm guessing that QUIC uses a unique header in its first few bytes, so the app could choose something different and not interfere with the reliable stream.
A better solution would be to expose the iroh send_datagram and read_datagram calls somehow. Maybe if dumbpipe accepted a datagram flag like -d, then a second connection to a peer could be opened. It would recognize that the peer has already been found and maybe reuse the iroh instance. Then the app could send over either stream when it needs to be reliable or best effort.
This missing datagram feature was the first thing I thought of too when I read the post, so it's disappointing that it doesn't discuss it. Mostly all proof of concept tools like this are MVP, so don't attempt to be feature-complete, which forces the user to either learn the entirety of the library just to use it, or fork it and build their own.
IMHO that's really disappointing and defeats the purpose of most software today, since developers are programmed to think that the "do one thing and do it well" unix philosophy is the only philosophy. It's a pet peeve of mine because nearly the entirety of the labor I'm forced to perform is about working around these artificial and unintentional limitations.
> It would be nice if dumbpipe revealed the local and remote IP and UDP port numbers via something like STDERR or a signal so that apps could send UDP datagrams on them with ordinary socket calls.
I believe this would be even more unreliable than UDP, since Iroh is also capable of using a relay server for when hole punching can't be performed, and Iroh also handles IP migration.
> it appears to be linux and macOS only
Iroh should work on Windows, IIUC, just the installer and possibly prebuilt binaries aren't provided. But dumbpipe isn't designed for UDP anyways, it's closer to a competitor for socat/nc.
The video is the epitome of applying solid engineering principles. Where most stop at a minimal viable working example, it sails past that and gets real work done solving foundational problems, using a waterfall approach that expands leverage at each level of abstraction. Complete with outside-the-box thinking and insights which help save the viewer from repeating unnecessary steps, so that the end goal is reachable using modest tools and readily available materials.
I would gladly work with Aaed, and I must admit that I am jealous that he and his friends are living out the dreams that my friends and I had in the 1990s.
-
The Boomers didn't quite understand what we were trying to accomplish, as they had already done the important work of manifesting the equality that had previously been a dream in the US, so these sorts of innovations were gravy. But Gen X was raised on Star Wars, so will not be withholding mental energy from future generations. Unfortunately we are also victims of trickle-down economics, so have little to pass forward except for wisdom gleaned from the school of hard knocks.
We had everything we needed to build these exact types of projects except for the time and resources that were hoarded by our elders. So the vast majority of us worked our lives away, barely making rent by prospecting the internet as miners for people with money, without ever striking gold in the vast majority of cases. Then watched as the fruits of our labors were used to build McMansions, expand monopolistic enterprises like private equity firms, or be simply wasted on frivilous expenses instead of reinvested in automations to create residual incomes. This led to us developing mental health issues like addiction due to the misalignment between the lives we had to live vs what might have been.
Then we lost our heroes, like when Y Combinator went from an indie startup funder to a vetting VC like all the rest. And when Elon Musk pulled up the ladder behind him after accomplishing so much, then used his wealth to dismantle the social safety nets which make indie work possible. I don't believe that these statements are political, as they objectively describe the unwritten history of how some became so wealthy while most struggled, and the irony that I write this as I stand on the shoulders of those fallen is not lost on me.
Now we have everything we need to build R2D2 and C3P0 right now, today.
So I'm hopeful that the next step will be to create a meta economy within the status quo that distributes resources outside of the artificial scarcity created by the previous dominant systems of capitalism, socialism and communism. I believe that a gift economy loosely resembling solarpunk has the potential to liberate humanity from forced labor so that every individual has the opportunity to self-actualize in a reasonable time frame and still experience the joys of leisure time and youth.
In practice, this will expand wealth redistribution models like Patreon and the WiX Toolset maintenance fee under an umbrella similar to the Humble Bundle, to level out long-tail effects and socialize gains while privatizing losses. Note that this works exactly the opposite of how most major economies work in the world, with the exception of nations like Norway which uses its sovereign wealth fund from nationalizing its oil companies to pay its citizens a pension that may someday become UBI.
I realize that these points are mostly excuses and platitudes. But they are in no way meant to diminish the efforts of hackers - on the contrary, they are intended to bolster them by adding meaning to the work and convey why it's so important to those that came before.
So I write this out today to record in the annals of history that the nature of the problems we face is no longer technical, but spritual.
Ah Urbana-Champaign, my alma mater. I was there from 1995-99, starting the year after Andreessen took NCSA Mosaic and made it into Netscape, when the internet hit the world. It warms my heart that KAMS is still there, at that time arguably one of the frat boy capitals of the world ..the wafting scent of vomit baked onto the sidewalk stands out vividly in my mind.
Back then the drinks of choice were Aftershock and Goldschläger, looks like the first was discontinued?! Obviously alcohol poisoning had nothing to do with it. Ice 101 was huge, Everclear (my roommate did a shot which turned to plastic foam in his mouth), and home-brewed wine made of frozen grape juice, sugar and bread yeast which tasted like ham and had to be thrown out if one substituted orange juice instead. Not that I condone such research.
My first cocktail was an Amaretto Stone Sour at Joe's (still there too it looks like?), which didn't make the IBA list:
Midwest beer stunk so our first binges were on MGD (notes of corn), the Beast - Milwaukee's Best (notes of grass clippings) and if we were desperate Miller High Life (notes of hay). Keystone Light was considered too crude due to the specially lined can which tasted of washed charcoal. The best we could get was a Black and Tan (Guinness and Bass) which was $4, ordinary beer like Bud was always $2 or less. A case of Coke was $4, Old Milwaukee $8, imagine the possibilities. Minimum wage was $4.25 but went up to $5.15 in 97, and a night out was $20, which none of us had because we lacked jobs or even cars, but saved by crashing events for free pizza and buying ramen.
I fondly remember the first time I rolled around on the ground after 3 free beers at the frat next door where there were actual girls and ended the night dancing in a circle, arms locked shoulder to shoulder with other drunks singing American Pie. That began a 4 year quest to get into a party every weekend or bust. The weeks consisted of pre-partying in the dorms, bar crawls, basement parties and after-hours parties until 4 am, with some studying mixed in. We often made our 8 am lectures.
One time I was in Chicago and an unhoused man (back during the politically correct era we still said homeless) was selling good booze out of a shopping cart for a newspaper promotion. I bought a bottle of Drambuie for $4 and began my excursion into "mixed drinks". I had no idea what I was doing, so crafted my own, my favorite of which was The Boot (whiskey and Diet Dr Pepper), but during the summers while camping I drank The Meagerita (tequila and Squirt). One can hardly call it a cocktail with 2 ingredients, but it is what it is.
The air today feels uncannily like 1995. There's a palpable feel that everything is wrong. Making fun of it all once again becomes our civic duty. AI just landed, which will disrupt established rich people as surely as the internet did then. There seems to be a 30 year generational cycle. So in the 90s we dressed like the 60s like kids today dress like the 90s. The counterculture is so overdue that it's already here. It hurts to dance as the world burns but history demands it for our own salvation.
Also it's technically Romneycare. The political right got everything it wanted with it, similarly to when Nixon passed HMO funding that tied insurance to work, basically forcing a work requirement in order to get healthcare:
This might be the biggest thing that caused US healthcare to double in cost vs the rest of the developed world, which provides universal or single payer healthcare to their citizens for free or nearly free.
Also the article got the numbers wrong. To be accurate, they should have said $400 or more, roughly $5000 per year for working adults pushing 40. More for families. That was 10 years ago when I was applying, I'm sure it's more now.
I'm surprised that a startup isn't providing transportation for medical tourism. A friend of mine lived in Costa Rica for a while decades ago, and care was covered even for noncitizens. So a round trip ticket and living abroad for weeks would still be cheaper than insurance plus deductible here.
It's to the point where half of you will block nationalized healthcare for the other half, even though we would continue paying double to cover you for free too. So much for objectivism.
Edit: I forgot my point, which is that I believe that healthcare may be a natural monopoly, which is when something that everyone needs must be regulated, or else it becomes a monopoly where the vendor charges as much as possible because people have no choice:
Ya I'm shocked by it too, said as a Gen Xer born in the late 1970s, occasionally a Xennial.
I partied for 4 years of college which is something like 30 years in sober adult terms. Our ragers were reminiscent of Animal House and Revenge of the Nerds, all of those old party movies that didn't age well. Scenes from Hackers, Fight Club, The Matrix, Trainspotting, Go, Swingers, Made, 200 Cigarettes, SLC Punk, Dazed and Confused, PCU, even Undergrads (a cartoon) were so spot-on for campus life, living for the weekend. Can't Hardly Wait, American Pie, Varsity Blues, Waiting, Superbad, etc came later, and I almost consider those watered down versions of the feral partying that happened earlier just as the internet went mainstream, but still canon.
A Friday night at my city's bar scene today looks like what our Sunday or Monday was. People half tipsy on 2 drinks, even though they're Ubering home later. The faint scent of ganja now instead of basements filled with smoke and first timers trying laughing gas. Nobody puking or disappearing around a corner to relieve themselves. No sound of bottles shattering. I feel like a curator of a museum now, a derelict from a forgotten time.
In fairness, I went to college in the midwest, where there was nothing else to do. Now the West Coast has effectively legalized drugs, awakening much of the country to the full human experience, and people have done the trips and plant medicine and maybe realize at a young age that alcohol and tobacco are rough drugs that tear you up. Which is admirable, but they also prepare you for getting torn up as an adult. To miss out on learning how to make your way home on drunk logic before you black out seems like a crucial rite of passage has been lost.
And it shows. In our country's embrace of puritanical politics like we saw in the jingoist 2000s, regentrified for the antivax era. In the worship of unspoiled beauty, idolizing of influencers, pursuit of financial security over visceral experience. In the fanboyism, bootlicking and drinking the kool-aid for every new evolutionary tech that cements the status quo instead of freeing the human spirit in a revolutionary manner. I gotta be honest, most of what's happening today is laughable to my generation. Blah I sound like a Boomer. Ok cryable then. We're in mourning. We worry about the kids today. All work and no play and all that. It's killing our souls, and theirs.
I guess my final thought after writing this is that partying is one of the most powerful reality-shifting tools in our arsenal. All of this can't be it. This can't be how America ends. You know what to do.
I remember a friend who was going to school in Boston coming to visit me at my college in western Massachusetts freshman year. I brought him to some off campus house in the woods, probably 200 or so people there, huge bonfire in the back, bands playing in the basement. We're passing a bottle of Jameson back and forth. Probably around 1 am everyone just heard someone screaming "that's my fucking couch!" from the outside deck as a few dudes tossed her couch into the bonfire. The flames were as high as the house and 15 minutes later the fire department was there. My friend couldn't believe what was going on, which honestly was a typical Friday night (aside from the couch burning).
I've lived in Brooklyn for about 20 years now, and while the parties still happen, most of them have become corporate. There are $50 covers and $15 beers, with wristbands you have to load a credit card onto instead of $5 covers and $2 beers in an illegal warehouse (cash only). The kids also seem to be taking ketamine a lot more than anything else, so they kinda disassociate and don't really dance that much at the clubs, whereas mdma and coke were things you ran into more when I was their age and people were not shy about grabbing someone on the dancefloor and grinding on each other for the night. They are definitely more sheltered and tame than we were as a whole, which isn't necessarily a bad thing I guess.
ketamine and whippets too. The whippets are getting quite worrisome. Basement parties are still alive and well, but yes it seems most venues have been demolished, killed by zoning or private-equitied. It's a tale as old as time (or at least as old as nimbyism), regulate something out of existence and then wonder where all the money, goodwill and life went. That and the fact that whenever anything out of the ordinary happens there's always a phone out. Always something to worry about.
I had never really considered partying as a reality-shifting tool, but as someone fond of regional burn events, yeah, it totally is.
Humans have partied for aeons. It's not just about letting off steam, it's about building social bonds, it's about traditions and rituals and marking key points in life.
This whole thread makes me rather sad, but in the same breath, makes me feel like there is real, actionable good to be done by promoting and helping run events. Not corporate pay-to-play curated experiences, which keep you on rails and only serve to condition more consumption behaviors, but relatively low cost, volunteer-run, do-it-yourself events. The latter, from my experience, have an absolutely infectious component of wanting to contribute, volunteer, create art, and drag others into the experience. But they are also a lot of work and not everyone is cut out for it.
It really has me thinking about lowering the bar to any sort of experience that gives folks a reprieve from the default world, however fleeting.
Ya that's what I wanted too. I'm actually flabbergasted that malloc() isn't a blocking call on most OSs, that would wait until the requested amount of memory was available before returning. That way programs would just suspend/resume as needed in low memory situations, rather that crashing. A process viewer could show which programs are blocked waiting for more memory, and users could optionally resume them manually. We could even serialize and unserialize entire programs and have them resume when enough memory is available.
This one simple thing would have freed users from running under the assumption that programs can crash at any time, and allowed them to operate at a higher level of abstraction to get more real work done.
This missed opportunity with a blocking malloc() is one of any number of obscene design decisions that I can't unsee in the tech status quo.
In my experience, basically all software bugs stem from asynchronous/nonblocking behavior. Because it's difficult to prove that async code is deterministic without coming full circle and restructuring it as sync. For example, higher-order methods like map/reduce and scatter/gather arrays can replace iterators. And this can now be done automagically by LLM code assistants, and static analyzers since the middle of last century. Once we see that this is possible, especially if we use all const variables to avoid mutation (mostly future-proofing our logic), it's hard to avoid asking ourselves why we're all doing it the hard way. We should be able to click a block of code and choose "make sync" or "make functional" and vice versa. So that beginners could write batch files and macros with familiar iteration syntax and transpile it to safe and reliable functional code. And experts could write pure functional code in shorthand and export it as imperative code for others to verify.
This was always another dream of mine, since I've been waiting for big companies to do it since the mid 1990s but they can't be bothered, yet I'll likely spend the rest of my life building CRUD apps to make rent.
> I'm actually flabbergasted that malloc() isn't a blocking call on most OSs, that would wait until the requested amount of memory was available before returning.
In many cases, the memory is "available" in the form of swap space (once other applications are swapped out). In other cases it's hard to tell how much memory is available (swap space can be variable, like with compressed swap space/compressed memory).
It's not that hard to write your own malloc() that does this, of course; all you need is a quick wrapper like
void* malloc_blocking(size_t size) {
void* out;
do {
out = malloc(size);
if (out == NULL) usleep(100000);
} while (out == NULL);
return out;
}
The problem with this "solution" is that malloc almost never fails. Applications don't typically handle malloc failure well, either; very few pre-allocate all the resources necessary to even log and notify the user of a memory allocation failure, let alone fail gracefully.
In my experience, Windows deals with low-memory situations a lot better. It has APIs notifying applications that memory is running low so they can clear caches (i.e. by using CreateMemoryResourceNotification) and will often try to extend the page file temporarily rather than kill applications if it can help it.
On Linux, you can measure the system memory state and application memory usage in a loop (do you check /proc/meminfo or the cgroup API? If so, which version? Good luck figuring that out!) but you're still left to the whims of the OOM daemon.
You cannot just "make sync" everything. Asynchronous code isn't written to make your life harder. It increases throughput and performance. That's why Rust still allows for asynchronous code in various methods (threads and async) with a whole type system to make memory exchange safe while maintaining performance.
Also, I've very rarely encountered bugs in any code I've debugged that are caused by asynchronous/nonblocking behaviour. You get a deadlock every now and then, but most bugs are either in the business logic layer or in code making assumption about things like "variables being initialized by the caller" and "pointers not being freed when used" or even just "pointers not being null". Using better languages and actually leveraging the many tools available these days can prevent most of those bugs.
Unfortunately the only answer that I know of is eternal vigilance, which is the price of liberty.
I decided to look up who that saying is attributed to, and apparently it's John Philpot Curran, not Thomas Jefferson. But I like Orwell's saying better, because it shows why all of you are just as ineffectual at steering government policy as I am:
https://www.socratic-method.com/quote-meanings-and-interpret...
reply