I can't get over the profound sense of waste when I do something and think "50,000 devs have probably done this before, and 100,000 more will do it again"... and yet it's the same shitty thing and it could easily be some open lib.

Every single one of us is standing on the shoulders' of every dev that preceded us, the least we can do is to help the next generation stand on our shoulders' too.

None of this is to suggest the politics aren't also important, but for me the waste trumps all.

Just look at MySQL, and derived DBs like MariaDB and Drizzle. There is much overlap with PostgreSQL, but PostgreSQL is objectively superior in just about every way. Then there is all of the effort put toward the various libraries and tools for each, toward other application support for each, and so forth. Is it not wasteful when people spend time, money and energy on or supporting the inferior open source project?

Then there is the duplication that is forced by the use of certain open source licenses. GPL-style licenses, for instance, have prevented a lot of reuse and sharing of effort due to their somewhat restrictive terms. Open source projects using more liberal licenses, or commercial projects, often must intentionally ignore and avoid such other open source software with more restrictive licensing.

Open source may help avoid duplicate effort and code, for instance, but it is still rife with inefficiencies.

All over the open source space, there are almost always two opposing forces in each camp, sometimes more, but whenever the competition dries up (VLC?) innovation slows down. VLC has become extremely refined as a media player, and I ask nothing more from it, but it hasn't tried any real radical UI departures or UX shifts and its mobile version is still slow on the uptake. I imagine if it had a media player competitor it would improve faster.

So the duplicated effort, although a "waste" of time, seems to drive developers to work harder, which produces better products in the end.

Compare that with exploring different paradigms and priorities (such as Postgres and Mysql who happen to have very different philosophies), which is absolutely OK in my book. Nobody wants a monoculture. Except malicious bystanders who first and foremost like to make this into some kind of faction warfare.

I don't really have a stake in this fake database conflict, but why is it that Postgres always seems to attract these kinds of people whereas Mysql users seem to be content doing their thing mostly in silence and without the vitriole?

You portray it as some sort of a "conflict" between open source projects, but that couldn't be any more distant from the truth. It's all about doing things properly. Some database systems do this, and there are some that don't. We should acknowledge this fact.

It surely is not a waste of time or effort to educate people about the reality of the situation, especially those who may ignorantly be using a certain inferior product without even realizing that there are better alternative out there.

I'm not writing this to attack you personally, but I want to call attention to a kind of behavior I think is not acceptable, and it sure as hell isn't productive in any setting - least of all a discussion with a diverse group of developers. You should not be able to get away with this so easily. Whether intentionally or not, you are poisoning the discourse.

You get to say all the reasons why you like product X, and all the reasons why you dislike product Y, and then we can have a discussion based on these points. Just asserting X is obviously superior, and Y users are ignorant nincompoops is not OK.

Regardless of what I may think or believe, the fact is that PostgreSQL is a superb relational database system. Of the open source database systems, it is by far the most capable and robust. Its price and very liberal licensing only make it better. This is indisputable.

I don't see how taking an unemotional look at the facts, especially when purely objective technical analysis is involved, is in any way "poisoning the discourse".

As professionals, we should be able to look at our tools and openly state which are good, and which are bad. Detaching oneself from emotions and personal convictions is one of the hallmarks of professionalism, regardless of the field in question.

If only it were so! We'd be talking about actual facts, instead you're doing this, posturing. You think because you say you're being objective makes it so? Nothing you said had any content besides an aggressive statement about your preferences badly hidden behind what you assert to be the objective truth. Listen, I tried. I don't know what else to say except repeating myself. I'm also not quite sure if you're not just trolling, so I think we should both spend our energy on something more rewarding.

PostgreSQL is a technologically superior database system, relative to the other open source database systems, and even compared to many commercial database systems. It does have extremely liberal licensing. It has a community that is exceptionally knowledgeable and extremely willing to help.

The facts are the facts. And the facts all prove the overall fact that PostgreSQL is the most capable, robust and trustworthy open source database system around, regardless of what we may personally believe.

When tackling a problem I decide wether:

A. This problem is niche, it's no use to someone else. I need to write it and it's not worth open sourcing it

B. This problem is of use to someone else - someone has probably already solved it better than I'm likely to and I should find their code

C. This problem is of use to someone else and I can't find code to solve it then there's a small chance that I can benefit other people (and perhaps also build my own profile) by writing something that does solve it. Like creating a product people use, it's an honour to create code people use.

Finally, as a founder I feel that if you produce code which isn't a tactical asset in the company but which really could be of use to a large group of other people then you stand a chance of making it tougher, more flexible and more efficient by letting other people use and improve it. It's a real power-up for the company itself.

I don't like the thought that one just totally waste the time and work of humans.

Open Source is working together with each one to advance the whole human race. Nothing is wasted. The whole human community benefits.

This is much more effective and makes more sense because you don't waste so much work resources for no reason.

It's also much more social.

I use because:

- I feel much more confident supporting something which I know I can dig into and observe the function of. This can give me and understand of what I'm doing wrong, allow me to work around it or possibly patch it entirely. Contrast this with the helplessness of troubleshooting the black box of closed software.

- It's cost effective. Money not spent on expensive licenses and implied support contracts can be applied elsewhere. Since much of my work deals with public funds, I feel particularly obligated to maximize value.

- I feel what I learn using OSS is more transferable, and applicable to other systems and problems than closed software.

I contribute because:

- I think sharing discovery and tools and building on the work of others is key to our success as a species. It feels completely natural.

- It opens the door for symbiotic improvements. Personally, I seem to be pretty good at find a novel perspective on a problem, but have little knowledge of how to code it efficiently. Sharing my code with someone who knows how to do those things benefits us both.

- To learn by practice and example.

PS. Participating in the O/S projects of others is the whole other deal. You may want to clarify the question as to what you are asking about.

Thats why I do open source. Hopefully one day I'll get lucky and one of my projects will catch on and become very popular. When that happens, I have guaranteed employment pretty much for the rest of my life. Although, only like 1% of all open source developers are lucky enough to have a project with enough "cred" to be in that position. It mostly has to do with luck. There are tons of popular projects that suck (PHP), and tons of projects that are amazingly written, but not popular.

Any great project made by humans has always been the work of a group of people working all together to solve a problem. This is how Egyptians built Pyramids, Alexender conquered lands and our people today built Linux and the Internet.

Sure history will always pick a person and gives him all the merit (Like father of the internet and visionary of that ...) but the truth is, every thing those people buit was done on top of what others have done before and the hard work of those who team up in groups and communities to build on top of it. If scientists haven't worked all together to understand Einstein theories and apply them, e=mc2 would be useless.

This is Open Source.

There's another really important aspect: the multiplicative productivity speed up of building systems by working with a larger community of smart engineers. It's just damn faster to build large tools / libraries when more people are helping out.

I know some S class epic level engineers, some of whom I'd like to consider friends. The engineering projects they do are ridiculous, amazing, and just damn bAller.

But If you ask them why they open source, it's not just all the work opportunities they get. Another large part is that no one person is smart enough and productive enough to do everything on their own. (As much as we all dream of being super human. )

This availability of knowledge, but also the freedom that comes from free software systems, are important values for me. I want to contribute to such systems being viable, modern environments.

A secondary motivation is that for purely hobby projects, there is a chance of finding collaborators that make the project more fun to work on and advance more quickly.

I've also released some code of which I'm the copyright holder as Free Software, mainly plugins for other applications. I've licensed it as such because I thought others could benefit from it.

I believe software should work the same way -- so if you purchase software from me, I think you should have the freedom to hire any interested software developer to make changes to that software, and not be locked in to hiring me again to make those changes.

proprietary software is an easy way for software companies to create a vendor lock-in situation, which I think is abusive.

Lets say you pick up a piece of Open Source software, implement it and make some bug fixes and new features along the way. By contributing those bug fixes you avoid the overhead of having to merge your own bug fixes and features into the code base again and again as you pull in other bug fixes and features from other developers. By releasing your bug fixes and features into the open source code base you have a competitive advantage over your competitors who maintain a private fork of the same software.

Open Source is not charity. It's profit driven. For those that say "it's the right thing to do," either they're doing it for the competitive advantage mentioned above without realizing it or they might just be wasting their time releasing useless open source code. That's not to say I don't think releasing your code is often the right thing to do, I'm CTO for a nonprofit bringing Open Source to Open Educational Resources (http://ole.org) and collaborative development in farming (http://farmhack.net), I'm clearly no crazy capitalist but we use open source for the competitive advantage because as a nonprofit, why not do the best you can do? Open source rocks. Thank you to the giants who pioneered Open Source.

This alone should be enough motivation to not just use, but contribute to open source. It empowers the users, and let's face it, we're all users these days. Beyond that, there is also the factor of giving back, or making a mark, making something of lasting worth. Sure, proprietary software is worth something - until you can't use it anymore because you don't have the source, and it either won't run on your platform, or has issues that make it unusable.

I myself am not a big open source developer - I mainly find bugs in products I use and contribute patches back (I'm a pretty good fixer ;). The difference between open and closed source? I can make a change to the software I use.

--

altruism: it feels good to help. plus, there are times you solve a problem that others are likely to encounter and want to save them the grief.

--

symbiosis: open source technologies make it easier to create our projects. to ensure these resources dont go away, we contribute back to them.

--

ego: it is quite an accomplishment to have a large number of other devs use your work. you also want to show your knowledge on a given subject.

--

profit: there is money in providing support to open source technologies. it also helps strengthen trust in a service.

2. Because it feels good to share. I use a lot of other people's open source code, so I want to give something back.

3. Because I like the idea of my work benefiting millions of people every day, even though it's in a very small way and without their knowledge.

4. Because it's much more effective than a CV / resume. Potential clients (mostly) come to me, rather than the other way round, because they've already seen (or are using) my work.

2. You can, mostly, work on ideas you think are worth doing. This is really awesome because worth is subjective term and only way to see you were right is to do it (your own way). "Talk is cheap, show me the code".

3. With right license you can take others code to fill parts of your project (either temporary or permanent). Helps in time consumption and agility. Ofc, you can always rewrite whenever and whatever you want. Some people think that reinwentihg the wheel is bad, but it all depends on situation. Sometimes it's even better then using bad code.

4. If you share code, someone will eventually modify it, so you can choose to integrate changes back (if worth merging). Only problem here is situation when people are using non-compatible licenses.

5. More hacker's culture in FLOSS in general.

One other project I made open was because it was a practice code and I wanted to show it off and maybe (again) help others who want to do something similar. I also made it open because I didn't intend to work on it anymore. One time job, so if someone else finds it of any use, they can commit to the main repo and improve the source repo instead of keeping the changes to themselves. And in the likely case that no one commits anything (no one has, yet) I wanted it to be somewhat of a demo code in case I wanted to show my skills to someone quick or impress someone.

-Access Databases have stupid problems, but SQLServer is too expensive? Postgres! -SAS is out of reach and Excel is dumb for multiple linear regressions? R! -Reporting is too manual in Excel and VBA is fragile? JasperReports! -Cognos/OBIEE is not possible? Mondrian!

Open Source amplified my power and productivity by 10-100x, and it didn't cost me a dime. Small contributions ended up making even bigger impacts. Open Source is a positive feedback loop, and you get a surprising amount back for the tiny bit that you put in.

I find traditional unit testing limiting because you have to design the tests (and may miss a corner case etc). Having other people use the code generally presents you with situations you may not have thought of or may not have planned to address until later.

Especially with a mammoth project, incremental development is easier when you know what aspects are most important. I have been doing this with http://niggler.github.io/js-xlsx/ and it proved to be effective when others started using it in ways I didn't plan to address until later.

Having source code helps me understand things from the computer's point of view.

Having open source tools and infrastructure to compile it helps me make my system behave better.

Being able to share my changes with the rest of the world helps me and others learn the wisdom of those changes.

All these things can still happen with closed source, but the process takes longer.

tl;dr - I'm only going to live for so long, so open source lets me live a more productive and fulfilling life.

Some of it because of project-specific reasons — there is some reason for that particular project to be open, and it's not a general reason.

Some of it because I'm paid to.

And some of it because I have an urge to write the code. That code is more or less art for art's sake, and art wants to be open.

Also, great (and many) companies are always looking for developers of specific Open Source projects like OpenGL, OpenCV, Rails, Django, and many others.

And if I do fix a bug myself, I can make a patch available to others so they can benefit as well.

Ultimately, it's about control of my resources.

Not to mention reading through code is interesting. Especially when you have a chance to change it to make it better.

It's a long read. I managed to get part of the way through it before getting distracted by other books, but a lot of it held true for me.

I also believe that writing open source software is a fantastic way to land more jobs as it proves to your employer that:

1. You can release software.

2. You can write clean code (your wrote clean code, right?)

3. You can use a DCVS like Git or Mercurial.

Where's the downside?

- Avoiding being alienated from my own work.

- To produce software which anyone can use, regardless of financial or geographical situation.

- To produce better quality software by not reinventing the wheel and by the ability to modify or reuse existing code.

I don't get to do that much for my usual contract work (yes, I'm one of those developers - sorry).

* it's nice to have people using code that you write, even if it isn't code that would make a lot of money (and making a little money isn't a big deal when you have a paycheck anyway). there's a sense of community and a pleasure in sharing - making other people happy with something you have made.

* it's a way of "giving back" to help support the community (whose open source code you have used yourself). if you're a programmer, the existence of a large body of code in your language (or, at a higher level, OS) is a huge help.

* it provides a way to show prospective employers / clients what you can do.

* it's the community norm and it gives you a certain amount of status within that same community.

* in some cases (eg https://github.com/andrewcooke/simple-crypt) the code is worthless without collaborative development (there's no way that code has/had a chance of being correct without feedback from many people).

* this one is a bit odd, but something i've felt for a while: there's a kind of evolutionary pressure from competing "technology ecologies". by putting code out there that uses "your choice" you support that particular technology (both by the code being useful and by simply adding to visibility). for example, a recent, small program i wrote (https://github.com/andrewcooke/id3img) was implemented in python 3 despite some requests for it to be in python 2, because i felt it was important to "support" python 3 (and we're talking "sub-ecologies here" - i guess others would feel anything python was helping support that against, say, ruby, or vice-versa). the motivation to help your particular "tech ecology" is that it is the one you have invested time learning, so it's to your advantage for it to flourish.

* sometimes it can be politically motivated (i'm just looking through my github repos and https://github.com/andrewcooke/GhettoNet was very much a political statement).

* sometimes there's a sense of frustration that a project does not exist. and while you know that a complete solution is probably more than you can implement yourself, you're trying to get something started. that is the case for https://bitbucket.org/isti/c-orm/wiki/Home for example, where i would love to bootstrap a community that supports and extends a decent "ORM" solution for C (that code is very much beta btw and i would appreciate feedback / curious users).

- credibility

I don't know what your technical level is so I'll aim for college-educated with a broad understanding of technology. Apologies if I come off as condescending; I don't know what you know.

Feel free to use this in your paper. You can also snip out profanity of course (or leave it in for flavor; that's up to you).

Email: michael.o.church at gmail

[Start here]

In my opinion, the economic problem of the 21st-century is convexity. Go here for further reading: http://michaelochurch.wordpress.com/2013/04/03/gervais-macle... . I have a whole slew of posts on economic topics that I've put together over the past 2 months. The effect of convexity is a movement from low-risk commodity work to risk-intrinsic work (i.e. its value is purely in its natural uncertainty, and not discomfort as with commodity labor) that is more fun and creative, but harder to make a living from. The overarching theme of economics from 1975 to now has been the offloading of commodity work to machines, who can do it more reliably. The hard and somewhat creative stuff, like programming these machines, is what's left for humans. (Actually, there's a lot of concave grunt work in coding; the goal is to have it done by programs called compilers that, again, can do that concave/boring stuff way better than we.) So the only thing left for humans is the risk-intrinsic convex stuff. I've been studying Convexity for years and it is a big f_king social problem. Convex work has high expectancy but lots of risk and it can't put forward the mediocre, regular income that average people are used to (and could not stand to be without).

Software is the leading edge of convex economics. It's the first battlefield between the old industrial regime (everyone gets a mediocre wage for mediocre work) and Convexity. With convexity, you have a long learning period before your earning period (in which, ideally, you keep learning). During that (poorly paid or unpaid, often) learning period, you build a lot of cool stuff. Eventually, you get to a level of expertise and product-quality that people will pay for it; but it's impossible to know (until you engage directly with the market) how close you are to that point. So you end up building lots of still-quite-cool (if not professional-grade) stuff in which there's no harm in giving it away for free. As programmers, we don't really fear people "stealing our ideas"; ideas are cheap and easy; code is hard. That's part of why we pretty much unanimously hate software patents.

Convexity creates risk for workers (because it makes full employment uncommon if not untenable) but also for institutions that need to hire people. Talent discovery is a massive problem. (I'm writing on that right now.) Companies have to pay about $10,000 to hire a 1.2-level (scale here: http://michaelochurch.wordpress.com/2012/01/26/the-trajector... ) programmer (interview time/opportunity cost) and about $40,000 to hire a 1.5 (add recruiting fees, hiring bonuses, perks and more interviewing). How much does it cost to hire a 2.2+? (Sometimes you need that level of talent.) Almost a million (mostly in R&D budgets to build an autonomy culture.) The only places where 2.2+ programmers want to work are companies with extremely high levels of autonomy where they can do 2.3+ level R&D work; no managerial meddling, full autonomy over time, plenty of resources. (The 2.2+ "repay" that $500-700k by doing excellent work, of course.) The reason Valve has an open allocation culture and Google used to have one (before ~2009) is that, even if it's "expensive" to hire engineers and have them doing R&D with full autonomy, that's what you have to do if you want to hire the (rare and picky) 2.2+.

So talent discovery is a hard problem, and there's a bilateral matching problem wherein companies find it really hard to hire the people they want, and even good engineers have significant job latency. The bilateral matching problem only gets worse as you move up the skill curve. (I'm a 1.8 and have had 3-month job searches; then again, I'm really picky.)

The old way for undiscovered (usu. young) talent to gate-crash this discovery problem was to pay 50 thousand goddamn dollars per year to some institution already sitting on billions, for 4 years, and then get discovered by another billionaire corporation, climb its organizational ladder doing low-paid boring work, and eventually show (around age 35) that you're ready to do real work (after 10+ mind-numbing years of order-taking that killed your creativity). Well, that's just too inefficient to hold water (get it? convexity, holding water? bad joke. anyway...) anymore.

Traditionally, convex labor with extreme talent-discovery problems (such as Hollywood acting and scientific research, where the natural talent us rare) fell to gatekeepers (literary agents, Ivy League admissions officers, graduate departments, athletic recruiters) who were imperfect and sometimes corrupt, but did the job. The problem with software is that no one is qualified to serve this "middleman" role because only an equal or superior engineer can judge another software engineer's work. So, it's not just that it's desirable to get around these middling agents; it's that the only people who can really do the job are software engineers, who'd rather write code.

So the new way to fix the talent-discovery problem seems to be to create this giant, amazing gift economy that we call "open source" (and blogging, print-and-play games, also are part of this). You participate because (a) it's more fun to build cool stuff and give it away than to spend 75+ percent of your time selling and <25 creating, and (b) it's a great way to overcome the talent-discovery problem.

So that's why we give cool stuff away for free.

I really don't understand why you think "convexity" is a new concept. Isn't this the way most if not all artisan crafts have worked since antiquity? Master-apprentice and guilds. Either software creation is an engineering discipline or it is a craft. You seem to be cherry-picking.

Also I believe that while your characterization of open-source projects as essentially ego trips may be accurate of some, it is so only of a minority and in any case will not withstand any serious feminist critique.

I find that your assumption that only superior software engineers can judge the work of other engineers to be patently false and it sets up a false dichotomy. That is the same as saying that only other artists can judge artistic merit. It has no basis in fact or history. What software engineers simply fail to do is provide objective basis for their work because they don't want to lose any bargaining power over management; much the way alchemists conned their own patrons.

I contribute to open source projects because it is the right thing to do. It is moral. Not to enhance my reputation in some false gift economy or portfolio waving. Perhaps I am "old school" but this is why reading Stallman was important to my development.

Finally, one does not get satisfaction from simply seeing your cool stuff in use. For example, seeing an evil regime use your cool stuff to repress its citizenry would give me no satisfaction at all.

Your interpretation is equally fair and interesting but seems off-topic because isn't the efficient creation of correct source code almost exclusively the domain of human work? I probably misunderstand you.

What I realized at some point was that labor tends to be split between "fun" work that has extreme inequality (e.g. writing novels) and volatility, and boring, commodity work. I started reading up on the economics of labor and S-curves (logistics) kept coming up and I realized what was going on.

An industrial enterprise is, in truth, much like an artificial neural net, because it's a bunch of logistic input/output relationships feeding into each other.

It's a simple matter of calculus to show that the old industrial optimization (maximum yield on capital, within bounded risk constraints) favors concave work, and that explains a lot about why businessmen have favored it. But now it's all being automated, because machines are even better at it (more precision) than we are.

That leaves us doing convex work, but there are some serious social problems in a world where (a) concave work is increasingly being gobbled up by machines, but (b) there's no basic income for people who can't supply convex work (again, the only stuff that needs humans) to the market at a living wage. See: http://michaelochurch.wordpress.com/2013/04/03/gervais-macle...

We're in the beginning stages of seeing Convexity on the larger scale, as people struggle to get the corporate jobs that our parents' generation took for granted.

Convexity's not new. What is new is the inability for humans to compete with machines on concave labor.

Convexity and concavity come (in theory, anyway) from the logistic form:

L(x; ...) = V + A/(1+exp(-S(x-D)))

It grows exponentially (convex) over x << D, becomes linear around x = D with an inflection point at (D, V + A/2), and then turns concave at x >> D, leveling off to meet a horizontal asymptote at x = V + A.

V is a vertical offset, A is maximum potential/saturation, S is precision, and D is difficulty. Those are constants per function; x is the input (performance). L computes the economic payoff.

Generally, values of x tend to be fairly narrowly distributed based on "the state of the art" and if the typical x << D, we have "hard" (concave) labor. If the typical x >> D, then we have concave/easy stuff. Since input is abstract here, it's typical to say the "typical" x is at 0.

What management has been doing for 200 years is increasing precision (S) and pushing D to the left... making labor concave/easier and precise. That is a good thing. That's exactly what you should do if you're managing a concave job. Incidentally, if you have a convex job, increasing S is not what you want to do because it increases your failure rate.

Now, concave work is (from an economic perspective) great because the ratio of the payoff to the risk (first derivative of L) is high, and that means that in the "knapsack problem" of trying to maximize value under risk constraints, you want to be doing as much of that as you can. Right. No argument there.

With machines, S → ∞ (step function) which you never see with humans, and not only that but the "step" level is usually quite low (in cost; cents per hour of electricity) once it's properly programmed. So what this means is that, for a given business enterprise, we can often give most or all of the concave work over to machines and there's still room in our "knapsack" for high-yield (convex) work. For convex work, value-per-risk is nearly constant and thus the traditional industrial optimization heuristic (value-per-risk) becomes irrelevant. Convex is risk-intrinsic, meaning that it is the risk (rather than unpleasantness) that explains the value.

Thus, the convex work is what's left for us. If it's concave, there's no need for humans to do. The good news? When work is convex, demand for it is effectively limitless. (Limits may exist; the S-curve may level off and turn concave. But if the work's convex, that means the state of the art isn't anywhere near there.)

Also I believe that while your characterization of open-source projects as essentially ego trips may be accurate of some

Oh no, that's is not what I am saying at all.

What is egotistical about doing great work and sharing it for free?

I am very much pro-OSS and would never say that.

I find that your assumption that only superior software engineers can judge the work of other engineers to be patently false and it sets up a false dichotomy.

We're just going to disagree on this one, because I think that there are attributes of work (especially presentation) that are easy to assess, and others that are hard, and, on the whole, management is about as equipped to assess code-quality issues as I am to go into a chip factory and start spouting off theories about how I think electrical engineering should work. (I have no EE experience.)

Imagine me going into an EE shop and saying, "No more use of imaginary numbers! I don't quite understand how something not real can be useful! If I catch you taking a square root of a negative number, you're fired!" Well, that's how management often comes across on software.

What software engineers simply fail to do is provide objective basis for their work because they don't want to lose any bargaining power over management

Sorry, but that's nonsense. If anything, we lack bargaining power over management, because most of us are terrible negotiators who are socially marginal and awkward, and our inability as a group to demand what we're worth is extremely detrimental.

I contribute to open source projects because it is the right thing to do. It is moral. Not to enhance my reputation in some false gift economy or portfolio waving. Perhaps I am "old school" but this is why reading Stallman was important to my development.

Finally, one does not get satisfaction from simply seeing your cool stuff in use. For example, seeing an evil regime use your cool stuff to repress its citizenry would give me no satisfaction at all.

That's awesome, and there's no disagreement coming from this corner.

>>> Also I believe that while your characterization of open-source projects as essentially ego trips may be accurate of some

Oh no, that's is not what I am saying at all. >>>

I apologize if I have misunderstood, but you argued above that one of the reasons for the explosive growth of OSS projects ("cool stuff for free") is because it is primarily a means to gain recognition to "shortcut" the monotony of apprenticeship. That seems to me not entirely but mostly an egotistic motivation ("Look at me, I'm good!").

>>> We're just going to disagree on this one [about whether the merits of the work of engineers can only be judged by superior engineers] >>>

And so you think that software which at its most fundamental level is a literary endeavor is immune from critique? That borders on the inane and almost requires a leap of faith. Nearly the same argument is made that the Bible is truth and not subject to any criteria except by those of its authors to be accepted by its reader and interpreted only by a sanctified priesthood. Do you also believe art, film, literary, etc criticism is devoid of value and entirely subjective?

Your example is also ridiculous. As if the management of any successful or functional technology company has as much relation to its product as that of a stupefied onlooker to a magic show. Most if not all managers of technical projects are previous engineers. That is one of the major criticisms of the profession by engineers. You reach a stage of ageism where career growth mandates the switch to management. Your analogy of you marching into a EE shop is I assume mirrored by a Dilbert-esque characterization of a pointy-haired boss marching into a technical meeting and arbitrarily dictating software decisions based on whether he likes the name ("Sharding sounds cool")? This may match your own experience but certainly doesn't match mine. What usually happens is after a failure is a conversation like...

Manager: You said sharding would solve our business problem.

Engineer: Yes but you didn't tell me about this feature which entails <magical incantation, abracadabra, hand wavy hocus pocus>.

Even chefs and cooks are judged based on the success of the output of their secret recipes; that it meets the requirements of the restaurant and it suits the taste level of its customers. Why is software intrinsically different? You haven't convinced me so far.

>>>> Sorry, but that's nonsense. If anything, we lack bargaining power over management, >>>>

And here I can not follow as you go off the rails. You just argued that technology managers have a complete inability to complete the work by any other means than using specific skilled labor they cannot understand. This is the very definition of bargaining leverage.

Moreover, you have spilled much ink arguing that the only reason to value skilled technology workers of high multipliers is that they are hard to replace (they have an innate value). That means second-tier and lower tech workers ability to remain relevant and increase their prospects is to emulate by whatever means their higher-value peers since they cannot hope to attain such a valuation on their innate skill. That is why average engineers seemingly go to great lengths to obscure results/defects and provide no objective means to judge the output of their software teams. It does not follow that such macroscopic categories by which to judge software quality do not exist.

Also as an off-topic aside in my humble opinion this is why we have seen such a dramatic rise of "agile", chaotic processes over more established, disciplined methods to manage software production. It is not that disciplined and established production techniques fail to predict accurately the outcome of software projects; it is that they do too good a job at illustrating how immature software engineering/craftsmanship actually is. To be fair, software manufacturing has had a relatively short go at it ;-)

No. There is no apprenticeship anymore. That's the problem with a convex economy. People have to get through a long "learning" period before the "earning" period in which the market will pay them a living wage.

It's even more convoluted now, because the learning and earning must be intermingled.

In the past, convex work was such a small part of what human society needed done that the effect of this (of making convex labor only available to a privileged class) wasn't such a problem. Now's different because the concave labor that supports the less fortunate is going away.

I'm going to step back from the management stuff because I don't want to start a flamewar. That's not to implicate you, but me. I'm sure good technology management exists, but in my experience it's goddamn rare. The only time I saw good management in software was in the public sector (including a govt. contractor). When people have (expensive for the govt.) security clearances, employees tend to be treated well.

I have once seen a business do software well, but it started to fall apart (culturally) as soon as it needed middle management. No idea if it has solved those problems since then. I left right at the inflection point (no, this wasn't Google) and it's still got a stirling reputation so it's quite possible that they actually got their middle management house in order. Certainly, their tech was excellent.

Ageism in technology is a problem, but there are a lot of badass older programmers who keep on going. But yeah, if you stop learning at 25, then by 40 management is your only option.

That will never happen. Compilers automate the tedious parts of code, and we still have jobs. There's almost limitless demand, at least now, for the use of technology to improve business processes.

It eliminates a lot of programming jobs.

It does. And it creates others.

Many programmers learn to be "modifiers" instead of creators and so they never really learn...

Well, if their bosses take too much out of them and they're stripped of the energy to learn, then they should demand higher salaries or change jobs. If they choose not to learn because they don't have the curiosity, then they don't really belong in technology because curiosity is the one inflexible requirement.

Is the plumber, the carpenter, the doctor, the lawyer giving you some service for free?

Fixed vs. variable costs. The marginal cost for a doctor to see a patient (in her time) is non-zero. The marginal cost for someone's repository to be copied is (effectively) zero. Different situation.

Programmers eat the fixed costs (mostly, time) because they enjoy the problem, want to learn something, or what to build something cool. They're not going to do ongoing support (variable costs) for free, though-- at least, not at scale.