I don't know what your technical level is so I'll aim for college-educated with a broad understanding of technology. Apologies if I come off as condescending; I don't know what you know.
Feel free to use this in your paper. You can also snip out profanity of course (or leave it in for flavor; that's up to you).
Email: michael.o.church at gmail
[Start here]
In my opinion, the economic problem of the 21st-century is convexity. Go here for further reading: http://michaelochurch.wordpress.com/2013/04/03/gervais-macle... . I have a whole slew of posts on economic topics that I've put together over the past 2 months. The effect of convexity is a movement from low-risk commodity work to risk-intrinsic work (i.e. its value is purely in its natural uncertainty, and not discomfort as with commodity labor) that is more fun and creative, but harder to make a living from. The overarching theme of economics from 1975 to now has been the offloading of commodity work to machines, who can do it more reliably. The hard and somewhat creative stuff, like programming these machines, is what's left for humans. (Actually, there's a lot of concave grunt work in coding; the goal is to have it done by programs called compilers that, again, can do that concave/boring stuff way better than we.) So the only thing left for humans is the risk-intrinsic convex stuff. I've been studying Convexity for years and it is a big f_king social problem. Convex work has high expectancy but lots of risk and it can't put forward the mediocre, regular income that average people are used to (and could not stand to be without).
Software is the leading edge of convex economics. It's the first battlefield between the old industrial regime (everyone gets a mediocre wage for mediocre work) and Convexity. With convexity, you have a long learning period before your earning period (in which, ideally, you keep learning). During that (poorly paid or unpaid, often) learning period, you build a lot of cool stuff. Eventually, you get to a level of expertise and product-quality that people will pay for it; but it's impossible to know (until you engage directly with the market) how close you are to that point. So you end up building lots of still-quite-cool (if not professional-grade) stuff in which there's no harm in giving it away for free. As programmers, we don't really fear people "stealing our ideas"; ideas are cheap and easy; code is hard. That's part of why we pretty much unanimously hate software patents.
Convexity creates risk for workers (because it makes full employment uncommon if not untenable) but also for institutions that need to hire people. Talent discovery is a massive problem. (I'm writing on that right now.) Companies have to pay about $10,000 to hire a 1.2-level (scale here: http://michaelochurch.wordpress.com/2012/01/26/the-trajector... ) programmer (interview time/opportunity cost) and about $40,000 to hire a 1.5 (add recruiting fees, hiring bonuses, perks and more interviewing). How much does it cost to hire a 2.2+? (Sometimes you need that level of talent.) Almost a million (mostly in R&D budgets to build an autonomy culture.) The only places where 2.2+ programmers want to work are companies with extremely high levels of autonomy where they can do 2.3+ level R&D work; no managerial meddling, full autonomy over time, plenty of resources. (The 2.2+ "repay" that $500-700k by doing excellent work, of course.) The reason Valve has an open allocation culture and Google used to have one (before ~2009) is that, even if it's "expensive" to hire engineers and have them doing R&D with full autonomy, that's what you have to do if you want to hire the (rare and picky) 2.2+.
So talent discovery is a hard problem, and there's a bilateral matching problem wherein companies find it really hard to hire the people they want, and even good engineers have significant job latency. The bilateral matching problem only gets worse as you move up the skill curve. (I'm a 1.8 and have had 3-month job searches; then again, I'm really picky.)
The old way for undiscovered (usu. young) talent to gate-crash this discovery problem was to pay 50 thousand goddamn dollars per year to some institution already sitting on billions, for 4 years, and then get discovered by another billionaire corporation, climb its organizational ladder doing low-paid boring work, and eventually show (around age 35) that you're ready to do real work (after 10+ mind-numbing years of order-taking that killed your creativity). Well, that's just too inefficient to hold water (get it? convexity, holding water? bad joke. anyway...) anymore.
Traditionally, convex labor with extreme talent-discovery problems (such as Hollywood acting and scientific research, where the natural talent us rare) fell to gatekeepers (literary agents, Ivy League admissions officers, graduate departments, athletic recruiters) who were imperfect and sometimes corrupt, but did the job. The problem with software is that no one is qualified to serve this "middleman" role because only an equal or superior engineer can judge another software engineer's work. So, it's not just that it's desirable to get around these middling agents; it's that the only people who can really do the job are software engineers, who'd rather write code.
So the new way to fix the talent-discovery problem seems to be to create this giant, amazing gift economy that we call "open source" (and blogging, print-and-play games, also are part of this). You participate because (a) it's more fun to build cool stuff and give it away than to spend 75+ percent of your time selling and <25 creating, and (b) it's a great way to overcome the talent-discovery problem.
I really don't understand why you think "convexity" is a new concept. Isn't this the way most if not all artisan crafts have worked since antiquity? Master-apprentice and guilds. Either software creation is an engineering discipline or it is a craft. You seem to be cherry-picking.
Also I believe that while your characterization of open-source projects as essentially ego trips may be accurate of some, it is so only of a minority and in any case will not withstand any serious feminist critique.
I find that your assumption that only superior software engineers can judge the work of other engineers to be patently false and it sets up a false dichotomy. That is the same as saying that only other artists can judge artistic merit. It has no basis in fact or history. What software engineers simply fail to do is provide objective basis for their work because they don't want to lose any bargaining power over management; much the way alchemists conned their own patrons.
I contribute to open source projects because it is the right thing to do. It is moral. Not to enhance my reputation in some false gift economy or portfolio waving. Perhaps I am "old school" but this is why reading Stallman was important to my development.
Finally, one does not get satisfaction from simply seeing your cool stuff in use. For example, seeing an evil regime use your cool stuff to repress its citizenry would give me no satisfaction at all.
He never said convexity is a new concept. In the context of the industrial economy, convexity is a new problem because the past 100 years of industrial economic development has relied on concave work that is now being outsourced to computers.
Ok. That was not my interpretation of his long essay which includes no context of existing literature on the subject the absence of which leads me to believe the author believes the term or concept is novel.
Your interpretation is equally fair and interesting but seems off-topic because isn't the efficient creation of correct source code almost exclusively the domain of human work? I probably misunderstand you.
I would say that I've "independently discovered" convexity. I'm the first (to my knowledge) to use it to describe human performance at work. I'm far from the first to have the idea. Traders talk about gamma (yep, one of those Greeks) which is the second derivative of an asset's sensitivity to an underlying price. If your portfolio has positive gamma with respect to an underlying, that means you benefit from (zero-mean) volatility in it.
What I realized at some point was that labor tends to be split between "fun" work that has extreme inequality (e.g. writing novels) and volatility, and boring, commodity work. I started reading up on the economics of labor and S-curves (logistics) kept coming up and I realized what was going on.
An industrial enterprise is, in truth, much like an artificial neural net, because it's a bunch of logistic input/output relationships feeding into each other.
It's a simple matter of calculus to show that the old industrial optimization (maximum yield on capital, within bounded risk constraints) favors concave work, and that explains a lot about why businessmen have favored it. But now it's all being automated, because machines are even better at it (more precision) than we are.
That leaves us doing convex work, but there are some serious social problems in a world where (a) concave work is increasingly being gobbled up by machines, but (b) there's no basic income for people who can't supply convex work (again, the only stuff that needs humans) to the market at a living wage. See: http://michaelochurch.wordpress.com/2013/04/03/gervais-macle...
We're in the beginning stages of seeing Convexity on the larger scale, as people struggle to get the corporate jobs that our parents' generation took for granted.
I really don't understand why you think "convexity" is a new concept.
Convexity's not new. What is new is the inability for humans to compete with machines on concave labor.
Convexity and concavity come (in theory, anyway) from the logistic form:
L(x; ...) = V + A/(1+exp(-S(x-D)))
It grows exponentially (convex) over x << D, becomes linear around x = D with an inflection point at (D, V + A/2), and then turns concave at x >> D, leveling off to meet a horizontal asymptote at x = V + A.
V is a vertical offset, A is maximum potential/saturation, S is precision, and D is difficulty. Those are constants per function; x is the input (performance). L computes the economic payoff.
Generally, values of x tend to be fairly narrowly distributed based on "the state of the art" and if the typical x << D, we have "hard" (concave) labor. If the typical x >> D, then we have concave/easy stuff. Since input is abstract here, it's typical to say the "typical" x is at 0.
What management has been doing for 200 years is increasing precision (S) and pushing D to the left... making labor concave/easier and precise. That is a good thing. That's exactly what you should do if you're managing a concave job. Incidentally, if you have a convex job, increasing S is not what you want to do because it increases your failure rate.
Now, concave work is (from an economic perspective) great because the ratio of the payoff to the risk (first derivative of L) is high, and that means that in the "knapsack problem" of trying to maximize value under risk constraints, you want to be doing as much of that as you can. Right. No argument there.
With machines, S → ∞ (step function) which you never see with humans, and not only that but the "step" level is usually quite low (in cost; cents per hour of electricity) once it's properly programmed. So what this means is that, for a given business enterprise, we can often give most or all of the concave work over to machines and there's still room in our "knapsack" for high-yield (convex) work. For convex work, value-per-risk is nearly constant and thus the traditional industrial optimization heuristic (value-per-risk) becomes irrelevant. Convex is risk-intrinsic, meaning that it is the risk (rather than unpleasantness) that explains the value.
Thus, the convex work is what's left for us. If it's concave, there's no need for humans to do. The good news? When work is convex, demand for it is effectively limitless. (Limits may exist; the S-curve may level off and turn concave. But if the work's convex, that means the state of the art isn't anywhere near there.)
Also I believe that while your characterization of open-source projects as essentially ego trips may be accurate of some
Oh no, that's is not what I am saying at all.
What is egotistical about doing great work and sharing it for free?
I am very much pro-OSS and would never say that.
I find that your assumption that only superior software engineers can judge the work of other engineers to be patently false and it sets up a false dichotomy.
We're just going to disagree on this one, because I think that there are attributes of work (especially presentation) that are easy to assess, and others that are hard, and, on the whole, management is about as equipped to assess code-quality issues as I am to go into a chip factory and start spouting off theories about how I think electrical engineering should work. (I have no EE experience.)
Imagine me going into an EE shop and saying, "No more use of imaginary numbers! I don't quite understand how something not real can be useful! If I catch you taking a square root of a negative number, you're fired!" Well, that's how management often comes across on software.
What software engineers simply fail to do is provide objective basis for their work because they don't want to lose any bargaining power over management
Sorry, but that's nonsense. If anything, we lack bargaining power over management, because most of us are terrible negotiators who are socially marginal and awkward, and our inability as a group to demand what we're worth is extremely detrimental.
I contribute to open source projects because it is the right thing to do. It is moral. Not to enhance my reputation in some false gift economy or portfolio waving. Perhaps I am "old school" but this is why reading Stallman was important to my development.
Finally, one does not get satisfaction from simply seeing your cool stuff in use. For example, seeing an evil regime use your cool stuff to repress its citizenry would give me no satisfaction at all.
That's awesome, and there's no disagreement coming from this corner.
I have no response to your economic theory of software labor. It seems convoluted and abstract and divorced from actual practice of work that I see. Doesn't mean it lacks value, just that I will think about it further.
>>> Also I believe that while your characterization of open-source projects as essentially ego trips may be accurate of some
Oh no, that's is not what I am saying at all.
>>>
I apologize if I have misunderstood, but you argued above that one of the reasons for the explosive growth of OSS projects ("cool stuff for free") is because it is primarily a means to gain recognition to "shortcut" the monotony of apprenticeship. That seems to me not entirely but mostly an egotistic motivation ("Look at me, I'm good!").
>>>
We're just going to disagree on this one [about whether the merits of the work of engineers can only be judged by superior engineers]
>>>
And so you think that software which at its most fundamental level is a literary endeavor is immune from critique? That borders on the inane and almost requires a leap of faith. Nearly the same argument is made that the Bible is truth and not subject to any criteria except by those of its authors to be accepted by its reader and interpreted only by a sanctified priesthood. Do you also believe art, film, literary, etc criticism is devoid of value and entirely subjective?
Your example is also ridiculous. As if the management of any successful or functional technology company has as much relation to its product as that of a stupefied onlooker to a magic show. Most if not all managers of technical projects are previous engineers. That is one of the major criticisms of the profession by engineers. You reach a stage of ageism where career growth mandates the switch to management. Your analogy of you marching into a EE shop is I assume mirrored by a Dilbert-esque characterization of a pointy-haired boss marching into a technical meeting and arbitrarily dictating software decisions based on whether he likes the name ("Sharding sounds cool")? This may match your own experience but certainly doesn't match mine. What usually happens is after a failure is a conversation like...
Manager: You said sharding would solve our business problem.
Engineer: Yes but you didn't tell me about this feature which entails <magical incantation, abracadabra, hand wavy hocus pocus>.
Even chefs and cooks are judged based on the success of the output of their secret recipes; that it meets the requirements of the restaurant and it suits the taste level of its customers. Why is software intrinsically different? You haven't convinced me so far.
>>>>
Sorry, but that's nonsense. If anything, we lack bargaining power over management,
>>>>
And here I can not follow as you go off the rails. You just argued that technology managers have a complete inability to complete the work by any other means than using specific skilled labor they cannot understand. This is the very definition of bargaining leverage.
Moreover, you have spilled much ink arguing that the only reason to value skilled technology workers of high multipliers is that they are hard to replace (they have an innate value). That means second-tier and lower tech workers ability to remain relevant and increase their prospects is to emulate by whatever means their higher-value peers since they cannot hope to attain such a valuation on their innate skill. That is why average engineers seemingly go to great lengths to obscure results/defects and provide no objective means to judge the output of their software teams. It does not follow that such macroscopic categories by which to judge software quality do not exist.
Also as an off-topic aside in my humble opinion this is why we have seen such a dramatic rise of "agile", chaotic processes over more established, disciplined methods to manage software production. It is not that disciplined and established production techniques fail to predict accurately the outcome of software projects; it is that they do too good a job at illustrating how immature software engineering/craftsmanship actually is. To be fair, software manufacturing has had a relatively short go at it ;-)
I apologize if I have misunderstood, but you argued above that one of the reasons for the explosive growth of OSS projects ("cool stuff for free") is because it is primarily a means to gain recognition to "shortcut" the monotony of apprenticeship. That seems to me not entirely but mostly an egotistic motivation ("Look at me, I'm good!").
No. There is no apprenticeship anymore. That's the problem with a convex economy. People have to get through a long "learning" period before the "earning" period in which the market will pay them a living wage.
It's even more convoluted now, because the learning and earning must be intermingled.
In the past, convex work was such a small part of what human society needed done that the effect of this (of making convex labor only available to a privileged class) wasn't such a problem. Now's different because the concave labor that supports the less fortunate is going away.
I'm going to step back from the management stuff because I don't want to start a flamewar. That's not to implicate you, but me. I'm sure good technology management exists, but in my experience it's goddamn rare. The only time I saw good management in software was in the public sector (including a govt. contractor). When people have (expensive for the govt.) security clearances, employees tend to be treated well.
I have once seen a business do software well, but it started to fall apart (culturally) as soon as it needed middle management. No idea if it has solved those problems since then. I left right at the inflection point (no, this wasn't Google) and it's still got a stirling reputation so it's quite possible that they actually got their middle management house in order. Certainly, their tech was excellent.
Ageism in technology is a problem, but there are a lot of badass older programmers who keep on going. But yeah, if you stop learning at 25, then by 40 management is your only option.
Isn't too much open-source also a bad thing? I think we are doing ourselves a disservice by creating open source software that automates the job of the programmer. It eliminates a lot of programming jobs. Many programmers learn to be "modifiers" instead of creators and so they never really learn...
And besides, look at other professions. Is the plumber, the carpenter, the doctor, the lawyer giving you some service for free? There is a one way relationship going on here. Of course, it's easier for us to give stuff away because code, like data, can be copied easily. It's not the same kind of service but it's still something to think about.
I think we are doing ourselves a disservice by creating open source software that automates the job of the programmer.
That will never happen. Compilers automate the tedious parts of code, and we still have jobs. There's almost limitless demand, at least now, for the use of technology to improve business processes.
It eliminates a lot of programming jobs.
It does. And it creates others.
Many programmers learn to be "modifiers" instead of creators and so they never really learn...
Well, if their bosses take too much out of them and they're stripped of the energy to learn, then they should demand higher salaries or change jobs. If they choose not to learn because they don't have the curiosity, then they don't really belong in technology because curiosity is the one inflexible requirement.
Is the plumber, the carpenter, the doctor, the lawyer giving you some service for free?
Fixed vs. variable costs. The marginal cost for a doctor to see a patient (in her time) is non-zero. The marginal cost for someone's repository to be copied is (effectively) zero. Different situation.
Programmers eat the fixed costs (mostly, time) because they enjoy the problem, want to learn something, or what to build something cool. They're not going to do ongoing support (variable costs) for free, though-- at least, not at scale.
Yep. Got an interview for a software engineer position after someone looked at my Github, when a repo of mine was trending overall. I've got an Art degree, and no formal CS training - but the internet is a powerful and beautiful place.
I don't know what your technical level is so I'll aim for college-educated with a broad understanding of technology. Apologies if I come off as condescending; I don't know what you know.
Feel free to use this in your paper. You can also snip out profanity of course (or leave it in for flavor; that's up to you).
Email: michael.o.church at gmail
[Start here]
In my opinion, the economic problem of the 21st-century is convexity. Go here for further reading: http://michaelochurch.wordpress.com/2013/04/03/gervais-macle... . I have a whole slew of posts on economic topics that I've put together over the past 2 months. The effect of convexity is a movement from low-risk commodity work to risk-intrinsic work (i.e. its value is purely in its natural uncertainty, and not discomfort as with commodity labor) that is more fun and creative, but harder to make a living from. The overarching theme of economics from 1975 to now has been the offloading of commodity work to machines, who can do it more reliably. The hard and somewhat creative stuff, like programming these machines, is what's left for humans. (Actually, there's a lot of concave grunt work in coding; the goal is to have it done by programs called compilers that, again, can do that concave/boring stuff way better than we.) So the only thing left for humans is the risk-intrinsic convex stuff. I've been studying Convexity for years and it is a big f_king social problem. Convex work has high expectancy but lots of risk and it can't put forward the mediocre, regular income that average people are used to (and could not stand to be without).
Software is the leading edge of convex economics. It's the first battlefield between the old industrial regime (everyone gets a mediocre wage for mediocre work) and Convexity. With convexity, you have a long learning period before your earning period (in which, ideally, you keep learning). During that (poorly paid or unpaid, often) learning period, you build a lot of cool stuff. Eventually, you get to a level of expertise and product-quality that people will pay for it; but it's impossible to know (until you engage directly with the market) how close you are to that point. So you end up building lots of still-quite-cool (if not professional-grade) stuff in which there's no harm in giving it away for free. As programmers, we don't really fear people "stealing our ideas"; ideas are cheap and easy; code is hard. That's part of why we pretty much unanimously hate software patents.
Convexity creates risk for workers (because it makes full employment uncommon if not untenable) but also for institutions that need to hire people. Talent discovery is a massive problem. (I'm writing on that right now.) Companies have to pay about $10,000 to hire a 1.2-level (scale here: http://michaelochurch.wordpress.com/2012/01/26/the-trajector... ) programmer (interview time/opportunity cost) and about $40,000 to hire a 1.5 (add recruiting fees, hiring bonuses, perks and more interviewing). How much does it cost to hire a 2.2+? (Sometimes you need that level of talent.) Almost a million (mostly in R&D budgets to build an autonomy culture.) The only places where 2.2+ programmers want to work are companies with extremely high levels of autonomy where they can do 2.3+ level R&D work; no managerial meddling, full autonomy over time, plenty of resources. (The 2.2+ "repay" that $500-700k by doing excellent work, of course.) The reason Valve has an open allocation culture and Google used to have one (before ~2009) is that, even if it's "expensive" to hire engineers and have them doing R&D with full autonomy, that's what you have to do if you want to hire the (rare and picky) 2.2+.
So talent discovery is a hard problem, and there's a bilateral matching problem wherein companies find it really hard to hire the people they want, and even good engineers have significant job latency. The bilateral matching problem only gets worse as you move up the skill curve. (I'm a 1.8 and have had 3-month job searches; then again, I'm really picky.)
The old way for undiscovered (usu. young) talent to gate-crash this discovery problem was to pay 50 thousand goddamn dollars per year to some institution already sitting on billions, for 4 years, and then get discovered by another billionaire corporation, climb its organizational ladder doing low-paid boring work, and eventually show (around age 35) that you're ready to do real work (after 10+ mind-numbing years of order-taking that killed your creativity). Well, that's just too inefficient to hold water (get it? convexity, holding water? bad joke. anyway...) anymore.
Traditionally, convex labor with extreme talent-discovery problems (such as Hollywood acting and scientific research, where the natural talent us rare) fell to gatekeepers (literary agents, Ivy League admissions officers, graduate departments, athletic recruiters) who were imperfect and sometimes corrupt, but did the job. The problem with software is that no one is qualified to serve this "middleman" role because only an equal or superior engineer can judge another software engineer's work. So, it's not just that it's desirable to get around these middling agents; it's that the only people who can really do the job are software engineers, who'd rather write code.
So the new way to fix the talent-discovery problem seems to be to create this giant, amazing gift economy that we call "open source" (and blogging, print-and-play games, also are part of this). You participate because (a) it's more fun to build cool stuff and give it away than to spend 75+ percent of your time selling and <25 creating, and (b) it's a great way to overcome the talent-discovery problem.
So that's why we give cool stuff away for free.