Hiring a programmer who has built stuff is like hiring a contractor who has shown an ability to do carpentry work. It's swell when all you need is the carpenter equivalent of a programmer; but often there is more involved than just nailing lines of code, classes and modules together from someone else's specification.

So, having a criterion of having built stuff is not sufficient, although I think it is reasonable. What qualifies as "stuff" is pretty subjective, though, and probably can't be anything but subjective. As I mentioned above: biases abound.

You also need to make sure they are able to figure out what to build and how, and sometimes what not to build -- whether that means they buy it, or it is a superfluous component.

You need to make sure they can think about what they are doing, rather than just being able to do things. The couple of outside projects I've become involved in so far, and some of the code I've read for other projects I'm not actively working on, suggest that some people don't do enough thinking. They've built plenty, but they didn't appear to build it with any goal but to build version 1 and be done. Think the software equivalent of a contractor coming in to fix what the first carpenter screwed up, because the first carpenter didn't care once the house passed inspection.

The code I've dealt with inside of organizations I've worked for is, if anything, worse than the code outside. But just looking at the end product wouldn't indicate anything is particularly wrong, excepting that maybe improvements to the site tend to occur only once every geologic era. Sometimes this can be indicative of quality of the developers; sometimes it is indicative of the quality of the management. Either way, it may not reflect on the developer you are talking to.

Depending, the developer may be able to think at many levels, and might be able to think lucidly about their work from a business strategy perspective. And depending on the organizational culture, this can either be a wonderful thing, or the most undesirable thing imaginable.

But this is all based on some amount of personal bias based on the kind of work I would want to be doing. I know there are places where programmers are expected to be nothing more than carpenters given specifications by the system analysts to turn into code. In these cases, I can see showing just an aptitude for building things to be enough. But I don't think these cases are common enough to warrant a single best practice.

Now, I'm not sure I agree with the author's point of any business person being able to spot a good programmer. But I think that if such a thing were possible, this would probably be how such a list would look.

But then again, we probably don't even agree on what a good programmer is. If there were an objective, standard way to find good programmers, articles like this wouldn't be necessary.

I feel guilty even typing this much, because I don't think you're being intellectually honest at all. The vast array of open-source and free projects in the world speaks plainly about the ability of people to build useful things outside of work.

Let "an institution" be "a place where one may work on projects safe in the knowledge that one's base needs for food and shelter will be met."

    I(x) := x is an institution

    S(x) := x seeks to join an institution

    A(x,y) := x will accept y

    P(x) := x has completed a project

    1. ∀xy( (S(x) ∧ I(y) ∧ A(y,x)) → P(x) )

    2. ∀xy∃z( (S(x) ∧ I(y) ∧ P(x)) → (I(z) ∧ y≠z ∧ A(z,x))) )

    ∴ ∀xy∃z( (S(x) ∧ I(y) ∧ A(y,x)) → (I(z) ∧ y≠z ∧ A(z,x))) )

That being said, I agree completely with your first paragraph. I was being half-satirical—not intending to find a flaw in your argument, so much as desiring to provoke you into strengthening your point by clarifying it (and trying to go one step beyond the common-sense assumption of "our current society" to see where this philosophy would take you.)

Do you think the world would be better if programmers could not transition into the workforce until they had created something that no one had asked them to create? The idea behind a company hiring you is that they need what you have, and so they pay you for it. If you're creating something and no one's paying you for it, then, presumably, either you aren't marketing it well enough, or it's something nobody needs (or you've made it as an act of goodwill, but the likelihood of Joe the Programmer writing FOSS as anything other than a prerequisite to a job—which, therefore, means that he really would rather have been paid, but just couldn't find a taker, again implying that he was underemployed—is slim to nil.)

Do you think the world would be better if programmers could not transition into the workforce until they had created something that no one had asked them to create?

I'm not sure if the world would be better. There's an overwhelming demand for competent programmers, and not anywhere close to enough supply to fill it. Is it better to lower our standards, resulting in shoddy and sometimes dangerously bad work, or to leave things undone?

However, for any given single institution, I am very confident that that institution would be better off hiring only people who had created things which nobody asked them to create. I agree that these are usually things which either nobody needs, or which are marketed poorly; but I don't see why that would indicate anything negative about the person in question (as far as the company is concerned.)

Forty hours a week is an theoretical standard; it says little about the actual hours put in by programmers. Urban lore suggests the accurate number is higher, sometimes much higher.

The free time a programmer isn't guaranteed to be available in large chunks for hacking. There are a number of things that can take up that time; some of them are bogus, but some are quite reasonable. Enough of that time gets chewed up, the programmer really doesn't have the kind of time they need to focus on deep, substantial projects. Some do find the time, but there are more factors involved than simply motivation.

As for primary and secondary school years, this too depends on a number of factors, not all of them matters of personal motivation and ability.

Assessment is hard enough when you don't have preconceived notions of the molds other people fit into; if you start assessing people already cherry-picking assumptions about them based on what societal fashion suggests is "normal," then your ability to fairly assess them is already screwed. You're not trying to fairly assess them for them; you're doing it for you, because doing it fairly is your best shot at not screwing up.

What he calls "indicators", I call "co-factors". They may or may not accompany a given programmer, and they predict nothing.

Generally speaking, good programmers are intelligent and do care about their work, but his positive indicators are much more a matter of personal preference than the quality of their work.

For each good programmer OP can identify who programs as a hobby, will talk your ear off, started programming before university, or has knowledge of a lots of technologies, I can cite a counterexample.

Similarly, I can also identify many good programmers who possess some of OP's "negative indicators" like having a day job, enjoying a single technology, not wanting to talk shop, or enjoying other activities.

There is one sure-fire way to identify a good programmer: their work. In fact, it is by definition the way that we should evaluate programmers. It is the great equalizer, and with web technology, exposed for all to see.

It really is as simple as cause and effect: good programmers produce good work; bad programmers don't. Anything else, including OP's positive and negative indicators, is just a detail.

Unless you are presuming a team of only a couple of developers doing their own thing, there is a real challenge in determining the extent of the programmer's responsibility for something that is exposed for all to see.

There are a number of other causes that might make a site a developer helped work on look or run like complete garbage. Organizational inertia might have forced the site into sucking, regardless of how good the programmers developing it were. The organization may have one or more bad programmers who were assigned to work on some critical component, screwing the whole site and all the good programmers.

We can wax on eternally about how the good programmers in this situation should've seen the writing on the wall and gotten out as soon as possible, but that's very easy to say seated in armchairs gazing from afar. Maybe they thought things were going better than they really were; maybe they saw things going bad, and thought they could motivate others to turn things around before things become a disaster -- and maybe they failed trying to do that[1]; maybe they have been trying to get out, but they are already tainted by the bad work of the organization that has become part of their image as well.

They could do work outside of their employer, for instance on an open source project, but these can succumb to the same sort of pressures that push profit-seeking organizations toward mediocrity. A single developer, even a really willful one, will have a hard time overcoming this inertia.

[1] At one of my prior employers, I was in this boat; I both tried and failed to get things turned around. I did quit, but probably not soon enough to convince people that the site was not a reflection of me and my standards. The site was absolutely shameful, and I'm sure anyone who used that site to infer things about my skill level would've sworn I had no skill or experience outside of a weekend PHP crash course. Granted, it would vet one organization I would probably not want to work at; but it would represent wasted potential based on presumption, which is a shame.

I really think the details matter, because they are a chance to determine if the work you are using to assess the programmer is spurious input, and reflect an influence largely made up of things other than that programmer.

Consider the common case of the average developer (especially so for a non-web developer). They've worked on several closed source projects on which the extent of their participation is non-trivial to determine. Ultimately you have to talk with them and suss out what they actually know, which is at least a day long process in order to attain any degree of accuracy.

Even if they've worked on open source projects, it may take a lot of effort to grovel through their commits to find out whether or not they're telling the truth on the size of their contributions.

Incidentally, all of this is a good reason to have your own personal solo project and your own personal blog, though it can take a lot of effort to do so.

speaking from experience, i've always had passion and personal projects, etc, but i used to be a sucky programmer. now i feel like i've mastered certain things, but i'm still developing and getting better.

for one thing, quality comes from sustained effort and committment. i've been working full time on startups for a while. that's a hell of a lot different than someone doing a different weekend toy project every month.

i'm friends with some pretty passionate "hackers," and i'm definitely not knocking that kind of activity---heck, i have side projects sometimes, too. but there's a big difference between hacking and building something serious. toying with tons of APIs and langauges doesn't make someone a good programmer.

finding a good programmer is still hard. even if programmers were completely honest, and the metrics all objective, would they even have the language to explain to business guys that they were good programmers, both in technical proficiency and quality? it's not impossible, but it's quite difficult.

I have to agree that metrics don't tell the whole story. It really depends on what the programmer is looking for. Some people may be attracted to new features & complexity in their projects, while others look to design a small simple project.

toying is part of the inventor spirit and different projects need different things. one project might be ok with one of the mockups architects make. another project might need the life sized building. and so on in between, changing as the project grows.

"If your potential programmer didn’t do any programming before university, and all his experience starts when she got her first job, she’s probably not a good programmer."

This is the only time he uses the female pronoun anywhere in his post, and he uses it to describe a programmer that started in college and isn't any good.

Conscious or not, it's somewhat accurate; female cs majors are less likely to have prior experience with programming going into college.

You could write a "How to recognize a good [X]" article about almost any profession if that's your thesis.

Let's face it, the vast majority of work programming isn't that hard. A lot of programmers, perhaps especially the good and smart ones, might rationally realize that having a life outside of work--in other words, working a 9-5--often makes more rational sense than putting all of one's considerable intellectual energy into writing yet another CRUD (or whatever application du jour one normally writes) remix.

For all of the great devs I know and have known programming is very much a passion for them, and they will engage that passion outside of work. But this doesn't mean their job is their life or that they don't maintain a healthy work/life balance. Indeed, working on projects they're passionate about outside of work can often provide a release from toiling with a brain-dead code base at work.

Also, the vast majority of software development may not be hard, but perhaps that's because the vast majority of developers aren't that good. The work being done by the best devs is orders of magnitude better (and more challenging) than the work being done by the daily-wtf generating paycheck programmers in the enterprise trenches.

Curiosity and genuine interest develop at different times for different people depending on the circumstances of their life. They might have not discovered programming until later in their life, or they may have been soured by an bad experience early on and only came back later on.

For some people, they don't discover programming until university, but then find the absolutely love it and quickly develop a considerable talent. For these people, who do exist, the pre-university test is a false negative.

A lot of these tests people come up with are based on the concept that past performance indicates future potential. The thing about these tests is that they suck really bad, except that for certain things there are no other tests.

A meteorologist I once heard talking about weather phenomena in Alaska joked in his talk about how there are two possibilites for available weather data there: data sparse, or data void. It is much the same in finding good programmers. There are a lot of important considerations, and getting a feel on each depends on sparse data at best. The best advice I can offer is don't take things like this too personally. Following advice from pg almost certainly applies to applying and interviewing for jobs as well.

Don't be put off if they say no. Rejection is almost always less personal than the rejectee imagines. Just move on to the next. (This applies to dating too.) - Paul Graham, Undergraduation.

Just take it as a sign that they probably would be judging you even after you are hired in ways that you might find unreasonable. Not every hacker is meant for every programming job.

Is that the point when you finally realised you had the freedom to do what you wanted in life? - If so, perhaps you are alright.

Is that when someone told you that you needed a job, and programming looked well paid? - Perhaps not so good.

Because at that time computers took entire floor of building and costs millions of dollars. By the time I got my hands on Apple II I was in graduate school...

The only reason I ask is that I consider myself a late-bloomer, and after reading Coders at Work (which I highly recommend) I was kind of bummed out that I squandered my opportunity in college by not hanging out with CS crowd. It seemed like so many of Seibel's interview subjects accomplished so much in this period and were more or less born to be programmers. . .it sounds trite when I read it aloud but I can no longer ignore this unspoken caste system amongst programmers.