You can say that software sucks because of poor programmers and poor project management, but the truth is that the code is not sufficiently wieldy. There's no way to manage the code. I can't query all places in the code where the UI interacts with a database column. Accounting systems can give you a variety of reports based on abstractions at a variety of layers, slicing the information in different ways (horizontally, vertically, etc). Software systems? Go fish.

Software sucks because at some point we got too excited about what we were doing and stopped (sufficiently) caring about how we were doing it.

expenses is a great example, it needs to accomodate so many different regulations, compliance issues, legacy compatibility needs within an implementation, that it can't be more than a generic shell in vanilla SAP. so you need consultants to set it up in the first place. what do they do? ask the client's project team to tell what they want...and this is why all enterprise software implementations produce shit outcomes. non-software people having a direct say on UI/UX.

sort of like homer simpson designing that car.

Consider internal software - there is literally no disconnect whatsoever, but very often the software is terrible. This is relevant because it raises the issue that the business simply may not care about quality, and thus putting in the work required to make the thing good might simply not be possible.

There are many, many factors involved in determining whether software sucks or not, and key is the understanding or lack thereof of what the users want and need, agreed. But keep in mind, there is a big difference between seeing what the users' need superficially vs. understanding clearly and deeply what the users' need and implementing it well.

For internal projects, investment is obviously a huge problem, and once you've built the wrong thing you're unlikely to get more money to do it right. But for a given price point, there's a huge range of quality you can obtain. Paper protoyping helps you maximize that at very low expense.

we also used a ticket tracking packages called, I think, CAMS. possibly a CA product. oh, and we used Harvest for source control. I still recall the joy in learning CAMS would be replaced, and the horror that followed as we came to grips with ManageNow. Never dreamt I'd wish for CAMS to return, but I did.

So, yes, I agree. I can't believe the companies that peddle this crap actually use it.

Anyone from SAP care to chime in?

Who's ever wished for a giant red "PLEASE STOP HELPING ME" button on their computer? Everyone, right? One of the causes of frustration we looked at was when the software helps you, but gets it wrong. Frustration is increased when the effort to fix the software's "help" is anything other than minor, and increased again when the computer repeats it's efforts at helping. Throw in no obvious way to get it to stop and you're in for some blood-boiling times.

The guideline we came up with was to not concentrate so much on how cool it is when things go well, but think about what happens when the software guesses user intent incorrectly. Consider:

* How often is the software likely to get it wrong?

* what was the level of irritation is caused by getting it wrong?

* How much effort is required to undo the computer's help?

and weight it up against how much effort you're saving the user when you get it right.

Thinking about this has caused us not to add a particular cool feature we had planned, because even though (we thought) it was very cool, there was about 30% chance or so of it being wrong, and the payoff wasn't worth it.

as in: if you're writing software for a client and the client drives the specs directly, the outcome will suck balls.

evidence: all enterprise software installations ever done. from ERP to CRM, the vanilla software packages might even have good UX/UI, but once implementation with all its "critical" customizations is done, you'll end up with a turd.

The main problem with "enterprise" software is that it's done cheaply, and violates levels of abstraction. Thus why it frequently requires specific versions of antiquated technology ("Our CRM requires IE6", "We can't upgrade past Windows 2000").

If it was designed properly, you could easily rip out both parts and iteratively redesign them to meet business and technical goals.

I've got a live PeopleSoft installation running on a server in my apartment that I'm outfitting with code to do exactly what you mentioned - split out the back-end from the front-end. So far it's going brilliantly - for some sadistic reason, I enjoy trying to reduce the complexities of these applications.

Re. the PeopleSoft copy - Oracle provides all of their software (and master license codes) for download for evaluation purposes through a portal called eDelivery. I had to read a few hundred pages of documentation, but after a month I was able to get all the components to talk together. I'm trying to convince Oracle to give me a non-support license so I can cover myself legally, but I'm getting the silent treatment since it's just me and I don't have the budget of a CTO lol.

a: "We always tear out everything, down to the rafters." b: "That sounds expensive." a: "It can be, especially if you find out the wiring or something isn't up to code." b: "What if someone wants only---" a: "No, there's no 'only'."

In my experience, if programmers always had a complete design specification before starting, and if we always "tore up" the old mess (down to something unquestionably stable) before making any improvements, our software would be much, much better. It'd probably also cost more.

The biggest cause of this that I see is that the people paying for it are not the people using it. In the consumer market we are starting to see that good design does sell (I think it was Gropius who predicted that the cost of design is amortized). But with enterprise software, the people paying for it are rarely the ones using it, so they're perfectly willing to buy a lousy program if it means saving cash in the short term.

And by the time the software was done, it'd be useless. A house doesn't get obsolete in ten years, but software - which is much more complex than a house - can be obsolete in ten months.

Not to mention that I'd rather not be reinventing the wheel every six months.

Generally the "lean startup" credo holds true here: Build an MVP fast, get feedback and iterate.

However, there is one technique which doesn't get enough airtime in my view and that is paper prototyping. It gives you a surprising amount of quality feedback with a fraction of the effort of coding something up (and no bugs!), and allows you to iterate there and then.

1) Why are all these people so confident, if only a minority of them can be right?

2) What procedure would make everyone converge on the same correct answer?

Developers trivialize things and interpret them in far "superior" ways that lead to huge gaps of "they never told us".

Development has to move from just coding, to learning to first understand what details are being managed, and how those details interact in a system in all the stages the details/data exist.

If we believe every business is becoming a software business, the reverse is true, all software developers must understand the business more and continually develop the skills to be the bridge between business goals and technology.

How to do that?

Shut the hell up and learn.

Ask (and learn) why things are done a certain way. Uncover any competitive advantages the business has from doing things a certain way before getting on the high horse and deciding to improve the world because it's so obvious. Developers may be surrounded by non-techs, but they certainly might be surprised to see the organization itself does have processes and competitive advantages that have to be maintained for the business to survive.

Avoiding the classic SAP-esq kiss of death of doing it the SAP way, wiping out the competitive advantage (seen first hand), and then spending tons of money customizing and automating any ERP to get back to what they had before (and more) seems silly, but I can't say the 70% of failed software projects fare much better.

So, before we think we understand something, shut up.

Before we think we know better, shut up.

Before we think we can simplify things, shut up.

Before we think we can make things more efficient, shut up.

Shut up, listen to the people using the current systems and processes and learn what is working for them, or not, first.

Shut up and learn. Don't finish people's sentences. Don't tune out. Don't think things are beneath you. Don't think you've seen it before, or built it before.

At each step ask them if you understand their process correctly before going off to formulate a faster way of doing things for their confirmation.

Software has the power to uplift the lives of people and help them get more done with less effort. If you don't value this, don't make the rest of us look bad for your laziness and inability to continually develop your own skills.

Once you have learnt why the business does what it does, the way it does, it's fair to ask the question "How should it be?", and see what differs. That, is the beginning of what you should start thinking about.

Having integrated custom systems and built new ones to replace existing ones since '99, this is the single worst thing I see. Enough developers simply don't have a healthy paranoia of their understanding. Knowing a little bit about something can make developers just as dangerous as the "business" folks they judge for doing the same. It all comes out in the wash with the 70% software failure rate.

Without understanding the data of the business, how it interacts, exists in different stages, and how it needs to be input/output, and why, amongst other things is the leading contributor to software failure.

It's as much "the customer didn't know what they wanted" as "developers failed to understand their job is to go learn the business first and then design something to build".

To be clear, this can mean working in people's positions first hand to see what they're going through / facing that they can't explain to you.

It can mean seeing what state a business is in, infancy (no systems or processes), adolescence (some systems or processes), or maturity (a mature system and process, even if it's all manual).

Do those three scenarios equal one approach to all of them? Hell no.

There is, though, a few common things to keep in mind:

- Ask customers to teach you the business as they know it first. Pretend you're the next apprentice, or the owner's right hand man. Ask to be taught not just how to do everything, but why it's done that way.

- Your goal is to get more done with less effort. The software you design and build should not simply make less work for some, and more for others. It should free people from BEING the tools and systems, to USING the tools and systems. The people of an organization should do what they know best, instead of being computers, they should be interacting with each other, and customers.

I could go on a long time about this. But it's Sunday and I hope the positive wishes come through.

And again: Paper prototyping. Sticking an interactive bit of paper in front of a user is an effective and inexpensive way to get him to explain the holes in your great idea, and you can adapt it there and then and maybe come up with something that fixes the problem that you were hoping to solve, without creating a ton of other issues.

A question I like asking is, "Teach me why this needs to be improved".

So when a question like this takes us 8 hours a week, every week, comes up, you can say we can save 416 man hours a year if you let me work on this for 40 hours.

I'm not sure whether or not there's paper prototyping. I use a lot of different tools, the most important of which helps me whittle down an idea to it's essence. For me that magic happens on whiteboards, and on paper first.

I've recently started doing it using a stylus on an iPad and a Galaxy Note with increasing success. I almost prefer the iPad or Galaxy Note because I can keep erasing and refining to get the perfect layout/design.

Give it a few thousand years. If we can manage to survive it, the miserable suckiness of our software will taper off.

If I was reading a book, and the author made an update, I wouldn't necessarily want the new update from the author. Notify me there's an update? Sure! Automatically replace the copy I'm reading with the update, simultaneously removing my ability to ever get back to the version I was reading? No thanks.

"What's there has blinded people to what is possible." As someone who has a good idea of what's possible, I tend to agree, and think of browser 'cookie' support as a good example. 'cookies' were lambasted as the embodiment of evil in the mid-late 90's. The hysteria around them was insane. Why? We had little control over the visibility of what a cookie was, what it said, and how to manage them. Even today, in 2012, cookies are little understood, and to 'manager' them requires multiple clicks in to layers of menus often marked 'advanced' or 'privacy' (both are silly labels for cookie data, imo). We could have given people easier access to view/manage/understand cookie data, and probably avoided national legislation like we see in some countries now, but that would have meant breaking the status quo. Far easier to focus on browser market share and "standards compliance" as the primary measurements of browser utility and acceptance.

It's easy to sit on a mountain top and spout off declarations about how things should be, but in no place is the old adage "the devil is in the details" more apropos then software development. Even the UI designer runs into these fundamental conflicts, but the low-level programmer is inundated with them. And worse yet, software stacks are so deep and CPUs so powerful that the scope of what software actually does is increasingly difficult to mentally model. The only way out of this is to ruthlessly narrow the scope and have an organization-wide emphasis on UX. This works for Apple, but consider that in order to do this they basically punt on all the hairiest business requirements because it just doesn't serve their vision. Nevertheless those requirements are real and unignorable which is why SAP et al thrive.

The reason most software is bad is not for lack of effort, it's just damn hard to do right. It's good to ask big questions, but then go try to solve some of them rather than admonish those of us who pour our lives into this.

A warning from experience if you are thinking of implementing Alan Cooper's theories on the topic of legacy UI: Be very careful when removing conventional things because they seem archaic and make sure that they respond to a genuine pain point and your dev team has the chops to fully implement the solution you propose. Otherwise you're going to end up with a half-arsed unconventional UI which will just confuse everyone.

But you are mostly right, software has far, far too many pointless layers of abstraction now, requiring vast resources just to do trivial tasks. There's nothing 99% of people use a wordprocessor or a spreadsheet for that you couldn't do on an 8-bit micro in the 80s. Games these days are just not fun, whereas the 8-bit days were a golden era of creativity. We need to take it back to the old school.

See guidelines 97 and 98 from Jakob Nielsen and Marie Tahir's book 50 Websites Deconstructed [1] for the primary reason why there has been little interest in removing the "refresh" button from browsers.

Technical limitations don't really exist (and if they do exist it'd be fairly easy to solve). Server-sent events[2] and WebSocket[3] are already implemented in the latest versions of popular browsers. Modules or implementations within popular HTTP servers already exist for doing HTTP push (they tend to use older AJAX-like techniques though).

If usability was no concern (or very carefully handled) it'd be fairly easy to write your own nginx module or "WebSocket server"[4] that uses inotify to check for file system changes. For each change that impacts an open WebSocket connection, a "refresh this page" notification can be sent to the browser (which then uses JavaScript to force a page refresh). There is a potential for smarter refresh mechanisms in browsers that maintain the current scroll state, field values, etc but you'd still be frustrating the user with severe usability problems.

[1] http://www.useit.com/homepageusability/guidelines.html

[2] https://en.wikipedia.org/wiki/Server-sent_events

[3] https://en.wikipedia.org/wiki/WebSockets

[4] http://altdevblogaday.com/2012/01/23/writing-your-own-websoc...

It can also be extremely wasteful - if I leave a tab open for hours or days, you'll have to waste your resources and mine to keep pushing me stuff I won't see, while now I just hit refresh when I want to see new content.

In terms of usability, it's often jarring to watch content change when you're interacting with it - that's why even sites that implement real time notifications often have a link or button that you have to press to update the UI. In many cases, doing that completely negates the benefits of pushing.

Also, see the thesis on "Architectural Styles and the Design of Network-based Software Architectures": http://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm

Polling is a good default for the web; it fits most use cases (content that rarely changes) in a simple and economical way. WebSockets are useful for the exceptions.

Web applications are a very small part of the web as a whole.

you can implement polling with interrupts

But why would you?

Are you aware that the OS is using an interrupt-driven system to poll the server?

Yes. So? In that case, interrupts are a great fit. In others, they aren't.

This is just dumb engineering.

Why?

And to say that it's good for "most things" is just lazy thinking.

It's not good: it's better for most things. There's nothing lazy about evaluating the options and choosing one.

"Most things" are that way precisely because of the crappy architecture. Rationalisations.

Sure, everyone who implemented this shit is dumb and lazy. It couldn't be that they have good reasons for doing what they did.

I see you criticized me for not providing technical points, yet now you refuse refute them. I don't see the point of this conversation anymore.

http://en.wikipedia.org/wiki/NeWS

And NeWS was better in the sense that it used PS for everything rather than an unholy mess of JavaScript, CSS, HTML, etc etc.

Well the web is supported everywhere because it was simple. The fact that it is available everywhere (which is actually pretty amazing and unique in the entire history of media if you stop and think about it) meant people kept pushing the boundaries. The limitations caused standards to be pushed forward. Wide use of the standards means no platform can break free from supporting it (Microsoft did their best for almost a decade!).

Obviously it looks like a Frankenstein monster from an engineering perspective, but it doesn't matter because that's where the investment is going. The good news is it will evolve and improve, and soon enough the grey beards who remember that there could have been a better way will all be dead; a lot like UNIX actually ;)

edit: I'd appreciate if bitter GUI developers would respond rather than misdirect their anger at the down vote button.

>The good news is it will evolve and improve, and soon enough the grey beards who remember that there could have been a better way will all be dead; a lot like UNIX actually ;)

That sounds like some sort of evil plot. For that matter my beard isn't anywhere close to being gray and I know that things could have been better. And if you didn't have +8000 karma I would think you were trolling.

That you're not just makes it worse.

You're the one who asked. If thats what I got out of it it's probably what the people downvoting got out of it too.

For what it's worth maybe the intended message would have come through better if you were saying it in your voice with tone and inflection rather than the way I read it as words on the screen.

So dev time, well the last THIRTY YEARS of dev time haven't gotten me any anything I didn't have already.

For example, what if you want to use your spreadsheet when you're not home? Never happened to you? Well, it happened to others.

Or to use my previous analogy, you're essentially saying that because you can build your own custom designed car from parts, pre-built cars are useless.

1. Software that is built to deadline will decay, even if the developers are good. That doesn't mean that an occasional deadline is the end, but if a long-term "deadline culture" sets in, get out. A long-standing deadline-oriented culture means you should be looking to jump to another project or company before the maintenance phase starts, because (1) the maintainers will be underappreciated (that's typical deadline culture) and (2) once the original architects get promoted it will be politically impossible to point out the real reason maintainers are unable to deliver in a timely fashion, and the slowest one to run away will get eaten by the bear. It means that technical debt will never be paid off; management will never budget time, and engineers will be too busy to clean up the code. Software engineers generally lack both the political pull and the broad-based knowledge to push back on deadlines and tease out which ones actually matter and which don't.

2. Entropy. Good software is less stable than bad software. Think of this as akin to the "broken windows" theory. Once software reaches a certain state of degradation, each change, although it might fix a bug or add a feature, will make the state of the software worse. There are creeping kinds of badness that can't be caught in incremental code reviews, such as adding 10 lines to a long for-loop or a "necessary" boolean parameter to a method that over time ends up with 15 boolean parameters. Often the managerial solution (once it's far past too late) is to put maintenance of this bad system on the calendar and make it someone's full-time job (instead of a shared responsibility) but no one wants that job and often that work is allocated to marginally skilled junior programmers with no clout. Then you get adverse selection: the more skilled people in that set will leave the project (or company) before they put in enough time to become decent at it.

3. "Pay as you go" maintenance, which includes periodic fixit spells, is always better than after-shit-breaks maintenance. That said, existing tools don't make it easy to revert quality degradation. IDEs really don't perform this function as commonly used. (I'm sure IDEs can be really powerful if well-learned, but people who are dedicated enough to master IDEs are also dedicated enough to jump wholesale to better languages for which IDEs are unnecessary and often poorly-supported. IDEs, in large part, exist to compensate for weak languages.) Code can rot in any language, but one advantage that languages like Scala and Python have is that, because they have REPLs, which are far more useful than any IDE, people can interact with the software at a code-level and fix things while the code is in that "moderately bad" state before it is too late. In 2012, I wouldn't start anything important in a REPL-less language. (C is not "REPL-less" because Unix is the C programming environment. C++ is, not on account of language intrinsics but because it has departed from the small-program Unix philosophy and is used for large-object programming which requires interactivity at a code level.) At least some programmers will have enough of a sense of ownership and citizenry to clean up failing code as they work with it, but if you deprive them of the REPL, the one tool that any good programmer will recognize as essential, they won't put in the work.

REPL or Fail: http://michaelochurch.wordpress.com/2012/02/07/repl-or-fail/

4. The transition from being a mediocre to a good and then to a great engineer is about moving away from being an "adder" (someone who increases codebase complexity and functionality, thereby having an additive business value-- ignoring long-term costs of complexity, which may or may not offset that additive value) to a "multiplier" (someone with broad-based positive effects that make the whole team more productive). Contemporary tools and programming environments (Java, C++, IDEs, IOC, dependency injection frameworks) are about helping more mediocre engineers become solid adders at the expense of the really great engineers, whose creativity is constrained by less powerful languages and tools. One of the goals behind Microsoft's professional certifications, the design of VB (and later, the hijacking of Java), and the attempted ghettoization of the command-line (which good engineers like) was to make it possible for huge teams of "commodity" programmers to be productive as adders, with the hope that "someone" would have the patience to staple together the zillion classes they cranked out. From an MBA perspective, this is a win, because 2-4 times more people are eligible to be adders, but it also holds people back from becoming multipliers. The long-term problem is that a team without any multipliers will accumulate complexity and the emergent design (because you want a solid engineer doing your design work, and you can't get them in commodity-programmer environments, "design" coming out of a commodity shop will be ad hoc) will be disastrous.

5. With a few exceptions, the real fuckups in software don't seem to be blameable on a single person. They usually emerge either from jobs no one does (because the people who care about them being done aren't in power) or that too many people do (once code has been passed over by too many hands, it turns to shit).

In addition, it's worth pointing out that many Java programmers use unit tests as a poor-man's REPL; it's not the same, but it serves a similar purpose: I want to write some code, then execute it to make sure it does what I think. It's less dynamic, but it has the advantage of leaving you with regression tests, and it does let you explore and quickly iterate your code. If I'm not sure how to use a library, I'll do exactly what you'd do with a REPL: I'll write some code to use the library, then write a simple test that executes that code, and then I'll iterate my way to a correct solution. Again, the integration of the IDE with the unit tests makes running, debugging, and bouncing between the test and the code much easier than it would be in, say, vim/emacs and a terminal.

My point is that good programmers in any language find a way to do the sort of iterative evolution and exploration of code that you act like is only possible with a REPL, allowing them to fix errors early.

Many of your other points here are good, I just really feel like your "the REPL is essential" argument is pretty misguided.

I find that I am generally faster in development (at least with new libraries) in Java, than I used to be in Ruby and Python. This is all thanks to the "discovery" ability of IDEs. (I admittedly never tried a Python or Ruby IDE.)

And I have the same feeling regarding refactoring. It is not merely limited to renaming a method. I find myself very often making major structural changes to my code. Moving packages around, introducing interfaces, changing type signatures. In this regard working with an IDE makes me feel like a "software architect", I get a big-picture of the project in a much faster and better way than I used to with purely a text editor.

I also feel I waste no time on boiler-plate code (which admittedly Java has a lot of). In Netbeans (I am sure it's the same in Eclipse and IDEA) the code generation abilities are terrific. For instance, I can just write "class C implements Interface", press Alt+Enter+Enter and see all interface methods written out and ready for me to fill in the implementation.

Regarding the REPL: I've found that with strict typing, I just end up knocking out the code that _I think_ should work, and then I test if it works. I can sometimes type for 200-300 lines without running the code, then test it and see that it actually works. Of course, sometimes it fails too: luckily Java debugging is easy and incredibly capable.

However, if you really want a Java REPL, you can have something a little bit similar with BeanShell (you can even embed it into NetBeans).

I think the important question here is, does IntelliJ help you simplify existing code? Do you routinely simplify existing code? Does your team routinely simplify code? Or even better, does IntelliJ help you write simpler code in the first place? Meaning, is code written with IntelliJ routinely simpler than code written with Eclipse or Emacs?

My experience with IDE overall, is that they are of tremendous help for navigating complexity. On the other hand, they are of very little help for reducing complexity. (You cite the method renaming as an example, but compiler errors keep track of broken references just fine)

Regarding the REPL, it's not the REPL itself which is essential. It's the tight feedback loop. There are other ways to provide such a loop. Some of them are much, much better than REPLs: http://vimeo.com/36579366

My objection has more to do with IDE-dependence. Also, Java is a language in which it's way too painful not to use an IDE-- I certainly use an IDE when I'm in Java-- and IDEs tend not to play nice with outside-of-IDE actors such as version control, so there tends to be a Mafia (once you're in, you can't get out) nature to them.

I don't dislike IDEs themselves. I dislike the fact that people use them to make insufficient languages less bad in lieu of using a better language, and worse yet, that business types end up with the impression that other languages are less mature/usable because they lack IDE support. When you have an expressive language, you don't need an IDE (or, at least, I've never found myself missing one).

That said, it may be that an IDE is pleasant to use with a better language, and makes it even better. A lot of what IDEs offer is useful. That said, I'd rather have a good language like Ocaml and no IDE than Java and the best IDE on the market. Java development is just not very "flow"-ful in my experience, and the productivity benefits conferred by an IDE are small compared to the astronomical bump conferred by an expressive language.

It's not exactly the same as a REPL (I can't call the method right then, of course, but I'll get to that in a minute), but it serves a different purpose, and your argument in that linked post about how IDEs aren't made to read code is, honestly, laughable: it's way, way easier to read and explore a Java code base within an IDE than it would be in a text editor and a REPL.

Ok, I see where you're coming from. I agree that reading Java code pretty much requires an IDE.

Now, you can argue that Java itself is verbose enough that reading it is painful because of all the boilerplate: sure, that's a fair point, but it has nothing to do with an IDE.

Not directly, but I think there's a cultural problem that might be enabled by the IDE. Just as it's said that 4-wheel drive helps a person get stuck in an even more inaccessible place, I feel like IDEs enable people to program who shouldn't be programming, inappropriate languages to just kinda work, and bad practices not to totally fall flat on people who use them. I can't prove this, but it seems like this is the case, taking an industry-wide perspective.

If you had a language with cleaner syntax and an IDE, that would be better than just a language with cleaner syntax and a REPL when it came to reading code.

I'd like to try this experiment. You could easily be right. My experience with IDEs is in weak languages and they seem not to be used in strong languages (Scala being an exception, although I haven't tried its IDE support).

In addition, it's worth pointing out that many Java programmers use unit tests as a poor-man's REPL; it's not the same, but it serves a similar purpose

Interactivity and unit tests serve different purposes. I don't think either is an acceptable substitute for the other (and yes, I've seen people attempt both substitutions).

For me, interactivity is the only thing that keeps me in a state of flow (instead of boredom) when I have to read code, especially because there's at least one library for which I really want a REPL so I can see what the calls do.

I just really feel like your "the REPL is essential" argument is pretty misguided.

I guess I should be saying "interactivity is essential". C technically doesn't have a REPL, but it succeeded as a language because the C/Unix philosophy encouraged small programs that could be used and explored at the command line, which keeps the C environment engaging and tractable as long as people aren't writing huge programs. I haven't seen interactivity superior to what the REPL provides, but I haven't seen enough to rule it out either.