There was a project that was crashing into a blue screen of death after a few hours. The crashing library had #defines that went 15 levels deep. The stack dump wasn't very instructive. Even though the library ISV was based in Boston, I had to fly out to Santa Cruz to talk to the developer directly. He was somewhat belligerent: "you should have figured it out yourself!". 15 levels of #defines. He was the perfect example of the other side of Dunning-Kruger thinking his arcane shit was obvious.

But, that didn't solve the problem. His fix didn't. So, I flew back to Boston. I started to trace everything with -ahem- #defines of the main APIs. It was turning into a Zen Koan: the sound of one app crashing. Everything worked with the Borland compiler. It was just the MSC compiler that failed. The company was one of the top 3 ISVs back then. Meetings about what to do would last for hours. There were technical politics: "Can we use Borland?" No; the company had standardized on MS. I could talk to their engineers directly, though. But, they didn't have a clue. So, I read though the docs to see what compiler switches were available. And there it was: best-fit. Fixed!

So, I wrote an analysis about why best-fit was absolutely the worst possible memory management and sent it to Microsoft. It took 8 years for MS to change the default. So, the reason why software would blow up? Best fit. Why early versions of Windows with go into blue-screen-of-death? Best fit. How many developers had to deal with this? Worst nightmare in aggregate.

> The simplest algorithm is first fit... A minor variation of first fit is next fit. ... Simulations by Bays (1977) show that next first gives slightly worse performance than first fit. ... Another well-known algorithm is best fit. ... Best fit is slower than first fit.... Somewhat surprisingly, it also results in more wasted memory than first fit or next fit because it tends to fill up memory with tiny, useless holes. First fit generates larger holes on the average. ... one could think about worst fit... Simulation has shown that worst first is not a very good idea. ...

> quick fit ... has the same disadvantage as all schemes that sort by hole size, namely, when a process terminates or is swapped out, finding its neighbors to see if a merge is possible is expensive. If merging is not done, memory will quickly fragment into a large number of small, useless holes.

What did I miss?

This is a whole topic in itself. There has to be some good medium between consistency and best tool for the job.

For example, IBM determined that, despite Windows dominating Desktop/laptop market share ( https://en.wikipedia.org/wiki/Usage_share_of_operating_syste... ), Macs were cheaper to use ( http://www.businessinsider.com/apple-says-ibm-saves-270-for-... ). Now you could go further into that and ask whether OS X is the right tool for everyone's job, as I'd argue in some cases it isn't. However, I think that it shouldn't be that difficult for a company to be open to letting their employees choose the hardware, software, and other tools to get the job done. If the budget is there, then why not have the discussion, and if it isn't, allow the employee to make the case for an exception.

The same could be said for priorities and tasks. Allow your employees to make the case for what they should be working on, and allow group discussion if appropriate. You may be surprised at what good could come of it.

When I was young, I would go into interviews sometimes where the other candidates where there in the lobby, too. Some were older and clearly had been in technology a long time... but never advanced past where they were in their "prime" as you made small talk. They weren't going to get the job nor should they have. They allowed themselves to become as relevant as token ring.

As technologists we have to keep ourselves current. That keeping up with the kids is really part of our profession: the technology evolves quickly and so must we. But this is the part I like about technology work. There's always something next to keep things fresh. A new technology, a new architectural style, new needs, unsolvable problems that become solvable. Our profession doesn't tolerate coasting. Good.

Other professions have this same problem, just not at our speed. The proverbial buggy whip makers had to evolve with changing realities... they just had more time to see the inevitable.

(As an aside, I play poker with a dedicated COBOL developer in his mid-50s... so sometimes you can make old school pay, too :-) ).

Side-tidbit: considering the way you said that it sounds like you might already know this, but some guys working on a variant of Smalltalk called Strongtalk were temporarily whisked off of that project and commissioned by Sun to write Java.

I understand that full compiler and VM source are now under the BSD license. It was last poked in '06. It probably needs cleaning up.

StrongTalk's a JIT, so it's going to be painful to port, unfortunately. And JITs have moved on a long way since then. I'd really like it to be resurrected and brought into the modern age, but in all honestly I think it's just dead.

Thinking about what you said gave me an idea.

Real-world native languages fall out of use forever on a daily basis. Linguistic historians want to preserve the unique attributes of these languages, because they each tell a story and add to the character of humanity's history.

Computer languages share some of these attributes in a broad sense, but these must also be offset with the technical merit(s) of the language in question: if a language's foundational concepts have become outdated, continuing to use that language outside of a job description is a liability.

However, if a language's fundamental design is still interesting, but the low-level technologies used to implement it have fallen out of favor (for example, targeting IA32, or only functioning on an obsolete processor, etc) that language will also still become effectively "dead," and arguably unfairly so.

It could be said that there are language designers and language implementers. The designers have sufficient scope in their mental model to sweep across the domain of language design and implementation and distill that into a functioning codebase. Implementors, on the other hand, just want to fulfil a spec.

I wonder what would happen if there were a website out there that listing all "dead but worthy" languages along with instructions on how to get them running? Does anything out there like this exist?

I'm envisaging students, language enthusiasts, etc, modernizing these old language specifications - which would allow the community to readily reasses the technical merits of the language in question. It might ultimately be demoted again, but we'd have a reasonable, modern justification of why.

I would worry about it and I'm only close to 30 but I'm self employed now.

I think it's more like Uncle Bob says: the number of programmer has doubled every five years. A consequence of that is that those of us who were in it for a long time (30 years in my case) appear to be surrounded by a sea of youngsters with no clue. Personally, I realized recently that not only I don't have a fear of losing my job - I'm starting to find it strange when others tell me about it. (I'm re-reading Clarke Ching's Rolling Rocks Downhill and I find it harder to empathize with the protagonist than it was a few years ago.)

I have done it twice in the last five years - in both cases because I wasn't allowed to improve the quality of the code. In the first case, it was because "the client isn't paying for that, they're paying for features". In the second case, because "that code was written by the NY office and they're going to be upset if we change it".

But I hear you on the "worse" technology sometimes winning. You mentioned Java; it was worse than just about all other major contenders, and is only finally losing popularity.

On a current technology fad: React seems to be designed to ignore 40 years of accumulated software best practices. [1] Separation of concerns? Who needs that any more? And the rationale for it is that it allows teams of 100 developers work together on an app. Open standards? Nah, how about lock-in to custom language extensions that will prevent you from migrating your code to the next web standard! Much better.

And how many app teams have 100 or more active developers? Probably fewer than a dozen, and I submit that none of them probably should. Certainly not the Facebook app: It has a lot of features, but not that many features, and yet it has a 150Mb footprint. When I hear things like that, I can't help but fill in "junior" or "mediocre" in front of "developers." React helps to prevent people from breaking each others' code when you have bloated development teams filled with junior developers. React has some cool ideas, but all told I think it's a step backward for software engineering, and certainly isn't as much of a help for small teams, especially if you want to have a CSS/SCSS/LESS expert styling your product without having to dig through JSX files, for instance.

The Java rationale was similar, IMO: You can use the Java abstractions to allow lots of mediocre developers to make contributions to a product without breaking each others' code. At least not as frequently as when they can poke right into random data structures and change things arbitrarily. If it weren't for Google's decision to require Java for Android, I think Java would be relegated to big company web backend development.

I do like React's idea of the Virtual DOM for optimization, but you can get that without using React. [2] React Native is great for using native components and driving them from JavaScript, but it's also not the only game in town. [3]

Back to the original point, though: You can stay on top of the Hot New Technologies, but when there are good technical reasons to use alternate technologies, stay on top of those as well. And then explain clearly to your clients (or employers) why the current fad is a fad, and how to get the key benefits of that stack without its drawbacks. Oh, and choose clients (or employers) who will listen to strong technical arguments. :)

[1] https://www.pandastrike.com/posts/20150311-react-bad-idea

[2] https://github.com/Matt-Esch/virtual-dom

[3] https://www.nativescript.org/

That's some strawman right here. First, JSX is optional. Sencondly, JSX is open. You may disagree whether it's a standard or not, but you know what? If tomorrow you end up with a large codebase where you want to get rid of JSX, you just apply a JSX transpiler on your existing codebase. Problem solved. As for separation of concerns, you have a little bit more of a case. React does allow you to put business logic in your views... just like 99% percent of templating languages out there. But React does not force you to do that. You can have only the minimum amount of logic you need, and put most of your frontend business logic in your store.

Templates using a "handlebars" syntax are pretty portable with minor tweaks, in general.

Web Components / HTML Imports kind of solve this though, but they're still not there. http://caniuse.com/#search=components

JavaScript has lexical scope, and that pretty much solves everything. CSS styles are now local variables and need to be passed around. Components are not just a dump of HTML, but opaque functions - their inner structure cannot be (accidentaly) accessed by outside components. Ah - finally, a sensible programming model.

Just imagine what would be possible if CSS class names were lexical, not global strings - and you could import them into HTML (perhaps with renaming). How big of a change that would be in terms of making things easier to separate.

Well... React users got tired of imagining things :) http://glenmaddern.com/articles/css-modules

You can't escape the fact that any non-trivial rendering needs some logic. If you don't let yourself get carried away, you get the exact amount of logic you need for your component in a reusable package. As someone currently porting a codebase from the dark ages of the web to React, I have difficulty expressing how nice that feels.

In for instance Angular you have the template seperated from the controller or directive. But in what way does that really help you? The controller and the template are inseperable. The variables inside of the template corrospond directly to the variables in the scope of the controller. You cannot see what the template is doing without having the controller next to it. In React you really see how the view is built up, without any magic.

The points you make on React make me really wonder if you did your research.

I don't see Angular great either, for what it's worth, so please don't think that I'm holding Angular up as a "better" solution. Both are heavy in the wrong ways. I got roped into using Angular because of its popularity, but I'm actively looking for a more solid and optimized replacement. React should be exciting to me as an option, but the more I read about it, the more wrong it feels. I just listened to a couple of the React creators on a podcast, for instance, and what they were saying about cross-platform development in the podcast is just demonstrably wrong. Source: I've been writing cross-platform apps for 20 years (mobile for the last 8 years or so).

As for Angular 2.0: I'm reserving judgment on that until it stabilizes, though it looks better.

I'm personally at least partly on the fence about whether the HTML template Really Should Be Separate, to be honest. My first exposure to Polymer was at 0.5, and the way that Polymer did things seemed profoundly wrong as well. I haven't tried again with Polymer 1.0, but I feel like using Polymer and React with all of their standard idioms will make an app less flexible rather than more. And React performance on mobile simply isn't good enough for complex apps [1]; for sufficiently simple apps, why should I even care what framework I use? Even back when Zuckerberg was claiming that HTML5 apps couldn't perform at native speed, Sencha showed them that, yes, they can, if they're done right. [2]

Where I still feel uncomfortable about spreading HTML into all the components, embedded in the JavaScript that instantiates them, is that it feels entirely too much like what people did with OOP when taking it to extremes. You end up with objects calling objects calling objects calling virtual functions calling whatever...and unless you've got a debugger open and you're stepping through the code as it runs, it's almost impossible to actually reason about the code, or even to understand how it all hooks together, unless you understand the entire execution path.

With HTML templates, you can look and see the overall app layout -- what goes where, how the hierarchy is shaped, what components are being instantiated, etc. -- without having to run the app and call up a DOM tree to see what really happened. When you embed the HTML in components, and those components rely on other components, you pretty much need to run the app and step through everything to see what gets instantiated where, and then you need to dig through the code to understand why.

To me it resonates as a "bad code smell." [3] I've been cranking out code for thirty years now, so I've got a certain amount of experience with code. And I'm still willing to admit that my gut reaction may be wrong, but I may have to write an app in that style to see if it confirms my bias, or if it actually feels OK in practice. It wouldn't be the first time I'd changed my opinion on a topic (see: OOP above, which I was one of the worst abusers of at one time).

I think that there is a place for isolated web components, but that the place is for things like the Google Map Component, or any component with nontrivial complexity like that, that really needs to be developed in isolation and/or is intended to be used in unrelated projects, and isn't expecting to be styled based on a global stylesheet. For self-contained complexity in components, the component idea is great, and for that why not use the web standard?

CSS being done in React-JavaScript for basically all components strikes me as a misfeature. First, it breaks the syntax for anyone who's done styling before (in a way that, for instance, SCSS doesn't). Second, it limits you in what styling tools you can use: There are tons of tools like Compass, Bourbon, and Susy that add useful styling functionality to your apps, and using React you've cut yourself off from such support (or waiting for React plug-ins to be created). Third, when styling is isolated to a particular group of files, you can drop in a different group of files and restyle your app.

P.S. I'm creating these various Walls Of Text in part because I'm trying to solidify my own thoughts on the matter. Feel free to ignore if what I've said doesn't hold any value for you, but writing it does hold value for me. Thanks for your comments.

[1] https://aerotwist.com/blog/react-plus-performance-equals-wha...

[2] https://www.sencha.com/blog/the-making-of-fastbook-an-html5-...

[3] https://en.wikipedia.org/wiki/Code_smell

In particular separation should be enforced among components which is what React encourages, not between facets of the same entity as we have practiced since the beginning. Coupling of styling and behavior with associated markup is pretty much inevitable for web applications and decoupling those just reduces the cohesion.

I have posted a more elaborate answer in my blog [1], but finally I do appreciate your suggestion to rationally vet new technologies before jumping onto the bandwagon.

[1] http://lorefnon.me/2015/11/22/an-answer-to-react-is-the-new-...

> React seems to be designed to ignore 40 years of accumulated software best practices.

Best practices in software is not set in stone. Sometimes you need to break from existing best practices of the past in order to arrive at truly powerful new approaches to solving problems that may not seem intuitive at first, but eventually manage to redefine best practices through their technical merits.

The Virtual DOM and re-rendering on every change is one such approach popularized by React, but it's definitely not the only one, nor is it even the most significant one, in my humble opinion.

Before React, mutable models was the only game in town. Angular imposed mutable models, Ember imposed mutable models, Backbone imposed mutable models, your average custom ad-hoc JQuery framework probably also imposed mutable models. Everyone blindly followed this "best practice" simply because this was the status quo ever since the earliest days of UI development.

React is the first JS UI framework that does not impose a mutable model. And as a result, the ClojureScript community was able to build on top of it and show the world that change detection can be implemented as a constant-time reference comparison when immutable models are used [1]. The immutable models approach was highly unintuitive (who'd have thought using immutable data structures in your model could make your app faster than using mutable data structures, which are intrinsically faster), but its results are clearly evident, and enables nothing short of a quantum leap in UI performance.

[1] http://swannodette.github.io/2013/12/17/the-future-of-javasc...

> Separation of concerns? Who needs that any more?

Your definition of separation of concerns seems to be the need to keep templates, styling and logic in separate files, which seems like a rather superficial distinction to me, to be honest. In any case, as mercurial already mentioned, JSX is completely optional, and React easily allows for you to keep your templates in a separate file from your logic and your styles.

Ironically enough considering your rather petty criticism, React and Flux-inspired architectures like Redux have in fact popularized a much more important separation of concern: the separation of application state vs operations on the application state.

This new separation of concern, along with Flux's insistence on a unidirectional data flow, has removed much incidental complexity from app state management, and enabled a whole new generation of developer tooling like component localized hot-reloading and time-traveling debugging [2].

[2] https://www.youtube.com/watch?v=xsSnOQynTHs

And this is the essence of why React has won over the likes of Angular, Ember, and Backbone. Developers have always been able to build great applications, even in the frameworks that come before React. But React and Flux-like architectures allows developers to manage complexity significantly better than in other frameworks that come before it.

This is why "it allows teams of 100 developers work together on an app". And as a front-end developer who has worked never worked in such 100 developer teams, and only on solo projects and in projects with small 2-10 dev teams, I can state with confidence that I can reap much of the same benefits of this reduced complexity. In fact, I probably benefit even more from this reduced complexity because as a solo/small-team developer, my complexity budget is absolutely tiny compared to what bigger teams can afford.

> Open standards? Nah, how about lock-in to custom language extensions that will prevent you from migrating your code to the next web standard!

The standardization process on the open web is painstakingly slow compared to the rate at which new technology is generally adopted and refined. This slow, methodical approach allows standardization committees plenty of time and input to think about every standard being proposed, but it's also one of the main reasons why very few libraries, even those with standardization as the end goal, begins as some kind of standard proposal.

It is much easier to gain traction as a standard if you already have an established implementation in a library/framework that is mature and well-adopted, and can demonstrate the merits of your approach. This is the approach taken by projects like TypeScript, and it's probably safe to assume that many aspects of React will be integrated into various standard tracks in the not too distant future.

A lot of your response seems to be emotionally laden. Bad form.

>Redux

I've been reading about Redux, and as soon as another app project comes along for me to try it out, I'm planning to give it a try.

As I've said more than once in this thread, to a large degree my own opinion is still forming on this topic. React does things that feel to me like "bad code smells," but it's possible that I need to adapt to the New Way of Thinking.

>And this is the essence of why React has won over the likes of Angular, Ember, and Backbone. Developers have always been able to build great applications, even in the frameworks that come before React. But React and Flux-like architectures allows developers to manage complexity significantly better than in other frameworks that come before it.

I've been writing software -- mostly games -- since I had to use assembly language for everything. Having a language (Java!) or framework (in this case React) explicitly protect me from myself almost always ends up slowing me down and slowing down the resulting app (yes, even React [1]).

Despite this I'm probably going to give React a real try at some point, even if it's in a toy project. I've been coding for 35 years, and it doesn't take me long to get the flavor of a new technology and its limitations when I finally stick my teeth in. It's all about finding the time...

[1] https://aerotwist.com/blog/react-plus-performance-equals-wha...

You're definitely right, I apologize for that. I cringed at parts of the post myself when I went back and read it again, but by then it was too late to edit.

> I've been writing software -- mostly games -- since I had to use assembly language for everything. Having a language (Java!) or framework (in this case React) explicitly protect me from myself almost always ends up slowing me down and slowing down the resulting app (yes, even React [1]).

Yes, micro-optimized, low-level code will always have an edge in terms of absolute raw performance, but there's a huge cognitive overhead involved with working with code like that, and you simply can't afford to do it on your entire codebase if you want to build new features and make changes quickly. What frameworks like React offers is a way to architect your application in such a way that makes it amenable to global optimizations that obsoletes entire classes of performance optimizations that you'd otherwise have to micro-optimize by hand on a case-by-case basis. This gives you more time to actually work on features and more time to thoroughly profile your code and micro-optimize the parts that actually lie on critical paths in your app.

Regarding your linked article:

If you take a look at the Vanilla benchmark, you can see the rendering time out-pacing JS compute time as we approach the 1200 mark, whereas for the React benchmark, the rendering time essentially stays constant. This is one example of a global optimization at work: the Virtual DOM spares us from having to micro-optimize DOM rendering for all of our components.

Regarding the JS performance scaling characteristics, I believe it probably has something to do with this:

> Did you set shouldComponentUpdate to false? Yeah that would definitely improve matters here, unless I need to update something like a “last updated” message on a per-photo basis. It seems to then become a case of planning your components for React, which is fair enough, but then I feel like that wouldn’t be any different if you were planning for Vanilla DOM updates, either. It’s the same thing: you always need to plan for your architecture.

This brings me to another example for a global optimization that React enables (this one doesn't come by default in React, but it's the first JS framework that made it possible): the use of immutable data in change detection. This allows you to implement shouldComponentUpdate as a single reference check for every component across your app. The change detection for the state array in the example would then become a constant time operation rather than the much more complex deep object comparison that React had to perform in the example, which is probably the root cause of the poor JS compute performance scaling as the number of photos increased. I strongly recommend taking a look at the first link in my original post if you're interested in more details.

I disagree. Code written to be optimized for a particular use case may itself be challenging to follow, but using it, if it's just a component and has a well documented API, doesn't have to be difficult at all. The Vanilla code in that benchmark article wasn't particularly hard to understand, for instance, and it could be wrapped in a Web Component to isolate the complexity.

Think about OpenGL/WebGL and the complexity and math and parallelization and optimization tricks that API conceals. At this point writing pixel shaders is almost easy, and yet they enable insane levels of parallel math with very little cognitive load.

I've written game SDKs that concealed a lot of complexity and yet were easy to use [1], so my gut reaction is to want to start with generic components that don't restrict what I can do, and then build up a DSL that is very easy to reason about that the app is assembled from. Based on my development history, I'm also likely to be making apps that are more than simple CRUD (with real-time interactive features, including games), so my personal requirements and point of view may be a bit different than the typical front-end developer.

>I strongly recommend taking a look at the first link in my original post if you're interested in more details.

OK, I'll take a look.

[1] Look at the listings for "Playground Game Engine"; the list is incomplete, but it gives you an idea: http://www.mobygames.com/developer/sheet/view/developerId,13...

I don't think we actually disagree. =)

By "working with code like that", I meant actually writing, understanding and changing micro-optimized, low-level code. Your example of building on top of a micro-optimized, lower-level SDK is the perfect example of a global optimization that alleviates some of the need for tedious, case-by-case micro-optimization from the code that uses it.

I'm just saying that micro-optimizing every single piece of code case-by-case is not the best use of our time, and that we should opt for global optimizations where ever possible, and that React and Flux-inspired architectures like Redux can enable some very practical global optimizations.

[2] http://lhorie.github.io/mithril/ (using its own virtual Dom); I think there are other implementations around, too.

[3] http://cycle.js.org/ (have a native driver)

FYI.

Virtual DOM as a panacea may be overrated, though. Maybe React's just isn't fast enough, but if you use another way to keep track of what parts of the DOM to change, simply pushing things into the DOM can be WAY faster than using a Virtual DOM. [1]

Cycle.JS still hits my "HTML in code BAD" reaction. An example from the front page:

      .map(name =>
        h('div', [
          h('label', 'Name:'),
          h('input.field', {attributes: {type: 'text'}}),
          h('h1', 'Hello ' + name)
        ])

A lot of popular frameworks are using this generated HTML approach, though (including apparently Mithril). I'm still trying to figure out if my negative reaction is justified.

[1] https://aerotwist.com/blog/react-plus-performance-equals-wha...

      .map(name =>
        div([
          label('Name:'),
          input('.field', {attributes: {type: 'text'}}),
          h1('Hello ' + name)
        ])
      )

That's what it is, just an automatic reaction without any real practical argument. Do yourself a favour and "unlearn" that. Open your mind. You're doing yourself a huge disservice by turning away from new ideas with these knee-jerk reactions. Working with views as functions of data gives you much more than it takes away.

On the other hand, every time I turn around I think of more things that it "takes away." This time it's an entire category of tool: The visual layout editor. I like it when my code can be data-driven, and data-driven using a standard format that can be saved using a (visual) tool.

I especially like it when my artists can tweak the page layout for me, but I don't like the idea of giving the artists files full of JavaScript or JSX to tweak.

Are there artist-friendly tools that can edit the layout of a React site/app, where the JSX can be modified in-place using the tool? It doesn't strike me as an impossible-to-solve problem, but it could be tricky to do well.

`h` is method that returns a JavaScript object; not HTML.

--

Just to clarify: React is not about killing the separation of concerns. React is about one thing: converting application state to a component tree and rendering that component tree to the DOM via highly optimised diffing.

At the base level, React turns state into component trees. Similarly, Redux is only about managing state and changes to that state. These are fairly straightforward ideas but having them as clearly defined building blocks with an extremely straightforward API (Redux moreso than React but in the trivial case a React component as of 0.14 can be a plain old pure function) radically simplifies application development.

I'll say it again: application development. I haven't said anything about HTML and CSS yet. This is as relevant to the web as it is to native application development and it's not an accident that React has been decoupled from ReactDOM with the latter now being merely the implementation detail of going the last mile and rendering things to the DOM (or to HTML strings) and maintaining DOM bindings and diffs.

--

A quick interlude: JSX is not HTML. It looks like XML and in many examples it has HTML tag names in it, but it's not HTML -- nor is it XML for that matter. It's just (entirely optional but very useful once you get over the initial visceral discomfort) syntactic sugar for defining component trees (not DOM subtrees). The syntax is obviously based on XML but it is a lot simpler and the familiarity obviously helps.

--

One thing most people don't understand about React (not least because React isn't vocal enough about it -- just like Flux wasn't vocal enough about certain concepts until Redux came along and showcased why they're important and useful) is that you can and should distinguish between presentational components and application logic.

If you use Redux (or Flux -- but with Flux there's the problem that most things called Flux aren't actually Flux because nobody really understood it) most of your components will be entirely presentational and the logic will live outside the components except for a few so-called "containers" which are just extremely thin wrappers.

So with React+Redux you then have your component structure living in React components and your logic living in a few functions that describe transitions of the immutable application state plus a few "containers" that just describe how the state is applied to the API of the few "smart" components that actually need interaction.

Sure, HTML strings intuitively feel more appropriate for describing these components and Web Components definitely look more "HTML-like" in that regard, but at the end of the day you're still writing something XML-like with made up elements (whether they're Web Components or React "components") that needs to be processed by JavaScript before being turned into the actual DOM.

--

One major change React brings to the table that isn't spoken nearly as much of as it should be is that React can be used to render applications to HTML, without the DOM, on the server. This usually gets mentioned in the context of "load times" or "SEO friendliness" but it's a pretty significant property of React.

Not only can React on the client "seamlessly" re-use the server-rendered DOM and attach itself like a jQuery plugin would but being able to render the application (and using Redux: render it with an arbitrary state) means you can truly embrace the idea of progressive enhancement without giving up the comfort of web application tooling.

Some years ago there was a lot of hype (well, not as much by a long stretch but some hype nevertheless) around the idea of making web apps work without JS (YUI[0] allowing server-side rendering of JS web apps and PJAX[1] enabling web sites to behave more like apps). Instead of mucking around with client-side templates you would render pages on the server and then intercept internal links to fetch and inject only the bits that changed between the current page and the next.

React+Redux is basically PJAX, but it's also the polar oppsite. Instead of rendering everything using server-side technologies, you render everything using client-side technologies. And instead of offloading re-renders to the server you keep them in the client when possible (where they can be further optimised thanks to DOM diffing).

Let me restate that: React+Redux allows you to build "Rich Internet Applications" in a way that makes it possible to support JavaScript-free fallbacks out of the box. There's nothing stopping developers from making the HTML output of their React apps richly semantic and accessible either (even though that represents the Eternal War of the web).

The idea is not new[2] but with React it's not only achievable but easily achievable. And unlike Web Components it only needs existing technologies that are widely supported (it even works in IE8).

--

The PandaStrike article you linked is not a rebuttal of React. It's a rebuttal of traditional client-side web applications. It doesn't matter whether they are built with Angular or Backbone or React -- the arguments are mostly universal. The difference is that React is the only option today that (despite not having HTML "templates") allows developers to do something smarter.

Of course React is a land grab. Every open source project is. But it's not an attack on the Open Web anymore than jQuery was an attack on the Open Web before querySelectorAll or XMLHttpRequest landed in a spec. That you're no longer writing straight-up HTML in a text file doesn't mean you're no better off than if you were using GWT[3].

And who says Web Components are the Right Choice just because they're becoming part of the native platform? Sometimes the thing we think we want[4] isn't what we really need[5], even if it's on track to become a standard.

--

[0] http://ajaxian.com/archives/server-side-rendering-with-yui-o...

[1] http://pjax.herokuapp.com/

[2] http://www.stevesouders.com/blog/2010/09/30/render-first-js-...

[3] http://www.gwtproject.org/gettingstarted.html

[4] http://readwrite.com/2014/07/24/object-observe-javascript-ap...

[5] http://www.infoq.com/news/2015/11/object-observe-withdrawn

Unfortunately technology is unlike some professions like, say, warehouse management or construction where techniques evolve with time, but there are more commonalities from one decade to the next. Technology sees highly disruptive change and so I would still argue that without keeping current in your knowledge, your experience has a very short half-life.

Incidentally, what you call "microservices on AWS" is architecturally identical to how mainframes worked in the 1970s, you would do it with CICS. Much of the computer science, algorithms and datastructures, in every day use, were discovered in the 70s if not before. So either you have very limited experience of technology, or this is just ageism.

As for me, I've run AS/400 shops (back when it was the AS400, but we did use newfangled 5250 emulators), been responsible for development in companies implementing highly distributed client/server based products, and most recently I've been involved in developing back end systems for web and iPhone applications (and, yes, on AWS in microservicey ways). I've been around technology plenty. I can tell you, too, that my early mistakes with AWS were precisely because I allowed my prior monolithic systems experience take too much precedence in my judgments when I first approached it. It was not the same. I needed to learn the new lessons; perhaps my only advantage was I recognized it quickly and recovered.

It's not ageism, however, but a realism that recognizes that not all experience is the same or applicable to every situation: it takes continuous professional development to make that experience worth a shit. However, I can understand why you'd be a little overly defensive in your response however... being a developer of a certain age. Don't worry. I'm about the same age as you, as are many of my colleagues, and I don't think we're too worried about finding work.

1) That's not a threat, if you think it is then call the Internet Police - they have a hashtag #hnwatch just for that

2) He is simply wrong that experience doesn't carry forwards. To use my Amazon example, the value in that business is understanding how people buy books and how a logistics operation to deliver them works. I don't know what technology Amazon actually uses in their warehouses but it's nothing that couldn't be done with terminals and line printers, I'll bet dollars to donuts, and nothing that would be unfamiliar to anyone working in the logistics field in the 70s.

I'd like to think that interviewing for positions that aren't entirely focused on new-hotness still leans on heavily trod areas of CS. Data structures, sorting and searching.

It always amuses me when I'm interviewing for some vanilla web app job and they start pitching maths questions. It's filtering on the wrong criteria.

If you don't, just find something else.

For the second advantage it depends where you want to do. If you want to stay a dev and only code then the best thing to do is to master your stack and learn complex things. By doing that you differentiate yourself and increase the barrier of entry. So for instance to learn C++, AWS, Machine Learning, Hadoop ecosystem, webgl... Depending your field and what can be the next big hard thing with it.

Else, the other way is to go up in the hierarchy to be a team leader and maybe to go higher. At first you have been (or should) start to learn how to manage people. You should know how to lead your team when things are doing well but also when things are at the worst. Sometimes you have deadlines that are hard to meet. Sometimes the stress is at the maximum, financially the company is not doing well, or goals and ideas diverge... It's all the experience you need to get to be able to manage these situations and start to be a great leader.

Also you -may- start to learn to master git to be a team leader, to be able to fix the problems, and manage the repo for the code reviews. Another point should be to increase your communication skill, every day by talking with your team, by enjoying to write well written emails and speeches. These books [1] are a good start. And also a last point that I think is important is to start to get financial skills with things like financial management, accounting, and if possible to be able to do simple DCFs...

[1] http://www.amazon.com/Elements-Style-4th-William-Strunk/dp/0...

http://www.amazon.com/Writing-Well-30th-Anniversary-Nonficti...

http://www.amazon.com/Pitch-Anything-Innovative-Presenting-P...

The true worst nightmare is, "it works, but it shouldn't", because that means your whole model of the domain was wrong, rather than some isolated, fixable component.

2. That doesn't happen on my machine.

3. That shouldn't happen.

4. What does that happen?

5. Oh, I see.

6. How did that ever work?

Stage six is the worst. Especially when I wrote the original code.

(http://plasmasturm.org/log/6debug/)

It comes with a nice follow up after the first 6 rules:

< MatthewWilkes> 7. svn blame

< miniwark> 8. one day we will write tests

Am I the only one who runs git blame (for the commit, not the author) 20 minutes into investigating the bug and git bisect an hour in? They are excellent tools to find when and how did the bug first occur

It turns out I was accidentally reimplementing a library function, and I had chosen the exact same names and method signatures for the (admittedly small and fairly obvious) class.

Everything is going well, update and configure bios, install Windows, install drivers and software. Then start configuring the server, sever needs to reboot, reboots to a blue screen of death. Can't get it to boot up normally. Ok must be bad software, driver, etc... Time for a clean install, everything is going fine and then reboot -> blue screen. Look up the bug check code, no help. Another clean install, this time no extra software, no drivers, same problem after a reboot. Finally figure out that it is only happening after I make the computer a domain controller. After Googling with this new information, I find one thread on some forum where people are having the same problem, turns out it was the solid state drive, if you use a Samsung 850 Evo SSD with Windows Server 2012, it will blue screen if you make it a domain controller. I never thought a problem like this was possible. Sure enough changed the installation drive and no more problems. Nearly drove me crazy, took me two days of troubleshooting.

At least with those (and to some degree optical media, but the write process there is more laborious) the storage and the RW logic is different pieces.

The closest I've come to that is tinkering around with random old hardware: a specific ISA (PnP) sound card and a specific 4GB IDE HDD I have here hate each other and if both are in the same system, when Linux sends the 'IDENTIFY' IDE command, it times out and never gets a response. This "works" across two different motherboards.

Thankfully I was just messing around, I knew the system I was using had used up 7 or 8 of its 9 lives and I was being careful, and I only changed one or two pieces of system state at a time so I could easily rollback (as nonsensical as "okay let's try removing the soundcard" is :P).

If anyone wants to know, model info shouldn't be too hard to locate, and pix would be fun to grab, I'd just have to go find all the parts.

Took me the whole day to figure out that the particular kernel version I updated to made my particular brand of router drop the connection to the ISP after ~15min. Had to downgrade the kernel to make it work again...

Fun times :)

When you're a sysadmin who's only dealt with one type of linux for a long time you forget what the different distro's do.. there really isn't that much, naming conventions/subtle commands etc;

But, apt's packages have the nature of 'always run the latest service if upgraded', a notion that does not exist in centos/rhel/fedora/bsd etc;

So, apt-get upgrade on our production database server means the server goes away for a while; not 'the packages are available to be used when you restart the service at midnight'*

:p oops.

I was working on a dev server (which was pretty identical to the prod servers), vmhost/kvm/ubuntu 10.04 LTS.

One day I'm staging my updates, to make sure they work when I go to prod.. I do my apt-get and reboot into the new kernel.. but- nothing can be mounted. No developers can work either, they depend on that machine..

I'm looking at it and I can mount the drives from busybox after the kernel panics.. No idea what's going on.

I'm asking around in #ubuntu on freenode, since googling is getting me nowhere. 'Just reinstall it' they say.

'Sometimes it's better not to know why these things happen'

:|

I found the issue, some person was smart enough to think lvm2 was called 'lvm' and then initramfs all my previous kernels during the upgrade procedure- so I couldn't boot into old kernels either.

Lost a day of dev time (although it was only 3 developers and me)

Problem was with the realtek soundchip on my card. Headphones stops working on windows (even though it detects it), but still works on ubuntu. Try all usual stuff (remove, update, reinstall drivers, etc), nothing works. One fine day, I find it starts working again. Then another day it stops working. Realise that sound stops working after i switch from Ubuntu to Windows. Weird. How does ubuntu have an effect on sound in windows? Google issue and find that its some strange ubuntu driver issue that puts audio driver in a weird state. Workaround is to shutdown laptop , remove battery, and hold down power button for 5 seconds to discharge capacitors and then boot into windows.

We had an HP server that was _randomly_ losing network connection with the virtual machines hosted on it!

Couldn't ping them, nothing, but you could log on through the hypervisor. If you rebooted the VM it'd be fine for a week to two weeks.

This server was pretty new from HP.

After some research, calling HP and MS, it was determined that it was a setting on the NIC that was causing the problem!

There's a KB on it now on MS's site.

This particular build was setup to be primarily a Hyper-V host, for the VM drive I chose an Intel 750 PCIe SSD. The drive is amazing, super fast, makes traditional SSDs look like spinning disk. Choose the EVO to save a few bucks for the customer, as the server OS wasn't going to doing any heavy lifting. I never imagined that a hard drive could be incompatible with an OS.

* Security : customers acccounts leaked, identities stolen, bank account depleted

* Undetected data corruption : customers' data is corrupted and it has seeped into the backup stream, even the off-site one, and that is corrupted as well now.

* Rarely occurring simingly random concurrency related bugs that manifest themselves once a month or longer.

* Time : making assumptions about it, dealing with it distributed systems etc. Nobody thinks about it until it stops working or systems' time gets snapped back somehow.

After writing a test script and letting it sit in a loop the bug became reproducible: 1152 times got a replication.

https://github.com/spf13/hugo/issues/1293

Instead of real observation, there's a lot of leaning on type theory, OOP, immutability, and other ideas that scratch some sort of neatness itch and/or advance the careers of opinionated language creators, but have little/no empirical support to prove they actually produce more reliable code.

Meanwhile formal techniques and languages that do have empirical support are sidelined and ignored.

Bugs like yours are not uncommon. If anything, because of Darwinian debugging - the obvious ones die first - you'd expect a good proportion of hard-to-reproduce bugs in any code base.

So they shouldn't be considered weird aberrations. I'm not sure why CS theory doesn't have a better handle on them.

Every anti-bug measure tends to work like yet another brush aimed at a wall. But most brushes overlap partially or even wholly with areas of the wall already painted with previous brushes. The only strategy that would really work would cover all of the wall in a very systematic fashion.

I once wrote an industrial control program, and used a time function that rolled over at midnight. To make matters worse, we never bothered to set the clocks on the computers, so it seemed like a completely random glitch, and was a devil to debug.

If so crash the program with a kill -9 set up to write the core to disk, then run gdb on the executable and reload that core file. Then run 'where'. This will tell you where you're hanging and how you got there as well as all the parameters to functions on the way down.

A ps -aux beforehand can give you another hint, look at the status letter.

Good luck!

The database person spent a month trying to figure out why there were entries with odd dates. Like one or two a month out of a few hundred thousand. Eventually figured out it wasn't his fault. His end was getting an occasional packet with a good crc and odd data. Most of the time the parser would kick those out. But a few were well formed enough to go through.

I chased it down eventually to a documentation error in the datasheet for the radio transceiver we were using. The datasheet very clearly stated that when a packet came in with a bad crc... nothing would happen. No end of packet interrupt. So I had to use a timer to detect a bad crc. (Okay whatever, I mean I hate those guys, but I've seen worse)

Turns out when the radio received a packet with a wrong/bad crc the radio silently reset the receiver and started waiting for a new packet. If a the radio received a packet before the timeout hammer came down then the driver would think the previously received packet was 'good'

Opps.

Was interesting because the symptom appeared at the farthest end of the system the tool graphing data for the customer and was caused by something at the other end. The error could have been literally anywhere in our system.

* gcc dies with internal compiler error when you try to compile with debug symbols.

By this time I've killed gcc, javac, ghc, and the .Net compiler. Worse, the ghc bug was the only one I could track.

I just read that. This guy certainly wasn't Ken Thompson. But this happened to me in 1991, seven years after the date of publication you posted, and the "login" portion did the exact same thing Ken was describing as far as the secret password. This was probably Thompson's genius idea implemented by the grad student. The mechanism was very different though in the compiler, but the outcome was still a poisoned compiler.

I had heard of self-modifying compiler before, too or I probably wouldn't have thought to look there myself. I'm not surprised to see Thompson behind the original idea.

As an ex-PHB rather than a programmer, I think perhaps that Dr Phelps could have had taken a bit more interest in the project &c. Perhaps had a look at the code now and again as simply asked "why does this look so unlike other people's code?". Might have sent a shot across the bows of the bad apple.

The thing that makes me tend to agree with you is that he hasn't mentioned any "finally, years later, I found out where that guy got his idea" moment in it.

"I suddenly realize it's in the compiler. It was the compiler. And every time you compile the original code and run it puts in the subliminal message code into the source code. I'd heard of this before."

Seeing the CPU do something it shouldn't, being traced by a hardware ICE, now that is a worst nightmare - and one I've actually experienced.

Anyway, after getting some really terrible error messages for 2 very long days, and digging way to deep I finally say: "This can't happen, the RAM on on this machine is clearly crap."

Swap out the ram and everything is fine after that. !@#$

PS: Not that I had a great understanding of OS 8 coding. I remember printing a help page for some function which said, "This is the documentation for _. Followed by a copyright notice." and that's it.

I know that pain. I was working on the NDIS driver (WinNT 3.5.1, later a 4.0 beta) for our HIPPI[1] PCI cards. The hardware was based on our very-reliable SBus cards, so when the PCI device started crashing, we assumed it must be a software error.

I probably spent 2+ months trying to find the cause of the crash. Trying to decide if your change had any affect at all when you have to wait anywhere from 5 minutes to >10 hours for the crash to happen will drive you insane. You have to fight the urge to "read the tea leaves"; you will see what you want to see if you aren't careful.

While, I never did find the problem, I did discover that MSVC was dropping "while (1) {...}" loops when they were in a macro, but compiled correctly when they macros were changed to "for (;;) {...}".

Later, a hardware engineer took the time to try to randomly capture the entire PCI interface in the logic analyzer, hoping to randomly capture what happened before the crash. After another month+ of testing, it worked. He discovered that the PCI interface chip we were using (AMCC) had a bug. If the PCI bus master deasserted the GNT# pin in exactly the same clock cycle that the card asserted the REQ# pin, the chip wouldn't notice that it lost the bus. The card would continue to drive the bus instead of switching to tri-state, and everything crashed.

Every read or write to the card was rolling 33MHz dice. Collisions were unlikely, but with enough tries the crash was inevitable.

[1] https://en.wikipedia.org/wiki/HIPPI

We found the bug, but the months of delay (and a few other problems like losing a big contract[1]) killed the startup a few months later. While I'm annoyed the SCSI3-over-{HIPPI,ATM,FDDI} switch I got to work on was never finished, the next job doing embedded Z80 stuff was a lot more fun... and a LOT easier to debug.

Incidentally, I found a picture[2] of the NIC. Note that HIPPI is simplex - you needed two cards and two 50-pin cables. This made the drivers extra "fun".

[1] "no, I'm not going to smuggle a bag of EEPROMs with the latest firmware through Malaysian customs in my carry-on" (still hadn't found the bug at the time)

[2] https://hsi.web.cern.ch/HSI/hippi/procintf/pcihippi/pcihipso...

Hehe :) That's a really neat one: if you've fixed your machine you can read it, otherwise you can't...

Awesome sleuthing, and the take home is if you get down deep enough digital is analog again.

[0] http://article.gmane.org/gmane.os.dragonfly-bsd.kernel/14518

[1] http://thread.gmane.org/gmane.os.dragonfly-bsd.kernel/14471

My money was on the psychologist the whole time. I still kind of think it was Dr. Phelps and maybe Mark the admin and the AT&T tech are grad students in disguise. So I guess my worst nightmare is finding myself in a similar situation and later finding out my boss set the whole thing up for funzies.

I have a somewhat related story. Circa 1995 or so, I was working on software compiled on a 486DX computer. Back then the computers had L2 cache in SDRAM chips you plugged into the motherboard.

I was moonlighting with a few guys to develop software and each night myself and two friends met up in office we were renting. There, on one of those fold out tables, we toiled away each night writing code.

The code we worked on was for off shore gambling (aka a bookie). Back in the 90s you could walk into a convenient store and pick up an magazine; usually autotrader magazine. On the back was an 800 number you could call to place bets on football, hokey, or baseball. (Called HOBs bets in the industry).

Anyway, because of the nature of the software and the people we dealt with, things were a bit hairy at times. The bookie we worked for paid us generously but, at the same time, expected perfection (i.e. software that just worked).

One evening, as usual, the three of us met to work toward the next release of the software. The next release of the software was to include a feature for boxing bets. Mike Tyson was about to be release from jail and the bookie was anticipating various future business in taking boxing bets.

So this one evening we were doing a software build to release as beta software. I did the software build and ran a battery of tests we normally run and all worked fine. To transfer the software to the offshore network we used a dial up connection.

I pushed the software out which my bookie contact would run and make sure all the logic was correct.

A few days go by and we get a phone call that the software is not functioning. Sometimes it crashes (which was rare) or sometimes just strange things would happen such as strange blinking characters on the screen.

I collected enough information to try and replicate the steps necessary. You can imagine my customer wasn't too happy. So the three of us basically stopped working on the code to track down the bug. We spent a week looking for the bug we were sure was in the code somewhere. Back then we didn't really use a code repository but we did have a diff of the beta release and previous release.

Pouring over the code changes we just could not find the problem. It even got to the point where I would recompile the beta version, run it, and could actually get the code to crash. Comparing to earlier release it would not crash.

It was by chance that my fellow coder, sitting right next to me, started running similar tests on his machine while I was off making a food run. It was sort of normal for us to group up around my workstation when something needed a collective look.

Long story short it turned out that one of the L2 cache chips was causing my compiles to become corrupt in just the right way to cause the code to be not as we typed it in.

Anyway, thought I'd share. Although not a malicious act, it was one of the worst things to track down. Fast forward to my current life... I work in the embedded field and recently solved a bit-flip problem in one of the products my company produces that relies on NAND memory. That little incident from my past certainly has served me.... ;)

We used something called a Pharlap DOS extender so our software could use beyond the 1MB boundary. It was in fiddling with that memory extender that I began to suspect a memory failure. Changing it's parameters eventually got me to fairly repeatable way to get the issue to show up not only in my app but in 3rd party software began failing too. Also, while we mostly did our work from a DOS prompt and a DOS editor (called qc) we would sometimes run Windows (Windows 3.11) which had it's own way of accessing Expanded vs. Extended memory through a DOS driver. So by swapping between the Pharlap driver and the Microsoft config.sys driver did I begin to suspect memory failure. In other words, the machine became more noticeably more unstable.

I don't recall the exact name but I began running something like memcheck on the computers. Basically a memory checker and although it did not reveal the problem in its entirety, the memchecker would crash on my machine and not any of the others. These were computers running just DOS with Novell Netware network drivers for DOS/Netwware only.

I reasoned that memory was failing before the memory walker/checker was getting corrupt. When I swapped main memory with another computer and the problem didn't follow the memory, my only other option was to pull out the L2 cache chips. When I put them in the computer next to mine, and finally saw the problem show up there I knew.

My friend owned the computer company that sold us the computers and he later was able to test and validate my findings.

See: http://dinaburg.org/bitsquatting.html

The year this happened probably disproves this theory as it sounds extremely intricate to prepare. Nowadays though I guess you could pull it off quite easily by making the "ex grad's machine" a VM...

This is how I explain why my technical job is difficult in a typical enterprise organization and why you need to be far better than average to produce merely mediocre results.

* https://news.ycombinator.com/item?id=10607029

This was a pure Java shop and I was almost the only engineer who even knew C++ since the reorganization that happened before I joined.

Turns out the affected installations all used custom compiled DLLs with hard coded specific crypto keys made up by someone long ago. We didn't really know how to deploy new versions of this.

But I found some version of the code at least, on some old file server, so I could sort of debug it. It took me a while to figure out but after working on it on and off for a few weeks I found a crucial race condition in the code path that looked up the crypto key.

Turns out if you were unlucky after a SQL Server reboot, a few entries could get encrypted with a "default" key. So I was able to decrypt the garbled entries on affected customers at least.

We decided to change the whole method of encryption since it was obviously hacked together by some cowboy coder a decade ago.

But in the meantime, the easiest way to fix the issue for the affected customers was a method I taught to one of the tech support guys.

I told him to RDP into the affected server, open the crypto DLL in Notepad++ and do a string substitution on a particular ASCII sequence that would change a certain character into a tilde. And then reload the DLL with a SQL Server command.

Because through disassembling the binary I found that this would sufficiently increase a certain value and fix the bogus comparison that caused the race condition.

The nightmare here was mostly because of custom DLLs running semi-unknown code on remote servers.

Once the performance was similar to competing OSes, the measurements from my performance experiments were varying by 30%! This was surprising because the benchmark was simple and the performance of competing OSes was completely stable.

Well after three or four days of frustration, I discovered that the problem was our test hardware: for whatever reason, immediately after BIOS POST, any memory loads executed for the next 10 seconds were 30% slower than memory loads executed almost exactly 10 seconds after BIOS POST. Initially I was shocked, but this may make sense -- it seems plausible that hardware manufacturers would run initialization code in the background resulting in quicker boot times.

Anyway, after we figured that out we simply waited 10 seconds after POST before we ran any experiments. I'm curious how common this slow-for-the-first-10-seconds thing is with other x86_64 machines.

Edited for clarity.

1. Uncommented code, but especially uncommented regular expressions 2. Unnecessary abstraction that runs too deep & so is hard to comprehend 3. Also, take any tractable problem and add timezones to it. Now you have an intractable problem!

Build .NET, prove the issue still exists with the local build, then add new instrumentation to the GC routines inside .NET's runtime.

Instrumentation never killed anybody... and if this issue persists too much longer...

I can't deny that it's a crazy idea though.

Never figured out what problems that particular IE had, but the workaround was simple... I added an extra hidden input field just to check if all was sent. The theory was that at least operator could repost the form if it went wrong - however, it never did again. Thank you IE for your obscure bugs and nonstandard behaviour! Rip. </rant>

I mean, who want to think that the main thing that would let you create code is the thing that is infected? And it seems like there weren't any other options for languages (which hopefully weren't compromised as well) so there would be no really getting around it.

I'd wager that people who work with esoteric embedded toolchains develop an instinctive distrust in that layer.

Or just if your project developed a show stopping bug at that point in general, since you'd get the feeling that 'oh god, we've been done by two days' or 'we've been serving malware for two days'.

Or maybe if you'd found a breaking bug that needed fixed, yet no answers were available online, no one in the room could help and the site was due to go live that night. All that time worrying, refreshing your Stackoverflow question and trying in vain to find a solution would be torture.

It's not the code that causes the nightmare, it's the result of the code. Say, a live production system, where something went wrong due to your coding, causing a great irreversible loss. You realize it the moment it happened, and you can't do anything about it.

Closest I got: data inconsistencies in a +5-year old, live core production database of a large company, caused by a coding error. You don't know where it is wrong, you don't have means to fix it (it was wrong from the start), you do know how much was lost because of it. Only 'good' thing was that it wasn't my fault.

LOL

https://pbs.twimg.com/media/Bp_KeZnCAAAijRe.jpg

A real nightmare for me was when I inherit a bunch of code full of uncommented regular expressions

Unless you're suggesting the entire system be under VCS?

Ah, well. The legends of self-modifying compilers are old at this point: http://www.ece.cmu.edu/~ganger/712.fall02/papers/p761-thomps...

[1] Reflections on Trusting Trust, Ken Thompson, Aug 1984. https://www.ece.cmu.edu/~ganger/712.fall02/papers/p761-thomp...

I'm scared of having developers with a little ops knowledge take over my job.

/devops/

Developers spend a lot more time thinking about abstractions, where to place things and creating & controlling nThings. If you cannot do these things well, you need to learn them.

The best way to do that will be to ride the bandwagon, work together with devs and learn from each other.

When I went to university in '94, if we'd had to deal with the same thing I'd have been in largely the same situation: Sun and SGI workstations that largely booted via NFS (some of the SGI's were exceptions, I think, but most parts of the system were mounted via NFS there too) where anyone trying to re-install the compiler etc. without the approval of the computer department would be in big trouble (and have to hack the system; though the protections were relatively weak).

And while we certainly could install compilers etc. locally in our accounts, getting hold of a cleanly compiled compiler for some of the systems we had would not have been all that trivial, even though the systems we had were all more mainstream than the 3B2 mentioned in the article.

I wouldn't be surprised about an outside contractor having a hard time getting the computing department to believe that this problem was a hacked system. I'd expect the reaction would be to assume that he was just trying to make excuses for why it was taking him so long to find the actual problem.

At the time of this story, based on the details (3B2 computers and Tymnet) RCS was likely still current and CVS likely didn't exist. There were other alternatives like SCCS, but speaking as someone who wrote code back then: It wasn't until the late 90's it started being weird to not see source control systems used even in a corporate setting. The last time I worked in settings were source control was not universally applied was as late as '99.