DevOps is a horribly misapplied term. I recently referred to a client systems administrator as a "sysadmin", because they were doing sysadmin things: provisioning and configuring production hardware, networking, backups, databases, etc. I was chastised by the [much-younger-than-I] CTO, who said he "hadn't heard the term sysadmin in 20 years" and that they were the DevOps team.
No, sorry. They were sysadmins. DevOps is valuable role, and I'm glad there's a focus on supporting developers and their needs, and in empowering developers to manage their own build and deployment infrastructure. In a larger organization, having a dedicated person or team working closely with developers on software build, test, and deployment automation is great.
But if your job is maintaining production hardware, network, operating system, DNS, SSL certificates, firewalls, backups, etc., then you're a sysadmin. It's not a pejorative term. In a small organization, or within a team, that may be combined with a DevOps role. You may even have one person who maintains all your infrastructure for production and development, and even doubles as a developer. That still doesn't mean you label the entire role DevOps.
In a way it’s like how people call themselves “software engineers” instead of “programmer” even if that might be more fitting. You sound more competent when you say devops so you might be able to get a larger salary than a sysadmin.
Writing code (aka programming) is only a part of being a software engineer. CI/CD, tests, deployment concerns, etc. are what make a software engineer a software engineer.
Ehh, a software engineer is a programmer. The name evolved, but I don't think there is any organization that has software engineering and programming roles separated.
I know software engineers that hardly write any code at all. They spend most of their time working on specs and design.
No separate job title, though. But similar things are happening in e.g. architecture and fields like mechanical and civil engineering. Used to be that you would have separate draftsman but nowadays you could say “an architect is a draftsman” and it would be about as accurate as saying “a software engineer is a programmer”.
A programmer as a job doesn’t exists because programming is in great majority of cases the easiest part of software development and/or engineering. Unsurprisingly it’s the people who are the most challenging at this job and the term ‘programmer’ is completely orthogonal to dealing with people.
I'm working with a big corporate client right now and not only is their DevOps team just sysadmin, but build automation is also a separate team.
For my money, the crux of DevOps isn't specifically about who does what task with code, it's about vertical alignment. The notion that your build pipeline and infrastructure are as much a part of product development as the coding, design and everything else and everyone up and down the stack is aligned to the same goals.
Most sysadmins have pretty much always coded to some extent.
That was one reason that Perl became so popular among sysadmin back in the early 90's: it was a really useful language in which you could do much the same as you could with (and had a syntax which was similar to) shell scripting, sed, and awk, which were tools that were popular among sysadmins up 'till then.
> The job description of a sysadmin always required knowing at least one interpreted and one compiled programming language.
You'd be surprised. A large chunk (say, more than 50%) of the sysadmin job market are guys that can set up a Windows LDAP and/or some Cisco and Wi-Fi equipment, but no more.
It's why the 'DevOps' moniker was invented, to differentiate the old-school Unix sysadmins from the LDAP-and-Exchange guys.
It's my understanding good sysadmins have always been able to script.
It's bridging the gap between software developers and sysadmins where devops falls into place. Taking applications and optimizing their environments, rollouts, versioning, reliability of deployments, reduction of downtime, automating server provisioning, etc.
IMO the best devops are software devs with some sysadmin experience.
Okay, this may what it has become but I've done both jobs at the same time.
To me DevOps: One must understand the build system, stack, and the minutiae related to maintaining that build stream. It's a lot of work and at least in the past was more of a developer responsibility and not a "System Admin" type responsibility.
System Administration: Security, User Accounts, Policy, Backups, Network Configuration, Automation, Reliability.
Sure there is some overlap but these used to be completely separate functions and with good reason there was a lot to understand about each workflow.
Modern workflows (K8s, AWS, Docker, VMs, Jenkins, CI, TDD) have mostly take the "System" out of system administration while improving reliability and speeding development while (I think) suffering security. But that's another post or maybe I will write an article about it.
One type writes a script to back up systems. Depending on their level of expertise they may also include error checking and tie it into the monitoring/alerting system so we know when the backups don't run correctly.
Another type buys backup software, and configures it to back up their systems.
Each type has pros and cons, and each may feel superior to the other. However the ones who haven't coded may find themselves ill-suited to moving to a job where they are expected to be more of the first type, the coding sysadmin.
When interviewing prospective sysadmins, I've tried to tease out whether they are a "maker" or "doer" type of person. The makers tend to be more scarce, and the doers may be ignorant of how much they could be learning and experiencing if they widened their horizons a little.
It may sound like I'm very dismissive of the doers. I'm not, a good team is going to have a mix and have different levels of expertise too.
I worked at 2 companies since graduating. The first company had a DevOps guy that was strictly not programming (setting up AWS things, CI, etc). It was clear that programming wasn't his skill. Although he was able to learn it, he wasn't hired to write code. The second company I worked at, DevOps engineers were software engineers/computer science majors. They could immediately switch teams to a write-code-only team and be qualified.
Technically, yes. But because the word had been misused for so long, it has taken on the meaning of a developer/sysadmin combination role.
The word “irregardless” also got misused long enough that it became acceptable.
irregardless
irregardless /ˌɪrɪˈɡɑːdləs /
▸ adverb non-standard regardless:
the photographer always says, irregardless of how his subjects are feeling, ‘Smile!’
irregardless, the song is a fine piece of work.
– ORIGIN mid 19th century : probably a blend of irrespective and regardless.
Irregardless means the same as regardless, but the negative prefix ir- merely duplicates the suffix -less, and is unnecessary. The word dates back to the 19th century, but is regarded as incorrect in standard English.
Your perception of a typical sysadmin and their sw-dev skills is widely wrong in that case.
Most sysadmins focus on keeping things running. They are usually taxed to the point that they don't have the luxury of sitting around and writing code to be more efficient.
From a career standpoint sysadmin is a pejorative term in 2019.
I'm the 'not feature code' director at a company that manufactures companies. Meaning 20% of my time is what you describe above.
This is along side security work, occasionally feature code for one aspect of our moonshot that I have more experience than everyone else on, architecture work for all the properties to make sure they can talk to each other (while still maintaining enough separation that we could sell any of them at any time), building out a small manufacturing facility, and managing their transition to docker (which they only pursued because I pushed it). I also manage 2 other people (one of whom is a remote contractor) who do bits and pieces of the same and yes, a little endpoint laptop crap.
If they had billed the position as a sysadmin I wouldn't have taken it. Sysadmin pigeonholes you into a very specific role in many peoples eyes, whether it's true or not. Labels matter.
The role of SRE was derived at Google. It has an obvious overlap with DevOps, especially where it concerns automation and scaling to reduce waste and eliminate toil. Regardless of the differences or similarities, in the eyes of the c-suite, those insisting upon using 'Sysadmin' as a role, displays intransigence; which will only embolden recruiters to either low ball potentially more experienced candidates or sideline them in favour of those who are more willing to tailor their attitude to fit the new paradigm.
They might prefer that, but it's misleading. "SRE" is closely associated with Google's approach, where SREs are supposed to spend the majority of their time writing high quality code. If you aren't doing that, you shouldn't call yourself an SRE.
I'm told that internally, SRE at google is divided into SRE-SW and SRE-OPS. At least, thats what the recruiter told me when setting up the interview panel.
Is there a good term for SRE-as-the-Google-inspired-philosophy to distinguish it from SRE-as-the-job-title? I'm trying to find a word that conveys e.g. "We ought to be measuring SLOs and defining error budgets" and not e.g. "We need to make sure the people we wake up at night are sysadmins who can code well and not just coders".
(In fairness, I think Google brought this on themselves by reusing SRE as both the job title and the philosophy.)
Another aspect of the whole devops thing is the idea that you don't silo your ops from dev. Calling someone 'just a sysadmin' because they do all the sysadmin work that needs to get done gives off the impression that they shouldn't attempt any devops or dev work.
Its not enough to just rename your IT guys devops. However, When you're a stickler about titles its also counterproductive to the goal of removing the wall between dev and ops.
I automate and work with Linux systems and I still consider myself a sysadmin -- I'm not DevOps, I'm not an SRE. My skillset and experience might coincide with what those roles require, but I am and always will be a sysadmin in my mind. Putting lipstick on a pig doesn't make it a supermodel.
Do you search for DevOps or sysadmin in job postings?
Whether you want it or not, sysadmin is left to designate the low end of the market, namely desktops and windows administration work, sometimes all the way to helpesk. Devops simply evolved as a separate term to designate the higher end of the market around linux and automation.
Maybe in the silicon valley every company is a tech company looking for linux sysadmins and developers, but in the rest of the world "sysadmin" will usually get one into windows or low end roles.
I search for neither in job postings. Whatever the market wants to call it is fine, but that doesn't change the fact that at the end of the day we're just sysadmins with bigger hats and different titles. Even in San Diego, I see job postings for Systems Administrators and they are in fact just that, a sysadmin (sorry, "DevOps Engineer") and are not just for Windows or helpdesk work. The world outside of San Francisco is pretty normal, generally. Even my previous role at Sony was for a Linux Systems Administrator, but I still automated and did way more involved work than someone just doing printers and helpdesk.
You also can't just rename your Dev team DevOps...
Ops people come from a background favoring stability, security, and connecting components.
Developers come from using new and great tools... Writing the connections between their own systems, and following API docs of their partners.
Ops people usually have less CS training, and developers frequently don't understand security aspects of running systems or containers. You should merge the teams, not drop additional responsibility's on one side or the other that are outside their training/experience
Please be careful with the prejudices here around the CS knowledge of operations professionals. As a computer scientist who builds software for operations automation, I often find that developers work in interpreted or virtual execution environments and have little understanding of the underlying execution infrastructure or even the compiled environments that execute their code. The ops team understands throughput,
I agree that cross-training and shared responsibility for both success and support are critical for success -- regardless of labels put on people and teams.
If you can rename "statistics" to "machine learning" and "data analyst" to "data scientist" [1], I don't see why you can't rename "IT Ops" to "Dev Ops".
[1] If you haven't heard it, there's a joke:
Q: What's the difference between a data analyst and a data scientist?
The distinguishing factor between those two roles that I have noticed is data scientists build predictive models and use statistics somehow, while analysts are relegated to the 'grunt' work like cleaning data, building reports, etc. Analysts are typically working to become data scientists. I only have 5 years of work experience at 2 companies, so I'm sure results vary. I make sure not to make any assumptions when I hear either title, however, I've met incompetent 'data scientists' and brilliant 'analysts'.
I'm glad I'm not the only one who thinks like this. Every time I read someone referring to, "a DevOps," I wonder what they think they're referring to. No one person is going to be everything you need for effective development and operations combined outside of small, extremely focused scope. DevOps is about avoiding the, "Just throw it over the wall," mentality, not making one person handle the whole thing.
>DevOps is about avoiding the, "Just throw it over the wall," mentality, not making one person handle the whole thing.
this are why it's now part of my interview process when I'm interviewing for a new role "how does your company/team define DevOps?" and I listen very closely for things that sound like 'over the wall-Ops'. Admittedly having the fortune at this point in life to be quite judicious about the answers given.
Ostensibly, if I'm applying for a 'DevOps' role, I shouldn't have to do this if the job description is written well enough, yet even then this has become a learned behavior even when coming across the rare well-articulated job req.
While I largely agree with that, I think it’s arguable that there are a set of technologies that enable that style of working, and that you can call an engineer that specializing in supporting those tools a devops engineer, and a team that supports them a devops team.
Having devops teams and engineers isn’t the same as practicing devops, but once you scale up your org, you’re going to end up with them, anyway.
For example, someone who is good with ci/cd pipelines, infra as code and just generally in automation and observability, as opposed to someone who is strong with linux internals or JavaScript
You can tear down silos all you want, but everybody can’t know everything and people will always specialize.
There's a big difference between people specializing and teams specializing. In every organization I've seen, when you take all of the individual specialists with the same skillset, put them on a team consisting of them (and only them), it ends up being an organizational disaster.
The organizational question is whether the ops folks are (physically/organizationally) colocated with developers on dev teams, or colocated with themselves on an overarching ops team.
I had this happen to me from the other end. I was the lone programmer (and my title reflected that). After a new hire, I was told that I was some kind of engineer because my title had changed to that and that we are now DevOps, hooray! Not coincidentally, this was to be my final awful Agile experience.
The thing is, I am not gifted in the area of system administration. I am not too shabby at it in Windows, quite pathetic in Linux, but being a sysadmin was a role I had escaped for a reason. And suddenly I am back in it. I am a very low-tooling programmer. Just let me stick my hands in the data and go away. I have great respect for people who are system administrators but it is a job where my talents lie below my meager skills, and my inclination far below that.
Meanwhile, the true system administrators, who had been doing that for quite some time, had been denied the actual title for some time, were going to be moved over just as ... bodies, I guess, to elsewhere. All just ... DevOps now or whatever.
I've had "here is an install of Jira, go play" with no configuration done or any admin existing too.
My greatest achievement in that job was making production a git repo so that we could regards against production before pushing. A sufficiently low bar you'll trip over it.
That's what my employer also did. In addition to a lot of metrics on which they base performance and awards. In other words easily and frequently gameable made up numbers
My recent experience in interviewing candidates is that they have all heard the word "DevOps" but what they understand by it is highly variable, and ranges from from a way of working, to a build pipeline in Azure to ... yup, a renamed Ops team.
Since there can't be a successful software delivery method that doesn't involve releasing software, the act of releasing software _cannot_ be new and unique to "DevOps".
IMO the new factor is in closing the feedback loop with the final link from live Operations back to Development, therefor if part of your cloud stack _must_ be singled out as the "DevOps" part, it's probably the logging / metrics / observability / alerting piece that does this, and not the good old CI and deploy pipelines.
> At least with DevOps, there is no doubt that it's about Linux systems and automation.
Is that what you understand by the term "Devops" ? That "it's about Linux systems" ? Really??
That definition is a new one on me, and you very much should doubt it since "it's about Linux systems" is neither necessary nor sufficient. The confusion is even worse than I thought.
Are you trying a joke on Linux vs Unix? in reference to companies using Solaris or FreeBSD maybe? (there aren't many left of those).
All the DevOps roles I've ever seen are Unix based. In my experience sysadmin roles were more likely to be a windows administration or a helpdesk role than that.
I think it's pretty good devops got split into a distinct title. It's not a great experience walking into a sysadmin interview only to find out they are looking for an exchange administrator.
No, I'm saying the the DevOps ideas have nothing to do with OS choice. You could use them in linux or Windows or in any other OS; or you could ignore them in linux or Windows or any other OS.
> At least with DevOps, there is no doubt that it's about Linux systems and automation.
Windows Server still exists and maintains some market share. Automated, continuous delivery, and monitoring of web applications to Windows Server machines is DevOps in the exact same sense that it is with Linux machines.
The essence of DevOps is that there's no fence. The same team that writes the code deploys and monitors it. This essence remains unchanged regardless of the hosting platform - Linux, Windows, or even proprietary platforms like AWS lambda.
Oh, another person telling the world what devops means.
If we take the 50 thousand or so other blog posts about the definition of devops and glue them together - it means whatever you want. Ops teams doing some code, devs building CI/CD infrastructure, whatever.
> another person telling the world what devops means....- it means whatever you want.
I think that devops (like agile) will have a definite meaning, but uncovering that is mostly a process of examining the early history of it - who first used it and when? The definitions used, the problems that it aimed to solve, and the methods used to solve them.
However, like "agile" it can and often is abused to mean everything and nothing. This does not make it inherently meaningless, though.
Cloud engineer. Network engineer. Software engineer.... I’ve never heard of medstudents calling themselves doctors, why do cs students/grads get a pass?
Up next chefs will be food engineers, tailors will be clothing engineers, and because I changed the oil in my Toyota once, I’m now a Toyota engineer; specializing in fluids.
Words mean literally nothing anymore (ha). Titles in tech are hilarious at almost every level, especially from the outside looking in.
This argument gets rehashed in HN pretty often. When I just worked in ops or support my job title had the word “technician” in it. When I was designing systems my job title had the word “engineer” in it, even if I also had ops duties. As far as I can tell this is a regional thing, where certain jurisdictions have laws that regulate who can call themselves engineers, just like certain jurisdictions have laws about who can call themselves hairdressers. That doesn’t stop people on HN from thinking that for some reason, we can all agree on a definition for “engineer”, even though we have regular disagreements about the definition about almost every other word in the English language.
“DevOps” is at least ill-defined from the very start, so if we have arguments about what “DevOps” means, at least I can learn something.
Well, at a certain level it applies. Part of doing my job well involves designing service quality checks. That requires:
1. understanding important engineering principals like queuing theory
2. understanding the system in question well enough to modify it to emit relevant metrics for applying stuff like queuing theory
3. collecting this data into timeseries form
4. applying statistical tests to them, e.g. for normality.
5. deciding between a variety of possible anomaly detection algorithms
Another portion involves capacity planning and datacenter design, which requires a certain amount of electrical engineering knowledge, as well as forecasting. A different part of my job simply involves writing software, testing it, and signing off on changes.
Overall it sounds fairly engineery, even if the powers that be would rather prefer it if the term be reserved those who pass an internship and test (but then refuse to expand said test to include new domains).
I've met many smart people from large enterprises who said, "I'm the DevOps [guy|team|person]" with a straight face. Generally, I believe they know it's wrong, but their leadership didn't, and they were just eager to move the needle a little bit that they accepted.
Maybe people shouldn’t get stuck up that much about words. Unfortunately they are so it’s smart to attach yourself to the currently fashionable title. If you do something with dev tools call yourself “devops”. If you run a few database queries better call yourself “data scientist “. And if your current buzzword falls out of favor then don’t change your work but find a new fashionable label.
It’s an unfortunate state of the industry that we put value and prestige into ill defined labels.
Yep. Similar notion applies with "Agile" -- you can't just throw out all requirements analysis, up-front design, and architecture work, and say "We're Agile". But people do it all the time, because they're enthralled with the buzzword and not the actual meaning behind the buzzword. Same applies for ( Agile | Devops | $WHATEVER). And the sad thing is, this in turn devalues the original idea and causes a backlash where people throw the baby out because somebody crapped in the bathwater.
It’s how tech and capitalism interface. The $whatever is actively marketed, the profits are arbitrage between the true value of the thing and the perception of the value. Since that value perception is easy to manipulate with network effects, and true innovation much more difficult, we then become stagnant. Maybe this is why we don’t see much productivity for the past several decades. Productivity gains are never realized because the tech stays constant while perceptions are maneuvered for profit. One reason China may win; less respect for IP.
Interesting, in my last company the team themselves thought DevOps was not a role, but more a responsibility that should be shared among all devs and they renamed themselves "Production Engineering".
They said it was admittedly not the best, clearest of the names, but the important thing is that DevOps should be shared, not owned. As a dev, it made a lot of sense to me
> They renamed themselves "Production Engineering"
That jibes with my experience, where there has generally been one or more groups that build and maintain infrastructure to enable dev teams to build, deploy, monitor, etc. But what do you call it? It's not a DevOps role per se, nor really an Ops role, but it enables DevOps. We called it Platform, but said we were hiring DevOps engineers.
That role /team is real, especially in larger orgs, and a better name for it is "platform" / "platform team" / "platform engineering" not "DevOps".
If you must call it "DevOps", it is "Devops enablement team", i.e. supplying a platform to enable dev teams to better Develop, build, deploy, monitor, Operate what they make, themselves - enabling others to do DevOps by supplying tools.
Hard to believe (or maybe not?) there's still ambiguity around "DevOps" so many years after its introduction.
And with ModelOps, DevSecOps, AIOps, GregOps, and GitOps recently entering the lexicon brace yourself for lots of confusion and "xOps explained" articles ahead.
By the way, one of those xOps isn't real, but who can even tell?
Role names in most software companies that I've worked at and know of are almost completely arbitrary.
They're useful if you have an organization that pays a specific rate for a specific job title, and they're useful in a hierarchical sense (e.g. managers, CTOs, etc). That's about it.
I install and maintain servers and desktops, databases, run backups, build infrastructure, source control servers, do low level programming, high level programming, frontend, backend, ... etc, it's too long to even enumerate.
As far as I'm concerned that's just being a (non-junior) software developer. It's all ultimately the same thing with different names to me, barely anywhere really needs people who are at the forefront in a tiny niche (i.e. _real_ specialists).
You can call me whatever you want as long as you're paying the bills.
DevOps, like most IT fads, is just trying to get more blood out of the stone and avoid letting anyone specialize in anything. That's obviously not the original intent, but it's what happens.
At least SRE has a working definition, which is the set of practices laid out in the Google SRE book. Defining DevOps is the proverbial blind men describing an elephant.
The sad reality in most organizations today is that buzz words are applied to existing jobs to make it look like progress. So yes, apparently you can rename your IT Ops team to DevOps and brag about "how you engaged your teams to move to a DevOps focus to better align them with company values while leveraging the low-hanging fruit within today's shifting economy" during an executive management meeting.
Agreed, but, perhaps, as a management strategy, it may help some teams transition. Since after all, for folks with careers in manual provisioning, devops truly is a new frame of mind.. swapping words in titles around is just part of operations development? :)
> You can't just rename your IT Ops team and call it DevOps
Sure you can. Companies can and will do whatever they want. It may not be semantically correct, but it definitely wont stop anyone. Just go on indeed and look up "DevOps jobs" and you'll see many companies who have done just that.
At public corporations (especially regulated) there are confounding factors that make DevOps a farce. For example audit, a point of view that IT is a cost center, and the non technical management decision making that go along with the previous challenge the spirit and proper execution to get DevOps right. But in reality DevOps is a symptom of a larger issue - functional reporting structures and goal misalignment.
Audit - the idea that your developer doesn’t have access to production systems and data is a challenge to DevOps. The idea that one person codes and one deploys can be a challenge. Some organizations like management visibility and an audit trail to prove it - this can challenge automation and frequent deployment. This is not just a DevOps issue, audit expectations for business requirements and QA signoff can be just as bad. Admittedly some of the audit rules are relics of the past but the need to be addressed in order for DevOps practices (and really for agile teams to be agile.
In organizations where IT is a supporting function of the business there is a view point that it’s a cost center only and doesn’t produce value and should be cost controlled and managed to ultimate efficiency. I’ve reminded CIOs that the work the IT team does is not “a factory” as they sometimes insist on calling it - no one in technology that I’ve ever met (except management) really wants to think they work in a factory.
The efficiency mindset rank orders roles with QA and analysts at the bottom, devops slightly higher, developers relatively high, architects at the top of the skilled food chain and management of course at the top. To some degree these functions can be centralized so they can be optimized and made efficient. Centralizing typically means you work for management rather than a particular business mission. Your job is to make management look good and keep everything running smoothly (remember you work in a factory). Management generally looks good when quality is achieved at a lower price point. Along with that tends to be a higher level of off shoring at the lower end of the skill rank where larger numbers means more efficiency at lower cost. Here lies the problem - the premise of DevOps is that the role is integrated in the team. When DevOps is a function reporting to management then their goals aren’t aligned to the development team. They might not even sit with the team and they might have their own orders. They might be in a 12-20 hour time zone offset.
To me the larger issue is management structures aligned by function instead of business. This leads to misalignment if goals and ultimately lack of integration in functions. It’s a bigger problem than DevOps and it’s a bigger problem than IT. I think the companies that get management reporting right will win in the end. I am curious to see how Google and Amazon respond to these issues at their scale. The pizza team doesn’t work when everyone has different goals.
