All this time I had thought that Rocky Linux was winning the fork war over Alma (in the fight to be the successor to CentOS), but this post might change that with a good chunk of the science community throwing their weight behind Alma.
Do we have anyone else in the audience that has any insights over whether Alma is more prevalent over Rocky or is it the other way around?
I know I can run Rocky Linux from DigitalOcean which I appreciate.
Rocky is "winning" if by "winning" you mean more widely used. Here are some graphs charting usage through EPEL statistics: https://rocky-stats.tiuxo.com.
The source used to generate the graphs is available at https://github.com/brianclemens/rocky-stats, however please be kind to the Fedora servers and don't download the stats database too often if you use it.
Also Rocky Linux is far larger in terms of community size, etc.
As someone who has had to use Oracle Linux for quite a few projects due to requirements, those graphs are actually a sobering look on things.
Of course, all of those distros are reasonably similar at the end of the day, but it's pretty clear that the popularity of Rocky Linux, Alma Linux, and even CentOS Stream are all formidable.
Also note that genuine RHEL users generally don't want to use EPEL that is out of support.
> The EPEL repository is used by a significant portion of Enterprise Linux users. The data they share are unbiased, and it is reasonable to assume approximately equal proportions of users of each Enterprise Linux distribution use the EPEL repository.
While it's true that a good number of RHEL users don't use EPEL due to the lack of vendor escalation, there are many that do use it. In fact, they tell Red Hat that specific EPEL packages being available in the next version of EPEL block them from migrating to the next major version of RHEL. The feedback finally got loud enough that Red Hat decided to provide headcount to the Community Platform Engineering (CPE) group to create an EPEL team focused on improving EPEL (this is my team at work).
That said, the "vendor escalate-able only" group is large enough that RHEL is significantly underrepresented in EPEL stats. My guess is that somewhere around 40-60% of RHEL customers use EPEL. For RHEL rebuilds, I would guess that it's probably in the high 90% range, so their numbers in EPEL countme stats are probably fairly accurate. CentOS Stream on the other hand is also underrepresented, as we have countme data for both EPEL 9 and CentOS Stream 9, which shows there are over twice as many instances without EPEL as instances with EPEL.
Keep in mind that the countme data only includes systems connecting directly to Fedora's MirrorManager. Sites that run their own local mirrors are not included. We will never have a complete picture of popularity between these distros. For example, Facebook runs "millions" (a direct quote from their engineers' conference talks, they don't publish exact numbers) of CentOS Stream instances, which is more than everything else in the EPEL countme metrics combined.
> Do we have anyone else in the audience that has any insights over whether Alma is more prevalent over Rocky or is it the other way around?
This is wrong reasoning. Distributions live on the shoulders of maintainers/developers. While popularity may be a factor attracting maintainers/developers it is not a deciding factor. If the less popular distribution attracts more maintenance/development effort (again, not necessarily head count) it will win back market share in medium term and once popular distribution will stagnate and fade.
It depends. As a counter example think of Scientific Linux [0] which was also an RH clone supported by Fermilab and CERN. It eventually got replaced by this "obscure" distro called CentOS.
While I didn't have many projects recently that focused on EL distros. Last year, I wanted to publish a public AMI built on top of Rocky. However, someone at the time decided to configure the published Rocky Linux AMIs to disallow republishing, so that's the only time I actually picked Alma over Rocky Linux.
After the dust settled behind the CentOS model changes, I think there might be a reasonable amount of users that also shifted to CentOS Stream and Fedora linux (Amazon Linux 2022 is also Fedora based instead of RedHat). For the scientific community something that changes way less is obviously a different story.
Alma is not exactly prevalent. It however releases faster than Rocky. Alma's mailing list is also more informative. Its Announce list post security advisory frequently. Rocky's Announce list in comparison is quiet.
Rocky Linux _is_ community oriented, and _is not_ beholden to a specific company, but it is not necessarily unpaid. The majority of the most active contributors to the project are being paid by their respective organizations (CIQ, OpenLogic / Perforce, etc) to do so.
Should be noted that AlmaLinux is a registered non-profit, whereas RockyLinux has not done such a thing and has bylaws that leave open the opportunity of being bought out by a for-profit
From RockyLinux:
> The Foundation is here for the benefit of the public community. We are a self imposed not-for-profit organization[^1] and thus we will never be driven, motivated, or manipulated by profit or monetary gain.
> [1]: This means the Foundation is a Delaware Public Benefit Corporation, with the objectives set forth in this Charter and the Foundation Bylaws. We do not have an objective to make money for shareholders. As of the time of this writing, the Foundation is NOT a 501(c)* US tax-exempt organization.
Also, AlmaLinux has its own infrastructure. It's not using CloudLinux infrastructure.
Most services are NOT on CL infra. There are a few minor things that need to be moved but all the core stuff is hosted on Alma's own infra including the build system/servers.
Strictly speaking, Greg Kurtzer didn't start CentOS. He started cAos, and people within cAos started CentOS to bootstrap cAos. When the CentOS and cAos people couldn't get along, CentOS left cAos to be its own thing. Lance Davis probably has more claim to being the founder than Greg Kurtzer, as he was the one that started actually creating the Red Hat Linux rebuild work the led to CentOS.
Rocky McGaugh started building cAos-EL under the cAos Foundation which I created and led.
Due to my role with the cAos Foundation, I was part of the planning, inception, architecture, setup, leadership, management and led the project itself, but Rocky did 99% of the engineering work for cAos-EL-2 (later renamed to CentOS-3) and after Rocky was working on that, John Newbigin started on what would become CentOS-2.
Lance was there since the early days of the cAos Foundation, and he suggested the name "CentOS" (and he squatted and held the domain from the Foundation, which was how he took over the project), and he got involved with engineering/development of CentOS-3 after Rocky passed away. While I would agree, he is a co-founder, he proved to be opportunistic and acted very unethically and has been further demonstrated with the open letter sent to him from Russ Herrold (another co-founder) and the rest of the CentOS contributors for going AWOL, still holding the domain, and taking the project donations for years.
CentOS split off from the cAos Foundation (501c3) due to Lance's stronghold on the domain. To be clear, the separation was not mutual but it was cordial and sugar coated for the good of the project. I always enjoyed being part of CentOS and working on Linux distributions so not being part of CentOS was hard on me. This is one of the reasons why I announced a new distribution (Rocky) within 2 hours of CentOS being killed off, I was excited to do a distribution again! You can also imagine how and why I setup the Rocky Enterprise Software Foundation different from the cAos Foundation to better protect the project(s).
One last point, just because I wasn't working on core engineering and development of CentOS doesn't mean I wasn't deeply involved for the first years of CentOS and not a reason to discredit my role as project leader and co-founder.
> The problem is that Cent 8/9 Stream has quicker critical CVE patches because it's essentially the source and is closer to mirroring RHEL.
This isn't entirely true, at least for embargoed CVEs (so the most critical vulnerabilities). RHEL will always get embargoed patches first. Once RHEL releases the patched packages, Alma/Rocky can rebuild them too. The patches may not be available in Stream yet at this point. This has happened a few times in the past.
Also, there are no advisories for Stream, so it's not always easy to tell if a Stream package includes a patch for a particular CVE. Sometimes you have to go hunting in the changelog to figure it out, as the version numbers don't always match with RHEL's.
Rocky Linux is developed by a large team, some who are paid too and some who are strictly volunteers, but yes, Gregory Kurtzer's own company (https://ciq.co) has invested quite a bit into Rocky Linux. (Many developers, FIPS validation alone is upwards of a million USD, etc)
He does not develop it whatsoever. Really all he does is talk, so basically zero development value. He certainly likes to take credit though for all the work all of rocky's volunteers do.
Yes, Gregory Kurtzer personally helped in development (primarily the packaging / tools during 8.3 and then 8.4). I know, I was there lol. For example, see the commit log to the early set of Rocky Linux devtools:
"All he does is talk" is unfairly dismissive. We all have our roles, and Greg's is not release engineering.
The "taking credit" bit is an unfortunate misconception, media likes to attribute the entire project to gmk since he's a notable personality, but he himself does not.
>For example, see the commit log to the early set of Rocky Linux devtools
And that's literally it. After that... Nothing. Not surprising.
>"All he does is talk" is unfairly dismissive.
Funny because he's the only one ever mentioned or talked to in any article. It'd be nice to hear from the actual developers and not a figure head. Notice how it's only him? His role is to talk. His other company is also there to take credit, thanks to him and the media. We can blame the media all we want, but the reality is he basks in the spotlight. If that's his actual role he's doing it well.
I certainly hope your upcoming board didn't drink his koolaid and keeps him out.
I do try to ensure that others are always getting credit, I'm not even mentioned in the release notes of who did the work.
And the board doesn't keep people out. If you read the bylaws and charter, you would know that it is all contributing members of the projects that will vote for the board members, and the board will elect the officers of the organization.
You are obviously not an RESF Member and I trust the Members of the projects to make the best decision for the RESF and Projects. If I am among them, cool, I will always do my best job there. If not, I will support the decision and enjoy knowing that the structure that I helped to create, is working, and will keep the project open, free, and in the community for decades to come.
Last point, it isn't cool to discredit non-technical contributions, every role in an open source project is important.
>And the board doesn't keep people out. If you read the bylaws and charter, you would know that it is all contributing members of the projects that will vote for the board members
Good to know, I forgot how voting works. I hope they do right by the community and not vote you anywhere near the officers of the org. In your words, that means the structure will be working. But that requires faith that there are members who didn't drink your koolaid.
>Last point, it isn't cool to discredit non-technical contributions, every role in an open source project is important.
You're right, advertising your company CIQ using Rocky Linux is definitely up there in important open source contributions. I forgot about that small point.
Do we have anyone else in the audience that has any insights over whether Alma is more prevalent over Rocky or is it the other way around?
I know I can run Rocky Linux from DigitalOcean which I appreciate.