You can try https://ci.sparrowhub.io as well, it’s yaml based but more flexible then standard yaml based CI systems, reporters https://github.com/melezhik/SparrowCI/blob/main/reporters.md allow you to integrate with any others third party systems ( for example with bugzilla , GitHub/GitLab , jira or IRC )

Absolutely, I feel like most CI systems are much harder to configure than necessary.

If I were to build my own CI system, then it would have these features.

- Simple config format, ideally flat and not nested, so maybe toml.

- Ability to run any step, task, job, pipeline locally. It should be possible to do something like `ci run -t build` and have it execute the build task locally. This would greatly help with building CI pipelines and make it possible to just use the CI tool as the local build tool as well instead of having to essentially setup the same thing twice.

- Easy to self host, so not a million micro-services, but just one (or max 3) executable.

- Modular, it should be possible to extend it with plugins using webassembly or containers. Look at how Concourse CI does this for ideas.

Edit: I would pay for a license to something like that, maybe not a lot but I could justify up to $200 as a once off for a self-hosted solution like that if it's well made.

This doesn't have everything you mention, but I recently learned about https://dagger.io/ which allows you to write your pipelines as code (Go, Python, etc) and then run them anywhere in containers. I always hated that in order to test my pipelines, I actually have to trigger it on some special machine (the CI server).

Dagger looks interesting, I will check it out.

I think these are all great points! I have came across Concourse CI, but have not looked too deeply into it yet.

I hadn't thought to use TOML for configuring CI pipelines, I'm curious what that might look like. TOML is indeed very flat, so it would be interesting to see what the equivalent TOML pipeline looks like compared to a YAML one.

I run Concourse CI for my personal stuff with the code in Fossil. It works decently enough, I like that it's somewhat extensible, but I have hit use cases that it can't really do like building an Alpine Linux cloud image.

As for YAML vs TOML, I am mostly sick of YAML, and I really like using TOML to configure Caddy, so I thought it might be good enough for CI config. I feel like it would be worth exploring, but maybe something slightly more complex or custom would be required.

Sounds a bit like TeamCity, except for the local execution. It has:

- Either GUI based configuration (serialized to XML), or Kotlin. So you can express configuration with a real programming language tracked with version control, or just use the GUI to define it.

- Can self host. There's one program to install for the master, and one for the agents.

- Can be extended with plugins. Not webassembly based, but those are theoretical anyway currently due to lack of better-than-C interfaces. You can write JVM plugins instead and get full GCd OOP.

The main thing it's lacking is the ability to run things locally. But then again, usually a TC step is just invoking a build system or script anyway so it's not that big of a deal.

In addition, it would need to try and solve for the 80% witch hatches for those that need the last 20%.

Part of the problem with current CI/CD solutions is trying to solve for everything.

And I think the local thing would be a killer if for no other reason than the ability to debug it locally.

That's kinda my idea with plugins, it should be possible to develop a plugin doing your custom thing if you need something in that last 20%.

Concourse CI has a design documented for a better resource/plugin system than what they currently have implemented that's worth checking out. But as far as I am aware they are really far from actually implementing it.

I've been thinking lately that how I could do some integration testing FOR my CI-pipelines. My client has very complex pipelines which has business logic inside them and every week we run into new bugs and regressions. I can't think any easy way to write tests for them to make it harder to break when developing new features.

Btw. Gitlab allows splitting your pipelines to multiple files using include[1]. Also we have used gitlab-ci-local[2] to run our pipelines locally when developing them.

1: https://docs.gitlab.com/ee/ci/yaml/includes.html 2: https://github.com/firecow/gitlab-ci-local

I havn't heard of any products or services that test the actual CI pipelines themselves, which seems like an interesting target market. I can't say how many times I have wanted to test out some CI workflow on some throw-away clone repo without mucking up the live repo.

It's good to know that you can use includes with Gitlab, And, from my quick glance at that second link, the setup process seems a bit involved. I don't like having to set up new user accounts just so that I can run a CI workflow locally.

I've been able to use the tool without setting up any accounts or other settings. Just run `gitlab-ci-local` in a folder where your `.gitlab-ci.yaml` lies.

The real meat of what CI/CD does is closely tied to the platform, I don't think there's a great way to improve that. Running on different configurations, uploading artifacts, deployment, etc.

Generally, I do as much as possible in the build system proper. The CI should mostly be running simple commands. Avoid GitHub Actions which can be trivially replaced with one or two shell commands (which is a lot of them).

It should be more achievable than it has ever been before with container tech being in a great place, qemu being a thing and the webassembly ecosystem getting better every day.

Make a bash file. Have the platform specific yaml call your bash file. call it a day

I’m not sure the niggles here are platform things - they’re hard lessons that come from experience in using CI for the most part.

Generally you want to aim for writing things in scripts that your CI system calls, so that they can be run locally trivially and so that migrating away from one CI provider (for whatever reason - cost, employer decides to shift, etc.) is not a huge piece of work.

So if we have to use a script anyway, then why do we have to write like 20+ lines of YAML to run the script?

Edit: I mean, why is this not the config if we are just doing a script?

  src: github.com/...
  run: test_and_build.sh

It's not that bad:

  jobs:
    unit-tests:
      runs-on: python:3.8.15-slim-buster
      - name: "Run tests"
          working-directory: ./bin
          run: unit-tests.sh

With GH actions you got everything. Conditionals, proper expressions, and goto refs, which allows loops.

I cannot seem to find any references to "goto refs" in GitHub Actions, where is this documented? I've never seen this before so I am curious

I'm pretty happy with GitHub actions