I can't believe the author doesn't mention gitlab. Setting up full CI on gitlab is amazing. They have the easily integratable gitlab runners i.e. travis-ci functionality + different build stages for automatic deployment via ansible/chef/whatever and unlimited free private projects to boot. If you're working on anything that can't be public I highly recommend gitlab. Its an amazing service.
GitLab is great and has come a very long way over the last several years. However, I still gladly pay GitHub $7 a month for my private repos because I feel that GitHub truly does offer a better code review and repository experience. GitLab in certain places is still very rough around the edges, not to mention performance has always been a concern.
To me, GitHub still is the gold standard and will likely be for quite some time.
Not sure how, it's got the same here is your changed files and the changes.
I find GitLab is better on a few point. You can theme it from about 4-5. I prefer the dark theme as it matches my IDE and easier on my eyes. You can also have comments marked as addressed. By default the changes are shown in the page but you can also choose to show side-by-side like in GitHub.
You can host it yourself. It ties in very well to it's own CI/CD service and comes with a docker registry. So your code can be tied easily to your deploys and you see this in the issues.
> , but Gitlab pages are also much easier to setup
uuh, what?
i think gitlab pages are wonderful, because you're not restricted to jekyll, but how is it easier to setup? you only need to toggle the checkpoint in the repository settings...
> can't believe the author doesn't mention gitlab.
I absolutely agree. We use Gitlab in our company for continuous deployment and it works like a charm.
Apart from the other tools I miss some performance monitoring. Sentry is great, but if your application suffers from bad performance after the latest deployment, you won't notice that soon enough.
I recently found Datadog for performance monitoring (it's like statsd++++) and I'm already a big fan. I haven't managed to get it working with my Heroku projects yet though.
Once you need something a bit more heavyweight than Travis (e.g. more CPU) I would highly suggest Buildkite (https://www.buildkite.com). It’s a “bring your own machines” solution, which means you can have beefy machines (e.g. via buildkite on EC2 spot autoscaling), or even machines that run inside production without handing buildkite itself any secrets. We’ve been using it at Ladder and it’s been phenomenal.
Heroku does have a free tier, but it's intended for experiments. It's very limited and wouldn't be acceptable for any serious website, certainly not a 'world class' setup. Unless you're very careful to have one free dyno per account, and avoid using features like staging apps, they will power down your server for several hours a day.
Yeah, the “world-class” hook was more about the quality of these tools than their applicability for use at scale for free - I encourage people to pay real money at various points in the article.
I’m currently running my blog on a single $7/month Heroku Dyno (plus $9/month for Heroku PostgreSQL) behind a free Cloudflare account - with the right caching headers I can easily handle huge amounts of read traffic. I was going to mention Cloudflare in this piece but I decided it didn’t fit the theme of continuous deployment as well as the other tools.
Codeship (https://codeship.com/features) allows forever free builds for public repos. It has support for both Continuous Integration and Delivery/Deployment. Maybe you can give it a spin as well?
I can second codeship - even though its a bit more clunky than Travis and with less integrations and command-line tools, its has way beefier machines, easily several times faster than what travis offers even to paid customers. And their tools are improving. Keep up the good work.
Codeship founder here. We support .yml file config for our container/Docker based product (called Codeship Pro). We're looking into offering a UI + yml file approach for both products because we see a 50-50 split when it comes to what people prefer. Thanks for the feedback :)
Only comment I would make is that you can use CircleCI instead of Travis to get for free running tests on private repos. The configuration is similar and the benefits as well.
Doesn't show any automated verification of code in deployed fashion, no load tests or verifications which would be part of a world class pipeline. Also no canary or red-blue tree type rollout hooked to sentry. At least it has a staging but don't know what they're using it for other than to test migrations before promoting to prod. Nice little write up but far from world class.
Exactly - continuous deployment is about moving some/most testing to production, not eliminating it altogether.
Acceptance testing the system once deployed is the cornerstone of organisational acceptance of the CD doctrine - errors will always happen, it's just that they can now be fixed much faster in production.
Using an acceptance test suite as a deployment safety harness should dramatically increase the quality of tests, and hopefully the application too.
or be willing to move vendor and re-implement the stuff.
For comparison my magic combination(and not all free or realistic for everyone):
- gitlab
- jenkins/looking into gl ci
- linode/cloudafrica(ah-hah hosting is not free)
- sentry is a tough one. We rolling a custom mqttwarn/zabbix solution
- irc(fine I will take your slack and use the irc-bridge pfftt)
> - irc(fine I will take your slack and use the irc-bridge pfftt)
We ditched Slack for Mattermost. It's not entirely up to par (yet), but works for most use-cases. And there's an IRC bridge for it, too (https://github.com/42wim/matterbridge). Oh, and Mattermost comes bundled with GitLab in their omnibus.
I really wanted to use Gitlab for my next side project, but Circle CI doesn't support them, and I really don't want to pay Travis for my shitty proj. Hopefully this changes some day soon.
GitLab's built-in CI is very good, and simpler than Travis for some things.
You also have the option (unlike Travis) of registering your own runners in your project settings, and you can do this either on gitlab.com or another deployment, without needing to be an admin of the deployment - you can get a key for it in your own project's settings. I'm using this at work, where we have an internal GitLab that I'm not an admin of (and where the GitLab admins probably have no interest in running a CI system), to do a small bit of CI on some things using one of my team's servers as a runner.
We use gitlab's built-in runners and pipelines for continuous deployment to kubernetes and it works very well. We self-host our repos but the same tools are available through the hosted version.
Heroku is awesome for this use case, but there are also great alternatives based on Cloud Foundry. Cloud Foundry is an open source PaaS with support for Heroku buildpacks + docker containers. At a median of ~5€/GB RAM per month they will also be a bit cheaper than Heroku.
Some providers that offer Cloud Foundry including a free tier for small Apps:
You can also do every step listed, for free using a single platform: VSTS (Visual Studio Team Services).
Online, free for small teams, Git source control, CI/build/release, supports every major build stack, not just MS. It even has a top quality agile work tracker, that I've found better (or at least more balanced) than Jira/Trello equivs.
Not sure bitbucket pipelines are mentioned here. But for free I manage to commit Hugo source files to a private git repository, generate the static files by spinning up a docker container and generating them using Hugo and finally rsyncing the static output to my Webserver. All for free. You only get 50 minutes of time now (for free) in the docker instance but for simple continuous deployment cases like mine it’s very effective.
You might want to look at Gitlab if you want to drop the need for a web server. Their pages support supports Hugo much better than GitHub (which leans toward Jekyll), and is pretty well documented.
Add Netlify and it will continue to be free. I use Hugo as well, Netlify compiles Hugo on it's own, so I don't need workflows, but it is cool to have them.
It’s fully automated. I do a commit and push from any machine that I have cloned the repo to and it gets published automatically.
Now I don’t need to download Hugo, remember the Hugo commands, remember my web server login details, etc. I think it’s a lot less fiddling as all I need to remember now is how to add a file to a git repo to get a post published.
Don't you miss out on Hugo's archetypes doing it that way, or at least need the templates locally? Also you shouldn't need your web server login details for rsync... Hey whatever works, it just seems like substantially more work to me.
I used to use team city, octopus and jenkins in previous jobs. Personally, it is really easy to use and you don't have to use the microsoft stack. It supports Python, Ruby, node etc... It feels to me like they waited to see all the good bits people have done and then just implemented them.
I didn't want to manage a server so it is perfect for me.
Does anyone have a solution for a situation where I have a Django backend and a VueJS front end and want to run the Vue build step before deploying? Currently doing this with Git hooks and committing the resultant build, but that is bad for a number of reasons. This is running on Heroku. Would their release phase allow for this without significantly slowing things down? Would it deal with npm installs?
You might be able to solve this using a bin/post_compile script - most standard Heroku buildpacks will run these as part of the build step, and they can be used to run commands that generate files which will then be included in the compiled app that gets deployed to Heroku.
I've done this for several apps. We have a large Jenkins 2 cluster and use declarative Jenkinsfile's. For Jenkins, you can accomplish this by simply setting up a Jenkinsfile that has an agent defined in each stage. Build your VueJS app in the first stage with a NodeJS container, and build the second stage with a Python container. Bonus points for building an actual Docker image as your artifact that has Django inside of it with Apache or nginx serving the VueJS bits too.
You do not need to commit your build to git if you are using heroku.
If you add the nodejs buildpack to your heroku project, it should detect and build your package.json automatically on heroku. Works with both yarn and npm.
You just need to make sure to set NODE_ENV=production in your heroku config.
We have this with GitLab CI, although with React and not Vue (but I don't think this matters). We have two stages, `build-assets` and `build-app`, first one runs Webpack etc, and the second one copies the build artifacts into a Docker image with Django and finalizes the image with collectstatic.
We're using Docker, but I'm sure the same approach can be used with simple tarballs, OS packages (.debs, etc) or deploy-specific git repo (for Heroku)
I use gitlab instead of GitHub and Travis and tend to use the free tiers of GAE and Heroku indistinctly. But man, I wasn't even aware of something like Sentry existing, I'll have to get me some of that.
I also need to think of the messaging system. I'll probably start using mattermost instead of slack, though; because GitLab.
I still have my stuff on AWS instead of Heroku and use Rollbar instead of Sentry (not for any specific reason), but overall I'm really happy to hear that I'm tracking well with this setup. I find it smooth as butter most days. Thanks so much for sharing!
Thanks for posting this. I'm looking to get any application and start deploying through a CI/CD process, to learn how this process works. Any application suggestions for a CI "Hello World"?
Travis is a third-party service. Jenkins is (usually) software you run yourself. Travis will run the infrastructure for you and will not charge you if your project is public on GitHub. So that makes it an extremely popular choice for open-source projects: sign into Travis with your GitHub credentials, click a checkbox on your repo, and add and commit a .travis.yml file, and now you have CI that is well-integrated with GitHub and you don't have to run any infrastructure or pay for any server hosting.
Of course, you have much less control over Travis than Jenkins. Two cases where Jenkins remains obviously better are when you need custom environments that aren't easily portable to Travis (Windows support is a big one, although there is a company called AppVeyor that is something like a Travis for Windows; licensed software is another big one), or if you have strong security/confidentiality requirements (e.g., you're in a regulated industry) and using random cloud services isn't a good option.
I think you mean CD “Hello World”, not CI. You can make a “hello world” web app (or even just an API endpoint) in your favorite language and set it up to deploy on each commit.
CD means each commit that passes tests is deployed
CI means your code is regularly integrated (merged) vs having long QA / code review cycles.
CI and CD work great together but you can have one without the other.
Yes, I want to better understand the process of committing code to the repository of an application, have it run automated tests, report via a communications tool the result, and automatically deploy if successful. Understanding this efficient process is high in demand and thus highly profitable.
I've created and worked on several projects that have involved APIs, reading from Zendesk, and writing to Slack or HTML, among other places. I've supported Jenkins and related tools multiple times in production. Now I want to fill this gap in my knowledge.
Travis is a lot more user-friendly than Jenkins and they run it for you, whereas Jenkins you usually self-host. (Jenkins is a lot more flexible and has a lot going for it, but Travis is much easier to get up and running)
Two hours after posting, and this is the only comment about Concourse? jacques_chester must be on holiday.
Is it possible get get Concourse running entirely on free tier stuff? I don't think there's a hosted Concourse service yet - airplane.ci is coming, but still in beta - and i imagine you'd struggle to get a whole setup on a t2.micro.
I like Bitbucket better for my personal private repo's. Also would like to mention few more tools I like, Netlify for static site hosting, SemaphoreCI as alternative to Travis, I think both are good.
When they say that AI is not going to take over dev jobs, show them this thread and tell them that the time cut down because of automation is going to be spent in something more meaningful, like coding!
Sentry also has self-hosted option available (and it's also opensource).
Instead of GitHub + Travis CI, you can use GitLab (it also has bundled CI capability, based on Docker).
with GitLab, you can also have Continuos Deployment, wired up with the CI solution. The best experience will be deploying to Kubernetes, as it will give you even monitoring, automatically, but you can code custom deployment logic as a step in the CI pipeline.
So you equate "world-class" with "enterprise"? I disagree. Tools that scale down can also be world-class, and in fact that ability to run cheaply is a big part of what allows startups to get off the ground and bootstrap their way to eventually challenge the big boys. Heck, even if they stay small, they can still have world-class tech. Once you get big and purchasing decisions are made far from the front lines I'd argue it gets harder to really have the best tech.
Well as part of my current world, sure I do. I have to satisfy the needs of not just dev, but operations, compliance, security, business, the people overseas that i've never met, etc. What I'd really like to see are startups that bootstrap this stuff to think that way prior to growing.
My original point was that this was cool for the dev crowd and perfect for the blog use case, but "world"-class seems hyperbolic. But if you are working for other people: people walking around who have strung together various automation stacks are a dime a dozen -- I've literally seen 4 different takes this month. Execution matters, and execution for your audience matters more.
>> Tools that scale down can also be world-class
We are of the same opinion. I love me some elegant free solutions, and I need to because then I can judge what I'm buying when I need to buy something.
>>Once you get big and purchasing decisions are made far from the front lines I'd argue it gets harder to really have the best tech.
I guess I see where you could be coming from, but it doesn't HAVE TO be that way does it? Isn't the whole point of work to empower yourself to affect the company decision-making. Failing at doing so is sort of, without offending you, you not doing your job. At least that's how I look at it.
FYI: I am in no way affiliated with Gitlab.