Can/will it be adapted to handle provisioning in cloud environments? - e.g. set up EC2 instances or is it your expectation that it will work with something that will handle that part?
So pyinfra itself won't do this - but because inventory/etc is written in Python it's possible to easily achieve this by using boto3 directly in the inventory file.
The use of Python for the inventory + deploy code makes it possible to integrate pyinfra with almost any external tool, without specific support in pyinfra (that's the theory, at least!).
Seems like a much more modern Ansible to me, more precise but more targeting Python users instead of trying to attract non-coders too, it's very promising. I wish I had found your project before I started my own too, it's shitty compared to yours but i'll swallow my shame and share it with you just in case you find an idea that you like yourlabs.io/oss/shlax
I've not used salt very much at all but I believe both tools provide similar functionality. pyinfra is only agentless vs. salt can do both which makes it more scalable at huge numbers of hosts. The major difference is the configuration in Python vs YAML, which I personally think is a big advantage (but can see arguments the other way too)!
YAML is not a programming language, and deployment is not a toy task. You need power.
The result is that ansible has pushed a markup language to its limit, then pushed the templating engine used in it to its limit too. Then used a bunch of duck tape + conventions to glue everything and ended up with 10% of an unclean verbose unexpressive real programming language with ridiculous limited tooling, testability and had to document + support it.
As with most DSL.
For what? For the quest of having something "declarative", and that any language could read.
Well you can have declarative API written in any language, and nobody except ansible ever read playbooks.
In the end, it's just a big weight at the ankle. I never ever used it and though, "damn, what a pleasant tech choice, I'm happy they didn't directly expose the API through Python".
It's the same reason I use "nox" and not "tox", or "doit" and not "make". DSL really seem like a great idea, but they fail most of the time. There is a reason only few of them - like SQL, CSS or regex patterns - became a success. 99% of the time, what you need is a good lib, with a well designed API. For the 0.9% of the time you do need multi-language communication, you may implement RPC. Then, only for the 0.01% case should you really consider a DSL.
But we geek love to create DSL. They are fun to write! They are so elegant in tutorials!
Is Pyinfra doing better? I don't know, but I'm sure to give it a try. I really, really want to leave ansible behind, but fabric 2 is not high level or declarative enough.
Ansible's pretend-declarative yaml doesn't add any value, while forcing you to use patterns that rarely match your use-case. It should have been a python library without all the yaml from the beginning.
That said, pyinfra seems to be making the same mistakes (being a tool instead of a library, prescribing a folder structure, not letting me create my own abstractions), so you are right that it provides no advantage over Ansible.
While it would be nice to have the core and the framework dissociated, stating that it provide no advantage over Ansible seems overly broad.
You can use python logic to compose your deployment scenario, use imports to reuse features, package modules the same way, etc. The whole ecosystem is also at your disposal, be it libraries, IDE support, linters, formatters, debuggers and so on.
It doesn't need to reinvent the wheel, and you don't need to learn a new syntax.
I would call that a win.
Now, is pyinfra well designed enough to be practical in production, that's another matter that needs testing.
Well then maybe this is more for you yourlabs.io/oss/shlax (it's still in development, check the pipeline build job to see some outputs)
But honestly pyinfra seems like the clean Ansible rewrite for people who like both devops and python programing, pyinfra looks like the next major version of Ansible.
Thanks for sharing Shlax, I didn't know about it, and at first glance, I like the design choices: encapsulate task, pass the target object so you can mock it for testing, use await to delegate I/O...
At a glance it seems more comparable to fabric than Ansible. Ansible is not really about controlling computers, it's about declarative configuration.
One major benefit over ansible is that you only need a posix shell on the remote side, instead of a compatible version of python and possibly some specific libraries. Which is a also a major downside if you want to manage systems that don't have a posix shell.
This is a bit like asking what the Marquis de Sade did that was so painful.
I don't even know where to begin explaining how frustrating Ansible is. Over time they've compounded one bad design decision on top of another, while never adding any actually useful functionality to be able to troubleshoot or iterate on the many random failures. They also don't provide guidance on how "if you use feature X, features a, b, c, d, and e will not work correctly". Finding a working example of core functionality, like an AWS inventory plugin, requires you to dig through all of the code and scour the internet and experiment for a day before you have a simple working configuration. The more features you use, the more fragile and shitty it becomes, to the point that nobody wants to actually change any roles or playbooks because deployments might stop working and it'll take you two days to figure out how to make them work again in Ansible's bass-ackwards design. And of course, it's Python, so you have to teach all your users how to use virtualenvs, freeze deps, and run it with Docker, or you'll constantly hear "it didn't run correctly for me".
It is easy to pick up and do easy tasks. Like installing a package on ubuntu and setup ufw. Then you throw in a few centos hosts and have to write if-else in yaml. Then u try to provision docker and end up with more if-else blocks to install the required python packages. Then you get to the part where you want to provision a database with presistent storage inside a docker container and just give up.
The two things that came into my mind when I first used Ansible was "bash" and "hudson/jenkins".
It seems like a tool which was written by an ops person (as in hacked together) for ops people (who don't mind stuff being hacked together), specifically for replacing manually distributed cryptic bash files. It's an improvement over that, for sure, but that's it.
Yes, if you freeze deps you can just install to the system paths in a Docker container. But most people who I've distributed Dockerized ansible git repos to don't read the README and don't try to run it in Docker, so they flail around trying to get it running without a venv or anything, until I finally convince them to just use the container (they're not familiar with containers either so it's almost as much of a slog).
Ansible error messages have gotten better, but there are still plenty of times where it will take a configuration error such as a missing module argument and report on it as a syntax error.
And then when you get those errors you need to understand what the object is and now you're running the debug on a ton of variables, some of which you first need to set facts on, then others you don't know what are available.
IMO the biggest problem with Ansible is that it feels like it was designed with a really good idea in mind, and then extended and modified by a bunch of different teams who had their own ideas. Which is fine if you've been paying attention, but explaining it all to someone new can be overwhelming.
ansible's declarative manifests got pretty complicated and even (potentially, not sure if it actually happened) achieved accidental turing completeness.
if your DSL gets to that point sometimes it's better to rewrite as a library.
Just checked out the NixOps manual https://nixos.org/nixops/manual/ and it seems able to deploy to NixOS machines on any cloud or LAN platform. But i also do not find any mention of it being able to deploy non-NixOS servers.
The question is why would you want to have non-nixos servers in that case. If the goal is to have super-fast automation, you still need to unify your setup around something common. I see more benefits in unifying around NixOS than in unifying around Ansible/Pyinfra, as the latter require me to specify my infra in terms of low-level OS- and distro-specific package managers, that would eventually define the same lack of "OS agnostic portability".
it supporting apt doesn't mean it "relies on apt". Looking at the list of options, it supports multiple package managers, including ones for Windows and MacOS.
but then, if one needs to use low-level primitives representing specific package managers, it's hardly aligned with a claim of "super fast infrastructure automation". With Nix I have exactly one package manager for any target platform.
- Pulumi [1], which supports amongst many other languages Python as well
- SaltStack [2], which has a broad range of execution and state modules, uses by default a "Master/Minion" architecture, but can be used push-based through "salt-ssh" [3] as well
- POP/Idem [4] - which originates in the concept of idempotent SaltStack states, but exposes this functionality as Python code and uses the POP paradigm [5] coined by SaltStack's founder Thomas Hatch. A lot of SaltStack itself will quite likely move towards this architecture in the foreseeable future as well
Staging works well in my experience. How can you benefit from a test pipeline and a staging system with tools that bypass it and ssh right into the production systems?
Keep a menacing hammer in the corner of the room. Walk in once a week and fondle it with a viscous grin just waiting and daring someone to make you use it.
The author of Pyinfra is just a dev like you and me working on a side project.
There is no relationship between pyinfra, pipenv and pip.
There is not even a relationship between pip and pipenv, they are completly different teams.
In fact, none of those projects are maintained by the Python core team or part of the Python project. Not even pip, which is separated from Python and provided using get-pip.py at install.
And deployment and crosscompilation (for native runtime dependencies) and performance, although those are probably not significant issues for this application.
Whats your issue with it? Asking as a noob to python package management. I've always used venv and pip and didn't even know pipenv existed until I read your post. Should it be avoided?
Personally I'd recommend Poetry [1] at this point for Python package management. 99% of the time it works perfectly and keeps out of your way. There's also a lot of good info in the comments here [2]
