When I said black magic[1] the last thing I was trying to convey was that using coreutils/bsdmainutils was complicated. Typed data is easy to work with, but creating a sophisticated unix pipes 2.0 is not. No matter how complicated you think coreutils/bsdmainutils mastery is, you have to admit its a lot easier than building unix pipes 2.0.
If you throw in numutils and moreutils you can go nuts with columns of data. What tasks would you like to accomplish on the command line with columns of typed data and pipes 2.0?
[1] On a side note I was surprised that we had different conceptions of what black magic. I was going for evil, nefarious, and/or unorthodox. Have I been using the term wrong? That's an honest question, it would not surprise me if I have been oblivious.
I don't expect every user to create unix pipes 2.0, so the difficulty of that is not really what needs to be compared. It will only have to be done once.
And once this is done any user can avoid having to painstakingly construct pipelines that try to cut out the right columns to treat as numbers, or avoid all the problems parsing strings that may contain spaces or other control characters. You can do an operation like:
filter out all processes with %cpu > 20 with uid > 1000 and sort by second cmdline arg as:
Obviously a made up example, but something like this is easy to read and write, whereas something working on tabular ascii data would be quite long and complicated.
As per black magic, I have about the same interpretation as you. I didn't really misunderstand it to be about how complicated it was. However, "black magic" certainly has a feel of "you should not do this", and arguing that you can then use that in order to do something which could instead be simple and obvious in a typed system seems kind of weird.
Your example would be about the same length with awk and sort, with the only caveat that you need to figure out the field numbers, and the upside that I can trust the tools are available pretty much everywhere.
First you have to handle the header specially (want it in the result but not in the comparisons).
In order to compare by uid you need numeric uids (-n), but that means you can't also get the readable username, so you need a custom output format.
Then you need to ensure the output format is such that nothing with possible spaces or control chars can end up in a column before the data you're looking at, as then finding the right column is hard.
Even then, extracting the first command line arg like in the example will fail in the case of a binary name that has a space in it (as there is no way to know which spaces in the commandline corresponds to actual spaces in the arguments or just delimiters).
As you admit your system is not simple to implement but you are correct the onus is on you to create pipes 2.0. But it also means that every other developer is going to have to implement pipes 2.0 compliant output for their program. Unless pipes 2.0 is going to auto-identify everything in addition to nulls?
Yes, the weak link here is obviously getting all kinds of output into the pipes 2.0 format. Triggering such output via format negotiation is possible, but you still would have to add support for actually outputting it.
If you throw in numutils and moreutils you can go nuts with columns of data. What tasks would you like to accomplish on the command line with columns of typed data and pipes 2.0?
[1] On a side note I was surprised that we had different conceptions of what black magic. I was going for evil, nefarious, and/or unorthodox. Have I been using the term wrong? That's an honest question, it would not surprise me if I have been oblivious.