Perl was created to do this stuff. Perl is like duct tape to hold together everything. Most people call it ugly because of that but Perl is the best language for quick dirty one liners that work. Only problem is that if you look at it later you will have problems understanding it.

Agree. Perl is terrific for quick text processing. You can make surprisingly powerful one-liners.

In most platforms you can install the rc shell from plan9. It's what I use exclusively for my shell scripts. It is only when I want to share some script with other people that I consider using /bin/sh, and even then I've gone for rc nevertheless. Here you can read about it: http://doc.cat-v.org/plan_9/4th_edition/papers/rc

Thanks for the suggestion. I've also been after a saner shell, and have been disappointed in one way or another with the approach of either using another language idiomatically (Python, Scheme, Haskell, Scala, etc.), since running commands, piping, etc. are all rather awkward; or using such languages with embedded shell-like libraries, which seem to have awkward edge-cases.

As a "real" shell, it looks like rc maintains the command, file and piping niceties of bash, whilst avoiding the edge-cases of shell-like embeddings.

In Perl, IPC::Run [1], while a bit on the quirky side sometimes, can do that sort of thing.

    use IPC::Run qw(run);
    IPC::Run::run(["echo", "abc"], "|", 
                  ["tr", "a-z", "A-Z"], "|", 
                  ["cat"],
                  \$out);
    print "out: $out";

will yield

    out: ABC

Useless use of cat here just to make a point, of course.

It's easy to use Perl strings as input or output, or files, and there's also ways to interact with streams as they go along.

Generally speaking the slightly more verbose interaction with shell is made up for by the savings when you can use Perl directly to do something rather than spawn a shell process for something simple. (Shell composition can be powerful, yes, but on the other hand spawning a full process and setting up a pipe for things like "tr" or "wc" is often just silly.)

I also personally believe there's a win in the syntax; you may say "What? 'tr a-z A-Z' is way more convenient than '["tr", "a-z", "A-Z"]' but I say one of the biggest and most pervasive errors in shell is to incorrectly set up the arguments by having something interpolated in incorrectly. Having an unambiguous syntax for separating arguments has often made it much safer for me to write the code that has to use the shell. Used correctly it can even be made to work in the face of user-supplied input, something that should generally not be combined with any form of implicit argument separation. (Although bear in mind that "used correctly" encompasses more than just "separating the arguments correctly.")

I am also NOT claiming exclusivity; this is just the thing I know off the top of my head. I'm sure the other scripting languages have nice libraries, too. Though watch out for them being too nice. There's a bit of an impedance mismatch between shell and scripting language. Anything that smooths it over too well is probably either giving up critical features or introducing subtle bugs, which can become security bugs in the face of arbitrary user input.

[1]: http://search.cpan.org/~toddr/IPC-Run-0.96/lib/IPC/Run.pm

Perl of course, even if the adjective "proper" seems weird for Perl ;-)

As a rule of thumb, if a script starts to need arrays or to handle spaces in filename, I migrate it to Perl.

Error handling is also easier and more natural in Perl.

Just found out that xon.sh http://xon.sh/tutorial . html does this.

Tcl lacks the piping syntax by default, but there are various ways to implement it: http://wiki.tcl.tk/17419