Oops sorry about that. Just found out about this wonderful program from this HN comment here(1). Should have been more careful to check it through algolia before posting though.
lintshell is an early prototype of a shell linter based on the Morbig trustworthy static parser for POSIX shell, based on the Why3 platform for deductive program verification.
The best best part of Shellcheck is that the errors it produces have a short code that can be looked up in their wiki. The wiki explains what you did, why it’s bad, and how to fix it. Using Shellcheck teaches you to write better Bash, rather than just yelling at you with inscrutable linter violations.
If you use iTerm2 (or I think this feature is available in other terminal software) you can use "Smart Selections" to transform any ShellCheck errors into links to the wiki.
1. Create a Smart Selection with the regular expression "SC\d+"
They should add the wiki URLs as links to those error codes then (if supported by the terminal) - GCC has done that recently for the -W flags shown in warning or error messages.
BASH is not an appropriate standard to use in writing a script.
ShellCheck recognizes three hashbangs, POSIX, BASH, and the Korn shell (as is explained in the manual page). I would argue and assert that BASH is the least relevant of the three.
I have occasionally needed to write to the POSIX standard, and remove advanced features from my scripts. This is most commonly needed for BusyBox (including Windows) and Ubuntu/Debian DASH. It is important to understand what is not in POSIX, for the time when an advanced shell isn't available. Please browse the POSIX definition, reported by a google of "POSIX shell":
The Korn shell lived and diverged for many years, beginning with ksh88 (issued before the first BASH release in 1989). David Korn retired, and AT&T was not pleased with ksh93 changes implemented by outsiders who impacted both compatibility and performance, and thus ksh93 was rewound.
Cygwin, notibly, only implements mksh for Windows. The mksh binary on all platforms is a fraction of the size of BASH (check the output of the "size" command if an "ls" does not persuade you).
However, the MirBSD Korn shell is one of the most lively shells for development activity, the latest release being in October of 2020. It is also on every Android phone (chosen as the system shell by Google), so its deployment is massive.
When I am purposely writing above the POSIX shell, I choose mksh from MirBSD as my standard. I do not think the BASH innovations to be well-thought.
> ShellCheck recognizes three hashbangs, POSIX, BASH, and the Korn shell (as is explained in the manual page). I would argue and assert that BASH is the least relevant of the three.
Depends on how you call "relevant", but I disagree regardless. If you're writing a shell script, you either need to be hard-core portable, in which case POSIX is your only option, or you don't, in which case you're almost certainly targeting a GNU/Linux system with BASH installed. Of course, if you know that you're targeting a specific system with a specific shell installed then by all means; if I were primarily an OpenBSD dev, I'd happily use ksh. But in general, the (unix) world is always going to give you POSIX, and BASH is the second most widely available option. And bluntly, for all that I love shell, if you're trying to argue for the most elegant or sophisticated language, sh/BASH/ksh is irrelevant because all of them kinda suck outside of their niche.
Oracle's list of "required packages" is annoyingly large but I cannot find a particular shell amongst them. The installation guide appears to have a paragraph devoted to each of bash, "bourne or korn", and C shells.
> However, the MirBSD Korn shell is one of the most lively shells for development activity, the latest release being in October of 2020.
The OpenBSD Korn shell is actively developed and there is a portable version of it available. The latest version which reflects the shell as available in OpenBSD 6.9 was released 3 days ago. https://github.com/ibara/oksh
> the MirBSD Korn shell is one of the most lively shells for development activity, the latest release being in October of 2020
I.e., you suggested the measure is release recency, not cadence. And after I pointed out that bash (sorry, BASH) has a more-recent release, you're moving the goalposts.
Anyways. It doesn't matter. Love the shell(s) you love; hate the shell(s) you hate. You don't need a quantitative basis. They can release as often or as rarely as they like with little impact on whether they're good shells or not.
As much of the scripting that I am required to maintain emerged from HP-UX, the "POSIX" shell on my legacy platform emerged from Korn (and I have the manual page to prove it - the text of the HP-UX "POSIX" shell manual page includes arrays and a number of other Korn features).
The reason that the POSIX shell is not Korn is the existence of Microsoft Xenix, where the maximum size of the code/text segment is 64k.
Debian chose the POSIX standard for precisely this reason: a minimal text segment.
One particular thing that I like about Korn is this:
Maybe it says something about the people I work with, but I've never seen a fully valid shell script, over 20 lines long, that handled spaces in paths and command errors properly, that didn't first go through ShellCheck. I've noticed that hardcore *nix people are usually some of the worst offenders, with their phobia of spaces, from years of broken shell scripts, completely preventing them from even thinking about the possibility of a space in a path, with some considering the space the bug, rather than its handling.
I've found that the best way to introduce someone to ShellCheck is to run it on their script, right in front of them, with most saying something like "How did I not know about this?". Linters are great!
The kind of issues shellcheck catches are mostly of the "didn't learn shell (and didn't believe that shell was a language that had to be learned)" kind. Or to put it another way, once you have a good hang of the shell language's peculiarities (which differ from most general-purpose languages, but are fairly self-consistent with a few high-frequency exceptions) ... well, less of your bugs are ones that shellcheck can catch.
I consider it an example of the general trend against learning domain-specific languages.
Did your regex get munged there? \f (form feed) is something I've never come across (well maybe back in BBC basic?) - what's the context of removing it/them (I'm guessing only pairs \f\f) from 7M files?
There are a few simple rules that, once you learn them, eliminate those kinds of problems. The main one is always use double-quotes around variable dereferences. The second is learning a few patterns, like find ... -print0 | xargs -0 ...
But even after you've learned those simple rules, shellcheck is still useful. It will find fewer problems, but that's a good thing, that means you learned something :-).
I can and do write shell scripts all the time which properly handle spaces, without ShellCheck. Not to say that it isn’t good to use, especially in a professional environment, but it’s not that hard. For parallel processing, xargs -0 and find -0 are your friends!
Agreed. I have it built into my vim editor, running after saves. Not only does it help catch errors, it catches potential errors, poor formatting and has become an amazing learning tool. Bash isn’t my favorite language, but many of the issues I had with it are no longer relevant.
ShellCheck and templates from the Shell Field Guide[1] is what keeps my bank running.
Rust cmd_lib[2] is tempting with safety, though tougher maintainability.
This and `pandoc` are solid proof that Haskell is a superb language for processing structured text (be it for compilation, or translation, or checking).
Unfortunately, last I checked, shellcheck is not compatible with M1 macs yet. I love the tool so much that I while I wait for compatibility, I manually check my shell scripts at the web UI.
Yeah, there are still issues with Haskell and the new Mac architecture. Pandoc and ShellCheck are two of the essential tools affected. In the case of ShellCheck, I just downloaded an x86 binary from the GitHub Releases page and put it in /usr/local/bin. Good enough for the time being.
ShellCheck is a great tool! If you're looking for an easy way to run static analysis on .sh files continuously, you might want to look at DeepSource. ShellCheck is supported. https://deepsource.io/gh/deepsourcelabs/demo-shell/issues/
One of the biggest value of shellcheck is that all of its errors are documented on their wiki, and easily searchable. If you don't understand why there is an error, the wiki page will tell you why an idiom is bad, and in some cases point you to examples of how it might fail, and you'll learn something new.
I wish the wiki was packaged into the tool for offline consumption.
I’ve written some pretty horrible Bash applications such as a load testing tool for a blockchain, probably hitting > 2kloc. ShellCheck and ShUnit made that possible.
Because the =~ and > and < operators aren't defined for [ in POSIX.
Also ShellCheck suggests similar improvement only when your shebang says that bash is the expected interpreter instead of sh (or other ways of denoting POSIX compat).
I can highly recommend this tool. A few years ago, I ran it over our shell scripts and it gave us many good recommendations and even found some bugs. I learned a lot about shell scripting during that time. Really, I cannot recommend it enough, what a fantastic tool.
(We mark stories as dupes if they've had significant attention in the last year or so).
Previous threads, for those who are curious:
Lessons learned from writing ShellCheck - https://news.ycombinator.com/item?id=22279585 - Feb 2020 (46 comments)
Shellcheck: a static analysis tool for shell scripts - https://news.ycombinator.com/item?id=9001931 - Feb 2015 (46 comments)
ShellCheck: a static analysis and linting tool for sh/bash scripts - https://news.ycombinator.com/item?id=8777705 - Dec 2014 (20 comments)
ShellCheck – Online shell script analyzer - https://news.ycombinator.com/item?id=8182745 - Aug 2014 (14 comments)