I set up my bash config to store history separately for each working directory (ie separate history files that I store under a $HOME/.bash_history_d directory) – so as I cd into a project directory, the history is populated with the commands that I've run within that directory.
I find this super-helpful to remind me of the context for projects – cd in, then up-arrow to whatever that command is I usually run to launch this server, grep this logfile, etc.
There are some downsides – sometimes I want to run the same command in directory X and also in subdirectory X/Y, but I love doing it this way.
I use zsh, and I have a thing in my profile that skips writing any `cd` command to history. Instead, it will rewrite the command using the absolute path of the directory I `cd`'d into and write that to history.
Thus, all of the `cd` commands that get written to my history use absolute paths and are re-usable no matter where I am.
It would be cool if this sort of functionality were generalizable to all commands where you type a relative path, not just `cd`.
In fish, suggested autocompletions are contextual. For example, if I am in /home/dual_basis/projectA and I type `cd some/subdir` then a bunch of time goes by when I come back to /home/dual_basis/projectA and start typing `cd` it will suggest `cd some/subdir`, even if I have typed a bunch of other `cd` commands while in other directories.
This isn't specific to `cd` commands either. It's useful if you are `rsync`ing different directories to different places, or checkout different `git` branches, etc.
You might enjoy checking out 'fasd', which, among other things, creates an alias ('zz') with autocomplete functionality for changing to frequently-used directories.
90% of the time, the bash history is super relevant to the current context. A small amount of the time, I remember that I issued a command, but not in which directory - in these cases, I can just search the history directory, and get both the context of what the command was, and also where I was when I originally issued it.
Thanks, reading over that code was useful. In particular I realized that my Oil project (a bash-compatible shell [1]) doesn't support PROMPT_COMMAND, which tons of people appear to be using.
It seems like a weird hack but apparently it works for tons of things!
[1] Latest status: Success With the Interactive Shell http://www.oilshell.org/blog/2019/02/05.html . Oil runs a bunch of interactive programs like bash-completion, virtualenv, and git-prompt.
zsh-histdb stores history inside a sqlite database instead of a simple text file.
The good thing is that it includes context: current directory, session, date.
I use it together with ssh-suggestions which can use SQL queries to find the most relevant command. You can program rules such as: use the most frequent command in that session, if you none do it for the current directory in the last 6 months, if not, search globally.
It's a great idea. If you additionally also like working with chroots you might find containers an interesting topic.
Basically a container let's you wrap a part of your filesystem, network, process space in a separate world of its own. And in that world you have only the context related stuff like history, processes, libs etc. Additionally most container environments come with tooling that enables you to make everything reproducable, so that you can share your container with others.
The great thing is you can always live in global space, while switching between what that global space means.
This is lovely, and I should integrate it with the chronology based approach I've been using.
I keep an infinite log by appending a timestamp to the beginning of the lines written to my history file, then merging this into a growing log with an hourly cronjob that runs sort -n | uniq on the file and the new version of the shell history. This way the shell history file is just a buffer. It doesn't get too big to cause corruption or other issues with reverse search on the command line. I just grep my way through the big history file to figure out what I was doing and when.
No because every line has a timestamp and the sort is done comparing numerical value, but still I question the use of uniq, unless it's invoked with the --skip-fields option.
I do something similar, but instead of current directory I have a notion of context that is carried by an environment variable set in a GNU screen process (and therefore inherited by its children) which drives logic in my .bashrc.
Could you elaborate? I can read docs, but it's not clear to me what correspondences you have in mind.
Specifically, I'm not sure what "this" is that fish does, as I described interaction of multiple pieces of software and I don't think fish includes a terminal multiplexer?
autocompletion in fish takes into account the directory where you are at, so it will autocomplete commands ran in the same directory before other commands.
Oh, I see. I interpreted "context" as current working directory. I took a look at your .bashrc but I couldn't detemine any sort of partitioned history. Are you saying that you have something like a virtual environment (as in Python) and the bash command history is separated according to the environment?
To echo what everyone else is saying, get FZF. Most of these linux productivity things are just some dumb pet project that solves a specific problem that a specific person finds annoying. FZF is not this - it makes something you do hundreds of times a day (history and file opening on zsh, bash, fish, whatever) indisputably better. If you find yourself sitting there tapping ctrl+r multiple times like an idiot for some thing that may or may not be there, do your stubborn beardy ass a favour and install fzf.
You can implement some or all of the tips in this HN post, it'll make fzf results better.
Yup, I tried the context aware history for couple of days and it got annoying pretty fast. I have everything in one file which I can just FZF reverse search and bam. No need for infinite up arrows.
I love this blog and learned so much about the Unix toolkit and the philosophy behind it by reading it. Too bad it doesn’t seem to be much active anymore, but there’s a lot of great material in the archives that pretty much never gets old since its focused mostly on standard Unix software. The author (Tom Ryder) is an very talented writer with a clear and concise style.
A couple of plugs for your consideration:
- I integrated all of these history options in my project Sensible Bash [1], an attempt at saner Bash defaults
- I made an ebook [2] out of a series which appeared on this blog, titled “Unix as IDE”.
promptFunc() {
# right before prompting for the
# next command, save the previous
# command in a file.
echo "$(date +%Y-%m-%d--%H-%M-%S) \
$(hostname) $PWD $(history 1)" \
>> ~/.full_history
}
PROMPT_COMMAND=promptFunc
This is in addition to standard bash history, and is much more reliable. Being able to see not only what command I ran but what directory I was in and when I ran it is very useful.
I do something very similar (just saved in files by month) and it has saved my bacon a few times. I was recently asked to re-run a process that I ran years ago. It would have taken me hours to re-discover how to run the process but instead took a few minutes after grepping the commands.
Yeah, this is how I do it - although I append the hostname to the end of the filename and sync the collection between my machines so that I can grep for history on any machine.
Edit: went to check and apparently I've overcomplicated this somewhat (it's about ten years old) but it is nice if empty lines are not logged. It's an old habit to ctrl-c then hit return repeatedly if a terminal is not responding.
PROMPT_COMMAND=storehist
storehist ()
{
CMDCNT=`history 1 | sed -e 's/\w* *\(.*\)/\1/'`;
if [ "$CMDCNT" != "$LASTCMDIND" ] && [ -n "$LASTCMDIND" ]; then
DATE=`date '+%Y%b%d %H%M'`;
if [ "$LASTCMDIND" != "$CMDCNT" ]; then
echo "$DATE $HOSTNAME:$BASHTTY $CMDCNT" >> ~/.custom_history;
fi;
LASTCMDIND="$CMDCNT";
fi;
LASTCMDIND="$CMDCNT"
}
Bash's broken history is what finally got me to switch to Zsh. The history behavior you want is: 1. each line of history is appended incrementally, but 2. each shell session only has access to its own history unless you specifically reload the history into your session.
Every combination of history commands in bash didn't do the right thing[1], but Zsh's works as desired.
[1] eg. putting 'history -a' in PROMPT_COMMAND as the article recommends appends your session's entire history each time, IIRC.
Yes what I wrote doesn't exclude that. The only distinction is that you generally don't want all your sessions' histories interleaved. I.e. when you up-arrow, you only want your current session's history, without random things from other active sessions in there too. If you do want "everything you've ever done" including things from other sessions that started after your current session, it's easy to reload your history into your session. I have a 'rlh' (reload history) alias for that.
Yeah I know, I was just making a distinction from
cortesoft's "I want everything I've ever done" in my history. I was saying "so do I, except I want active sessions separate" (which as you point out, they are by default).
> [1] eg. putting history -a in PROMPT_COMMAND as the article recommends appends your session's entire history each time, IIRC.
No, only the last command. Very handy because that way you can start a
new terminal and have your current history available immediately. I
also have an alias r='history -n'. With this you can load the history
of another terminal session directly into the current session.
I have a global bash history, having set up bash/zsh to [immediately share without any annoying race conditions.][2] I use FZF mapped to CTRL+R to search through my history.
The unique thing is that I use a [python script (in my dotfiles)][1] to clean the history in a more advanced way than is possible with HISTIGNORE.
The script removes duplicates, leaving the most recent copy and removes lines matching several regexes.
If one is only interested to remove duplicates (your script does more) I've found this one liner in a StackExchange post:
awk '!x[$0]++' ~/.zsh_history
Note that this only keeps the first copy, not the most recent copy. To make it so that it only keeps the most recent, you would need to run this through `tac` twice.
This is a great idea, thanks for it! I've done similar things with using fzf to git add, git diff, and so on, but I hadn't thought of using it to pick a commit to rebase from.
I made a few changes so I figured I'd post them back here in case you're interested:
pick-commit = # equivalent to your git log... cut...
rif = "!f() { rev=$(git pick-commit); [[ $rev ]] && git ri $rev~; }; f"
"rif" = "rebase interactive fzf/fuzzyfind". The function and test just makes it exit cleanly if you escape out of the filter instead of giving "fatal: invalid upstream '~'" (and I changed it from ^ to ~ because I think ~ is usually what one should use over ^, but I'm willing to be corrected). And obviously I already have 'ri' aliased to 'rebase -i'.
While I'm here, here are my similar git aliases for adding, diffing, etc. files using fzf:
pick-status-files = "!f() { git status -z | xargs -0n1 | cut -c4- | fzi --print0 | xargs -0to git "$@"; }; f"
af = !git pick-status-files add
# and so on for diff, difftool, checkout, etc.
Oh, and 'fzi' (fuzzy-inline) is a small shell script that just calls:
>By default, Bash only records a session to the `.bash_history` file on disk when the session terminates. This means that if you crash or your session terminates improperly, you lose the history up to that point.
I kept loosing my history consistently on my new installation because of this. I usually put my laptop to sleep, and pretty much never intentionally terminate my x-session except on kernel updates. But because its a cheapo laptop with a crappy battery, I would occasionally run out of battery when asleep or it just wont wake up.
I have a fairly expensive ThinkPad and it also runs out if battery if asleep for more than about 4 days, and frequently freezes on suspend or resume when it's been both suspended and resumed docked and undocked. (Linux, of course.)
Because this comes up every so often, I'll post this link again: I wrote a little widget that stores your bash history in a sqlite db. I haven't touched it in a while because I now rely on fzf and this doesn't integrate with it, though maybe I could make it do...
I will add PROMPT_COMMAND to Oil, but I'm curious if there is something else I should add to support end-user customization.
One thing that might be interesting is to support a hook for the ! syntax, so the sqlite DB could be searched. I think that would solve some of the problems you mention in the code's comments, and maybe the fzf issue. (Although that literally just occurred to me, I don't promise to do this :) )
I haven't looked at how fzf works exactly. But if you want to chat about it please chime in on Github :)
[1] Latest status: Success With the Interactive Shellhttp://www.oilshell.org/blog/2019/02/05.html . Oil runs a bunch of interactive programs like bash-completion, virtualenv, and git-prompt!
> This is cool, somebody recently suggested that I add history to sqlite in Oil
I think a shell with an embedded sqlite, which makes sqlite accessible to scripts as shell builtins, and in which command line history is stored in a sqlite database instead of a plain text file, would be pretty cool. You could store command history (and maybe even other stuff) in something like $HOME/.shell_db As well as just a list of lines, other useful information could be stored for each line, such as current directory, timestamp, username (maybe I su to root and am running commands as root but still going into my own account's history database), hostname (if $HOME is on NFS, putting aside the issue that databases, including sqlite, don't work well over NFS due to file locking issues), etc.
> somebody recently suggested that I add history to sqlite in Oil
what a disgusting, perturbed mind! Shell history is probably the kind of data that fits better as a text file. It is naturally textual and it can never grow too long, even if you are typing commands continuously for your entire life. Please, keep the dirty hands of sql far, far away from the shell!
Ha, thanks for the feedback... Yeah I don't want the sqlite dependency by default.
The fact that somebody managed to bolt it onto bash after the fact is interesting. Oil is all about having a better/sane shell programming language, so it should have better hooks that make such customization. But yes it doesn't need to be in the core.
Yeah I agree. In Oil I'm hoping to have the best of both worlds. That is, no silly quoting and parsing of strings, but also being able to use plain tools like "grep", and probably SQL.
"As if there was no tomorrow" - exactly, because once you add another column suddenly $4 points to something completely different and you have to rewrite everything.
Not to mention $4 is completely arbitrary and it's impossible to tell without prior knowledge which field it's referencing.
The original query `SELECT pwd, cmd WHERE exit-code != 0` is both descriptive and won't break with future updates.
I've had this in my .bashrc for years. I thought it came from this post judging from my commments, but looks different, so I wonder whether it came from HN comments when this was previously posted.
## Better bash history
# Avoid duplicates
export HISTCONTROL=ignoredups:erasedups
# When the shell exits, append to the history file instead of overwriting it
shopt -s histappend
# After each command, append to the history file and reread it
export PROMPT_COMMAND="${PROMPT_COMMAND:+$PROMPT_COMMAND$'\n'}history -a; history -c; history -r"
I remember reading this article back when I was still using Bash. I've finally got the history I like only after switching to ZSH.
You can get decent history in ZSH just by setting the right options. If you want great history, you'll need to write a bit of code. Right now my history works as follows.
History is written to disk after every command. Up and Down keys go through the local history (from the same session). Ctrl+Up and Ctrl+Down go through the shared history (from all sessions). Ctrl+R also uses shared history (it's easy to add another binding for local history but I don't have it). Pressing Up/Ctrl+Up after typing something will go over history entries that have the matching prefix. For example, `git<Up>` will show the last command from the current session that starts with `git`. Here's my config: https://old.reddit.com/r/zsh/comments/bsa224/how_to_setup_a_....
In addition, my history is stored in Git. History from the local machine comes first, and from other machines second. This is fairly easy to do it Bash, too.
Using PROMPT_COMMAND to log every command in a separate timestamped log file. This allows you to use a much better format than bash history has to be in.
Easily readable to see what you've been doing and extremly efficient to grep for.
that's exactly what my setting is for history, nice post. The only difference is that I unset HISTCONTROL, don't recall why I did that.
shopt -s histappend is the default already I believe.
shopt -s cmdhist is also the default now.
HISTSIZE=-1 will be unlimited, which is fine for today's computers.
can anyone please explain how BASH history on current Ubuntu, is different when logging in from different desktop machines, to the same login ? the dot-local directory or dot-gconf directory is part of this ? consternation..
I find this super-helpful to remind me of the context for projects – cd in, then up-arrow to whatever that command is I usually run to launch this server, grep this logfile, etc.
There are some downsides – sometimes I want to run the same command in directory X and also in subdirectory X/Y, but I love doing it this way.
Anyone else done something similar?