I find myself always having to arrow up and going backwards through the command line to change one part of an earlier command that is pipped to a later grep or head or whatever command.

For a crude example: searching dmesg for a given string and only wanting the last 5 occurrences.

I would do the above such that:

dmesg | grep -i USB | tail -n 5

but then if I want to change the search term or what I am searching for or the source, I am waiting for my cursor to backtrace to the relevant part of the line. I'd like to move that to the end of the line such that the above example could be represented as (I know this is incorrect):

tail -n 5 < dmesg | grep -i USB (and I could then search for sda)

as I said, this is a crude example and is indicative of the sort of thing I'd LIKE to do, but not actually what I want to do, ie. this isn't about dmesg or the use of grep and tail, but how to interact between these programs placing the "variable" at the end.

This could be further exampled on getting the seek time on a DNS enquiry such that:

dig google.com | grep msec (let's move google.com to the end so that I can then test another domain) via
grep msec < dig google.com

again, poor examples. I am talking more about when I have input/outputs flowing between multiple programs

I do admit I only have a basic understanding of piping and redirection which could be summarised as:

  • piping (|) using the stdout of one program to feed into the stdin of another
  • redirect (>) taking the stdout of one program and feeding it into a file (aside: could it be redirected elsewhere other than the only obvious ones I can think of
    > /dev/null or
    > /dev/sda or
    > {said file}
    whereas piping is program output as program input going left to right.
  • redirect (<) (which I call "file as input") taking the contents of a file and using those as if they'd been typed at the command line
  • Are you talking about running the same command often, but with one part (not at the end) that varies? Or about editing the command you've just run to change one thing in the middle? Or something else? I can't work out what the general thing is that you're after, from your examples. – Gilles Mar 6 '16 at 16:26
  • both of those really. It's usually as I'm trying to search for something or (as was recently the case) take two files sort them together on one field, pipe that thru grep, run a uniq on particular characters over that then sort that by a different field with a different delimiter, but then I'd want to change the grep term. In the end I solved that particular one by making a script file which took the grep term as an argument. Last nights example was similar but I'm testing a DNS and needed to keep changing the url (since it was now cached) – Madivad Mar 7 '16 at 1:56

I'm often annoyed by the same thing, and there are a few ways of dealing with it:

  1. Just accept it, sometimes it's not worth spending any time to work around it.

  2. as PSkocik mentioned, you can use e.g. tail -n 5 <(dmesg | grep USB)

  3. Become more adept with your shell's editing capabilities - e.g. in bash (and many other programs that use readline) you can use CtrlA to get to beginning of line, ESCf and ESCb to move forward and back a "word", and Ctrl-XCtrl-E to edit the current line in $EDITOR (e.g. vim)

There are many more editing commands available and readline is fully documented in .info files. On a Debian system, install the readline-doc package. Other distros may include the documentation in the readline package itself or may separate it as Debian does.

I also recommend installing and using pinfo for a more lynx web-browser-like experience (IMO the GNU info browser is ghastly and almost unusable). If it's not already in your distro, you can find it at http://pinfo.alioth.debian.org/

  1. readline also has a vi mode for editing (the default is an emacs-like mode), which some people prefer.

  2. In simple cases, you can use quick substitution: e.g. if the last command you entered was:

dmesg | grep -i USB | tail -n 5

then typing ^USB^sda^Enter would result in this being executed:

dmesg | grep -i sda | tail -n 5

For more details on this, see man bash and search for HISTORY EXPANSION, especially the section Event Designators.

and yes, doing this repeatedly also becomes annoying.

  1. For more complex cases, the best solution is to write a shell script or function that does what you want and run that instead of your long complicated pipeline of commands.

e.g. write a shell script called dmesg-grep that looks something like this:

#! /bin/bash

# regexp to search for is arg 1.  needs to be an extended regexp
# because we're using grep -E aka egrep.

# number of lines to output is optional arg 2 (default 5)

dmesg | grep -iE "$re" | tail -n "$lines"

Then you can just run dmesg-grep usb or dmesg-grep sda.

If you do this a lot, make a bin subdirectory of your home directory and add ~/bin to your default PATH (e.g. in ~/.bash_profile or ~/.bashrc) and save your scripts in there.

  • I do think your first point is most apt! (deal with it lol). I knoew about CTRL+A, but not the ESC sequences. Thx. #5 is PERFECT, again learning new things. Big thanks. I've done #6 but that (your version) is even better. Although @PSkocik did come up with an acceptable answer first, I am going to go for this for it's completeness. Thanks very much :) – Madivad Mar 7 '16 at 2:21
  • BTW, I forgot to mention that bash also has .info documentation. Depending on your distro, you might need to install pinfo and bash-doc packages before you can run pinfo bash. It has an entire chapter called Command Line Editing. Also BTW, I use all of the above methods (except readline's vi-mode) at different times. – cas Mar 7 '16 at 2:31

In shell that support <() (bash, ksh, zsh), you can transform

$ cmd1 | cmd2 


$ cmd2 < <(cmd1)

These two should be equivalent in terms of piping. (However, I think there are some minor differences in terms of how the two forms handle things like process group and signals).

For better command line editing, I recommend you explore your shell's facilities for that. E.g., bash has set -o vi which allows you to use vi-like shortcuts or switch into vim directly to edit your command more efficiently. The default set -o emacs uses emacs shortcuts (search man bash for readline to learn more about them (/readline)).

  • I was so close, I'd tried a double < but I had a space in there and then spaces within as well. As both you and cas have mentioned, I do think it's an example of learning my shell better (I use bash) with all default settings. – Madivad Mar 7 '16 at 2:03

You may think this is a sideways answer, but often the solution is to simply use variables in your command line. They are not just for shell scripts. Eg you are developing a complex "one-liner" by trial-and-error and have:

 $ some complex command using value 23 at this point

then you want to vary just one value in the middle of the command line. A single edit and then the line becomes easy to recall without it needing any further editing:

 $ a=23
 $ some complex command using value "$a" at this point
 $ a=37
 $ a=42

It's even better when you need $a in 2 or more places in the command, such as for a directory or filename.

  • I have to admit, command line variables are not something that I've ever given a lot of thought to. I have experienced it playing with other command line "structures" including python and mysql, but have always viewed the command line as being "outside" that scope. Thanks for reminding me that that is not the case! – Madivad Mar 7 '16 at 13:06

Your Answer

