This week-of-unix-tools is intended to be a high concentration of
information with little fluff. I'll be covering only GNU versions of the
tools, for the sake of choosing only one version for sanity sake.
xargs is used to take input to generate a list of arguments and run a
program with those arguments. GNU xargs is extremely useful. You can:
- Use xargs as a 'threadpool' for parallel processing of input.
- Take a list of things on input, and run a command with one thing
as the argument, for all things input.
- Reformat word lists.
No "and then".
xargs [flags] [command [args]]
'command' is optional. If unspecified, it defaults to '/bin/echo' or
something very similar.
- -t
- trace. Show the commands being run before they are executed
- -P{N}
- Specify number (where {N} is) of forked processes for parallelism; ie, -P5 for 5 processes.
- -n{N}
- Call the command with {N} arguments
- -I{REPLSTR}
- Specify that '{REPLSTR}' will be replaced with the argument
Find some files and delete them.
% mkdir i; touch i/a.png i/b.png i/c.png i/otherfile
% find ./i/ -name '*.png' | xargs
./i/a.png ./i/b.png ./i/c.png
# Default action above was to echo, so it output as if you had typed:
# echo ./i/a.png ./i/b.png ./i/c.png
# Actually delete the png files:
% find ./i/ -name '*.png' | xargs rm
% ls ./i
otherfile
Sometimes the "thing" you want to do with this argument list is not just
one command. You know how express what you need in shell, but you don't
know how to get xargs to play nice? Not to fear. Simply use "sh" as your
command, like this example:
Rename *.foo -> *.bar
% touch a.foo b.foo c.foo
% ls *.foo | xargs -n1 -I@ sh -c 'x="@"; mv $x ${x%.foo}.bar'
# ${x%.foo} means remove '.foo' from the end of $x.
% ls *.bar
a.bar b.bar c.bar
This example shows using replacement with the character '@'. This means
any instance of '@' in the command string will be replaced with the
current value (a.foo, for example).
This example expects 'somehosts' to a line-delimited list of hosts to connect to.
% echo kenya scorn | tr ' ' '\n' > /tmp/somehosts
% cat /tmp/somehosts \
| xargs -P10 -I"HOST" -n1 ssh HOST uptime
1:28AM up 11 days, 3:44, 4 users, load averages: 0.13, 0.08, 0.02
4:28AM up 2 days, 18:54, 10 users, load averages: 0.10, 0.05, 0.05
There's a slight problem with the above invocation. You don't know what
host is outputting what data! This becomes much more clear of a problem
when you aren't executing 'uptime' but instead are doing something that
outputs many lines.
An easy solution is to use sed (or awk, et al) to prefix everything with the hostname.
% cat /tmp/somehosts \
| xargs -P10 -I"HOST" -n1 sh -c 'ssh HOST uptime | sed -e "s/^/HOST: /"'
kenya: 1:36AM up 11 days, 3:52, 4 users, load averages: 0.07, 0.07, 0.03
scorn: 4:36AM up 2 days, 19:03, 10 users, load averages: 0.01, 0.02, 0.02
