如何避免 bash 指令替代删除换行符?

为了加快 bash 脚本的执行速度,我希望使用指令替代将命令的结果保存在一个变量中,但是指令替代用一个空格替换了 0x0A换行符。例如:

a=`df -H`

或者

a=$( df -H )

当我想进一步处理 $a时,换行符被一个空格替换,所有的行现在都在一行上,这样更难于进行 grep:

echo $a

避免换行符被指令替代删除的简单技巧是什么?

34307 次浏览

Non-trailing newlines are not removed

The newlines you are looking for are there, you just don't see them, because you use echo without quoting the variable.

Validation:

$ a=$( df -H )
$ echo $a
Filesystem Size Used Avail Use% Mounted on /dev/sda3 276G 50G 213G 19% / udev 2.1G 4.1k 2.1G 1% /dev tmpfs 832M 820k 832M 1% /run none 5.3M 0 5.3M 0% /run/lock none 2.1G 320k 2.1G 1% /run/shm
$ echo "$a"
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda3       276G   50G  213G  19% /
udev            2.1G  4.1k  2.1G   1% /dev
tmpfs           832M  820k  832M   1% /run
none            5.3M     0  5.3M   0% /run/lock
none            2.1G  320k  2.1G   1% /run/shm
$

Trailing newlines are removed

As @user4815162342 correctly pointed out, although newlines within the output are not removed, trailing newlines are removed with command substitution. See experiment below:

$ a=$'test\n\n'
$ echo "$a"
test




$ b=$(echo "$a")
$ echo "$b"
test
$

In most cases this does not matter, because echo will add the removed newline (unless it is invoked with the -n option), but there are some edge cases where there are more that one trailing newlines in the output of a program, and they are significant for some reason.

Workarounds

1. Add dummy character

In these case, as @Scrutinizer mentioned, you can use the following workaround:

$ a=$(printf 'test\n\n'; printf x); a=${a%x}
$ echo "$a"
test




$

Explanation: Character x is added to the output (using printf x), after the newlines. Since the newlines are not trailing any more, they are not removed by the command substitution. The next step is to remove the x we added, using the % operator in ${a%x}. Now we have the original output, with all newlines present!!!

2. Read using process substitution

Instead of using command substitution to assign the output of a program to variable, we can instead use process substitution to feed the output of the program to the read built-in command (credit to @ormaaj). Process substitution preserves all newlines. Reading the output to a variable is a bit tricky, but you can do it like this:

$ IFS= read -rd '' var < <( printf 'test\n\n' )
$ echo "$var"
test




$

Explanation:

  • We set the internal field separator for the read command to null, with IFS=. Otherwise read would not assign the entire output to var, but only the first token.
  • We invoke read with options -rd ''. The r is for preventing the backslash to act as a special character, and with d '' set the delimiter to nothing, so that read reads the entire output, instead of just the first line.

3. Read from a pipe

Instead of using command or process substitution to assign the output of a program to variable, we can instead pipe the output of the program to the read command (credit to @ormaaj). Piping also preserves all newlines. Note however, that this time we set the lastpipe shell optional behavior, using the shopt builtin. This is required, so that the read command is executed in the current shell environment. Otherwise, the variable will be assigned in a subshell, and it will not be accessible from the rest of the script.

$ cat test.sh
#!/bin/bash
shopt -s lastpipe
printf "test\n\n" | IFS= read -rd '' var
echo "$var"
$ ./test.sh
test




$

I was trying to wrap my head around this because I was using bash to stream in the result of running the interpreter on an F# script. After some trial and error, this turned out to solve the problem:

$ cat fsi.ch
#!/bin/bash
echo "$(fsharpi --quiet --exec --nologo $1)"


$ fsi.ch messages.fsx
Welcome to my program. Choose from the menu:
new | show | remove

Assuming, of course that you need to run a terminal program. Hope this helps.

Another "neat trick" is to use the carriage return character, which prevents the newline from being stripped but doesn't add anything to the output:

$ my_func_1 () {
>     echo "This newline is squashed"
> }
$ my_func_2 () {
>     echo "This newline is not squashed"
>     echo -n $'\r'
> }
$ echo -n "$(my_func_1)" && echo -n "$(my_func_2)" && echo done
This newline is squashedThis newline is not squashed
done
$

But buyer beware: as mentioned in the comments this can work nicely for output that is simply going to the terminal, but if you are passing this on to another process you might confuse it as it probably won't be expecting the weird terminating '\r'.