我如何执行任何命令编辑其文件(参数)“就地”使用 bash?

我有一个 temp.txt 文件,我想用 bash 中的 sort命令对它进行排序。

我希望将排序后的结果替换为原始文件。

例如,这不起作用(我得到的是一个空文件) :

sortx temp.txt > temp.txt

这是否可以在一行中完成,而不需要复制到临时文件?


编辑: -o选项对于 sort来说非常酷。我在问题中使用了 sort作为例子。我在使用其他命令时遇到了同样的问题:

uniq temp.txt > temp.txt.

有更好的通用解决方案吗?

61581 次浏览

Read up on the non-interactive editor, ex.

Here you go, one line:

sort temp.txt > temp.txt.sort && mv temp.txt.sort temp.txt

Technically there's no copying to a temporary file, and the 'mv' command should be instant.

sort temp.txt -o temp.txt

Use the argument --output= or -o

Just tried on FreeBSD:

sort temp.txt -otemp.txt

If you insist on using the sort program, you have to use a intermediate file -- I don't think sort has an option for sorting in memory. Any other trick with stdin/stdout will fail unless you can guarantee that the buffer size for sort's stdin is big enough to fit the entire file.

Edit: shame on me. sort temp.txt -o temp.txt works excellent.

Many have mentioned the -o option. Here is the man page part.

From the man page:

   -o output-file
Write output to output-file instead of to the  standard  output.
If  output-file  is  one of the input files, sort copies it to a
temporary file before sorting and writing the output to  output-
file.

A sort needs to see all input before it can start to output. For this reason, the sort program can easily offer an option to modify a file in-place:

sort temp.txt -o temp.txt

Specifically, the documentation of GNU sort says:

Normally, sort reads all input before opening output-file, so you can safely sort a file in place by using commands like sort -o F F and cat F | sort -o F. However, sort with --merge (-m) can open the output file before reading all input, so a command like cat F | sort -m -o F - G is not safe as sort might start writing F before cat is done reading it.

While the documentation of BSD sort says:

If [the] output-file is one of the input files, sort copies it to a temporary file before sorting and writing the output to [the] output-file.

Commands such as uniq can start writing output before they finish reading the input. These commands typically do not support in-place editing (and it would be harder for them to support this feature).

You typically work around this with a temporary file, or if you absolutely want to avoid having an intermediate file, you could use a buffer to store the complete result before writing it out. For example, with perl:

uniq temp.txt | perl -e 'undef $/; $_ = <>; open(OUT,">temp.txt"); print OUT;'

Here, the perl part reads the complete output from uniq in variable $_ and then overwrites the original file with this data. You could do the same in the scripting language of your choice, perhaps even in Bash. But note that it will need enough memory to store the entire file, this is not advisable when working with large files.

This would be highly memory constrained, but you could use awk to store the intermediate data in memory, and then write it back out.

uniq temp.txt | awk '{line[i++] = $0}END{for(j=0;j<i;j++){print line[j]}}' > temp.txt

To add the uniq capability, what are the downsides to:

sort inputfile | uniq | sort -o inputfile

Here's a more general approach, works with uniq, sort and whatnot.

{ rm file && uniq > file; } < file

Tobu's comment on sponge warrants being an answer in its own right.

To quote from the moreutils homepage:

Probably the most general purpose tool in moreutils so far is sponge(1), which lets you do things like this:

% sed "s/root/toor/" /etc/passwd | grep -v joey | sponge /etc/passwd

However, sponge suffers from the same problem Steve Jessop comments on here. If any of the commands in the pipeline before sponge fail, then the original file will be written over.

$ mistyped_command my-important-file | sponge my-important-file
mistyped-command: command not found

Uh-oh, my-important-file is gone.

I like the sort file -o file answer but don't want to type the same file name twice.

Using BASH history expansion:

$ sort file -o !#^

grabs the current line's first arg when you press enter.

A unique sort in-place:

$ sort -u -o file !#$

grabs the last arg in the current line.

An alternative to sponge with the more common sed:

sed -ni r<(command file) file

It works for any command (sort, uniq, tac, ...) and uses the very well known sed's -i option (edit files in-place).

Warning: Try command file first because editing files in-place is not safe by nature.


Explanation

Firstly, you're telling sed not to print the (original) lines (-n option), and with the help of the sed's r command and bash's Process Substitution, the generated content by <(command file) will be the output saved in place.


Making things even easier

You can wrap this solution into a function:

ip_cmd() { # in place command
CMD=${1:?You must specify a command}
FILE=${2:?You must specify a file}
sed -ni r<("$CMD" "$FILE") "$FILE"
}

Example

$ cat file
d
b
c
b
a


$ ip_cmd sort file
$ cat file
a
b
b
c
d


$ ip_cmd uniq file
$ cat file
a
b
c
d


$ ip_cmd tac file
$ cat file
d
c
b
a


$ ip_cmd
bash: 1: You must specify a command
$ ip_cmd uniq
bash: 2: You must specify a file

Another solution:

uniq file 1<> file