强制在 bash 脚本仍在运行时刷新输出到文件

小开

This isn't a function of bash, as all the shell does is open the file in question and then pass the file descriptor as the standard output of the script. What you need to do is make sure output is flushed from your script more frequently than you currently are.

In Perl for example, this could be accomplished by setting:

$| = 1;

See perlvar for more information on this.

小开

最佳答案

bash itself will never actually write any output to your log file. Instead, the commands it invokes as part of the script will each individually write output and flush whenever they feel like it. So your question is really how to force the commands within the bash script to flush, and that depends on what they are.

小开

I don't know if it would work, but what about calling sync?

小开

well like it or not this is how redirection works.

In your case the output (meaning your script has finished) of your script redirected to that file.

What you want to do is add those redirections in your script.

小开

I had this problem with a background process in Mac OS X using the StartupItems. This is how I solve it:

If I make sudo ps aux I can see that mytool is launched.

I found that (due to buffering) when Mac OS X shuts down mytool never transfers the output to the sed command. However, if I execute sudo killall mytool, then mytool transfers the output to the sed command. Hence, I added a stop case to the StartupItems that is executed when Mac OS X shuts down:

start)
if [ -x /sw/sbin/mytool ]; then
# run the daemon
ConsoleMessage "Starting mytool"
(mytool | sed .... >> myfile.txt) &
fi
;;
stop)
ConsoleMessage "Killing mytool"
killall mytool
;;

小开

Would this help?

tail -f access.log | stdbuf -oL cut -d ' ' -f1 | uniq

This will immediately display unique entries from access.log using the stdbuf utility.

小开

You can use tee to write to the file without the need for flushing.

/homedir/MyScript 2>&1 | tee some_log.log > /dev/null

小开

script -c <PROGRAM> -f OUTPUT.txt

Key is -f. Quote from man script:

-f, --flush
Flush output after each write.  This is nice for telecooperation: one person
does 'mkfifo foo; script -f foo', and another can supervise real-time what is
being done using 'cat foo'.

Run in background:

nohup script -c <PROGRAM> -f OUTPUT.txt

小开

How just spotted here the problem is that you have to wait that the programs that you run from your script finish their jobs.
If in your script you run program in background you can try something more.

In general a call to sync before you exit allows to flush file system buffers and can help a little.

If in the script you start some programs in background (&), you can wait that they finish before you exit from the script. To have an idea about how it can function you can see below

#!/bin/bash
#... some stuffs ...
program_1 &          # here you start a program 1 in background
PID_PROGRAM_1=${!}   # here you remember its PID
#... some other stuffs ...
program_2 &          # here you start a program 2 in background
wait ${!}            # You wait it finish not really useful here
#... some other stuffs ...
daemon_1 &           # We will not wait it will finish
program_3 &          # here you start a program 1 in background
PID_PROGRAM_3=${!}   # here you remember its PID
#... last other stuffs ...
sync
wait $PID_PROGRAM_1
wait $PID_PROGRAM_3  # program 2 is just ended
# ...

Since wait works with jobs as well as with PID numbers a lazy solution should be to put at the end of the script

for job in `jobs -p`
do
wait $job
done

More difficult is the situation if you run something that run something else in background because you have to search and wait (if it is the case) the end of all the child process: for example if you run a daemon probably it is not the case to wait it finishes :-).

Note:

wait ${!} means "wait till the last background process is completed" where $! is the PID of the last background process. So to put wait ${!} just after program_2 & is equivalent to execute directly program_2 without sending it in background with &

From the help of wait:

Syntax
wait [n ...]
Key
n A process ID or a job specification

小开

I found a solution to this here. Using the OP's example you basically run

stdbuf -oL /homedir/MyScript &> some_log.log

and then the buffer gets flushed after each line of output. I often combine this with nohup to run long jobs on a remote machine.

stdbuf -oL nohup /homedir/MyScript &> some_log.log

This way your process doesn't get cancelled when you log out.

小开

Buffering of output depends on how your program /homedir/MyScript is implemented. If you find that output is getting buffered, you have to force it in your implementation. For example, use sys.stdout.flush() if it's a python program or use fflush(stdout) if it's a C program.

小开

Thanks @user3258569, script is maybe the only thing that works in busybox!

The shell was freezing for me after it, though. Looking for the cause, I found these big red warnings "don't use in a non-interactive shells" in script manual page:

script is primarily designed for interactive terminal sessions. When stdin is not a terminal (for example: echo foo | script), then the session can hang, because the interactive shell within the script session misses EOF and script has no clue when to close the session. See the NOTES section for more information.

True. script -c "make_hay" -f /dev/null | grep "needle" was freezing the shell for me.

Countrary to the warning, I thought that echo "make_hay" | script WILL pass a EOF, so I tried

echo "make_hay; exit" | script -f /dev/null | grep 'needle'

and it worked!

Note the warnings in the man page. This may not work for you.

小开

alternative to stdbuf is awk '{print} END {fflush()}' I wish there were a bash builtin to do this. Normally it shouldn't be necessary, but with older versions there might be bash synchronization bugs on file descriptors.