强制 bash 展开从文件加载的字符串中的变量

我正在尝试解决如何使 bash (力?)在字符串中展开变量(从文件中加载)。

我有一个名为“ something. txt”的文件,其内容如下:

hello $FOO world

然后我就跑

export FOO=42
echo $(cat something.txt)

返回:

   hello $FOO world

即使设置了变量,它也没有展开 $FOO。我不能对文件进行 eval 或者源代码-因为它会尝试并执行它(它不是可执行的,因为它是-我只想要带有内插变量的字符串)。

有什么想法吗?

68451 次浏览

You don't want to print each line, you want to evaluate it so that Bash can perform variable substitutions.

FOO=42
while read; do
eval echo "$REPLY"
done < something.txt

See help eval or the Bash manual for more information.

you can try

echo $(eval echo $(cat something.txt))

Another approach (which seems icky, but I am putting it here anyway):

Write the contents of something.txt to a temp file, with an echo statement wrapped around it:

something=$(cat something.txt)


echo "echo \"" > temp.out
echo "$something" >> temp.out
echo "\"" >> temp.out

then source it back in to a variable:

RESULT=$(source temp.out)

and the $RESULT will have it all expanded. But it seems so wrong !


Single line solution that doesn't need temporary file :

RESULT=$(source <(echo "echo \"$(cat something.txt)\""))
#or
RESULT=$(source <(echo "echo \"$(<something.txt)\""))

Many of the answers using eval and echo kind of work, but break on various things, such as multiple lines, attempting to escaping shell meta-characters, escapes inside the template not intended to be expanded by bash, etc.

I had the same issue, and wrote this shell function, which as far as I can tell, handles everything correctly. This will still strip only trailing newlines from the template, because of bash's command substitution rules, but I've never found that to be an issue as long as everything else remains intact.

apply_shell_expansion() {
declare file="$1"
declare data=$(< "$file")
declare delimiter="__apply_shell_expansion_delimiter__"
declare command="cat <<$delimiter"$'\n'"$data"$'\n'"$delimiter"
eval "$command"
}

For example, you can use it like this with a parameters.cfg which is really a shell script that just sets variables, and a template.txt which is a template that uses those variables:

. parameters.cfg
printf "%s\n" "$(apply_shell_expansion template.txt)" > result.txt

In practice, I use this as a sort of lightweight template system.

I stumbled on what I think is THE answer to this question: the envsubst command:

echo "hello \$FOO world" > source.txt
export FOO=42
envsubst < source.txt

This outputs: hello 42 world

If you would like to continue work on the data in a file destination.txt, push this back to a file like this:

envsubst < source.txt > destination.txt

In case it's not already available in your distro, it's in the GNU package gettext.

@Rockallite

  • I wrote a little wrapper script to take care of the '$' problem.

(BTW, there is a "feature" of envsubst, explained at https://unix.stackexchange.com/a/294400/7088 for expanding only some of the variables in the input, but I agree that escaping the exceptions is much more convenient.)

Here's my script:

#! /bin/bash
## -*-Shell-Script-*-
CmdName=${0##*/}
Usage="usage: $CmdName runs envsubst, but allows '\$' to  keep variables from
being expanded.
With option   -sl   '\$' keeps the back-slash.
Default is to replace  '\$' with '$'
"


if [[ $1 = -h ]]  ;then echo -e >&2  "$Usage" ; exit 1 ;fi
if [[ $1 = -sl ]] ;then  sl='\'  ; shift ;fi


sed 's/\\\$/\${EnVsUbDolR}/g' |  EnVsUbDolR=$sl\$  envsubst  "$@"

If you only want the variable references to be expanded (an objective that I had for myself) you could do the below.

contents="$(cat something.txt)"
echo $(eval echo \"$contents\")

(The escaped quotes around $contents is key here)

Following solution:

  • allows replacing of variables which are defined

  • leaves unchanged variables placeholders which are not defined. This is especially useful during automated deployments.

  • supports replacement of variables in following formats:

    ${var_NAME}

    $var_NAME

  • reports which variables are not defined in environment and returns error code for such cases




TARGET_FILE=someFile.txt;
ERR_CNT=0;


for VARNAME in $(grep -P -o -e '\$[\{]?(\w+)*[\}]?' ${TARGET_FILE} | sort -u); do
VAR_VALUE=${!VARNAME};
VARNAME2=$(echo $VARNAME| sed -e 's|^\${||g' -e 's|}$||g' -e 's|^\$||g' );
VAR_VALUE2=${!VARNAME2};


if [ "xxx" = "xxx$VAR_VALUE2" ]; then
echo "$VARNAME is undefined ";
ERR_CNT=$((ERR_CNT+1));
else
echo "replacing $VARNAME with $VAR_VALUE2" ;
sed -i "s|$VARNAME|$VAR_VALUE2|g" ${TARGET_FILE};
fi
done


if [ ${ERR_CNT} -gt 0 ]; then
echo "Found $ERR_CNT undefined environment variables";
exit 1
fi
  1. If something.txt has only one line, a bash method, (a shorter version of Michael Neale's "icky" answer), using process & command substitution:

    FOO=42 . <(echo -e echo $(<something.txt))
    

    Output:

    hello 42 world
    

    Note that export isn't needed.

  2. If something.txt has one or more lines, a GNU sed evaluate method:

    FOO=42 sed 's/"/\\\"/g;s/.*/echo "&"/e' something.txt
    

The following works: bash -c "echo \"$(cat something.txt)"\"

$ eval echo $(cat something.txt)
hello 42 world
$ bash --version
GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin17)
Copyright (C) 2007 Free Software Foundation, Inc.

envsubst is a great solution (see LenW's answer) if the content you're substituting is of "reasonable" length.

In my case, I needed to substitute in a file's content to replace the variable name. envsubst requires that the content be exported as environment variables and bash has a problem when exporting environment variables that are more than a megabyte or so.

awk solution

Using cuonglm's solution from a different question:

needle="doc1_base64" # The "variable name" in the file. (A $ is not needed.)
needle_file="doc1_base64.txt" # Will be substituted for the needle
haystack=$requestfile1 # File containing the needle
out=$requestfile2
awk "BEGIN{getline l < \"${needle_file}\"}/${needle}/{gsub(\"${needle}\",l)}1" $haystack > $out

This solution works for even large files.

foo=45
file=something.txt       # in a file is written: Hello $foo world!
eval echo $(cat $file)
expenv () {
LF=$'\n'
echo "cat <<END_OF_TEXT${LF}$(< "$1")${LF}END_OF_TEXT" | bash
return $?
}


expenv "file name"