[Linux][Shell] Linux中Shell的進程替換(Process Substitution)和命令替換(Command Substitution)

基本上只要是Linux的作業系統就會有Command Subsitution 或 Process Substitution的情狀發生(這兩樣的中文翻譯滿亂的),這邊紀錄兩者之間差別在哪裡,

Command Substitution (命令替換):

原文解釋:

Command substitution allows the output of a command to replace the command itself.

Command substitution occurs when a command is enclosed like this:

$(command)

or like this using backticks:

`command`

Bash performs the expansion by executing COMMAND in a subshell environment and replacing the command substitution with the standard output of the command, with any trailing newlines deleted. Embedded newlines are not deleted, but they may be removed during word splitting. The command substitution $(cat file) can be replaced by the equivalent but faster $(< file).

In computing, command substitution is a facility that allows a command to be run and its output to be pasted back on the command line as arguments to another command.

從上面可以得知命令替換有兩個特性,

  • 特性1:bash會將子命令標準輸出的最後換行符(\n)全部刪除掉。
    # Eample 1
    echo -n -e $(echo -n -e "a\nb\nc\n\n\n")
    
    ## Result 1
    a b c
    
    # Example 2
    echo -n -e "$(echo -n -e "a\nb\nc\n\n\n")"
    
    ## Result 2
    a
    b
    c
    

    會發現兩個例子的stdout不太一樣,第一個例子的命令替換將最後的三個\n吃掉了,中間的兩個\n是由於後續的單詞分割(Word Splitting)根據變量IFS的值替換成了空格 (0x20)。

    變數 IFS (Internal Field Separator) 是 bash 在做 Word Splitting 時做為分隔符(delimiter)用,其預設值(default value)是 space, tab 與 newline,這三個字元每一個都可做為分隔符(delimiter)用。

    Word Splitting 是 bash 做完 expansions (parameter expansion, command substitution, arithmetic expansion) 後,針對非雙引號 (double quote) 且有發生 expansions 的部分,進一步以變數 IFS 來做 Word Splitting。

    而第二個例子會換行符號會沒有被替換的原因如下圖,

LinuxShell_Substitution_001

當使用雙引號的時候,bash跳過了單詞分割 (Word Splitting) 和路徑擴展 (Pathname Expansion) 這兩項擴展,自然就保留了中間的兩個\n

其實在bash手冊的最後也給出了解釋:

If the substitution appears within double quotes, word splitting and filename expansion are not performed on the results.

  • 特性2:$(cat file)在命令替换中更有效的形式是$(< file)

    盡量使用$(command)會比較好,

    # Example 1
    echo `echo `ls``      # INCORRECT
    # Result 1
    ls
    
    # Example 2
    echo `echo \`ls\``    # CORRECT
    # Result 2
    bin boot dev etc home lib lib64 media misc mnt net opt proc root run sbin srv sys tmp usr var
    
    # Example 3
    echo $(echo $(ls))    # CORRECT
    # Result 3
    bin boot dev etc home lib lib64 media misc mnt net opt proc root run sbin srv sys tmp usr var
    

    第一個例子,因為shell自動把前兩個 ` 和後兩個 ` 當成了命令替換,因為這前後兩個命令替換的結果都為空,所以最後只剩ls了。

上面的特性有了了解後,這邊來看看一個例子,

#!/bin/bash

DATE=`date`
echo "Date is ${DATE}"

USERS=$(who | wc -l)
echo "Logged in user are ${USERS}"

UP=`date ; uptime`
echo "Uptime is ${UP}"

他會顯示如下,

Date is Wed Jun 12 15:29:11 CST 2019
Logged in user are        2
Uptime is Wed Jun 12 15:29:11 CST 2019
15:29  up 6 days, 21 mins, 2 users, load averages: 2.27 2.45 2.53

這邊的Command Substitution就是把command的Standard Output (stdout)變成下一個指令或變數的值來使用或是搭配echo指令來輸出,也可以搭配for loop等script。

Process Substitution (進程替換):

原文解釋:

Process substitution is supported on systems that support named pipes (FIFOs) or the /dev/fd method of naming open files. It takes the form of

<(LIST)

or

>(LIST)

The process list is run asynchronously, and its input or output appears as a filename. This filename is passed as an argument to the current command as the result of the expansion. If the >(list) form is used, writing to the file will provide input for list. If the <(list) form is used, the file passed as an argument should be read to obtain the output of list. Note that no space may appear between the < or > and the left parenthesis, otherwise the construct would be interpreted as a redirection. Process substitution is supported on systems that support named pipes (FIFOs) or the /dev/fd method of naming open files.

Process Substitution的模板如下,

<(<some command> <args>)

來看看例子,我們有兩個文件,文件各自的內容如下:

# A.txt
orange
apple
watermelon

# B.txt
Bitter gourd
watermelon
apple

利用diff指令來比對看看兩份文件的差異性,應該會顯示如下:

diff A.txt B.txt

--------------------Result--------------------
1,2c1
< orange
< apple
---
> Bitter gourd
3a3
> apple

可是如果我希望兩份文件是可以先sort完後在進行比對呢?

diff <(sort A.txt) <(sort B.txt)

--------------------Result--------------------
0a1
> Bitter gourd
2d2
< orange

可以看到所謂的Process Substitution就是把命令(Command)的輸出(stdout)當作文件(file)來使用,如果你的指令是原本接受的參數(parameter)是文件名稱(filename)的話,就可以使用Process Substitution來將指令的輸出當作是文件內容處理。

這是因為系統會創建一個臨時的檔案描述符(File Descriptor)來關聯指令的輸出,可以通過下面的方法來驗證:

# 輸入
echo <(sort A.txt)

# 顯示
/dev/fd/63

進程替代並不能和檔案完全等價,進程替代所建立的"對象",是不能進行寫入和隨機讀取操作的,因爲如果進行寫入操作,將會寫入到那個臨時的檔案描述符裏面去,而這個臨時檔案描述符會被迅速地釋放掉。

在來看看另一個用來比較檔案內容的指令,

comm -1 -2 <(sort list1.txt) <(sort list2.txt)

以上。

Reference

Bash Guide for Beginners 3.4. Shell expansion

Unix / Linux - Shell Substitution

Process substitution and pipe

變數值代換(variables substitution)精簡筆記

How To Use Bash Parameter Substitution Like A Pro

Scope of variables in a process substitution

Advanced Bash-Scripting Guide: Chapter 23. Process Substitution

Command substitution

Linux Shell技巧: 进程替代(Process Substitution)

bash之命令替换(command substitution)

Word Splitting

关于 Word Splitting 和 IFS 的三个细节

Bash IFS & word splitting

Difference between subshells and process substitution

Bash Shell | 如何印出 IFS 變數的內容值與 word splitting

Compare different IPs in two files? closed

Add a Comment