Apuntes

# IO Models

# Date: 2019/04/07

# 60 seconds

# Operating System Concepts


Operating System Concepts 10e - 3.6.2 Synchronization


# 60 minutes

# UNIX networking I/O models

首先來看一下UNIX這本書上對I/O模型的介紹:

There are normally two distinct phases for
an input operation:

  1. Waiting for the data to be ready

  2. Copying the data from the kernel to the process

IO發生時通常會有兩個階段, 等待資料準備好 及 將資料從 核心 中複製到 行程 中.

以socket上的IO來說, 第一步是等待資料抵達; 當封包抵達以後, 把它複製到 核心 裡的緩衝區(kernel buffer).
第二步才是再把資料從核心緩衝區複製到應用程式的緩衝區裡.

因為這兩階段中的不同情況, 於是有了下列的5種I/O模型:


  • Blocking I/O 阻塞式IO

blocking

書上有特別提到使用UDP當作範例的原因:

We use UDP for this example instead of TCP because with UDP,

the concept of data being
"ready" to read is simple: either an entire datagram has been received or it has not. With
TCP it gets more complicated, as additional variables such as the socket's low-water mark
come into play.

相較於TCP, UDP在 資料準備好被讀取的概念比較簡單, 只要考慮資料接受與否即可;
TCP比較複雜, 還會牽扯到一些額外的變數等等.

在表中,可以看到這個行程呼叫了 recvfrom 並執行了系統呼叫 ;
可是一直到資料傳送跟複製資料到緩衝區的動作完成之前, 或是錯誤發生, 內核 才會回傳結果,
這個行程才會解除阻塞狀態, 繼續運行下去.

We say that our process is blocked the entire time from when it calls recvfrom until it
returns.

When recvfrom returns successfully, our application processes the datagram.

整個 行程 在 recvfrom回傳成功之前都是阻塞的狀態, 只有recvfrom回傳成功, 應用程式才會繼續處理資料.


  • Nonblocking I/O 非阻塞IO

non-blocking

When we set a socket to be nonblocking, we are telling the kernel "when an I/O operation
that I request cannot be completed without putting the process to sleep, do not put the
process to sleep, but return an error instead."

當socket被設為非阻塞IO時, 其實就是告訴內核"當一個IO請求在不讓行程 sleep 的情況下便無法完成時, 以回傳錯誤代替讓行程睡眠".

如上圖所示, 可以看到前三次呼叫recvfrom時, 資料都還沒準備好, 內核這時候會馬上回傳 EWOULDBLOCK.


✍️ Linux手冊上對EWOULDBLOCK的說明:

Error name Meaning
EAGAIN Resource temporarily unavailable (may be the same value as EWOULDBLOCK)
EWOULDBLOCK Operation would block (may be same value as EAGAIN)

第4次呼叫recvfrom時, 資料準備好了並複製到應用程式緩衝區中, 此時recvfrom回傳成功, 程式開始處理回傳的資料.
像這種程式一直在迴圈當中等待呼叫成功就叫做 polling; 它會不斷地去詢問核心資料好了沒, 極為耗費CPU時間.

應用場景可以參考這一篇 (opens new window)的介紹.


  • I/O multiplexing (select and poll)

I/O multiplexing (select and poll)

在I/O複用裡面, 先使用**select()**將行程停下來, 等候任何一個socket變成可讀之後,

將這個可讀的socket回傳回來, 呼叫 recvfrom() 複製資料並繼續執行.

與Blocking IO做比較的話, 好像沒什麼優點, 甚至還比Blocking IO多做了一次系統呼叫;
可是因為它使用了select(), 能夠在多個 描述符 裡有任何一個準備就緒時就處理,
讓它能夠以單一的process/thread處理多個連線.

SELECT的介紹

select function allows the process to instruct the kernel to wait for any one of multiple eventsto occur and to wake up the process only when one or more of these events occurs or when a specified amount of time has passed.

select() 告訴內核在指定的事件發生時, 再喚醒這個行程並繼續執行.
也就是說, 我們能跟內核說我們有興趣的 描述符 (在這個案例裡就是可讀的socket)以及等待的時間.
當然, 這些描述符並不僅僅限制於socket.

更詳細的解說可以參考知乎 (opens new window) 與 以Redis為範例的I/O Multiplexing詳盡說明 (opens new window).

  • Signal-Driven I/O (SIGIO)

Signal driven I/O

訊號驅動IO與SIGIO關係密切, 可以先到Linux Manual閱讀一下signal的說明 (opens new window)

✍️ Linux手冊上對SIGNAL的說明:

Term: Default action is to terminate the process.

Signal Standard Action Comment
SIGIO - Term I/O now possible (4.2BSD)
SIGPOLL P2001 Term Pollable event (Sys V). Synonym for SIGIO

首先使用sigaction來建立一個訊號處理器(singal handler), 並馬上返回, 程式可以不被阻擋繼續執行.
當資料準備好的時候, SIGIO訊號便會被拋出.

Regardless of how we handle the signal, the advantage to this model is that we are not blocked while waiting for the datagram to arrive.
The main loop can continue executing andjust wait to be notified by the signal handler that either the data is ready to process or thedatagram is ready to be read.

此時我們可以:

  1. 呼叫recvfrom從訊號處理器讀取資料, 並通知主迴圈資料準備完成, 可被處理
  2. 通知主迴圈並讓它自己去讀取資料

此模型的優點就是, 不論我們如何處理資料, 在等待資料抵達的過程中, 程式都是不被阻擋的.
主迴圈可以持續執行並等候訊號通知.


  • Asynchronous I/O (the POSIX aio_functions)

Asynchronous I/O

Asynchronous I/O is defined by the POSIX specification, and various differences in thereal-time functions that appeared in the various standards which came together to form the current POSIX specification have been reconciled.
In general, these functions work bytelling the kernel to start the operation and to notify us when the entire operation(including the copy of the data from the kernel to our buffer) is complete. The main difference between this model and the signal-driven I/O model in the previous section is that with signal-driven I/O, the kernel tells us when an I/O operation can be initiated, butwith asynchronous I/O, the kernel tells us when an I/O operation is complete.

大致上來說, asynchoronous告訴內核執行一整個操作(其中也包括了從內核複製資料到緩衝區裡), 並在操作完成之時通知我們.

與signal-driven I/O最大的差別是, signal-driven是內核告訴我們I/O動作可以開始了;
asynchronous是內核告訴我們I/O已經完成了.

We call aio_read (the POSIX asynchronous I/O functions begin with aio_ or lio_) and pass the kernel the descriptor, buffer pointer, buffer size (the same three arguments for read), file offset (similar to lseek), and how to notify us when the entire operation is complete.
This system call returns immediately and our process is not blocked while waiting for the I/O to complete. We assume in this example that we ask the kernel togenerate some signal when the operation is complete.
This signal is not generated until the data has been copied into our application buffer, which is different from the signal-driven I/O model.

我們呼叫aio_read(), 並且傳入descriptor, buffer pointer, buffer size(也就是read (opens new window)的三個參數), file offset (與lseek (opens new window)參數一致), 以及當操作完成時該如何通知.
在這邊我們假設操作完成時, 內核會回傳一個訊號. 此訊號直到資料被複製進入應用程式的緩衝區以後才會被送出.

傳給**aio_read()**的參數可以參考Linux manual (opens new window).

✍️ Linux手冊上對AIO_READ的說明:

#include <aio.h>

int aio_read(struct aiocb *aiocbp);
1
2
3

可以注意到AIO_READ接收的參數是一個結構, 在繼續查下去可以看到AIO的頁面上 (opens new window)有這個結構的原始碼

#include <aiocb.h>
struct aiocb {
    /* The order of these fields is implementation-dependent */

    int             aio_fildes;     /* File descriptor */
    off_t           aio_offset;     /* File offset */
    volatile void  *aio_buf;        /* Location of buffer */
    size_t          aio_nbytes;     /* Length of transfer */
    int             aio_reqprio;    /* Request priority */
    struct sigevent aio_sigevent;   /* Notification method */
    int             aio_lio_opcode; /* Operation to be performed;
                                        lio_listio() only */

    /* Various implementation-internal fields not shown */
};

/* Operation codes for 'aio_lio_opcode': */

enum { LIO_READ, LIO_WRITE, LIO_NOP };
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

參數詳細說明:

Description


一次5種IO模型的比較

comparison

# Synchronous I/O v.s. Asynchronous I/O

POSIX defines these two terms as follows:

  • A synchronous I/O operation causes the requesting process to be blocked until that I/O operation completes.
  • An asynchronous I/O operation does not cause the requesting process to be blocked.

Using these definitions, the first four I/O models blocking, nonblocking, I/O multiplexing,and signal-driven I/O are all synchronous because the actual I/O operation (recvfrom) blocks the process.
Only the asynchronous I/O model matches the asynchronous I/O definition.

根據POSIX的定義, 最大的差異就是在於IO operation到底會不會讓請求被阻擋:

  • 同步IO操作會導致請求的程序在IO操作完成之前都被阻擋住.
  • 異步IO操作則不會造成請求程序被堵住.

可以搭配這張圖片服用:

Sync v.s. Async


# References


# 其他參考資料

Carl. (2018, Jan 31). 淺談I/O Model (opens new window). Medium.

xianyunyh. (2018, Dec 29). PHP-Interview - LinuxIO模型.md (opens new window). xianyunyh/PHP-Interview.

findumars (2017, Feb 02). 5种网络IO模型（有图，很清楚） (opens new window) 博客园.

二毛儿 (2019, Jan 10). 5种网络IO模型（有图，很清楚）(備份) (opens new window) 知乎.

calidion. (2016, Jul 8). 同步，异步，阻塞，非阻塞等关系轻松理解 #40 (opens new window). calidion/calidion.github.io.

Stevens, R & Fenner, B & Rudoff, A (2003, Nov 21). UNIX Network Programming Volume 1, Third Edition: The Sockets Networking API (opens new window). Amazon.

Silberschatz, A & Galvin, P (2018, May 2). Operating System Concepts 10e (opens new window). Amazon.

Linux Programmer's Manual (2017, Sep 15). SLEEP(3) (opens new window) Linux Programmer's Manual.

Linux Programmer's Manual (2017, Mar 06). ERRNO(3) (opens new window) Linux Programmer's Manual.

Wang, G. T. (2014, Jan 16). WebSocket 通訊協定簡介：比較 Polling、Long-Polling 與 Streaming 的運作原理 (opens new window) G. T. Wang 的個人部落格

知乎用户 (2015, Jun 27). 知乎 - I/O多路复用技术（multiplexing）是什么？ (opens new window) 知乎

Draveness (2016, Nov 26). Redis 和 I/O 多路复用 (opens new window) Draveness's Blog

Last Updated: 2022/7/7 上午8:34:36