Linux下三种配置iscsi的方法

Linux下三种配置iscsi的方法

linux下面配置iscsi有多种方法，用户态的有tgt，内核态的有scst和lio，如果只是普通的使用来说，没有太大的区别，对于某些特殊的场景就存在区别了

比如tgt不支持pacemaker里面的 Persistent (SCSI-3) Reservations，而scst支持
tgt不支持vmware的vaai的某些属性，而lio支持
比如scst还支持srpt的功能，而tgt也不支持

所以在某些场景下，我们需要根据自己的需要来选择服务，总体上来说iscsi算是一个中间的组件，并不影响数据本身，只是一个对外的服务，满足需求即可

tgt的配置

因为tgt是支持librbd的，也是支持krbd的，所以这个地方存在两种配置，默认的tgt是不支持rbd的，需要自己重新打包支持开启

1
[root@lab103 ~]# tgtadm --lld iscsi --mode system --op show | grep rbd

打包的文档见这里：打包一个支持rbd的tgt

安装打好的包

1
2
3
4
5
[root@lab103 ~]# rpm -ivh scsi-target-utils-*
Preparing...                          ################################# [100%]
Updating / installing...
   1:scsi-target-utils-1.0.84-4.el7   ################################# [ 50%]
   2:scsi-target-utils-rbd-1.0.84-4.el################################# [100%]

检查librdb的支持情况

1
2
[root@lab103 ~]# tgtadm --lld iscsi --mode system --op show | grep rbd
    rbd (bsoflags sync:direct)

支持librbd的配置方式

修改配置文件

1
2
3
4
5
[root@lab103 ~]# cat /etc/tgt/conf.d/sample.conf
<target iqn.2022-01.com.example:server.target1>
    bs-type rbd
    backing-store rbd/testrbd
</target>

启动服务

1
[root@lab103 ~]# systemctl start tgtd

查看生成的服务

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
[root@lab103 ~]# tgt-admin -s
Target 1: iqn.2022-01.com.example:server.target1
    System information:
        Driver: iscsi
        State: ready
    I_T nexus information:
    LUN information:
        LUN: 0
            Type: controller
            SCSI ID: IET     00010000
            SCSI SN: beaf10
            Size: 0 MB, Block size: 1
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            SWP: No
            Thin-provisioning: No
            Backing store type: null
            Backing store path: None
            Backing store flags:
        LUN: 1
            Type: disk
            SCSI ID: IET     00010001
            SCSI SN: beaf11
            Size: 32212 MB, Block size: 512
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            SWP: No
            Thin-provisioning: No
            Backing store type: rbd
            Backing store path: rbd/testrbd
            Backing store flags:
    Account information:
    ACL information:
        ALL

可以看到这里多了个

1
2
Backing store type: rbd
Backing store path: rbd/testrbd

客户端登录后查看磁盘

1
2
3
4
5
6
7
[root@lab101 ~]# parted -s /dev/sdc print
Error: /dev/sdc: unrecognised disk label
Model: IET VIRTUAL-DISK (scsi)
Disk /dev/sdc: 32.2GB
Sector size (logical/physical): 512B/4194304B
Partition Table: unknown
Disk Flags:

可以看到物理扇区的大小为4194304B，这个在很多环境，rac，vmware都可能不识别，需要修改扇区的大小，这个是在tgt的配置文件里面控制的

修改配置文件

1
2
3
4
5
6
[root@lab103 ~]# cat /etc/tgt/conf.d/sample.conf
<target iqn.2022-01.com.example:server.target1>
    bs-type rbd
    backing-store rbd/testrbd
    lbppbe 0
</target>

这个lbppbe 0就是关键配置，设置以后客户端的连接后看到的磁盘扇区就是512b的了

1
2
3
4
5
6
7
[root@lab101 ~]# parted -s /dev/sdc print
Error: /dev/sdc: unrecognised disk label
Model: IET VIRTUAL-DISK (scsi)
Disk /dev/sdc: 32.2GB
Sector size (logical/physical): 512B/512B
Partition Table: unknown
Disk Flags:

支持krbd的配置方式

修改配置文件

1
2
3
<target iqn.2022-01.com.example:server.target1>
    backing-store /dev/rbd/rbd/testrbd
</target>

启动服务

1
[root@lab103 ~]# systemctl start tgtd

查看生成的服务

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
[root@lab103 ~]# tgt-admin -s
Target 1: iqn.2022-01.com.example:server.target1
    System information:
        Driver: iscsi
        State: ready
    I_T nexus information:
    LUN information:
        LUN: 0
            Type: controller
            SCSI ID: IET     00010000
            SCSI SN: beaf10
            Size: 0 MB, Block size: 1
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            SWP: No
            Thin-provisioning: No
            Backing store type: null
            Backing store path: None
            Backing store flags:
        LUN: 1
            Type: disk
            SCSI ID: IET     00010001
            SCSI SN: beaf11
            Size: 32212 MB, Block size: 512
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            SWP: No
            Thin-provisioning: No
            Backing store type: rdwr
            Backing store path: /dev/rbd/rbd/testrbd
            Backing store flags:
    Account information:
    ACL information:
        ALL

这个地方注意下看下客户端的映射的盘的扇区大小

1
2
3
4
5
6
7
[root@lab101 ~]# parted -s /dev/sdc print
Error: /dev/sdc: unrecognised disk label
Model: IET VIRTUAL-DISK (scsi)
Disk /dev/sdc: 32.2GB
Sector size (logical/physical): 512B/512B
Partition Table: unknown
Disk Flags:

LIO配置

使用命令行配置

创建一个block设备

1
2
[root@lab103 ~]# targetcli /backstores/block/ create disk01 /dev/rbd/rbd/testrbd
Created block storage object disk01 using /dev/rbd/rbd/testrbd.

查看当前配置

1
2
3
4
5
6
7
8
9
10
11
12
[root@lab103 ~]# targetcli ls /
o- / ............................................................ [...]
  o- backstores ................................................. [...]
  | o- block ..................................... [Storage Objects: 1]
  | | o- disk01  [/dev/rbd/rbd/testrbd (30.0GiB) write-thru deactivated]
  | |   o- alua ...................................... [ALUA Groups: 1]
  | |     o- default_tg_pt_gp .......... [ALUA state: Active/optimized]
  | o- fileio .................................... [Storage Objects: 0]
  | o- pscsi ..................................... [Storage Objects: 0]
  | o- ramdisk ................................... [Storage Objects: 0]
  o- iscsi ............................................... [Targets: 0]
  o- loopback ............................................ [Targets: 0]

创建一个target

1
2
3
4
5
[root@lab103 ~]# targetcli /iscsi/ create iqn.2022-01.org.linux-iscsi.lab103.x8664:sn.192f7d06da4a
Created target iqn.2022-01.org.linux-iscsi.lab103.x8664:sn.192f7d06da4a.
Created TPG 1.
Global pref auto_add_default_portal=true
Created default portal listening on all IPs (0.0.0.0), port 3260.

创建一个lun

1
2
[root@lab103 ~]# targetcli  /iscsi/iqn.2022-01.org.linux-iscsi.lab103.x8664:sn.192f7d06da4a/tpg1/luns create /backstores/block/disk01
Created LUN 0.

设置target的属性

1
[root@lab103 ~]# targetcli  /iscsi/iqn.2022-01.org.linux-iscsi.lab103.x8664:sn.192f7d06da4a/tpg1 set attribute authentication=0 demo_mode_write_protect=0 generate_node_acls=1 cache_dynamic_acls=1

保存配置文件

1
2
3
[root@lab103 ~]# targetcli /  saveconfig
Last 10 configs saved in /etc/target/backup/.
Configuration saved to /etc/target/saveconfig.json

启动target

1
[root@lab103 ~]# systemctl start target

scst的配置

配置fileio模式

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
[root@lab103 ~]# cat /etc/scst.conf
HANDLER vdisk_fileio {
    DEVICE disk01 {
        filename /dev/rbd/rbd/testrbd
        nv_cache 1
    }
}

TARGET_DRIVER iscsi {
    enabled 1
    TARGET iqn.2022-05.net.vlnb:tgt {
        LUN 0 disk01
        enabled 1
    }
}

配置blockio模式

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
[root@lab103 ~]# cat /etc/scst.conf
HANDLER vdisk_blockio {
    DEVICE disk01 {
        filename /dev/rbd/rbd/testrbd
        nv_cache 1
    }
}

TARGET_DRIVER iscsi {
    enabled 1
    TARGET iqn.2022-05.net.vlnb:tgt {
        LUN 0 disk01
        enabled 1
    }
}

启动服务

1
[root@lab103 ~]# systemctl start scst