oracle 19c 3node rac 구성== 내용 참고만 하셍

[생짜초보]

Oracle has released the patches for Oracle 19c (19.8) and 19c (19.9) for ACFS/AFD drivers.

1. 30590023 UEK6 SUPPORT FOR ACFS/AFD (System Patch) - > 19.9.0.0.0ACFSRU

2. 30590023 UEK6 SUPPORT FOR ACFS/AFD (System Patch) - > 19.8.0.0.0ACFSRU

Please refer for more details:

ACFS Support On OS Platforms (Certification Matrix). (Doc ID 1369107.1)

Click Additional notes: 

OL8 UEK6: 30590023 OL8 UEK6 SUPPORT FOR ACFS/AFD (NOTE: 19.x and later Grid only)

Environment:

1. Installed OEL 8.2

2. Upgraded Kernel "kernel-uek-4.18.0-193.13.2.el8_2.x86_64" to kernel-uek-5.4.17-2036.100.6.1.el8uek.x86_64

3. Installed GI 19.3 (while GI installation applied Oracle 19c GI (19.8) RU)

4. Applied "p30590023_198000ACFSRU_Linux-x86-64.zip" patch on Oracle 19c (19.8) for GRID

19c-rac1:

[oracle@19c-rac1 ~]$ ps -ef | grep pmon

oracle     12651       1  0 18:30 ?        00:00:00 asm_pmon_+ASM1

oracle     14408       1  0 18:31 ?        00:00:00 mdb_pmon_-MGMTDB

oracle     35737   35364  0 18:39 pts/1    00:00:00 grep --color=auto pmon

[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ uname -r

5.4.17-2036.100.6.1.el8uek.x86_64

[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ uname -a

Linux 19c-rac1.localdomain 5.4.17-2036.100.6.1.el8uek.x86_64 #2 SMP Thu Oct 29 17:06:00 PDT 2020 x86_64 x86_64 x86_64 GNU/Linux

[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ cat /etc/oracle-release

Oracle Linux Server release 8.2

[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ . oraenv

ORACLE_SID = [+ASM1] ?

ORACLE_HOME = [/home/oracle] ? /u01/app/19.3.0/grid

The Oracle base remains unchanged with value /u01/app/oracle

[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ echo $ORACLE_HOME

/u01/app/19.3.0/grid

[oracle@19c-rac1 ~]$ cd $ORACLE_HOME/OPatch

[oracle@19c-rac1 OPatch]$ ./opatch lspatches

31335188;TOMCAT RELEASE UPDATE 19.0.0.0.0 (31335188)

31305087;OCW RELEASE UPDATE 19.8.0.0.0 (31305087)

31304218;ACFS RELEASE UPDATE 19.8.0.0.0 (31304218)

31281355;Database Release Update : 19.8.0.0.200714 (31281355)

OPatch succeeded.

[oracle@19c-rac1 OPatch]$

[oracle@19c-rac1 ~]$ chmod 777 p30590023_198000ACFSRU_Linux-x86-64.zip

[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ scp p30590023_198000ACFSRU_Linux-x86-64.zip oracle@19c-rac2:/home/oracle/

p30590023_198000ACFSRU_Linux-x86-64.zip        100%  686MB  36.5MB/s   00:18

[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ scp p30590023_198000ACFSRU_Linux-x86-64.zip oracle@19c-rac3:/home/oracle/

p30590023_198000ACFSRU_Linux-x86-64.zip        100%  686MB  37.3MB/s   00:18

[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ unzip p30590023_198000ACFSRU_Linux-x86-64.zip

Archive:  p30590023_198000ACFSRU_Linux-x86-64.zip

   creating: 30590023/

   creating: 30590023/automation/

  inflating: 30590023/automation/messages.properties

  inflating: 30590023/automation/bp1-out-of-place-non-rolling-switchback.xml

  inflating: 30590023/automation/bp1-out-of-place-switchback.xml

[oracle@19c-rac1 ~]$ echo $ORACLE_HOME

/u01/app/19.3.0/grid

[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ cd $ORACLE_HOME/OPatch

[oracle@19c-rac1 OPatch]$ ./opatch lspatches

31335188;TOMCAT RELEASE UPDATE 19.0.0.0.0 (31335188)

31305087;OCW RELEASE UPDATE 19.8.0.0.0 (31305087)

31304218;ACFS RELEASE UPDATE 19.8.0.0.0 (31304218)

31281355;Database Release Update : 19.8.0.0.200714 (31281355)

OPatch succeeded.

[oracle@19c-rac1 OPatch]$

[oracle@19c-rac1 OPatch]$ ./opatch version

OPatch Version: 12.2.0.1.21

OPatch succeeded.

[oracle@19c-rac1 OPatch]$

[oracle@19c-rac1 OPatch]$ cd

[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ su - root

Password:

[root@19c-rac1 ~]#

[root@19c-rac1 ~]# . oraenv

ORACLE_SID = [root] ? +ASM1

ORACLE_HOME = [/home/oracle] ? /u01/app/19.3.0/grid

The Oracle base has been set to /u01/app/oracle

[root@19c-rac1 ~]#

[root@19c-rac1 ~]# sh /u01/app/19.3.0/grid/OPatch/opatchauto apply /home/oracle/30590023/

OPatchauto session is initiated at Fri Dec 18 19:24:30 2020

System initialization log file is /u01/app/19.3.0/grid/cfgtoollogs/opatchautodb/systemconfig2020-12-18_07-24-35PM.log.

Session log file is /u01/app/19.3.0/grid/cfgtoollogs/opatchauto/opatchauto2020-12-18_07-25-12PM.log

The id for this session is P258

Executing OPatch prereq operations to verify patch applicability on home /u01/app/19.3.0/grid

Patch applicability verified successfully on home /u01/app/19.3.0/grid

Bringing down CRS service on home /u01/app/19.3.0/grid

CRS service brought down successfully on home /u01/app/19.3.0/grid

Start applying binary patch on home /u01/app/19.3.0/grid

Binary patch applied successfully on home /u01/app/19.3.0/grid

Starting CRS service on home /u01/app/19.3.0/grid

CRS service started successfully on home /u01/app/19.3.0/grid

OPatchAuto successful.

--------------------------------Summary--------------------------------

Patching is completed successfully. Please find the summary as follows:

Host:19c-rac1

CRS Home:/u01/app/19.3.0/grid

Version:19.0.0.0.0

Summary:

==Following patches were SUCCESSFULLY applied:

Patch: /home/oracle/30590023/30590023

Log: /u01/app/19.3.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-12-18_19-27-43PM_1.log

OPatchauto session completed at Fri Dec 18 19:31:02 2020

Time taken to complete the session 6 minutes, 32 seconds

[root@19c-rac1 ~]#

[root@19c-rac1 ~]# exit

logout

[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ . oraenv

ORACLE_SID = [+ASM1] ?

ORACLE_HOME = [/home/oracle] ? /u01/app/19.3.0/grid

The Oracle base remains unchanged with value /u01/app/oracle

[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ cd $ORACLE_HOME/OPatch

[oracle@19c-rac1 OPatch]$ ./opatch lspatches

30590023;ACFS Interim patch for 30590023

31335188;TOMCAT RELEASE UPDATE 19.0.0.0.0 (31335188)

31305087;OCW RELEASE UPDATE 19.8.0.0.0 (31305087)

31281355;Database Release Update : 19.8.0.0.200714 (31281355)

OPatch succeeded.

[oracle@19c-rac1 OPatch]$ cd

[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ ps -ef | grep pmon

oracle     87342       1  0 19:30 ?        00:00:00 asm_pmon_+ASM1

oracle     97416   35364  0 19:48 pts/1    00:00:00 grep --color=auto pmon

[oracle@19c-rac1 ~]$

19c-rac2:

[oracle@19c-rac2 ~]$ . oraenv

ORACLE_SID = [cdbrac2] ? +ASM2

ORACLE_HOME = [/home/oracle] ? /u01/app/19.3.0/grid

The Oracle base remains unchanged with value /u01/app/oracle

[oracle@19c-rac2 ~]$

[oracle@19c-rac2 ~]$ cd $ORACLE_HOME/OPatch

[oracle@19c-rac2 OPatch]$

[oracle@19c-rac2 OPatch]$ ./opatch lspatches

31335188;TOMCAT RELEASE UPDATE 19.0.0.0.0 (31335188)

31305087;OCW RELEASE UPDATE 19.8.0.0.0 (31305087)

31304218;ACFS RELEASE UPDATE 19.8.0.0.0 (31304218)

31281355;Database Release Update : 19.8.0.0.200714 (31281355)

OPatch succeeded.

[oracle@19c-rac2 OPatch]$

[oracle@19c-rac2 OPatch]$ ./opatch version

OPatch Version: 12.2.0.1.21

OPatch succeeded.

[oracle@19c-rac2 OPatch]$

[oracle@19c-rac2 OPatch]$ cd

[oracle@19c-rac2 ~]$

[oracle@19c-rac2 ~]$ ps -ef | grep pmon

oracle     15057       1  0 18:32 ?        00:00:00 asm_pmon_+ASM2

oracle     48512   29638  0 19:24 pts/0    00:00:00 grep --color=auto pmon

[oracle@19c-rac2 ~]$

[oracle@19c-rac2 ~]$ su - root

Password:

[root@19c-rac2 ~]# . oraenv

ORACLE_SID = [root] ? +ASM2

ORACLE_HOME = [/home/oracle] ? /u01/app/19.3.0/grid

The Oracle base has been set to /u01/app/oracle

[root@19c-rac2 ~]#

[root@19c-rac2 ~]# sh /u01/app/19.3.0/grid/OPatch/opatchauto apply /home/oracle/30590023/

OPatchauto session is initiated at Fri Dec 18 19:47:58 2020

System initialization log file is /u01/app/19.3.0/grid/cfgtoollogs/opatchautodb/systemconfig2020-12-18_07-48-02PM.log.

Session log file is /u01/app/19.3.0/grid/cfgtoollogs/opatchauto/opatchauto2020-12-18_07-48-14PM.log

The id for this session is ZJLN

Executing OPatch prereq operations to verify patch applicability on home /u01/app/19.3.0/grid

Patch applicability verified successfully on home /u01/app/19.3.0/grid

Bringing down CRS service on home /u01/app/19.3.0/grid

CRS service brought down successfully on home /u01/app/19.3.0/grid

Start applying binary patch on home /u01/app/19.3.0/grid

Binary patch applied successfully on home /u01/app/19.3.0/grid

Starting CRS service on home /u01/app/19.3.0/grid

CRS service started successfully on home /u01/app/19.3.0/grid

OPatchAuto successful.

--------------------------------Summary--------------------------------

Patching is completed successfully. Please find the summary as follows:

Host:19c-rac2

CRS Home:/u01/app/19.3.0/grid

Version:19.0.0.0.0

Summary:

==Following patches were SUCCESSFULLY applied:

Patch: /home/oracle/30590023/30590023

Log: /u01/app/19.3.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-12-18_19-49-48PM_1.log

OPatchauto session completed at Fri Dec 18 19:53:11 2020

Time taken to complete the session 5 minutes, 13 seconds

[root@19c-rac2 ~]#

19c-rac3:

[oracle@19c-rac3 ~]$ . oraenv

ORACLE_SID = [cdbrac3] ? +ASM3

ORACLE_HOME = [/home/oracle] ? /u01/app/19.3.0/grid

The Oracle base remains unchanged with value /u01/app/oracle

[oracle@19c-rac3 ~]$

[oracle@19c-rac3 ~]$ cd $ORACLE_HOME/OPatch

[oracle@19c-rac3 OPatch]$

[oracle@19c-rac3 OPatch]$ ./opatch lspatches

31335188;TOMCAT RELEASE UPDATE 19.0.0.0.0 (31335188)

31305087;OCW RELEASE UPDATE 19.8.0.0.0 (31305087)

31304218;ACFS RELEASE UPDATE 19.8.0.0.0 (31304218)

31281355;Database Release Update : 19.8.0.0.200714 (31281355)

OPatch succeeded.

[oracle@19c-rac3 OPatch]$

[oracle@19c-rac3 ~]$ ps -ef | grep pmon

oracle     14810       1  0 18:32 ?        00:00:00 asm_pmon_+ASM3

oracle     48694   29385  0 19:24 pts/0    00:00:00 grep --color=auto pmon

[oracle@19c-rac3 ~]$

[oracle@19c-rac3 ~]$ su - root

Password:

[root@19c-rac3 ~]# . oraenv

ORACLE_SID = [root] ? +ASM3

ORACLE_HOME = [/home/oracle] ? /u01/app/19.3.0/grid

The Oracle base has been set to /u01/app/oracle

[root@19c-rac3 ~]#

[root@19c-rac3 ~]# sh /u01/app/19.3.0/grid/OPatch/opatchauto apply /home/oracle/30590023/

OPatchauto session is initiated at Fri Dec 18 19:54:35 2020

System initialization log file is /u01/app/19.3.0/grid/cfgtoollogs/opatchautodb/systemconfig2020-12-18_07-54-39PM.log.

Session log file is /u01/app/19.3.0/grid/cfgtoollogs/opatchauto/opatchauto2020-12-18_07-54-50PM.log

The id for this session is KS92

Executing OPatch prereq operations to verify patch applicability on home /u01/app/19.3.0/grid

Patch applicability verified successfully on home /u01/app/19.3.0/grid

Bringing down CRS service on home /u01/app/19.3.0/grid

CRS service brought down successfully on home /u01/app/19.3.0/grid

Start applying binary patch on home /u01/app/19.3.0/grid

Binary patch applied successfully on home /u01/app/19.3.0/grid

Starting CRS service on home /u01/app/19.3.0/grid

CRS service started successfully on home /u01/app/19.3.0/grid

OPatchAuto successful.

--------------------------------Summary--------------------------------

Patching is completed successfully. Please find the summary as follows:

Host:19c-rac3

CRS Home:/u01/app/19.3.0/grid

Version:19.0.0.0.0

Summary:

==Following patches were SUCCESSFULLY applied:

Patch: /home/oracle/30590023/30590023

Log: /u01/app/19.3.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-12-18_19-57-06PM_1.log

OPatchauto session completed at Fri Dec 18 20:03:37 2020

Time taken to complete the session 9 minutes, 2 seconds

[root@19c-rac3 ~]#

Checking drivers in 19c-rac1:

[oracle@19c-rac1 ~]$ su - root

Password:

[root@19c-rac1 ~]# . oraenv

ORACLE_SID = [root] ? +ASM1

ORACLE_HOME = [/home/oracle] ? /u01/app/19.3.0/grid

The Oracle base has been set to /u01/app/oracle

[root@19c-rac1 ~]#

[root@19c-rac1 ~]# cd $ORACLE_HOME/bin

[root@19c-rac1 bin]# ./afddriverstate supported

AFD-9200: Supported

[root@19c-rac1 bin]# ./acfsdriverstate -orahome /u01/app/19.3.0/grid installed

ACFS-9203: true

[root@19c-rac1 bin]# ./acfsdriverstate -orahome /u01/app/19.3.0/grid loaded

ACFS-9203: true

[root@19c-rac1 bin]# ./acfsdriverstate -orahome /u01/app/19.3.0/grid supported

ACFS-9200: Supported

[root@19c-rac1 bin]# ./acfsdriverstate -orahome /u01/app/19.3.0/grid version

ACFS-9325:     Driver OS kernel version = 5.4.17-2011.0.7.el8uek.x86_64.

ACFS-9326:     Driver build number = RELEASE.

ACFS-9212:     Driver build version = 19.0.0.0.0 (19.8.0.0.0).

ACFS-9547:     Driver available build number = RELEASE.

ACFS-9548:     Driver available build version = 19.0.0.0.0 (19.8.0.0.0).

[root@19c-rac1 bin]#

[root@19c-rac1 bin]#  asmcmd afd_configure

ASMCMD-9523: command cannot be used when Oracle Clusterware stack is up

[root@19c-rac1 bin]#

[root@19c-rac1 bin]# crsctl stop crs

CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on '19c-rac1'

CRS-2673: Attempting to stop 'ora.crsd' on '19c-rac1'

CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on server '19c-rac1'

CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on '19c-rac1'

CRS-33673: Attempting to stop resource group 'ora.asmgroup' on server '19c-rac1'

CRS-2673: Attempting to stop 'ora.OCRVD.dg' on '19c-rac1'

CRS-2673: Attempting to stop 'ora.LISTENER_SCAN3.lsnr' on '19c-rac1'

CRS-2673: Attempting to stop 'ora.cvu' on '19c-rac1'

CRS-2673: Attempting to stop 'ora.chad' on '19c-rac1'

CRS-2677: Stop of 'ora.OCRVD.dg' on '19c-rac1' succeeded

CRS-2673: Attempting to stop 'ora.asm' on '19c-rac1'

CRS-2677: Stop of 'ora.LISTENER.lsnr' on '19c-rac1' succeeded

CRS-2673: Attempting to stop 'ora.19c-rac1.vip' on '19c-rac1'

CRS-2677: Stop of 'ora.LISTENER_SCAN3.lsnr' on '19c-rac1' succeeded

CRS-2673: Attempting to stop 'ora.scan3.vip' on '19c-rac1'

CRS-2677: Stop of 'ora.asm' on '19c-rac1' succeeded

CRS-2673: Attempting to stop 'ora.ASMNET1LSNR_ASM.lsnr' on '19c-rac1'

CRS-2677: Stop of 'ora.19c-rac1.vip' on '19c-rac1' succeeded

CRS-2677: Stop of 'ora.cvu' on '19c-rac1' succeeded

CRS-2677: Stop of 'ora.scan3.vip' on '19c-rac1' succeeded

CRS-2677: Stop of 'ora.chad' on '19c-rac1' succeeded

CRS-2677: Stop of 'ora.ASMNET1LSNR_ASM.lsnr' on '19c-rac1' succeeded

CRS-2673: Attempting to stop 'ora.asmnet1.asmnetwork' on '19c-rac1'

CRS-2677: Stop of 'ora.asmnet1.asmnetwork' on '19c-rac1' succeeded

CRS-33677: Stop of resource group 'ora.asmgroup' on server '19c-rac1' succeeded.

CRS-2672: Attempting to start 'ora.19c-rac1.vip' on '19c-rac3'

CRS-2672: Attempting to start 'ora.scan3.vip' on '19c-rac2'

CRS-2672: Attempting to start 'ora.cvu' on '19c-rac2'

CRS-2676: Start of 'ora.cvu' on '19c-rac2' succeeded

CRS-2676: Start of 'ora.19c-rac1.vip' on '19c-rac3' succeeded

CRS-2676: Start of 'ora.scan3.vip' on '19c-rac2' succeeded

CRS-2672: Attempting to start 'ora.LISTENER_SCAN3.lsnr' on '19c-rac2'

CRS-2676: Start of 'ora.LISTENER_SCAN3.lsnr' on '19c-rac2' succeeded

CRS-2673: Attempting to stop 'ora.ons' on '19c-rac1'

CRS-2677: Stop of 'ora.ons' on '19c-rac1' succeeded

CRS-2673: Attempting to stop 'ora.net1.network' on '19c-rac1'

CRS-2677: Stop of 'ora.net1.network' on '19c-rac1' succeeded

CRS-2792: Shutdown of Cluster Ready Services-managed resources on '19c-rac1' has completed

CRS-2677: Stop of 'ora.crsd' on '19c-rac1' succeeded

CRS-2673: Attempting to stop 'ora.asm' on '19c-rac1'

CRS-2673: Attempting to stop 'ora.crf' on '19c-rac1'

CRS-2673: Attempting to stop 'ora.drivers.acfs' on '19c-rac1'

CRS-2673: Attempting to stop 'ora.mdnsd' on '19c-rac1'

CRS-2677: Stop of 'ora.crf' on '19c-rac1' succeeded

CRS-2677: Stop of 'ora.drivers.acfs' on '19c-rac1' succeeded

CRS-2677: Stop of 'ora.mdnsd' on '19c-rac1' succeeded

CRS-2677: Stop of 'ora.asm' on '19c-rac1' succeeded

CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on '19c-rac1'

CRS-2677: Stop of 'ora.cluster_interconnect.haip' on '19c-rac1' succeeded

CRS-2673: Attempting to stop 'ora.ctssd' on '19c-rac1'

CRS-2673: Attempting to stop 'ora.evmd' on '19c-rac1'

CRS-2677: Stop of 'ora.ctssd' on '19c-rac1' succeeded

CRS-2677: Stop of 'ora.evmd' on '19c-rac1' succeeded

CRS-2673: Attempting to stop 'ora.cssd' on '19c-rac1'

CRS-2677: Stop of 'ora.cssd' on '19c-rac1' succeeded

CRS-2673: Attempting to stop 'ora.gipcd' on '19c-rac1'

CRS-2673: Attempting to stop 'ora.gpnpd' on '19c-rac1'

CRS-2677: Stop of 'ora.gpnpd' on '19c-rac1' succeeded

CRS-2677: Stop of 'ora.gipcd' on '19c-rac1' succeeded

CRS-2793: Shutdown of Oracle High Availability Services-managed resources on '19c-rac1' has completed

CRS-4133: Oracle High Availability Services has been stopped.

[root@19c-rac1 bin]#

[root@19c-rac1 bin]# asmcmd afd_configure

AFD-627: AFD distribution files found.

AFD-634: Removing previous AFD installation.

AFD-635: Previous AFD components successfully removed.

AFD-9294: updating file /etc/sysconfig/oracledrivers.conf

AFD-636: Installing requested AFD software.

AFD-637: Loading installed AFD drivers.

AFD-9321: Creating udev for AFD.

AFD-9323: Creating module dependencies - this may take some time.

AFD-9154: Loading 'oracleafd.ko' driver.

AFD-649: Verifying AFD devices.

AFD-9156: Detecting control device '/dev/oracleafd/admin'.

AFD-638: AFD installation correctness verified.

Modifying resource dependencies - this may take some time.

[root@19c-rac1 bin]#

[root@19c-rac1 bin]# asmcmd afd_state

ASMCMD-9526: The AFD state is 'LOADED' and filtering is 'ENABLED' on host '19c-rac1.localdomain'

[root@19c-rac1 bin]#

[root@19c-rac1 bin]# crsctl start crs

CRS-4123: Oracle High Availability Services has been started.

[root@19c-rac1 bin]#

[root@19c-rac1 bin]#

[root@19c-rac1 bin]# ls -lrt afd*

-rwxr-xr-x 1 root oinstall     1000 Jun 20  2016 afdroot

-rwxr-xr-x 1 root oinstall   280896 Nov 15 09:17 afdtool.bin

-rwxr-xr-x 1 root oinstall 87596064 Nov 15 09:17 afdboot

-rwxr-xr-x 1 root oinstall     2860 Dec 18 20:07 afdtool

-rwxr-xr-x 1 root oinstall      760 Dec 18 20:07 afdload

-rwxr-xr-x 1 root oinstall     3084 Dec 18 20:07 afddriverstate

[root@19c-rac1 bin]#

[root@19c-rac1 bin]# lsmod|grep oracle

oracleafd             221184  1

oracleacfs           4997120  0

oracleadvm            745472  0

oracleoks             729088  2 oracleadvm,oracleacfs

[root@19c-rac1 bin]#

[root@19c-rac1 bin]# cat /etc/os-release | head -n2

NAME="Oracle Linux Server"

VERSION="8.2"

[root@19c-rac1 bin]#

[root@19c-rac1 bin]# ./acfsdriverstate -orahome /u01/app/19.3.0/grid version

ACFS-9325:     Driver OS kernel version = 5.4.17-2011.0.7.el8uek.x86_64.

ACFS-9326:     Driver build number = RELEASE.

ACFS-9212:     Driver build version = 19.0.0.0.0 (19.8.0.0.0).

ACFS-9547:     Driver available build number = RELEASE.

ACFS-9548:     Driver available build version = 19.0.0.0.0 (19.8.0.0.0).

[root@19c-rac1 bin]#

Checking drivers in 19c-rac2:

[root@19c-rac2 ~]# cd $ORACLE_HOME/bin

[root@19c-rac2 bin]# ./afddriverstate supported

AFD-9200: Supported

[root@19c-rac2 bin]#

[root@19c-rac2 bin]# ./acfsdriverstate -orahome /u01/app/19.3.0/grid installed

ACFS-9203: true

[root@19c-rac2 bin]# ./acfsdriverstate -orahome /u01/app/19.3.0/grid loaded

ACFS-9203: true

[root@19c-rac2 bin]# ./acfsdriverstate -orahome /u01/app/19.3.0/grid supported

ACFS-9200: Supported

[root@19c-rac2 bin]# ./acfsdriverstate -orahome /u01/app/19.3.0/grid version

ACFS-9325:     Driver OS kernel version = 5.4.17-2011.0.7.el8uek.x86_64.

ACFS-9326:     Driver build number = RELEASE.

ACFS-9212:     Driver build version = 19.0.0.0.0 (19.8.0.0.0).

ACFS-9547:     Driver available build number = RELEASE.

ACFS-9548:     Driver available build version = 19.0.0.0.0 (19.8.0.0.0).

[root@19c-rac2 bin]#

[root@19c-rac2 bin]# asmcmd afd_configure

ASMCMD-9523: command cannot be used when Oracle Clusterware stack is up

[root@19c-rac2 bin]# crsctl stop crs

CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on '19c-rac2'

CRS-2673: Attempting to stop 'ora.crsd' on '19c-rac2'

CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on server '19c-rac2'

CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on '19c-rac2'

CRS-2673: Attempting to stop 'ora.LISTENER_SCAN3.lsnr' on '19c-rac2'

CRS-2673: Attempting to stop 'ora.cvu' on '19c-rac2'

CRS-2673: Attempting to stop 'ora.qosmserver' on '19c-rac2'

CRS-33673: Attempting to stop resource group 'ora.asmgroup' on server '19c-rac2'

CRS-2673: Attempting to stop 'ora.OCRVD.dg' on '19c-rac2'

CRS-2673: Attempting to stop 'ora.chad' on '19c-rac2'

CRS-2677: Stop of 'ora.OCRVD.dg' on '19c-rac2' succeeded

CRS-2673: Attempting to stop 'ora.asm' on '19c-rac2'

CRS-2677: Stop of 'ora.LISTENER.lsnr' on '19c-rac2' succeeded

CRS-2673: Attempting to stop 'ora.19c-rac2.vip' on '19c-rac2'

CRS-2677: Stop of 'ora.LISTENER_SCAN3.lsnr' on '19c-rac2' succeeded

CRS-2673: Attempting to stop 'ora.scan3.vip' on '19c-rac2'

CRS-2677: Stop of 'ora.cvu' on '19c-rac2' succeeded

CRS-2677: Stop of 'ora.19c-rac2.vip' on '19c-rac2' succeeded

CRS-2677: Stop of 'ora.scan3.vip' on '19c-rac2' succeeded

CRS-2677: Stop of 'ora.asm' on '19c-rac2' succeeded

CRS-2673: Attempting to stop 'ora.ASMNET1LSNR_ASM.lsnr' on '19c-rac2'

CRS-2677: Stop of 'ora.chad' on '19c-rac2' succeeded

CRS-2677: Stop of 'ora.qosmserver' on '19c-rac2' succeeded

CRS-2677: Stop of 'ora.ASMNET1LSNR_ASM.lsnr' on '19c-rac2' succeeded

CRS-2673: Attempting to stop 'ora.asmnet1.asmnetwork' on '19c-rac2'

CRS-2677: Stop of 'ora.asmnet1.asmnetwork' on '19c-rac2' succeeded

CRS-33677: Stop of resource group 'ora.asmgroup' on server '19c-rac2' succeeded.

CRS-2672: Attempting to start 'ora.qosmserver' on '19c-rac1'

CRS-2672: Attempting to start 'ora.19c-rac2.vip' on '19c-rac3'

CRS-2672: Attempting to start 'ora.scan3.vip' on '19c-rac1'

CRS-2672: Attempting to start 'ora.cvu' on '19c-rac1'

CRS-2676: Start of 'ora.cvu' on '19c-rac1' succeeded

CRS-2676: Start of 'ora.19c-rac2.vip' on '19c-rac3' succeeded

CRS-2676: Start of 'ora.scan3.vip' on '19c-rac1' succeeded

CRS-2672: Attempting to start 'ora.LISTENER_SCAN3.lsnr' on '19c-rac1'

CRS-2676: Start of 'ora.LISTENER_SCAN3.lsnr' on '19c-rac1' succeeded

CRS-2676: Start of 'ora.qosmserver' on '19c-rac1' succeeded

CRS-2673: Attempting to stop 'ora.ons' on '19c-rac2'

CRS-2677: Stop of 'ora.ons' on '19c-rac2' succeeded

CRS-2673: Attempting to stop 'ora.net1.network' on '19c-rac2'

CRS-2677: Stop of 'ora.net1.network' on '19c-rac2' succeeded

CRS-2792: Shutdown of Cluster Ready Services-managed resources on '19c-rac2' has completed

CRS-2677: Stop of 'ora.crsd' on '19c-rac2' succeeded

CRS-2673: Attempting to stop 'ora.storage' on '19c-rac2'

CRS-2673: Attempting to stop 'ora.crf' on '19c-rac2'

CRS-2673: Attempting to stop 'ora.drivers.acfs' on '19c-rac2'

CRS-2673: Attempting to stop 'ora.mdnsd' on '19c-rac2'

CRS-2677: Stop of 'ora.crf' on '19c-rac2' succeeded

CRS-2677: Stop of 'ora.drivers.acfs' on '19c-rac2' succeeded

CRS-2677: Stop of 'ora.storage' on '19c-rac2' succeeded

CRS-2673: Attempting to stop 'ora.asm' on '19c-rac2'

CRS-2677: Stop of 'ora.mdnsd' on '19c-rac2' succeeded

CRS-2677: Stop of 'ora.asm' on '19c-rac2' succeeded

CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on '19c-rac2'

CRS-2677: Stop of 'ora.cluster_interconnect.haip' on '19c-rac2' succeeded

CRS-2673: Attempting to stop 'ora.ctssd' on '19c-rac2'

CRS-2673: Attempting to stop 'ora.evmd' on '19c-rac2'

CRS-2677: Stop of 'ora.ctssd' on '19c-rac2' succeeded

CRS-2677: Stop of 'ora.evmd' on '19c-rac2' succeeded

CRS-2673: Attempting to stop 'ora.cssd' on '19c-rac2'

CRS-2677: Stop of 'ora.cssd' on '19c-rac2' succeeded

CRS-2673: Attempting to stop 'ora.gipcd' on '19c-rac2'

CRS-2673: Attempting to stop 'ora.gpnpd' on '19c-rac2'

CRS-2677: Stop of 'ora.gpnpd' on '19c-rac2' succeeded

CRS-2677: Stop of 'ora.gipcd' on '19c-rac2' succeeded

CRS-2793: Shutdown of Oracle High Availability Services-managed resources on '19c-rac2' has completed

CRS-4133: Oracle High Availability Services has been stopped.

[root@19c-rac2 bin]#

[root@19c-rac2 bin]# asmcmd afd_configure

AFD-627: AFD distribution files found.

AFD-634: Removing previous AFD installation.

AFD-635: Previous AFD components successfully removed.

AFD-9294: updating file /etc/sysconfig/oracledrivers.conf

AFD-636: Installing requested AFD software.

AFD-637: Loading installed AFD drivers.

AFD-9321: Creating udev for AFD.

AFD-9323: Creating module dependencies - this may take some time.

AFD-9154: Loading 'oracleafd.ko' driver.

AFD-649: Verifying AFD devices.

AFD-9156: Detecting control device '/dev/oracleafd/admin'.

AFD-638: AFD installation correctness verified.

Modifying resource dependencies - this may take some time.

[root@19c-rac2 bin]#

[root@19c-rac2 bin]# asmcmd afd_state

ASMCMD-9526: The AFD state is 'LOADED' and filtering is 'ENABLED' on host '19c-rac2.localdomain'

[root@19c-rac2 bin]#

[root@19c-rac2 bin]# crsctl start crs

CRS-4123: Oracle High Availability Services has been started.

[root@19c-rac2 bin]#

[root@19c-rac2 bin]# ls -lrt afd*

-rwxr-xr-x 1 root oinstall     1000 Nov 12 21:27 afdroot

-rwxr-xr-x 1 root oinstall   280896 Nov 15 09:17 afdtool.bin

-rwxr-xr-x 1 root oinstall 87596064 Nov 15 09:17 afdboot

-rwxr-xr-x 1 root oinstall      760 Dec 18 20:16 afdload

-rwxr-xr-x 1 root oinstall     2860 Dec 18 20:16 afdtool

-rwxr-xr-x 1 root oinstall     3084 Dec 18 20:16 afddriverstate

[root@19c-rac2 bin]#

[root@19c-rac2 bin]# lsmod|grep oracle

oracleafd             221184  1

oracleacfs           4997120  0

oracleadvm            745472  0

oracleoks             729088  2 oracleadvm,oracleacfs

[root@19c-rac2 bin]# cat /etc/os-release | head -n2

NAME="Oracle Linux Server"

VERSION="8.2"

[root@19c-rac2 bin]#

Checking drivers in 19c-rac3:

[root@19c-rac3 ~]# cd $ORACLE_HOME/bin

[root@19c-rac3 bin]#

[root@19c-rac3 bin]# ./afddriverstate supported

AFD-9200: Supported

[root@19c-rac3 bin]# ./acfsdriverstate -orahome /u01/app/19.3.0/grid installed

ACFS-9203: true

[root@19c-rac3 bin]# ./acfsdriverstate -orahome /u01/app/19.3.0/grid loaded

ACFS-9203: true

[root@19c-rac3 bin]# ./acfsdriverstate -orahome /u01/app/19.3.0/grid supported

ACFS-9200: Supported

[root@19c-rac3 bin]# ./acfsdriverstate -orahome /u01/app/19.3.0/grid version

ACFS-9325:     Driver OS kernel version = 5.4.17-2011.0.7.el8uek.x86_64.

ACFS-9326:     Driver build number = RELEASE.

ACFS-9212:     Driver build version = 19.0.0.0.0 (19.8.0.0.0).

ACFS-9547:     Driver available build number = RELEASE.

ACFS-9548:     Driver available build version = 19.0.0.0.0 (19.8.0.0.0).

[root@19c-rac3 bin]#

[root@19c-rac3 bin]# asmcmd afd_configure

ASMCMD-9523: command cannot be used when Oracle Clusterware stack is up

[root@19c-rac3 bin]# crsctl stop crs

CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on '19c-rac3'

CRS-2673: Attempting to stop 'ora.crsd' on '19c-rac3'

CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on server '19c-rac3'

CRS-33673: Attempting to stop resource group 'ora.asmgroup' on server '19c-rac3'

CRS-2673: Attempting to stop 'ora.OCRVD.dg' on '19c-rac3'

CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on '19c-rac3'

CRS-2673: Attempting to stop 'ora.LISTENER_SCAN2.lsnr' on '19c-rac3'

CRS-2677: Stop of 'ora.OCRVD.dg' on '19c-rac3' succeeded

CRS-2673: Attempting to stop 'ora.asm' on '19c-rac3'

CRS-2677: Stop of 'ora.LISTENER_SCAN2.lsnr' on '19c-rac3' succeeded

CRS-2673: Attempting to stop 'ora.scan2.vip' on '19c-rac3'

CRS-2677: Stop of 'ora.LISTENER.lsnr' on '19c-rac3' succeeded

CRS-2673: Attempting to stop 'ora.chad' on '19c-rac1'

CRS-2673: Attempting to stop 'ora.chad' on '19c-rac2'

CRS-2673: Attempting to stop 'ora.chad' on '19c-rac3'

CRS-2677: Stop of 'ora.scan2.vip' on '19c-rac3' succeeded

CRS-2677: Stop of 'ora.asm' on '19c-rac3' succeeded

CRS-2673: Attempting to stop 'ora.ASMNET1LSNR_ASM.lsnr' on '19c-rac3'

CRS-2677: Stop of 'ora.chad' on '19c-rac1' succeeded

CRS-2677: Stop of 'ora.chad' on '19c-rac2' succeeded

CRS-2677: Stop of 'ora.chad' on '19c-rac3' succeeded

CRS-2673: Attempting to stop 'ora.mgmtdb' on '19c-rac3'

CRS-2677: Stop of 'ora.ASMNET1LSNR_ASM.lsnr' on '19c-rac3' succeeded

CRS-2673: Attempting to stop 'ora.asmnet1.asmnetwork' on '19c-rac3'

CRS-2677: Stop of 'ora.asmnet1.asmnetwork' on '19c-rac3' succeeded

CRS-33677: Stop of resource group 'ora.asmgroup' on server '19c-rac3' succeeded.

CRS-2677: Stop of 'ora.mgmtdb' on '19c-rac3' succeeded

CRS-2673: Attempting to stop 'ora.MGMTLSNR' on '19c-rac3'

CRS-2677: Stop of 'ora.MGMTLSNR' on '19c-rac3' succeeded

CRS-2673: Attempting to stop 'ora.19c-rac3.vip' on '19c-rac3'

CRS-2672: Attempting to start 'ora.MGMTLSNR' on '19c-rac1'

CRS-2677: Stop of 'ora.19c-rac3.vip' on '19c-rac3' succeeded

CRS-2676: Start of 'ora.MGMTLSNR' on '19c-rac1' succeeded

CRS-2672: Attempting to start 'ora.mgmtdb' on '19c-rac1'

CRS-2676: Start of 'ora.mgmtdb' on '19c-rac1' succeeded

CRS-2672: Attempting to start 'ora.chad' on '19c-rac1'

CRS-2672: Attempting to start 'ora.chad' on '19c-rac2'

CRS-2676: Start of 'ora.chad' on '19c-rac2' succeeded

CRS-2676: Start of 'ora.chad' on '19c-rac1' succeeded

CRS-2672: Attempting to start 'ora.19c-rac3.vip' on '19c-rac2'

CRS-2672: Attempting to start 'ora.scan2.vip' on '19c-rac1'

CRS-2676: Start of 'ora.19c-rac3.vip' on '19c-rac2' succeeded

CRS-2676: Start of 'ora.scan2.vip' on '19c-rac1' succeeded

CRS-2672: Attempting to start 'ora.LISTENER_SCAN2.lsnr' on '19c-rac1'

CRS-2676: Start of 'ora.LISTENER_SCAN2.lsnr' on '19c-rac1' succeeded

CRS-2673: Attempting to stop 'ora.ons' on '19c-rac3'

CRS-2677: Stop of 'ora.ons' on '19c-rac3' succeeded

CRS-2673: Attempting to stop 'ora.net1.network' on '19c-rac3'

CRS-2677: Stop of 'ora.net1.network' on '19c-rac3' succeeded

CRS-2792: Shutdown of Cluster Ready Services-managed resources on '19c-rac3' has completed

CRS-2677: Stop of 'ora.crsd' on '19c-rac3' succeeded

CRS-2673: Attempting to stop 'ora.asm' on '19c-rac3'

CRS-2673: Attempting to stop 'ora.crf' on '19c-rac3'

CRS-2673: Attempting to stop 'ora.drivers.acfs' on '19c-rac3'

CRS-2673: Attempting to stop 'ora.mdnsd' on '19c-rac3'

CRS-2677: Stop of 'ora.crf' on '19c-rac3' succeeded

CRS-2677: Stop of 'ora.drivers.acfs' on '19c-rac3' succeeded

CRS-2677: Stop of 'ora.mdnsd' on '19c-rac3' succeeded

CRS-2677: Stop of 'ora.asm' on '19c-rac3' succeeded

CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on '19c-rac3'

CRS-2677: Stop of 'ora.cluster_interconnect.haip' on '19c-rac3' succeeded

CRS-2673: Attempting to stop 'ora.ctssd' on '19c-rac3'

CRS-2673: Attempting to stop 'ora.evmd' on '19c-rac3'

CRS-2677: Stop of 'ora.ctssd' on '19c-rac3' succeeded

CRS-2677: Stop of 'ora.evmd' on '19c-rac3' succeeded

CRS-2673: Attempting to stop 'ora.cssd' on '19c-rac3'

CRS-2677: Stop of 'ora.cssd' on '19c-rac3' succeeded

CRS-2673: Attempting to stop 'ora.gipcd' on '19c-rac3'

CRS-2673: Attempting to stop 'ora.gpnpd' on '19c-rac3'

CRS-2677: Stop of 'ora.gpnpd' on '19c-rac3' succeeded

CRS-2677: Stop of 'ora.gipcd' on '19c-rac3' succeeded

CRS-2793: Shutdown of Oracle High Availability Services-managed resources on '19c-rac3' has completed

CRS-4133: Oracle High Availability Services has been stopped.

[root@19c-rac3 bin]#

[root@19c-rac3 bin]# asmcmd afd_configure

AFD-627: AFD distribution files found.

AFD-634: Removing previous AFD installation.

AFD-635: Previous AFD components successfully removed.

AFD-9294: updating file /etc/sysconfig/oracledrivers.conf

AFD-636: Installing requested AFD software.

AFD-637: Loading installed AFD drivers.

AFD-9321: Creating udev for AFD.

AFD-9323: Creating module dependencies - this may take some time.

AFD-9154: Loading 'oracleafd.ko' driver.

AFD-649: Verifying AFD devices.

AFD-9156: Detecting control device '/dev/oracleafd/admin'.

AFD-638: AFD installation correctness verified.

Modifying resource dependencies - this may take some time.

[root@19c-rac3 bin]#

[root@19c-rac3 bin]# asmcmd afd_state

ASMCMD-9526: The AFD state is 'LOADED' and filtering is 'ENABLED' on host '19c-rac3.localdomain'

[root@19c-rac3 bin]#

[root@19c-rac3 bin]# crsctl start crs

CRS-4123: Oracle High Availability Services has been started.

[root@19c-rac3 bin]#

[root@19c-rac3 bin]# ls -lrt afd*

-rwxr-xr-x 1 root oinstall     1000 Nov 12 21:27 afdroot

-rwxr-xr-x 1 root oinstall   280896 Nov 15 09:17 afdtool.bin

-rwxr-xr-x 1 root oinstall 87596064 Nov 15 09:17 afdboot

-rwxr-xr-x 1 root oinstall     2860 Dec 18 20:25 afdtool

-rwxr-xr-x 1 root oinstall      760 Dec 18 20:25 afdload

-rwxr-xr-x 1 root oinstall     3084 Dec 18 20:25 afddriverstate

[root@19c-rac3 bin]#

[root@19c-rac3 bin]# lsmod|grep oracle

oracleafd             221184  1

oracleacfs           4997120  0

oracleadvm            745472  0

oracleoks             729088  2 oracleadvm,oracleacfs

[root@19c-rac3 bin]#

[root@19c-rac3 bin]# cat /etc/os-release | head -n2

NAME="Oracle Linux Server"

VERSION="8.2"

[root@19c-rac3 bin]#

Creating diskgroup ACFSDG using asmca:

[oracle@19c-rac1 ~]$ . oraenv

ORACLE_SID = [cdbrac1] ? +ASM1

ORACLE_HOME = [/home/oracle] ? /u01/app/19.3.0/grid

The Oracle base remains unchanged with value /u01/app/oracle

[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ asmcmd

ASMCMD> lsdg

State    Type    Rebal  Sector  Logical_Sector  Block       AU  Total_MB  Free_MB  Req_mir_free_MB  Usable_file_MB  Offline_disks  Voting_files  Name

MOUNTED  EXTERN  N         512             512   4096  4194304     49136    25104                0           25104              0             Y  OCRVD/

ASMCMD>

ASMCMD> exit

[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ asmca &

[1] 166449

[oracle@19c-rac1 ~]$

Screenshots: 19cr8-ASMFD-1.JPG to 19cr8-ASMFD-4.JPG

[oracle@19c-rac1 ~]$ . oraenv

ORACLE_SID = [+ASM1] ?

ORACLE_HOME = [/home/oracle] ? /u01/app/19.3.0/grid

The Oracle base remains unchanged with value /u01/app/oracle

[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ cd $ORACLE_HOME/bin

[oracle@19c-rac1 bin]$

[oracle@19c-rac1 bin]$ pwd

/u01/app/19.3.0/grid/bin

[oracle@19c-rac1 bin]$

[oracle@19c-rac1 bin]$ asmcmd dsget

parameter:/dev/sd*

profile:/dev/sd*

[oracle@19c-rac1 bin]$

[oracle@19c-rac1 bin]$ asmcmd dsset /dev/sd*,'AFD:*'

[oracle@19c-rac1 bin]$

[oracle@19c-rac1 bin]$ asmcmd dsget

parameter:/dev/sd*, AFD:*

profile:/dev/sd*,AFD:*

[oracle@19c-rac1 bin]$

[oracle@19c-rac1 ~]$ su - root

Password:

[root@19c-rac1 ~]# . oraenv

ORACLE_SID = [root] ? +ASM1

ORACLE_HOME = [/home/oracle] ? /u01/app/19.3.0/grid

The Oracle base has been set to /u01/app/oracle

[root@19c-rac1 ~]#

[root@19c-rac1 ~]# cd $ORACLE_HOME/bin

[root@19c-rac1 bin]#

[root@19c-rac1 bin]#  ls -alrt /dev/oracleafd/disks/*

ls: cannot access '/dev/oracleafd/disks/*': No such file or directory

[root@19c-rac1 bin]# ./acfsload start

ACFS-9391: Checking for existing ADVM/ACFS installation.

ACFS-9392: Validating ADVM/ACFS installation files for operating system.

ACFS-9393: Verifying ASM Administrator setup.

ACFS-9308: Loading installed ADVM/ACFS drivers.

ACFS-9325:     Driver OS kernel version = 5.4.17-2011.0.7.el8uek.x86_64.

ACFS-9326:     Driver build number = RELEASE.

ACFS-9212:     Driver build version = 19.0.0.0.0 (19.8.0.0.0).

ACFS-9547:     Driver available build number = RELEASE.

ACFS-9548:     Driver available build version = 19.0.0.0.0 (19.8.0.0.0).

ACFS-9549:     Kernel and command versions.

Kernel:

    Build version: 19.0.0.0.0

    Build full version: 19.8.0.0.0

    Build hash:    9256567290

    Bug numbers:   NoTransactionInformation

Commands:

    Build version: 19.0.0.0.0

    Build full version: 19.8.0.0.0

    Build hash:    9256567290

    Bug numbers:   NoTransactionInformation

ACFS-9327: Verifying ADVM/ACFS devices.

ACFS-9156: Detecting control device '/dev/asm/.asm_ctl_spec'.

ACFS-9156: Detecting control device '/dev/ofsctl'.

ACFS-9294: updating file /etc/sysconfig/oracledrivers.conf

ACFS-9322: completed

[root@19c-rac1 bin]#

[root@19c-rac1 bin]# ls -alrt /dev/oracleafd/disks/*

ls: cannot access '/dev/oracleafd/disks/*': No such file or directory

[root@19c-rac1 bin]#

[root@19c-rac1 bin]# ./asmcmd afd_label ACFSDG01 /dev/sdp1

[root@19c-rac1 bin]# ./asmcmd afd_label ACFSDG02 /dev/sdq1

[root@19c-rac1 bin]# ls -alrt /dev/oracleafd/disks/

total 8

drwxrwx--- 3 oracle dba      80 Dec 18 20:07 ..

-rw-rw-r-- 1 oracle oinstall 10 Dec 18 20:51 ACFSDG01

-rw-rw-r-- 1 oracle oinstall 10 Dec 18 20:51 ACFSDG02

drwxrwxr-x 2 oracle dba      80 Dec 18 20:51 .

[root@19c-rac1 bin]#

Note: Again executed ASMCA and created ACFSDG

[root@19c-rac1 bin]# exit

logout

[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ . oraenv

ORACLE_SID = [+ASM1] ?

ORACLE_HOME = [/home/oracle] ? /u01/app/19.3.0/grid

The Oracle base remains unchanged with value /u01/app/oracle

[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ sqlplus / as sysasm

SQL*Plus: Release 19.0.0.0.0 - Production on Fri Dec 18 20:52:11 2020

Version 19.8.0.0.0

Copyright (c) 1982, 2019, Oracle.  All rights reserved.

Connected to:

Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production

Version 19.8.0.0.0

SQL>  SELECT group_number, name, compatibility, database_compatibility FROM v$asm_diskgroup;

GROUP_NUMBER NAME

------------ ------------------------------

COMPATIBILITY

------------------------------------------------------------

DATABASE_COMPATIBILITY

------------------------------------------------------------

           1 OCRVD

19.0.0.0.0

10.1.0.0.0

SQL> exit

Disconnected from Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production

Version 19.8.0.0.0

[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ sqlplus / as sysasm

SQL*Plus: Release 19.0.0.0.0 - Production on Fri Dec 18 20:59:54 2020

Version 19.8.0.0.0

Copyright (c) 1982, 2019, Oracle.  All rights reserved.

Connected to:

Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production

Version 19.8.0.0.0

SQL> set lines 100 pages 100

SQL> col GROUP_NUMBER format 99

SQL> col NAME format a10

SQL> col COMPATIBILITY format a10

SQL> col DATABASE_COMPATIBILITY format a20

SQL> SELECT group_number, name, compatibility, database_compatibility FROM v$asm_diskgroup;

GROUP_NUMBER NAME       COMPATIBIL DATABASE_COMPATIBILI

------------ ---------- ---------- --------------------

           1 OCRVD      19.0.0.0.0 10.1.0.0.0

           2 ACFSDG     19.0.0.0.0 10.1.0.0.0

SQL>  alter diskgroup ACFSDG set attribute 'compatible.rdbms'='19.0.0.0.0';

Diskgroup altered.

SQL> SELECT group_number, name, compatibility, database_compatibility FROM v$asm_diskgroup;

GROUP_NUMBER NAME       COMPATIBIL DATABASE_COMPATIBILI

------------ ---------- ---------- --------------------

           1 OCRVD      19.0.0.0.0 10.1.0.0.0

           2 ACFSDG     19.0.0.0.0 19.0.0.0.0

SQL> exit

Disconnected from Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production

Version 19.8.0.0.0

[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ asmcmd

ASMCMD> volcreate -G ACFSDG -s 19G acfs_vol

ASMCMD> volinfo --all

Diskgroup Name: ACFSDG

         Volume Name: ACFS_VOL

         Volume Device: /dev/asm/acfs_vol-296

         State: ENABLED

         Size (MB): 19456

         Resize Unit (MB): 64

         Redundancy: UNPROT

         Stripe Columns: 8

         Stripe Width (K): 1024

         Usage:

         Mountpath:

ASMCMD> exit

[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ su - root

Password:

[root@19c-rac1 ~]#

[root@19c-rac1 ~]# . oraenv

ORACLE_SID = [root] ? +ASM1

ORACLE_HOME = [/home/oracle] ? /u01/app/19.3.0/grid

The Oracle base has been set to /u01/app/oracle

[root@19c-rac1 ~]#

[root@19c-rac1 ~]# modprobe  oracleacfs

[root@19c-rac1 ~]# modprobe  oracleadvm

[root@19c-rac1 ~]# modprobe  oracleoks

[root@19c-rac1 ~]# lsmod | grep oracle

oracleafd             221184  1

oracleacfs           4997120  1

oracleadvm            745472  7

oracleoks             729088  2 oracleadvm,oracleacfs

[root@19c-rac1 ~]#

[root@19c-rac1 ~]# mkfs -t acfs /dev/asm/acfs_vol-296

mkfs.acfs: version                   = 19.0.0.0.0

mkfs.acfs: on-disk version           = 46.0

mkfs.acfs: volume                    = /dev/asm/acfs_vol-296

mkfs.acfs: volume size               = 20401094656  (  19.00 GB )

mkfs.acfs: Format complete.

[root@19c-rac1 ~]#

[root@19c-rac1 ~]# chown -R oracle:oinstall /acfs_vol

chown: cannot access '/acfs_vol': No such file or directory

[root@19c-rac1 ~]#

[root@19c-rac1 ~]# cd /

[root@19c-rac1 /]# mkdir acfs_vol

[root@19c-rac1 /]#

[root@19c-rac1 /]# chown -R oracle:oinstall /acfs_vol

[root@19c-rac1 /]#

[root@19c-rac1 /]# chmod -R 777 /acfs_vol

[root@19c-rac1 /]#

[root@19c-rac1 /]# acfsutil registry -a /dev/asm/acfs_vol-296 /acfs_vol

acfsutil registry: mount point /acfs_vol successfully added to Oracle Registry

[root@19c-rac1 /]#

[root@19c-rac1 /]# acfsutil registry -c /dev/asm/acfs_vol-296 /acfs_vol -u oracle

acfsutil registry: successfully modified ACFS registration for '/dev/asm/acfs_vol-296'

[root@19c-rac1 /]#

[root@19c-rac1 /]# mount -t acfs /dev/asm/acfs_vol-296 /acfs_vol

[root@19c-rac1 /]#

[root@19c-rac1 /]# df -h

Filesystem             Size  Used Avail Use% Mounted on

devtmpfs               4.8G  8.0K  4.8G   1% /dev

tmpfs                  4.8G  845M  3.9G  18% /dev/shm

tmpfs                  4.8G   18M  4.8G   1% /run

tmpfs                  4.8G     0  4.8G   0% /sys/fs/cgroup

/dev/mapper/ol-root     50G   26G   25G  51% /

/dev/mapper/ol-home     42G  7.4G   35G  18% /home

/dev/sda1             1014M  291M  724M  29% /boot

tmpfs                  968M  1.2M  967M   1% /run/user/42

tmpfs                  968M  4.6M  964M   1% /run/user/54321

/dev/asm/acfs_vol-296   19G  828M   19G   5% /acfs_vol

[root@19c-rac1 /]#

[root@19c-rac1 /]# mount | grep acfs

/dev/asm/acfs_vol-296 on /acfs_vol type acfs (rw,relatime,device,rootsuid,ordered)

[root@19c-rac1 /]#

[root@19c-rac1 /]# cat /etc/fstab

#

# /etc/fstab

# Created by anaconda on Sun May 17 21:14:39 2020

#

# Accessible filesystems, by reference, are maintained under '/dev/disk/'.

# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info.

#

# After editing this file, run 'systemctl daemon-reload' to update systemd

# units generated from this file.

#

/dev/mapper/ol-root     /                       xfs     defaults        0 0

UUID=539eae26-4e39-4888-8697-cb027503ef1b /boot                   xfs     defaults        0 0

/dev/mapper/ol-home     /home                   xfs     defaults        0 0

/dev/mapper/ol-swap     swap                    swap    defaults        0 0

[root@19c-rac1 /]#

[root@19c-rac1 /]# vi /etc/fstab

[root@19c-rac1 /]# cat /etc/fstab

#

# /etc/fstab

# Created by anaconda on Sun May 17 21:14:39 2020

#

# Accessible filesystems, by reference, are maintained under '/dev/disk/'.

# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info.

#

# After editing this file, run 'systemctl daemon-reload' to update systemd

# units generated from this file.

#

/dev/mapper/ol-root     /                       xfs     defaults        0 0

UUID=539eae26-4e39-4888-8697-cb027503ef1b /boot xfs     defaults        0 0

/dev/mapper/ol-home     /home                   xfs     defaults        0 0

/dev/mapper/ol-swap     swap                    swap    defaults        0 0

/dev/asm/acfs_vol-296   /acfs_vol               acfs    defaults        0 0

[root@19c-rac1 /]#

[root@19c-rac1 /]# crsctl stat res -t | grep acfs

ora.acfsdg.acfs_vol.acfs

               ONLINE  ONLINE       19c-rac1                 mounted on /acfs_vol

               ONLINE  ONLINE       19c-rac2                 mounted on /acfs_vol

               ONLINE  ONLINE       19c-rac3                 mounted on /acfs_vol

[root@19c-rac1 /]#

[root@19c-rac1 /]# crsctl stat res -t | grep ACFS

ora.ACFSDG.ACFS_VOL.advm

ora.ACFSDG.dg(ora.asmgroup)

[root@19c-rac1 /]#

[root@19c-rac1 /]# srvctl status diskgroup -diskgroup ACFSDG -verbose

Disk Group ACFSDG is running on 19c-rac1,19c-rac3,19c-rac2

[root@19c-rac1 /]#

[root@19c-rac1 /]# exit

logout

[oracle@19c-rac1 ~]$ . oraenv

ORACLE_SID = [+ASM1] ?

ORACLE_HOME = [/home/oracle] ? /u01/app/19.3.0/grid

The Oracle base remains unchanged with value /u01/app/oracle

[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ sqlplus / as sysasm

SQL*Plus: Release 19.0.0.0.0 - Production on Fri Dec 18 21:10:08 2020

Version 19.8.0.0.0

Copyright (c) 1982, 2019, Oracle.  All rights reserved.

Connected to:

Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production

Version 19.8.0.0.0

SQL> set lines 100 pages 100

SQL>  col name for a20

col path for a18

col library for a60

set lines 160

SQL> select name, path, library from v$asm_disk where group_number <> 0;

NAME                 PATH               LIBRARY

-------------------- ------------------ ------------------------------------------------------------

OCRVD_0002           /dev/sdl1          System

OCRVD_0003           /dev/sdm1          System

OCRVD_0000           /dev/sdj1          System

OCRVD_0001           /dev/sdk1          System

ACFSDG01             AFD:ACFSDG01       AFD Library - Generic , version 3 (KABI_V3)

ACFSDG02             AFD:ACFSDG02       AFD Library - Generic , version 3 (KABI_V3)

6 rows selected.

SQL> exit

Disconnected from Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production

Version 19.8.0.0.0

[oracle@19c-rac1 ~]$ cd $ORACLE_HOME/bin

[oracle@19c-rac1 bin]$ ./asmcmd afd_lsdsk

--------------------------------------------------------------------------------

Label                     Filtering   Path

=========================================================================ACFSDG01                    ENABLED   /dev/sdp1

ACFSDG02                    ENABLED   /dev/sdq1

[oracle@19c-rac1 bin]$

[oracle@19c-rac2 ~]$ . oraenv

ORACLE_SID = [+ASM2] ?

ORACLE_HOME = [/home/oracle] ? /u01/app/19.3.0/grid

The Oracle base remains unchanged with value /u01/app/oracle

[oracle@19c-rac2 ~]$

[oracle@19c-rac2 ~]$ cd $ORACLE_HOME/bin

[oracle@19c-rac2 bin]$ asmcmd dsget

parameter:/dev/sd*, AFD:*

profile:/dev/sd*,AFD:*

[oracle@19c-rac2 bin]$

[oracle@19c-rac2 bin]$ cd

[oracle@19c-rac2 ~]$ su - root

Password:

[root@19c-rac2 ~]#

[root@19c-rac2 /]# df -h

Filesystem             Size  Used Avail Use% Mounted on

devtmpfs               4.8G  8.0K  4.8G   1% /dev

tmpfs                  4.8G  845M  3.9G  18% /dev/shm

tmpfs                  4.8G   26M  4.7G   1% /run

tmpfs                  4.8G     0  4.8G   0% /sys/fs/cgroup

/dev/mapper/ol-root     50G   25G   26G  49% /

/dev/sda1             1014M  290M  725M  29% /boot

/dev/mapper/ol-home     42G  1.8G   40G   5% /home

tmpfs                  968M  1.2M  967M   1% /run/user/42

tmpfs                  968M     0  968M   0% /run/user/54321

/dev/asm/acfs_vol-296   19G  828M   19G   5% /acfs_vol

[root@19c-rac2 /]#

[root@19c-rac2 /]# cat /etc/fstab

#

# /etc/fstab

# Created by anaconda on Sun May 17 21:14:39 2020

#

# Accessible filesystems, by reference, are maintained under '/dev/disk/'.

# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info.

#

# After editing this file, run 'systemctl daemon-reload' to update systemd

# units generated from this file.

#

/dev/mapper/ol-root     /                       xfs     defaults        0 0

UUID=539eae26-4e39-4888-8697-cb027503ef1b /boot                   xfs     defaults        0 0

/dev/mapper/ol-home     /home                   xfs     defaults        0 0

/dev/mapper/ol-swap     swap                    swap    defaults        0 0

[root@19c-rac2 /]#

[root@19c-rac2 /]# vi /etc/fstab

[root@19c-rac2 /]# cat /etc/fstab

#

# /etc/fstab

# Created by anaconda on Sun May 17 21:14:39 2020

#

# Accessible filesystems, by reference, are maintained under '/dev/disk/'.

# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info.

#

# After editing this file, run 'systemctl daemon-reload' to update systemd

# units generated from this file.

#

/dev/mapper/ol-root     /                       xfs     defaults        0 0

UUID=539eae26-4e39-4888-8697-cb027503ef1b /boot                   xfs     defaults        0 0

/dev/mapper/ol-home     /home                   xfs     defaults        0 0

/dev/mapper/ol-swap     swap                    swap    defaults        0 0

/dev/asm/acfs_vol-296   /acfs_vol               acfs    defaults        0 0

[root@19c-rac2 /]#

[oracle@19c-rac3 ~]$ . oraenv

ORACLE_SID = [+ASM3] ?

ORACLE_HOME = [/home/oracle] ? /u01/app/19.3.0/grid

The Oracle base remains unchanged with value /u01/app/oracle

[oracle@19c-rac3 ~]$

[oracle@19c-rac3 ~]$ cd $ORACLE_HOME/bin

[oracle@19c-rac3 bin]$

[oracle@19c-rac3 bin]$ asmcmd dsget

parameter:/dev/sd*, AFD:*

profile:/dev/sd*,AFD:*

[oracle@19c-rac3 ~]$ df -h

Filesystem             Size  Used Avail Use% Mounted on

devtmpfs               4.8G  8.0K  4.8G   1% /dev

tmpfs                  4.8G  845M  3.9G  18% /dev/shm

tmpfs                  4.8G   34M  4.7G   1% /run

tmpfs                  4.8G     0  4.8G   0% /sys/fs/cgroup

/dev/mapper/ol-root     50G   25G   26G  49% /

/dev/mapper/ol-home     42G  1.8G   40G   5% /home

/dev/sda1             1014M  290M  725M  29% /boot

tmpfs                  968M  1.2M  967M   1% /run/user/42

tmpfs                  968M     0  968M   0% /run/user/54321

/dev/asm/acfs_vol-296   19G  828M   19G   5% /acfs_vol

[oracle@19c-rac3 ~]$

[oracle@19c-rac3 ~]$ su - root

Password:

[root@19c-rac3 ~]#

[root@19c-rac3 ~]# cat /etc/fstab

#

# /etc/fstab

# Created by anaconda on Sun May 17 21:14:39 2020

#

# Accessible filesystems, by reference, are maintained under '/dev/disk/'.

# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info.

#

# After editing this file, run 'systemctl daemon-reload' to update systemd

# units generated from this file.

#

/dev/mapper/ol-root     /                       xfs     defaults        0 0

UUID=539eae26-4e39-4888-8697-cb027503ef1b /boot xfs     defaults        0 0

/dev/mapper/ol-home     /home                   xfs     defaults        0 0

/dev/mapper/ol-swap     swap                    swap    defaults        0 0

[root@19c-rac3 ~]#

[root@19c-rac3 ~]# vi /etc/fstab

[root@19c-rac3 ~]# cat /etc/fstab

#

# /etc/fstab

# Created by anaconda on Sun May 17 21:14:39 2020

#

# Accessible filesystems, by reference, are maintained under '/dev/disk/'.

# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info.

#

# After editing this file, run 'systemctl daemon-reload' to update systemd

# units generated from this file.

#

/dev/mapper/ol-root     /                       xfs     defaults        0 0

UUID=539eae26-4e39-4888-8697-cb027503ef1b /boot xfs     defaults        0 0

/dev/mapper/ol-home     /home                   xfs     defaults        0 0

/dev/mapper/ol-swap     swap                    swap    defaults        0 0

/dev/asm/acfs_vol-296   /acfs_vol               acfs    defaults        0 0

[root@19c-rac3 ~]#

Final Status