Problem
I have successfully upgraded the ASMlib from v2 to v3.1 in a 2-Node Oracle RAC 19c, originally running with Oracle Linux 8.10 and Kernel UEK6 (5.4.17). Since the cluster was properly working with the new ASMlib, I decided to change the Kernel to UEK7 (5.15.0) to take advantage of the new interface KABI_V3 based on IO_URING.
However, after set the new Kernel and reboot the server, the CRS didn’t started because the CSSD process was unable to find the Vote Disks:
[root@c02db01 ~]# tail -100f /u01/app/grid/diag/crs/c02db01/crs/trace/alert.log......2026-04-03 11:09:34.097 [CSSDAGENT(10013)]CRS-8500: Oracle Clusterware CSSDAGENT process is starting with operating system process ID 100132026-04-03 11:09:39.369 [OCSSD(10125)]CRS-8500: Oracle Clusterware OCSSD process is starting with operating system process ID 101252026-04-03 11:09:40.620 [OCSSD(10125)]CRS-1713: CSSD daemon is started in hub mode2026-04-03 11:09:41.066 [OCSSD(10125)]CRS-1714: Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /u01/app/grid/diag/crs/c02db01/crs/trace/ocssd.trc2026-04-03 11:09:56.093 [OCSSD(10125)]CRS-1714: Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /u01/app/grid/diag/crs/c02db01/crs/trace/ocssd.trc2026-04-03 11:10:11.109 [OCSSD(10125)]CRS-1714: Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /u01/app/grid/diag/crs/c02db01/crs/trace/ocssd.trc2026-04-03 11:10:26.122 [OCSSD(10125)]CRS-1714: Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /u01/app/grid/diag/crs/c02db01/crs/trace/ocssd.trc2026-04-03 11:10:41.134 [OCSSD(10125)]CRS-1714: Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /u01/app/grid/diag/crs/c02db01/crs/trace/ocssd.trc2026-04-03 11:10:56.153 [OCSSD(10125)]CRS-1714: Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /u01/app/grid/diag/crs/c02db01/crs/trace/ocssd.trc2026-04-03 11:11:11.167 [OCSSD(10125)]CRS-1714: Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /u01/app/grid/diag/crs/c02db01/crs/trace/ocssd.trc2026-04-03 11:11:26.180 [OCSSD(10125)]CRS-1714: Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /u01/app/grid/diag/crs/c02db01/crs/trace/ocssd.trc
So I opened a second SSH session and checked if the disks was properly listed by ASMlib, and it looks fine here:
[root@c02db01 ~]# oracleasm listdisksDATA1DATA2DATA3DATA4DATA5DATA6RECO1RECO2
But when I executed the discover command, only one disk was listed:
[root@c02db01 ~]# oracleasm discoverUsing ASMLIB from /opt/oracle/extapi/64/asm/orcl/1/libasm.so[ASM Library - Linux, version 3.1.1 (KABI_V3), 2025-07-24 22:52]Discovered disk: ORCL:RECO2 [16775168 blocks (8588886016 bytes), blksz 512/512, maxio 1310720 bytes, integrity unsupported][root@c02db01 ~]#
Solution
So I tried just to manually rescan the disks and check the discover again.
Now all the 6 DATA and 2 RECO disks were properly discovered:
[root@c02db01 ~]# oracleasm scandisksReloading disk partitions: doneCleaning any stale ASM disks...Scanning system for ASM disks...[root@c02db01 ~]# [root@c02db01 ~]# [root@c02db01 ~]# oracleasm discoverUsing ASMLIB from /opt/oracle/extapi/64/asm/orcl/1/libasm.so[ASM Library - Linux, version 3.1.1 (KABI_V3), 2025-07-24 22:52]Discovered disk: ORCL:DATA6 [33550336 blocks (17177772032 bytes), blksz 512/512, maxio 1310720 bytes, integrity unsupported]Discovered disk: ORCL:RECO1 [16775168 blocks (8588886016 bytes), blksz 512/512, maxio 1310720 bytes, integrity unsupported]Discovered disk: ORCL:DATA2 [33552384 blocks (17178820608 bytes), blksz 512/512, maxio 1310720 bytes, integrity unsupported]Discovered disk: ORCL:DATA3 [33552384 blocks (17178820608 bytes), blksz 512/512, maxio 1310720 bytes, integrity unsupported]Discovered disk: ORCL:DATA4 [33552384 blocks (17178820608 bytes), blksz 512/512, maxio 1310720 bytes, integrity unsupported]Discovered disk: ORCL:DATA5 [33550336 blocks (17177772032 bytes), blksz 512/512, maxio 1310720 bytes, integrity unsupported]Discovered disk: ORCL:RECO2 [16775168 blocks (8588886016 bytes), blksz 512/512, maxio 1310720 bytes, integrity unsupported]Discovered disk: ORCL:DATA1 [33552384 blocks (17178820608 bytes), blksz 512/512, maxio 1310720 bytes, integrity unsupported][root@c02db01 ~]#
When I came to the first session where I was monitoring the CRS alert log, I noticed the CSSD was able to find the vote disks during the automatic retries:
...2026-04-03 11:13:56.325 [OCSSD(10125)]CRS-1714: Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /u01/app/grid/diag/crs/c02db01/crs/trace/ocssd.trc2026-04-03 11:14:11.343 [OCSSD(10125)]CRS-1714: Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /u01/app/grid/diag/crs/c02db01/crs/trace/ocssd.trc2026-04-03 11:14:30.592 [OCSSD(10125)]CRS-1601: CSSD Reconfiguration complete. Active nodes are c02db01 c02db02 .2026-04-03 11:14:32.153 [OCSSD(10125)]CRS-1720: Cluster Synchronization Services daemon (CSSD) is ready for operation.2026-04-03 11:14:32.859 [OCTSSD(17151)]CRS-8500: Oracle Clusterware OCTSSD process is starting with operating system process ID 171512026-04-03 11:14:32.926 [OCTSSD(17151)]CRS-2403: The Cluster Time Synchronization Service on host c02db01 is in observer mode.2026-04-03 11:14:34.752 [OCTSSD(17151)]CRS-2401: The Cluster Time Synchronization Service started on host c02db01...
And the CRS successfully started after it:
[root@c02db01 ~]# crsctl check crsCRS-4638: Oracle High Availability Services is onlineCRS-4537: Cluster Ready Services is onlineCRS-4529: Cluster Synchronization Services is onlineCRS-4533: Event Manager is online[root@c02db01 ~]# [root@c02db01 ~]# crsctl check cluster -all**************************************************************c02db01:CRS-4537: Cluster Ready Services is onlineCRS-4529: Cluster Synchronization Services is onlineCRS-4533: Event Manager is online**************************************************************c02db02:CRS-4537: Cluster Ready Services is onlineCRS-4529: Cluster Synchronization Services is onlineCRS-4533: Event Manager is online**************************************************************
Final tips
If you are doing a similar upgrade, I would recommend follow the approach that I used in the second node in this cluster.
First, check some basic requirements I shared here “How to Update Your Existing ASM Installation to ASMlib v3 in Oracle Linux 8.10“
Disable the CRS autostart before reboot the server, so the Clusterware will not start automatically after the server reboot.
crsctl stop crs crsctl disable crs
Do your kernel update or changes and reboot activity here.
After the reboot, you can double check if the ASMlib has recognized all disks properly:
oracleasm status oracleasm listdisks oracleasm discover
If you didn’t executed a reconfiguration in the ASMlib after the upgrade, you can do this now, which should update the ASMlib configuration file with new parameters:
oracleasm exit oracleasm configure -i
Accept all existing configurations as default, and start the ASMlib again:
oracleasm init oracleasm status oracleasm discover
If everything looks good at ASMlib level, you can enable and start the Clusterware again:
crsctl enable crs crsctl start crs -wait