Patching an ExaXS VM Cluster

Introduction

The ExaXS is an Exadata VM Cluster based on shared Exascale Infrastructure on OCI. When you are connected to an ExaXS VM, you will find everything is similar to any other virtualized Exadata, such as when you are connected to an ExaCS or ExaCC. The main difference with ExaXS is that you don’t have any visibility into the Exadata Infrastructure itself, such as the physical compute and storage nodes; you only see the VMs.

Based on my own experience testing this process multiple times, when you need to patch an ExaXS VM Cluster, the maintenance workflow and tasks are pretty much the same as what we have for DomU on ExaCS and ExaCC: we need to patch the OS in the VMs, the Grid Infrastructure, and the databases. All this work is automated by OCI, but we need to start it manually.

In this blog post, I will demonstrate the process to patch a VM Cluster with 2 VMs. In subsequent posts, I will cover the process to patch the Grid Infrastructure and the databases using the out-of-place approach.

Patch Workflow

The workflow to completely patch a ExaXS, or even on other Exadata Cloud services like ExaCS/ExaCC, usually follow this order:

• Patch the Operating System
• Patch the Grid Infrastructure
• Patch the Databases

We can patch the OS before the Grid and Database most of the time, but sometimes we can face restrictions regarding the minimum Grid version required by the new OS image. In those cases, we need to patch the Grid Infrastructure before the OS.

To identify those cases, we just need to run the precheck for the target OS image. If the current Grid version isn’t supported, the OCI console will tell you that you need to patch the Grid to a minimum RU first.

Rolling Behavior and Driving Node Concept

Rolling Behavior

The automation usually patch in rolling mode from the first to the last node. For example, if you have a 4 node cluster, the nodes will be patched in this order: 1,2,3,4.

Driving Node

Behind of scene, the OCI automation will execute the patchmgr utility to apply the Exadata image update. The patchmgr needs to be executed from a remote server, so the utility can reboot the node during the patch and resume the work.

For a 2-node VM Cluster, the automation will usually execute the patchmgr from node 2 to patch the node 1, and then will use the node 1 to patch the node 2. If you have more than 2 nodes, the automation will continue executing the patchmgr from node 1 to patch all the other nodes.

Checking the Current Version

We can find the current version in the VM Cluster Information menu in the OCI console:

So here we have the Exadata image running on version 24.1.14 and the Grid on 23.7.

We can also check the Exadata image version using the imageinfo utility with the following command:

dcli -g ~/dbs_group -l opc "sudo /opt/oracle.cellos/imageinfo"

Example:

[opc@guob011-rpdrb ~]$ dcli -g ~/dbs_group -l opc "sudo /opt/oracle.cellos/imageinfo"
guob011: 
guob011: Kernel version: 5.4.17-2136.343.5.3.el8uek.x86_64 #3 SMP Mon Jun 30 00:11:58 PDT 2025 x86_64
guob011: Image kernel version: 5.4.17-2136.343.5.3.el8uek
guob011: Image version: 24.1.14.0.0.250706
guob011: Image activated: 2025-08-02 19:14:24 -0300
guob011: Image status: success
guob011: Exadata software version: 24.1.14.0.0.250706
guob011: Node type: GUEST
guob011: System partition on device: /dev/mapper/VGExaDb-LVDbSys1
guob011: 
guob012: 
guob012: Kernel version: 5.4.17-2136.343.5.3.el8uek.x86_64 #3 SMP Mon Jun 30 00:11:58 PDT 2025 x86_64
guob012: Image kernel version: 5.4.17-2136.343.5.3.el8uek
guob012: Image version: 24.1.14.0.0.250706
guob012: Image activated: 2026-02-21 17:09:52 -0300
guob012: Image status: success
guob012: Exadata software version: 24.1.14.0.0.250706
guob012: Node type: GUEST
guob012: System partition on device: /dev/mapper/VGExaDb-LVDbSys1
guob012: 

Precheck

Go to Updates (OS), choose the target version you want to apply and click in the 3 point at right, then click in Precheck to run the online prereqs for this patch.

Just confirm to proceed:

This process will take around 30 minutes per VM.

You can monitor the progress in the Work requests menu, but don’t expect a very accurate % here. You will see the process rapidly showing 65% and then still in this percent until the precheck is completed.

If you are interested to really check the progress of precheck in more detail, you can connect in the VM using SSH and check the patchmgr logs that are stored in the /u02 filesystem.

Applying the OS Patch

When the precheck is successfully completed, you can now click in the Apply Exadata OS Image Update:

And you should see a confirmation menu like this:

Don’t worry about the warning related to last precheck if you have executed this recently, like in the past few hours.

At this point, the VM Cluster state should be “Updating”. Like I mentioned in the precheck section, you can monitor the progress checking the Worklogs, but if I really need to monitor the progress, I usually do this by checking the patchmgr logs.

How to Monitor the Patchmgr Logs

TIP: The same is valid for ExaCS and ExaCC

Please be aware the automation will always execute a precheck prior apply the patch, even if we have completed a precheck manually recently, so the overall maintenance time will be longer. Also the first logs you see will be probably related to precheck and not the apply itself.

Since the automation usually patch in rolling mode from the first to the last node, so in the very beginning of process, you should see the logs in the second node first, where patchmgr will being executed to analyze the node 1.

Here I’m connected in the node 1 as root user:

List the content under /u02 filesystem, and you should see some folder starting with dbserver.patch.zip_exadata with the specific version you selected in the OCI console:

[root@guob011-rpdrb ~]# ls -lrt /u02/
total 36
drwx------ 2 oracle oinstall 16384 Apr 12  2025 lost+found
drwxr-xr-x 4 oracle oinstall  4096 Apr 12  2025 opt
drwxr-x--x 3 root   root      4096 Apr 12  2025 oracle.ahf
drwxrwxr-x 3 oracle oinstall  4096 Apr 12  2025 app
drwxr-xr-x 3 root   root      4096 Jun 25 13:07 dbserver.patch.zip_exadata_ol8_25.2.9.0.0.260515.1_Linux-x86-64.zip
-rw-r--r-- 1 root   root      1262 Jun 25 17:44 corruption_check

Checking under this folder, you can find the extracted patch folder, and then find the patchmgr folder where all the logs are written:

[root@guob011-rpdrb ~]# 
[root@guob011-rpdrb ~]# ls -lrt /u02/dbserver.patch.zip_exadata_ol8_25.2.9.0.0.260515.1_Linux-x86-64.zip/
total 8564
-rw-r--r-- 1 root root 8765372 Jun 25 13:07 dbserver.patch.zip
drwxr-xr-x 5 root root    4096 Jun 25 17:34 dbserver_patch_260515.1
[root@guob011-rpdrb ~]# 
[root@guob011-rpdrb ~]# 
[root@guob011-rpdrb ~]# ls -lrt /u02/dbserver.patch.zip_exadata_ol8_25.2.9.0.0.260515.1_Linux-x86-64.zip/dbserver_patch_260515.1/
total 13720076
-r-xr-x--- 1 root root        353 Feb 22  2023 dbserver_backup_excludes.list
-r--r--r-- 1 root root      50247 May 15 23:07 imageLogger
-r-xr-x--- 1 root root       5824 May 15 23:07 get_hardware_property.sh
-r-xr-x--- 1 root root      72600 May 15 23:07 exadata.img.hw
-r--r--r-- 1 root root      14693 May 15 23:07 exadata.img.env
-r--r----- 1 root root       6133 May 15 23:07 ExaXMLNode.pm
-r-xr-xr-x 1 root root       7903 May 15 23:08 exa_json_utils.py
-r-xr-xr-x 1 root root      16503 May 15 23:08 error_code_json_functions
-r-xr-xr-x 1 root root      75757 May 15 23:08 dcli
-r-xr-xr-x 1 root root       1570 May 15 23:08 ExadataSendNotification.pm
-r-xr-xr-x 1 root root      63754 May 15 23:08 ExadataImageNotification.pl
-r-xr-xr-x 1 root root      13811 May 15 23:08 timing_rpt_template.html
-r-xr-xr-x 1 root root      68795 May 15 23:08 timing_report
-r-xr-xr-x 1 root root      14860 May 15 23:08 timing_record.sh
-r-xr-xr-x 1 root root      27284 May 15 23:08 patchmgr_functions
-r-xr-xr-x 1 root root     110955 May 15 23:08 patchReport.py
-r-xr-xr-x 1 root root      83753 May 15 23:08 master_error_code.json
-r-xr-xr-x 1 root root     717620 May 15 23:27 patchmgr
-r-xr-xr-x 1 root root        677 May 15 23:31 install.sh.sample
-r-xr-xr-x 1 root root     123790 May 15 23:31 dbserver_backup_mandatory.list
-rw-r--r-- 1 root root        580 May 15 23:31 README.txt
-rw-r--r-- 1 root root 7211849728 May 16 00:05 exadata_ol8_base_repo_25.2.9.0.0.260515.1.iso
-rw-r--r-- 1 root root     493346 May 16 00:14 README.html
-rw-r--r-- 1 root root       1002 May 16 13:48 md5sum_files.lst
-rw-rw-r-- 1 root root    8463356 May 16 13:48 dbnodeupdate.zip
-r--r--r-- 1 root root 6826984887 Jun 25 13:08 exadata_ol8_25.2.9.0.0.260515.1_Linux-x86-64.zip
drwxr-xr-x 6 root root       4096 Jun 25 14:00 patchmgr_log_4d17e696-db6a-4818-a1f0-2c98c96c97b4_guob011-rpdrb
drwxr-xr-x 2 root root       4096 Jun 25 17:33 patch_states_data
-rw-r--r-- 1 root root         37 Jun 25 17:34 node_list_01d288b8-f559-415d-8b82-7fd8dad93e25
drwxr-xr-x 5 root root       4096 Jun 25 17:37 patchmgr_log_01d288b8-f559-415d-8b82-7fd8dad93e25

Check the latest patchmgr_log* folder, where is potentially used by the current patchmgr session:

[root@guob011-rpdrb dbserver_patch_260515.1]# cd patchmgr_log_01d288b8-f559-415d-8b82-7fd8dad93e25/
[root@guob011-rpdrb patchmgr_log_01d288b8-f559-415d-8b82-7fd8dad93e25]# 
[root@guob011-rpdrb patchmgr_log_01d288b8-f559-415d-8b82-7fd8dad93e25]# 
[root@guob011-rpdrb patchmgr_log_01d288b8-f559-415d-8b82-7fd8dad93e25]# ls -lrt
total 192
-rw-r--r-- 1 root root     90 Jun 25 17:34 patchmgr.stderr
-rw-r--r-- 1 root root    979 Jun 25 17:34 drivernode_guob011-rpdrb_Precheck_details_250626173414.json
drwxr-xr-x 2 root root   4096 Jun 25 17:34 notifications
drwxr-xr-x 2 root root   4096 Jun 25 17:37 timing
-rw-r--r-- 1 root root   1592 Jun 25 17:37 patchmgr.stdout
-rw-r--r-- 1 root root   1973 Jun 25 17:37 PatchmgrConsole.out
-rw-r--r-- 1 root root  22499 Jun 25 17:37 ExadataImageNotification.pl.trc
-rw-r--r-- 1 root root   4947 Jun 25 17:37 ExadataImageNotification.pl.log
-rw-r--r-- 1 root root      0 Jun 25 17:37 _wip_stdout
-rw------- 1 root root      0 Jun 25 17:37 _wip_stdout.5YaWU22C
-rw-r--r-- 1 root root  12833 Jun 25 17:37 patchmgr.log
-rw-r--r-- 1 root root 116438 Jun 25 17:37 patchmgr.trc
drwx------ 2 root root   4096 Jun 25 17:37 tmp

Example:

From node 2:

After the precheck is completed for all nodes, the automation will execute the patchmgr again with the -upgrade option, this is the patch itself. Note that each patchmgr execution results in a new dedicated log folder. You should go back to the main folder and check for the new logs, for example:

[root@guob012-hxuig patchmgr_log_2087ddd5-bd71-4ee6-b52f-f90393b36a3b]# cd ..
[root@guob012-hxuig dbserver_patch_260515.1]# 
[root@guob012-hxuig dbserver_patch_260515.1]# pwd
/u02/dbserver.patch.zip_exadata_ol8_25.2.9.0.0.260515.1_Linux-x86-64.zip/dbserver_patch_260515.1
[root@guob012-hxuig dbserver_patch_260515.1]# 
[root@guob012-hxuig dbserver_patch_260515.1]# ls -lrt | grep patchmgr_log
drwxr-xr-x. 6 root root       4096 Jun 25 14:14 patchmgr_log_4d17e696-db6a-4818-a1f0-2c98c96c97b4_guob012-hxuig
drwxr-xr-x. 6 root root       4096 Jun 25 18:10 patchmgr_log_01d288b8-f559-415d-8b82-7fd8dad93e25_guob012-hxuig
drwxr-xr-x. 5 root root       4096 Jun 25 18:19 patchmgr_log_2087ddd5-bd71-4ee6-b52f-f90393b36a3b

You should see -upgrade instead of -precheck here:

When all nodes are completed, after a few minutes, you should see the worklogs showing completed in OCI console:

And the information confirming the patch has been applied under Update History:

Conclusion

In this blog post we completed the steps to run the precheck and apply the OS updates in the VM Cluster. In next posts, I will cover the steps in how to patch the Grid and database.