To perform maintenance on Ceph that brings Ceph down, all VMs must be stopped beforehand and restarted after maintenance has been finished.
Step by Step Guide
Mark all hypervisors as not accepting workloads so no new resources can be created.
curl -sS -X GET -H "apikey: $ADMIN_APIKEY" https://$API_HOST/v1/base-operator/host/list | \
jq -r '.[]|.uuid' | \
xargs -n 1 -I {} curl -X PUT -H "apikey: $ADMIN_APIKEY" https://$API_HOST/v1/base-operator/admin/host_flags -d uuid={} -d is_accepting_workloads=0
Get a list of all currently running VMs, store the result to a file.
curl -sS -H "apikey: $ADMIN_APIKEY" -X GET https://$API_HOST/v1/user-resource/vm/all?status=running > running_vms.json
Stop all running VMs. This can take time. Not all VMs might agree to stop. These must be stopped forcefully.
cat running_vms.json | jq -r '.[]|.uuid' | \
xargs -n 1 -I {} curl -X POST -H "apikey: $ADMIN_APIKEY" https://$API_HOST/v1/.../stop -d uuid={}
It is now safe to perform maintenance and bring Ceph offline.
Maintenance-maintenance-maintenance-...
Once Ceph is available again, mark hosts as accepting workloads.
curl -sS -X GET -H "apikey: $ADMIN_APIKEY" https://$API_HOST/v1/base-operator/host/list | \
jq -r '.[]|.uuid' | \
xargs -n 1 -I {} curl -X PUT -H "apikey: $ADMIN_APIKEY" https://$API_HOST/v1/base-operator/admin/host_flags -d uuid={} -d is_accepting_workloads=1
Finally, start all VMs that were running before.
cat running_vms.json | jq -r '.[]|.uuid' | \
xargs -n 1 -I {} curl -X POST -H "apikey: $ADMIN_APIKEY" https://$API_HOST/v1/.../start -d uuid={}