ai-center
2022.4
true
- Getting Started
- Network requirements
- Single-node requirements and installation
- Multi-node requirements and installation
- Post-installation
- Accessing AI Center
- Provision an AI Center tenant
- Updating Orchestrator and Identity Server certificates
- Resizing PVC
- Adding a new node to the cluster
- ML packages offline installation
- Configuring the cluster
- Configuring the FQDN post-installation
- Backing up and restoring the cluster
- Using the monitoring stack
- Setting up a Kerberos authentication
- Provisioning a GPU
- Using the configuration file
- Node scheduling
- Migration and upgrade
- Basic Troubleshooting Guide
AI Center Installation Guide
Last updated Jun 6, 2024
Rollback on error
Note:
Reverting to the previous version may be needed in the following scenarios:
- There is failure during upgrade, which is beyond recovery;
- There is an accidental data loss or corruption;
- Upgraded cluster is unusable.
To roll back to a previous version, take the following steps:
- Make a copy of
cluster_config.json. - Verify that the etcd backup data is present on the primary server at the following location:
/mnt/backup/backup/<etcdBackupPath>/<node-name>/snapshots.etcdBackupPath– the same as the one specified inbackup.jsonwhile enabling the backup.node-name– the hostname of the server node.
- To check the
backupstorevolume backup directory, run the following command:ls /mnt/backup/backupstorels /mnt/backup/backupstore - Uninstall AI Center on all the nodes.
- Now restore to the previous version of AI Center. Make sure to follow the Backing up and restoring the cluster documentation available for that specific version.
- Once the restore is complete, make sure you disable the maintenance mode:
/path/to/old-installer/configureUiPathAS.sh disable-maintenance-mode/path/to/old-installer/configureUiPathAS.sh disable-maintenance-mode - Verify that maintenance mode is disabled by executing the following command:
/path/to/old-installer/configureUiPathAS.sh is-maintenance-enabled/path/to/old-installer/configureUiPathAS.sh is-maintenance-enabled - During the upgrade, scheduled backups were disabled on the primary node. To enable them again, run the following command:
/path/to/old-installer/configureUiPathAS.sh resume-scheduled-backups/path/to/old-installer/configureUiPathAS.sh resume-scheduled-backups
