MCM Platform — Helper Scripts & CommandsCopy Anchor Link

This guide serves as a comprehensive operational runbook for managing, diagnosing, and maintaining the Multi-Cloud Management (MCM) platform. It details environment-specific control commands (3-Tier vs. Single VM), host-level resource diagnostics, container platform statistics, SSL/TLS certificate inspections, and the mcm command-line tool.

1. Platform Control & Monitoring (Docker Swarm & Systemd)Copy Anchor Link

These commands are used to check the cluster topology, service health, scheduling, and to manage the platform's background system daemon.

Cluster Node StatusCopy Anchor Link

Verify that all virtual machines are joined, ready, and active inside the cluster management plane (run on Access Node for 3-tier):

docker node ls

• Explanation: Lists all nodes participating in the orchestrated cluster, their status (Ready/Down), availability (Active/Drain), and their role (e.g., Access Node as Leader Manager, Worker nodes as Worker).

Service Replica HealthCopy Anchor Link

Verify that all backend application and database services are running with their target replicas (run on Access Node for 3-tier):

docker service ls

• Explanation: Returns a high-level list of all deployed services, their scheduling mode, and replica counts. Healthy services display 1/1 (indicating the service task is running and has passed health checks). Initialization tasks (prefixed with mcm_init-) transition to 0/1 or exit once they complete their one-time database and search-index setup.

Stack Task MappingCopy Anchor Link

Inspect the scheduling and actual node mapping of container instances across the cluster (run on Access Node for 3-tier):

docker stack ps mcm

• Explanation: Displays exactly which node each service container is currently running on, its current state (Running, Preparing, or Shutdown), and error logs if a container failed to start.

Tail Service Application LogsCopy Anchor Link

To inspect application-level outputs and debug issues, tail the stdout/stderr logs of a specific cluster service (run on Access Node for 3-tier):

# General service tailing
docker service logs mcm_<service-name> --tail 100 --follow

# View Keycloak IAM initialization logs
docker service logs mcm_keycloak --tail 100

# View Backend Core API connectivity logs
docker service logs mcm_mcm-api --tail 100

• Explanation: docker service logs fetches aggregated logs from all container replicas associated with that service across all cluster hosts.

Restart a Specific Service / ContainerCopy Anchor Link

To restart any individual service or container in the Swarm cluster (run on Access Node for 3-tier):

docker service update --force mcm_<service-name>

• Explanation: Performs a rolling restart of all container replicas for the designated service without taking down the rest of the platform.

Tear Down the StackCopy Anchor Link

To safely stop all services, containers, and networks in the platform stack (run on Access Node for 3-tier):

sudo mcm stop

• Explanation: Removes the orchestrated stack deployment. This tears down running containers and virtual networks without deleting persistent volumes (databases remain intact). Use sudo mcm restart to bring it back. See Section 5 for details.

MCM Management Service (Systemd)Copy Anchor Link

The mcm command talks to a small background service, mcm.service, that the installer sets up on the host (on VM1 for 3-tier). It does not run the application containers itself — use mcm stop and mcm restart to stop or restart the platform.

# Check the management service
sudo systemctl status mcm.service

# Restart the management service (the platform containers keep running)
sudo systemctl restart mcm.service

# Enable auto-start at system boot
sudo systemctl enable mcm.service

• Explanation: Manages the host operating system service that powers the mcm commands, the in-app upgrade and domain/TLS changes. Stopping it does not stop the platform.

Service Log InspectionCopy Anchor Link

View logs generated by the management service:

# View live real-time log streams
sudo journalctl -u mcm.service -f

# View the last 100 lines of log output
sudo journalctl -u mcm.service -n 100 --no-pager

# View logs generated within a specific timeframe
sudo journalctl -u mcm.service --since "30 minutes ago" --no-pager

• Explanation: Displays output captured from the management service, which is helpful for troubleshooting installs, upgrades and domain/TLS changes.

2. Host-Level Resource Diagnostics (Common)Copy Anchor Link

Before running the installer or when diagnosing performance degradation, use these commands on any VM to inspect host hardware resources.

Checking System MemoryCopy Anchor Link

Verify available, used, and cached RAM on the host virtual machine:

free -h

• Explanation: Displays the total RAM, how much is consumed, how much is free, and the swap usage. The -h flag converts bytes into human-readable formats (e.g., GB, MB).

Checking Disk SpaceCopy Anchor Link

Verify storage capacity and disk usage across all mounted file systems:

df -h

• Explanation: Shows disk partition allocations. Ensure the root partition (/) has at least 15–20% free space to prevent write blocks on database and container volumes.

CPU Core CountCopy Anchor Link

Identify the number of logical processor cores available on the VM:

nproc

• Explanation: Returns the active processor count, which helps verify if the instance size matches the minimum required specifications.

Process CPU & Memory UtilizationCopy Anchor Link

Inspect real-time CPU and memory usage of the top running processes:

top -bn1 | head -22

• Explanation: Runs the system monitor in batch mode (-b), performs a single iteration (-n1), and prints the header plus the top 15 resource-consuming processes to diagnose CPU spikes or memory leaks.

3. Container Platform Diagnostics (Common)Copy Anchor Link

Use these commands on any cluster VM to inspect the local container engine daemon, storage pools, and performance.

Container Engine InfoCopy Anchor Link

Inspect the container platform daemon's configuration, active runtime, and overall state:

docker info

• Explanation: Returns system-wide information including the number of running/paused/stopped containers, active storage driver (usually overlay2), and network plugin configurations.

Container Storage AllocationCopy Anchor Link

Check disk space usage allocated to images, container runtimes, and local volumes:

docker system df

• Explanation: Displays a breakdown of space consumed by images, active container writable layers, local volumes, and build cache, highlighting reclaimable space.

Live Container StatisticsCopy Anchor Link

View real-time CPU, memory, network, and disk I/O usage statistics for all running containers on the current host:

docker stats --no-stream

• Explanation: Outputs a single snapshot (--no-stream) of resource consumption for all active local containers, helping identify memory-heavy or CPU-unbounded services.

Host Network Port BindingsCopy Anchor Link

Identify which processes or sockets are occupying specific network ports:

ss -tlnp | grep -E ":(80|443)"

• Explanation: Lists active TCP sockets in listening mode (-l) with numerical ports (-n) and owning process IDs (-p). Helpful for verifying that ports 80 and 443 are free before deploying the access node gateway.

4. SSL/TLS Certificate and Keystore Inspections (Common)Copy Anchor Link

These commands help inspect and validate TLS certificate files and Java Keystores directly on the VM filesystems.

Gateway Certificate and Domain StatusCopy Anchor Link

Check which domain and HTTPS certificate the gateway is serving, who issued it and when it expires:

mcm tls status

• Explanation: Shows the active domain, the certificate issuer and expiry date, and any domain/TLS change in progress. No sudo is required.

Certificate File VerificationCopy Anchor Link

List the certificates directory contents for a specific service:

# E.g., for the Core API service
ls -la /var/lib/mcm/mcm-api/certs/

• Explanation: Lists the private keys, certificate files, and truststores generated for the microservice.

Java Keystore InspectionCopy Anchor Link

Inspect the certificates and credentials contained inside a Java keystore (keystore.p12):

keytool -list -v -keystore /var/lib/mcm/mcm-api/certs/keystore.p12 -storepass <KEYSTORE_PASSWORD>

• Explanation: Lists keystore entries in verbose mode (-v) to verify that the private key and certificate chain are correctly imported and valid. The keystore password is stored in /etc/mcm/secrets.env.

5. Using the mcm Command-Line ToolCopy Anchor Link

The MCM deployment installs a single mcm command on the host (on VM1, the Access Node, for 3-tier). It replaces the v1.2 helper scripts.

Check Platform Status and HealthCopy Anchor Link

mcm status
mcm healthcheck
mcm version

• Explanation: mcm status shows the platform version, its URL and the replica status of every service. mcm healthcheck is a point-in-time snapshot in which a service below its expected replica count is reported as unhealthy. mcm version prints the installed version. None of these need sudo.

Upgrade the PlatformCopy Anchor Link

MCM provides Option 1 (Automated UI Upgrade), which is recommended, and mcm upgrade for redeploying the platform on the server.

Option 1: Automated Upgrade via MCM UI (Recommended)Copy Anchor Link

1. Log in to your MCM UI as an administrator.
2. Navigate to Settings -> MCM App tab.
3. Check the list of available platform versions. Click Upgrade on the target release version.
4. The server will begin downloading the release package automatically. Once the download and extraction complete:
   • A 10-second warning banner will appear notifying that platform upgrade is starting.
   • The web interface will automatically redirect to the Maintenance Page.
5. Once the upgrade completes, you will be automatically redirected to the updated MCM interface. Your domain, SSL certificates and user configuration are preserved.

   Note: If the Maintenance Page remains active for longer than 15 minutes, refresh your browser. If issues persist, contact Revdau support at mcm@revdau.ai.

Option 2: Redeploy from the Server (mcm upgrade)Copy Anchor Link

# Redeploy every service from the platform's current configuration
sudo mcm upgrade

# Roll just one service (for example after changing its configuration)
sudo mcm upgrade --service=mcm-api

• Explanation: mcm upgrade re-applies the configuration in /etc/mcm and redeploys the platform without deleting database tables or volumes. It works out whether this is a Single VM or 3-Tier host automatically. --service=<name> is a faster way to restart a single service onto its configured image.

Stop the PlatformCopy Anchor Link

sudo mcm stop

• Explanation: Removes the platform stack so all services stop. Docker volumes (your data) and configuration are kept.

Restart the PlatformCopy Anchor Link

sudo mcm restart

• Explanation: Stops the platform, re-applies configuration, secrets and certificates, and redeploys it. Use this after editing files in /etc/mcm.

Start the PlatformCopy Anchor Link

There is no separate start command. After mcm stop, run sudo mcm restart (or sudo mcm upgrade) to deploy the platform again.

Configure Domain and HTTPS CertificateCopy Anchor Link

sudo mcm tls configure --domain=mcm.example.com --cert-type=letsencrypt --email=admin@example.com
mcm tls status

• Explanation: See the Domain and TLS Certificate Configuration Guide for all options, including custom certificates.

Configure the Forward ProxyCopy Anchor Link

sudo mcm proxy configure \
  --http-proxy=http://<PROXY_HOST>:<PORT> \
  --https-proxy=http://<PROXY_HOST>:<PORT> \
  --no-proxy=<COMMA_SEPARATED_DESTINATIONS>
mcm proxy status

• Explanation: Routes the platform's outbound traffic through your corporate proxy. You can also manage it under Settings > Proxy in the MCM console. --no-proxy is optional and lists destinations that bypass the proxy. mcm proxy status shows the current settings and the state of the change.

Remove the PlatformCopy Anchor Link

# Safe cleanup: prunes unused Docker images and networks only
sudo mcm remove

# Delete all MCM data (databases and files) — irreversible
sudo mcm remove --delete-data

# Delete data and configuration
sudo mcm remove --full

• Explanation: mcm remove first prints exactly what it will remove and keep, then asks for confirmation (--yes skips the prompts in scripts). The default mode never touches the running platform, volumes or files. --delete-data and --full permanently delete your data, so back up first. On 3-Tier installs they also clean VM2 and VM3.