Skip to main content

Observability Requirements

To ensure reliable operation of your Flexify IO deployment and timely troubleshooting it is essential to be prepared.

The checklist below must be completed for each production-ready self-hosted Flexify IO instance.

Access​

  • You’re able to sign in to Flexify IO Management Console as administrator

  • You’re able to locate and access all Flexify IO resource in Azure portal:

    • Management Server (API) App Service

    • Web Console App Service

    • SQL Database

    • Engines VMSS

    • Load balancer

  • You’re able to access App Service Diagnostic site (AKA Advanced Tools or SCM)

  • You’re able to download Management Server logs from the SCM site

  • You’re able to establish SSH session to the Management Server container via the SCM site

  • You’re able to connect to the SQL Database and run custom read-write queries

  • You’re able to send S3 requests to each engine and to the load balancer

  • You’re able to SSH to each engine, locate and download diagnostic information (logs, crash dumps, etc.) if needed

  • You’re able to copy and send/share files with diagnostic information (such as logs) to Flexify for analysis

  • You’re able to run basic network diagnostic tools (ping, traceroute, nslookup, nc, curl, ss) from the SSH sessions on the Management Server and Engines

Monitoring and Alerts​

  • Management Server monitoring and alerts are configured according to Management Server Metrics

  • Management Server email alerts are configured, tested, and monitored

  • Engine monitoring and alerts are configured for each engine according to Engine Metrics

  • Monitoring tools allows sorting and viewing exemplars associated with metrics

Health checks​

  • The load balancer is configured to preform health checks on each engine

  • Automated end-to-end health checks are configured. The health check should include uploading and/or downloading an object from an attached storage account via the load balancer

  • [Recommended] The health check includes HTTP GET request (rather then TCP connection only) to a public object in a virtual bucket

  • You will receive an alert if any of those health checks fail and this is periodically tested/verified

  • You will receive an alert if health checks are not performed anymore

Diagnostic and audit logs​

  • Management Server is configured to send HTTP logs and App Service Application Logs to Azure Log Analytics workspace, Azure Storage Account, and/or another similar service.

  • You’re able to access those logs for the period of 90 days or more

  • Engine diagnostic logs (Engine logs) with at least INFO level are configured to be automatically uploaded to Azure Storage Account

  • Engine poxy reports and migration reports (if you run migrations) with ALL are configured to be automatically uploaded to Azure Storage Account

  • You can access both engine diagnostic logs and proxy/migration reports for a period of at 90 days or more

  • You can access both engine diagnostic logs and proxy/migration reports directly on each engine (if they are not uploaded to the storage account yet)