Continuous Delivery and Continuous Compliance

  • Continuous Delivery should be atomic
    • Complete delivery does not need to be “atomic” in the sense of instant deployment (see rollouts)
    • Blue/Green is a common building block of atomic deploys
    • Atomic means every commit corresponds to a deploy (deployed branches are always green)
  • Canary rollouts, Staged rollouts, and A/B rollouts should be used when they make sense.
    • Canary rollouts: Essentially part of a end-to-end test. Deploy to error-prone yet fault tolerant targets. Monitor success. Auto full deploy if it passes, auto revert otherwise.
    • Staged rollouts: Prevent overwhelming initial load from crashing servers. Used when one time setups are required (e.g., downloading software updates).
    • A/B rollouts: Used to test different implementations of a single feature. Used to test UI/UX, user engagement, performance in real world scenarios, etc.
  • CD should be fully auditable
    • Action. Timestamp. User. Permission profile. Reason (issue in tracker)
    • Audit trail should be implicit and automatic (eg git commit logs. CI logs. CD logs. Metric tracking)
  • CD should have two giant red buttons
    • Oh shit nevermind: Reverts the last change and instantly redeploys the last working version. Complete with audit log.
    • Sudo doTheThing™: Bypasses the CI and triggers a fully manual, audited deploy. Used for critical patches.