# Continuous Integration

• CI should be stateless
• CI should be transient and ephemeral
• CI should be idempotent
• CI should be quick to start up cold
• CI should ideally be minimal
• CI should be sound. Completeness is impossible
• Testing pipelines should be quick. They should finish by the time someone commits, opens a browser, goes to the repository, and looks at the results of their pipeline. 20-30 seconds, ideally.
• Not everything needs to be tested every time
• Not every test needs to run on every code change
• Compiling can be done incrementally if caches are configured well
• There are 3 tiers of tests. Each corresponds to the context required to test a falsifiable assertion of a Good Explanation.
• Warning: This article shows why it’s important to write even the logic tests within the context of Good Explanations.
• As a general rule, the more stateless an application is, and the less IO it does, the more its total complexity can be tested with pure logic tests.
• Logic: Stateless, idempotent. No IO or threading. (<1ms) 70%
• Ex: Unit tests, Generative testing
• Interface: Stateless, Idempotent. (<1s) 20%
• Composition of logic preserves semantics
• Tests composition invariants that can’t be represented in the type system
• Environment/Context. “The full thing” (no limit) 10%
• Tests things as fully and interactively as possible from the perspective of the end user (ie their full environment with their full context)
• Schedule tests and CI runs. Run Env/Context tests nightly.
• Use branches as control flow. Example: Use CI to auto merge to the “staged” branch. Test the staged branch. Auto merge to master if it passes the long nightly test suite.
• Compiling should ideally be reproducible
• CI test results should be ergonomically reproducible in a development environment
• Don’t break the build/
• Many large projets see to follow the optimization of calculating changed files, then calculating the minimal sets of targets to build and then building only those targes. Particularly for monorepos.