Differences Between Build Systems, Package Managers, Compilers, and IDEs

One might ask, “what’s the difference between build systems and compilers?”. But that would be silly, right? There’s tons of differences!

With a build system you write a program in its language, ideally iteratively and following the process of Developing Good Explanations. On the other hand, with a compiler you write a program in its language, ideally iteratively and… wait a minute.

So, are they the same thing? For the most part, I think they are.

There’s no better example of how tightly these can intertwine than the development of ghcide. What you have here is a compiler being interfaced with by a build system that’s acting as a real-time and interactive compiler in order to implement an IDE backend.

There’s also the salsa crate for rust that the rust-analyzer language-server uses as an IDE backend. Additionally, roslyn, the C# compiler, began life as wanting to share codebases between the IDE and the compiler; they were both constantly sharing the same problems.

Currently, IDEs and compilers have a lot more attention being paid to them than build systems, but fundamentally, the underlying strengths and issues are very much the same. Luckily, the situation is changing with the advent of the Build Server Protocol, which is focused around improving integration between language servers and build tools.

Going even further, the differences in the tooling and even the classical separation of tools may disappear as things slowly converge.

Unfortunately, I don’t know what the correct API is, but here is a strawman proposal: every compiler and build system writer should have an alternative mode which lets a user ask the query, “How do I make $output file?” This mode returns (1) the dependencies of that file, and (2) a recipe for how to make it. The idea is to place the dependency-finding logic in the compiler (the canonical place to put it), while letting an external tool actually handle building the dependencies.

Not sufficient, but better.

one way to look at the buildsome is that it is implementing a limited form of continuation passing, where the call back to the build system is the compiler says, “I need dependency foo” (we only need one-shot continuations, in which case a continuation system is basically just multithreading with blocking communication.) So in this way, a system like buildsome is very close to what you might actually want. But, when I have conversations with people who build build systems, they would say a language which behaves this way is being actively hostile to useful builds.

Tools/workflows that improve developer productivity in the build/package/compile/ide space:

  • code formatters
  • auto complete / LSP
  • refactoring APIs
  • error reporting
  • debugging
  • compiler extensions/plugins
  • live REPLs with hot code swapping

why rust doesn’t expose a single file compiler