A support library for building packages' native components

[thblt]

Description

Some Emacs package use native components, written in C, C++ or any other language, that require building on the user's computer. Such packages include irony-mode, mu4e, pdf-tools, intero and a lot more. The common approach is to use a shell script or ad-hoc elisp to install the dependencies using the adequate method for the running OS and try to produce a binary. This is horrible.

The envisioned library would provide a standard way for packages to expose their native component's dependencies and build system and would handle building and rebuilding

Things to Implement [0/5]

• Some basic abstraction over common build systems, at the very least autotools and cmake.

• Some basic abstraction over package management, supporting both the "old school method" (apt-get install) and the Nix/Guix "shell" approach (where the package manager creates a shell with the required packages available)

• Some basic abstraction over common dependencies. Eg, Debian's libexpat-dev = Arch's libexpat = nix's expat. Runtime dependencies must be distinguished from compilation deps. Having a stupid database of name equivalences is no big deal and makes things a lot easier for everyone.
  (the general rule that ∀x Debian = libX-dev, Arch=libX, Nix=X can be a good start)

• Updates support: store the built version and compare with the library's version.

• Support for generating binaries and Emacs native modules.

Interface for package writers

It could look like this:

(require 'whatever-name-you-want-to-give-the-lib)

(declare-native-component your-package-name
  :type 'program
  :build 'cmake
  :lang 'c++ ; must accept a symbol or a list, determines the binding language for deps.
  :src "server" ; a subdirectory or your-package-name's root dir
  :build-deps '(compiler poppler png zlib)) ;; May be either a list of an alist of (language . deps), so the correct bindings are installed for each language.

From that, the library would create the build environment for the current OS and build the components. Options for installing the components should probably be provided, with reasonable defaults (= just take this binary and put it in your-package-name/bin or /lib

Support for including a simple test could help automate validation of the configuration in all supported OSs and build systems.

Prior art

Packages with a native component:

• https://github.com/Sarcasm/irony-mode
• https://github.com/politza/pdf-tools (uses a complex shell script)
• https://github.com/djcb/mu (no support at all for building from Emacs, usually used as an OS package)
• https://github.com/commercialhaskell/intero (the best implementation, but very specific)

Package managers with support for building:

• Borg has good support for building packages, both the elisp part and external modules. It provides a good abstraction over build systems and OS so all specifics are handled through plugins, eg my own nix-shell support plugin. Build configuration is in the user's .gitmodules, an example can be seen here. But the level of abstraction is so high that setups are not really reusable.

[thblt]

I won't have time to do it myself, but I can be of some help with build systems and subshell-based dependency managers, like Nix. (where you don't just install the dependencies then build the program, but the package manager creates a shell into which the build sequence should run)

[rememberYou]

Thank you for the idea.