Nix packaging

[bemyak]

🐛 Bug Report

Hi!

I'm trying to make supertokens work in devenv, so I'm building a Nix package for supertokens-core.

I managed to get it running, but I had many issues on every step, so reporting them here in hope the situation can be improved 🙏

1. Non-universal "binary" packages

The official website offers 4 download links for various operating systems. It means that the download link needs to be constructed and 4 different sha256 hashes computed and updated for every version bump.

    if system == "x86_64-linux" then
      {
        os = "linux";
        hash = "sha256:0r08hxvirzjxhxqc2jlp4pv2wkyknr4vvw8hkw5ggygh7qln3f17";
      }
    else
      {
        os = "mac";
        hash = "sha256:1p9vk555v8dykm3x15b1j66n7ibpi8wkm3z27rz8mjs44y71qi4x";
      };

(I went with 2 for now)

I suppose the main difference between these versions is the bundled JRE, so it would be nice if there would be a universal package without it.

2. Dependecy downloads

The install script runs the downloader to fetch the dependencies. This is not ideal, since the build now requires Internet connection and thus not hermetic and reproducible.

A workaround is to disable Nix sandbox with --option sandbox false.

Vendoring dependencies together with the installer seems like a good solution.

3. Writing supertokens script to /usr/bin

The install fails every time at the end when it tries to write to /usr/bin, which is not allowed.

The workaround is to ignore the failure:

./jre/bin/java -classpath "./cli/*" io.supertokens.cli.Main true --path=$out || true

Allowing the script installation path to be configured could solve the issue.

4. Forking

StartHandler spawns a new process and for some reason on my machine it fails to do so:

java.io.IOException: Cannot run program "/nix/store/p1rq5gvpnj5igv42w089clq4pbjhisw6-supertokens-core-9.3.0/jre/bin/java": error=13, Permission denied
	at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1142)
	at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1073)
	at io.supertokens.cli.commandHandler.start.StartHandler.doCommand(StartHandler.java:105)
	at io.supertokens.cli.commandHandler.CommandHandler.handleCommand(CommandHandler.java:31)
	at io.supertokens.cli.Main.start(Main.java:101)
	at io.supertokens.cli.Main.main(Main.java:50)
Caused by: java.io.IOException: error=13, Permission denied
	at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
	at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:313)
	at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:244)
	at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1109)
	... 5 more

This is the output of supertokens start; supertokens list works just fine, so there are no issues with file permission for java.

As a workaround, I wrote a script, which executes io.supertokens.Main directly, without relying on cli.

$out/jre/bin/java -Djava.security.egd=file:/dev/urandom -classpath \"$out/core/*:$out/plugin-interface/*:$out/ee/*\" io.supertokens.Main /tmp configFile=$out/config.yaml foreground=true \$@

5. Supertokens require the installation directory to be writable

A workaround is to pass any other directory to io.supertokens.Main (./ in the example above).

A proper solution could be to treat the installation directory as an immutable location.

6. Path to version.yaml is not configurable

With the last workaround, we change the working directory to ./. This results in config.yaml and version.yaml reported as missing.

Luckily, the location of config.yaml can be specified with configFile=$out/config.yaml, but there is no such option for version.yaml.

The solution depends on how issue 5 is solved, but an easy fix would be to add a similar versionFile option.

My Derivation

The final derivation can be checked here:

supertokens-core.nix

{
  stdenv,
  system,
  ...
}:

let
  params =
    if system == "x86_64-linux" then
      {
        os = "linux";
        hash = "sha256:0r08hxvirzjxhxqc2jlp4pv2wkyknr4vvw8hkw5ggygh7qln3f17";
      }
    else
      {
        os = "mac";
        hash = "sha256:1p9vk555v8dykm3x15b1j66n7ibpi8wkm3z27rz8mjs44y71qi4x";
      };
in
stdenv.mkDerivation rec {
  pname = "supertokens-core";
  version = "9.3.0";
  src = builtins.fetchTarball {
    url = "https://api.supertokens.com/0/user/app/download?pluginName=postgresql&os=${params.os}&core=${version}";
    sha256 = params.hash;
  };

  postUnpack = ''
    chmod +x source/jre/bin/*
  '';

  buildPhase = ''
    ./jre/bin/java -classpath "./downloader/*" io.supertokens.downloader.Main
  '';

  installPhase = ''
    mkdir -p $out
    # This command always fails, since it tries to install a script into /usr/bin
    ./jre/bin/java -classpath "./cli/*" io.supertokens.cli.Main true --path=$out || true
    chmod +x $out/jre/bin/*

    mkdir $out/bin
    echo "#!/usr/bin/env bash" >> $out/bin/supertokens
    echo "$out/jre/bin/java -classpath \"$out/cli/*\" io.supertokens.cli.Main false $out/ \$@" >> $out/bin/supertokens
    chmod +x $out/bin/supertokens

    echo "#!/usr/bin/env bash" >> $out/bin/supertokens-start
    echo "$out/jre/bin/java -Djava.security.egd=file:/dev/urandom -classpath \"$out/core/*:$out/plugin-interface/*:$out/ee/*\" io.supertokens.Main /tmp configFile=$out/config.yaml foreground=true \$@" >> $out/bin/supertokens-start
    chmod +x $out/bin/supertokens-start
  '';
}

Please let me know if any additional information is needed!

[zihenzzz]

If it's a bug, I can try to fix it