Continue Messing With Nix

Certainly the clearest way to get through my current pandoc issues would be to switch to a Lua filter…but where's the fun in that.

Running the filter through runhaskell presumably needs pandoc-types to be available and discoverable to the compiler/linker, however that works in nix.

Alternatively I could try compilng the plugin beforehand which may also help with some of my concerns about the use of haskell filters for pandoc. Unfortunately a quick experiment resulting in a version mismatch due to the pandoc-types installed on my system being a newer version than that available within the nix environment.

Error in $: Incompatible API versions: encoded with [1,23,1] but
attempted to decode with [1,22,2]

Inspecting Packages

The search interface for nix doesn't reveal much, but viewing the source for the packages should provide all a comprehensive picture of what's happening. The produced nix files are fairly large, and I'm also getting up to speed on the language itself, but that information should allow me to track down some way or another around the current impasse.

Within the GitHub UI, searching for pname = "..." seems to be a reasonable way to find the relevant packages. Based on what is defined the pandoc packages include pandoc-types so that seems like it should be usable once any edges are smoothed out.

I'd hypothesize that the problem is that while pandoc is configured, the ghc package is not using the libraries it exposes.

Getting Things Working

This effort will probably be split in half in terms of getting something usable, and then looking to optimize it. To get things working I'll just find a command that works but may execute a fair amount more within the nix environment. Later I'll likely look at producing a nix package if one doesn't exist defining my requirements. This mirrors the split between producing an image and invoking more logic within a container.

Using stack may be promising, but it seems like it could take quite a while to produce the desired environment. The digest package requires pkg-config so that was added to the nix environment.

There's the prospect of using yet another package management tool as an interim approach, or even using a container if supported in the CI environment. These options all seem pretty ugly but at least provide something usable.

Use of ghcWithPackages

The nix wiki provides some docs around using Haskell which describes the use of ghcWithPackages. Now I'm hitting an issue around Data.Text being in a hidden package but I feel like I'm getting very close (while it seems like adding that package would resolve the issue it doesn't seem to).

While I'm at it I'll fix up my use of that package to be qualified…and the pandoc package for that matter (motivation to be covered separately).

That package specifically is pulled in as it is used by pandoc, so this is not something I've introduced that could be easily removed.

Adjust File Mode

One issue that's caused a bit of churn is that my filter file was not executable. Therefore the shebang within the file was being ignored and Pandoc was deriving an interpretter based on the extension.

The call to runhaskell can now be adjusted to expose the text package, but I'm getting a more arcane error around that main is not in scope. This most likely seems to be a problem with the adjusted runhaskell command not actually interpretting the file contents (and therefore whining about the missing entrypoint).

Adjusted Interpretter

Using stack to interpret the file seems to work, though it presumably may take a bit longer. In any case it seems to offer a path forward for now and can be revisited based on how long the builds take.

The updated filter is now:

#!/usr/bin/env stack
-- stack script --resolver lts-21.13 --package pandoc-types --package text
{-# LANGUAGE OverloadedStrings #-}

import qualified Data.Text as T
import qualified Text.Pandoc.JSON as P

main :: IO ()
main = P.toJSONFilter linkFixer

linkFixer :: P.Inline -> P.Inline

linkFixer (P.Link attr inline (href, title))
  | ".org" `T.isSuffixOf` href =
    P.Link attr inline (fixedHref, title)
      where fixedHref = (T.dropEnd 4 href) <> ".html"

linkFixer e = e

which was usable with an invocation of:

nix-shell --pure --packages hut 'haskellPackages.ghcWithPackages (pkgs: with pkgs; [pandoc-cli stack])' --run 'make --file generate.mk publish'

This should be incorporated into the CI build. The makefile should also be updated to make sure the filter is executable, but that should ideally be part of the post-tangle automation.