towards Nupm workspaces

[amtoine]

i was once again working on NGM and was thinking about a few things related to Nupm 😏

• installing multiple modules in a single package:
  in allow installing multiple modules from a single package #33, we made it possible to install multiple distinct modules from a single package, e.g. nu-git-manager and nu-git-manager-sugar in NGM. However, after using this feature, it sounds a bit strange... it's a bit like having two separate src/ directories in a single Rust crate 🤔
  i think we should revert that feature and keep it simpler, i.e. a single module per module package, and move towards a "workspace" or "bundle" model
  • Revert "allow installing multiple modules from a single package (#33)" #50
• dependencies resolution:
  in NGM, i always have to write things like use ../../foo/bar.nu in the modules themselves or use ../src/nu-git-manager/ in the tests, which is not that great 🤔
  i think it would be great to have a nupm run that would resolve the dependencies and do that for the user, e.g. use module/foo/bar.nu and use nu-git-manager respectively: this would only be a trick with NU_LIB_DIRS to expose modules and activate overlays
  • resolve pkg keyword #52
• "workspace" or "bundle" support:
  would be cool to have multiple module packages in a repo and a central top-level package.nuon that would define a "workspace" and allow maybe to install everything at once?, e.g. nupm install --workspace
• rename package.nuon to nupm.nuon
  • rename package file to nupm.nuon #51

[amtoine]

another quick thought, we might want to change package.nuon to Nupm.nuon?

[kubouch]

Sounds good. I'm also fine renaming package.nuon to nupm.nuon (no need for the capital IMO).

[amtoine]

Sounds good.

nice, i'll start working on all this then 😌

I'm also fine renaming package.nuon to nupm.nuon (no need for the capital IMO).

yeah, nupm.nuon is great 👌
the inspiration was Cargo.toml but the capital is not required 😉

[amtoine]

progress on point 2

i'm trying to use a keyword, pkg to act as crate:: in Rust

i've defined the following simple module

foo/
|-- lib
|   `-- lib.nu
`-- mod.nu

with

# mod.nu
use pkg/lib/lib.nu         # tries to use an internal _library_ command
#   ___
#    `-> the example is a bit contrived, because one could just `use lib/lib.nu` here.
#        but, because `use lib/lib.nu` would mean `use ./lib/lib.nu` and we want to
#        abstract away from the exact location of the root of the module => imagine
#        a submodule deep inside, you'd have to write `use ../../../lib/lib.nu` but
#        here we want the same `use pkg/lib/lib.nu` everywhere!
 
export def main [] {
    print "this is foo"
    lib                    # here
}

# lib/lib.nu               # defines a _library_ command
export def main [] {
    print "this is lib"
}

use foo obviously gives

Error: nu::parser::module_not_found

  × Module not found.
   ╭─[/tmp/foo/foo/mod.nu:1:1]
 1 │ use pkg/lib/lib.nu
   ·     ───────┬──────
   ·            ╰── module not found
 2 │
   ╰────
  help: module files and their paths must be available before your script is run as parsing occurs before anything is evaluated

which is expected

now, when i try to make pkg available, it does not work either 🤔

• copy the directory into target/foo/<git-rev>/pkg/

let pkg = open package.nuon | get name

# NOTE: just to wait for 0.88 and `std null-device`
def null-device []: nothing -> path {
    "/dev/null"
}

let head = ^mktemp -t -d nupm_install_XXXXXXX
^git worktree add --detach $head HEAD out+err> (null-device)
rm --recursive ($head | path join ".git")
^git worktree prune out+err> (null-device)

let target = "target" | path join $pkg (^git rev-parse HEAD)
if ($target | path exists) {
    rm --recursive $target
}
mkdir $target

cp --recursive ($head | path join $pkg) ($target | path join "pkg")

• update NU_LIB_DIRS to include pkg

$env.NU_LIB_DIRS = [
    ("target" | path join (open package.nuon | get name) (^git rev-parse HEAD))
]

• run either use foo or use pkg

Error: nu::parser::module_not_found

  × Module not found.
   ╭─[/tmp/foo/target/foo/a6f3db10c11291d697cc0d4aeca52285bba16d9a/pkg/mod.nu:1:1]
 1 │ use pkg/lib/lib.nu
   ·     ───────┬──────
   ·            ╰── module not found
 2 │
   ╰────
  help: module files and their paths must be available before your script is run as parsing occurs before anything is evaluated

[kubouch]

I think I misunderstood your point 2, we don't have anything like Rust's crate in Nushell. Modules currently don't have an idea of being a part of a package, we'd have to build it into the language. I purposefully kept the modules as flexible as possible to avoid rigid module lock-in situations like in Python, but maybe something dumb like substituting the pkg in pkg/lib/lib.nu for the first parent directory containing nupm.nuon (or throwing an error) could work. It would most likely need to be implemented as a part of Nushell, though.

[amtoine]

@kubouch
i've managed to fix my dumb issue => i needed to put the full absolute path in $env.NU_LIB_DIRS

i've created a draft PR to try things out 😋

[amtoine]

in NGM, i've defined a top-level nupm.nuon and i use it here to generate the documentation of the two modules in the workspace.