Please update dub package

[MartinNowak]

Now that dub-0.9.25 will be compatible with packages used as gitsubmodules, it's time to update the dub config and tag a new release :).
dlang/dub#735

[CyberShadow]

Yep, good stuff, but maybe someone more familiar with writing dub configs could do it.

What needs to be updated in the dub config?

[MartinNowak]

What needs to be updated in the dub config?

I think you just need to change the import path to ...
Any maybe update list of source files if that's outdated.

[CyberShadow]

@MartinNowak @Paxa I updated dub.sdl and now the demos and external apps using path="..." work, but attempting to use it without path downloads an old version, apparently "1.0.1" though I have never designated such a version. Also I didn't have to set importPath to "..", maybe that is required for when downloads it, but that's not close to working right now. The ae package on code.dlang.org is not registered in my name and I'm not sure what the next step would be here.

[CyberShadow]

I tried setting importPath to "..", but that causes Dub to scan everything in ae's parent directory. In my filesystem I keep my other projects there, so this does not work. I assume it's also incompatible with the submodule approach.

[dkgroot]

Hi Vladimir,

If you create a new subdirectory 'ae' and move:

• 'net' to 'ae/net'
• 'sys' to 'ae/sys'
• 'ui' to 'ae/ui'
• 'utils' to 'ae/utils'

Plus these changes to sub.sdl

diff --git a/dub.sdl b/dub.sdl
index 3975006..40edbe9 100644
--- a/dub.sdl
+++ b/dub.sdl
@@ -3,55 +3,55 @@ description "CyberShadow's ArmageddonEngine utilites for almost everything"
 authors "Vladimir Panteleev <vladimir@thecybershadow.net>"
 homepage "https://github.com/CyberShadow/ae"
 license "MPL-2.0"
-targetType "sourceLibrary"
+targetType "library"
 
 ---------------------------
 
 # The main package has no special dependencies.
 
-sourcePaths "sys" "utils" "net"
-importPaths "sys" "utils" "net"
+sourcePaths "ae/sys" "ae/utils" "ae/net"
+importPaths "."
 
 # In ae:sqlite
-excludedSourceFiles "sys/sqlite3.d"
-excludedSourceFiles "sys/persistence/keyvalue.d"
+excludedSourceFiles "ae/sys/sqlite3.d"
+excludedSourceFiles "ae/sys/persistence/keyvalue.d"
 
 # In ae:openssl
-excludedSourceFiles "net/ssl/openssl.d"
-excludedSourceFiles "sys/net/system.d"
+excludedSourceFiles "ae/net/ssl/openssl.d"
+excludedSourceFiles "ae/sys/net/system.d"
 
 # In ae:sdl2
-excludedSourceFiles "utils/graphics/sdl2image.d"
+excludedSourceFiles "ae/utils/graphics/sdl2image.d"
 
 # Needs SDL
-excludedSourceFiles "utils/graphics/sdlimage.d"
+excludedSourceFiles "ae/utils/graphics/sdlimage.d"
 
 # Needs X11
-excludedSourceFiles "sys/sendinput.d"
+excludedSourceFiles "ae/sys/sendinput.d"
 
 # In ae:windows
-excludedSourceFiles "sys/windows/pe/package.d"
-excludedSourceFiles "sys/windows/pe/pe.d"
-excludedSourceFiles "sys/windows/pe/resources.d"
-excludedSourceFiles "sys/windows/pe/versioninfo.d"
+excludedSourceFiles "ae/sys/windows/pe/package.d"
+excludedSourceFiles "ae/sys/windows/pe/pe.d"
+excludedSourceFiles "ae/sys/windows/pe/resources.d"
+excludedSourceFiles "ae/sys/windows/pe/versioninfo.d"
 
 # Triggers on import
-excludedSourceFiles "sys/benchmark.d"
+excludedSourceFiles "ae/sys/benchmark.d"
 
 # Deprecated
-excludedSourceFiles "sys/vfs_curl.d"
-excludedSourceFiles "utils/meta/misc.d"
+excludedSourceFiles "ae/sys/vfs_curl.d"
+excludedSourceFiles "ae/utils/meta/misc.d"
 
 # Triggers on import
-excludedSourceFiles "sys/benchmak.d"
+excludedSourceFiles "ae/sys/benchmak.d"
 
 # Requires non-standard compiler features
-excludedSourceFiles "utils/alloc.d"
-excludedSourceFiles "utils/container/hashtable.d"
-excludedSourceFiles "utils/container/list.d"
-excludedSourceFiles "utils/container/package.d"
-excludedSourceFiles "utils/serialization/*.d"
-excludedSourceFiles "utils/xmldom.d"
+excludedSourceFiles "ae/utils/alloc.d"
+excludedSourceFiles "ae/utils/container/hashtable.d"
+excludedSourceFiles "ae/utils/container/list.d"
+excludedSourceFiles "ae/utils/container/package.d"
+excludedSourceFiles "ae/utils/serialization/*.d"
+excludedSourceFiles "ae/utils/xmldom.d"
 
 ---------------------------

Then everything should still work (even without the side-effect you were experiencing with importPaths ".." on your local system). Added benefit, ae is now exported as a library instead of a sourceLibrary. So when I import/include ae into my project and run the unittests, it will only run my local unittests and not include ae unittests as well.

If you would like a pull request for this change, just let me know. The change should not have a negative side effect when ae is used by another project (either via git clone or dub dependency), as far as i can tell. It does however fix the targetType "library" and gitsubmodule issues.

[CyberShadow]

If you create a new subdirectory 'ae' and move:

Sorry, that's out of the question - it would break using ae as a package+submodule, which is the primary intended way to use ae.

[dkgroot]

The importpaths ".." method would work for everyone else, but only require a change on your local system.

Another option would be to create a link/shortcut "./ae/ -> ." (pretty ugly), which would be similary to include a git submodule for on subdirectory "./ae".

Your original dub.sdl can remain and would only need this change:
targetType "library"
sourcePaths "sys" "utils" "net"
importPaths "."

Unrelated:
Changing the targetType would make sense even when it is used as a gitsubmodule, right ?
Are there any other dub packages that have a similar usage pattern ?

[CyberShadow]

The importpaths ".." method would work for everyone else, but only require a change on your local system.

Well, no, it would break building every project using ae as a package+submodule.

Well, okay, but I really don't want to work around a Dub deficiency in such an ugly way as to break the convention I use for my machines.

Another option would be to create a directory link / shortcut "./ae/ -> ." (pretty ugly), which would be similary to include a git submodule for on subdirectory "./ae".

Unfortunately symlinks are still an issue on Windows.

Changing the targetType would make sense even when it is used as a gitsubmodule, right ?

Not sure what this would achieve.

Are there any other dub packages that have a similar usage pattern ?

I think there was at least one other.

[dkgroot]

Unfortunately symlinks are still an issue on Windows.

True

Not sure what this would achieve.

It would replicate the symlink situation where 'ae' is available in the root directory from the perspective of the dub.sdl. And replicates the intended use of 'ae' via git submodule. I would however separate the dub.sdl (root directory) and 'ae' as a git submodule (yuck):

• ./ae-dub # https://github.com/CyberShadow/ae-dub
• ./ae-dub/ae # git submodule -> https://github.com/CyberShadow/ae

All the potential solutions are messy and ugly. Not sure if any of this was any help (Sorry).

Other projects like dfmt and libdpart don't seem to have this problem. I guess this has to do with them using 'src' or 'source' from the beginning.

[CyberShadow]

The root of the problem is that Dub is not a very good build tool.

D is designed with a module system from the very beginning, and its module system is designed to map to a filesystem hierarchy: one file is one module, and one directory is one package.

The correct way to build D programs is to discover dependencies through module imports, and map the module names to filesystem paths through the directory include list. This is the approach used by rdmd and the new dmd -i.

Dub eschews all that and uses dumb globbing. The result is bloated binaries containing irrelevant code, slower than necessary build times, and problems like those seen here.

Essentially, I really don't think that library authors need to bend over backwards to satisfy a broken design. Let's please fix Dub instead.