🛠 Tooling: Switch to Tinyglobby

[lishaduck]

Tooling Report Checklist

• I have tried restarting my IDE and the issue persists.
• I have pulled the latest main branch of the repository.
• I have searched for related issues and found none that matched my issue.

Overview

Tinyglobby has a lot more configuration options than fs.glob which would be much desired for #2638. It's also faster, not that it matters, and is already in most people's dependency trees, so it's not heavier in practice, even if e18e will try to tell us otherwise.

Additional Info

No response

[JoshuaKGoldberg]

configuration options

Which in particular are you thinking of?

[lishaduck]

Which in particular are you thinking of?

dot

[JoshuaKGoldberg]

nodejs/node#61012 should fix that, I think?

[lishaduck]

nodejs/node#61012 should fix that, I think?

I'd think so, yes. We're using v24.15.0 though and I'm not seeing that behavior, which was released in 24.13.1. I'll take a closer look, Amp wrote some harnesses to test it and I eyeballed the harnesses and read the results myself, but didn't run them myself. Might be a path issue or similar 🤷🏻‍♂️

[auvred]

tinyglobby is indeed faster than fs.glob (behchmark):

packages/*/tsconfig.json (typescript-eslint)
┌─────────┬────────────────┬───────────────────┬────────────────────┬────────────────────────┬────────────────────────┬─────────┐
│ (index) │ Task name      │ Latency avg (ns)  │ Latency med (ns)   │ Throughput avg (ops/s) │ Throughput med (ops/s) │ Samples │
├─────────┼────────────────┼───────────────────┼────────────────────┼────────────────────────┼────────────────────────┼─────────┤
│ 0       │ 'tinyglobby'   │ '508658 ± 1.55%'  │ '477329 ± 25693'   │ '2031 ± 0.53%'         │ '2095 ± 116'           │ 1966    │
│ 1       │ 'fast-glob'    │ '661853 ± 2.41%'  │ '623110 ± 34514'   │ '1581 ± 0.61%'         │ '1605 ± 90'            │ 1511    │
│ 2       │ 'glob'         │ '780551 ± 2.93%'  │ '726127 ± 42885'   │ '1352 ± 0.72%'         │ '1377 ± 81'            │ 1282    │
│ 3       │ 'node:fs glob' │ '1505245 ± 2.59%' │ '1515192 ± 143257' │ '683 ± 1.03%'          │ '660 ± 58'             │ 665     │
└─────────┴────────────────┴───────────────────┴────────────────────┴────────────────────────┴────────────────────────┴─────────┘
**/* (typescript-eslint)
┌─────────┬────────────────┬─────────────────────┬───────────────────────┬────────────────────────┬────────────────────────┬─────────┐
│ (index) │ Task name      │ Latency avg (ns)    │ Latency med (ns)      │ Throughput avg (ops/s) │ Throughput med (ops/s) │ Samples │
├─────────┼────────────────┼─────────────────────┼───────────────────────┼────────────────────────┼────────────────────────┼─────────┤
│ 0       │ 'tinyglobby'   │ '29652602 ± 1.87%'  │ '29379318 ± 384781'   │ '34 ± 1.64%'           │ '34 ± 0'               │ 64      │
│ 1       │ 'fast-glob'    │ '57514468 ± 1.95%'  │ '54984459 ± 1020568'  │ '17 ± 1.80%'           │ '18 ± 0'               │ 64      │
│ 2       │ 'glob'         │ '71516257 ± 1.36%'  │ '71731233 ± 2851123'  │ '14 ± 1.34%'           │ '14 ± 1'               │ 64      │
│ 3       │ 'node:fs glob' │ '250222129 ± 1.52%' │ '249481772 ± 8514922' │ '4 ± 1.49%'            │ '4 ± 0'                │ 64      │
└─────────┴────────────────┴─────────────────────┴───────────────────────┴────────────────────────┴────────────────────────┴─────────┘

I'll also post this here because otherwise I'll forget: there is a nice way to resolve files ignored by git without trying to re-implement .gitignore semantics on the JS side - https://e18e.dev/docs/replacements/globby.html#tinyglobby. As far as I remember Flint's .gitignore resolver doesn't support core.excludesFile and nested .gitignore files.

[JoshuaKGoldberg]

Is it faster enough to justify the extra node_modules/ size? At those speeds it seems pretty equivalent to me.

[lishaduck]

At those speeds it seems pretty equivalent to me.

It is 8 times slower... I doubt it's the bottle neck but we do effectively glob **/*.js, so I'm just now realizing that could actually make a difference....

[JoshuaKGoldberg]

This is another reason why #2627 excites me. It'll be nice to be able to get a real definitive answer. I'll try to set that up soon.

[lishaduck]

We're using v24.15.0 though and I'm not seeing that behavior, which was released in 24.13.1.

I'm not sure what that PR was doing, but there's now this PR to add a dot option, the description of which agrees with "my" analysis that fs.glob() currently excludes dotfiles.

This is pretty core functionality, and although we've got #2978, and maybe nodejs/node#63205 would land in time, I think I'd rather get it fixed and drop a dependency later if it's not a substantial performance problem 🤷🏻‍♂️

[michaelfaith]

I think I'd rather get it fixed and drop a dependency later if it's not a substantial performance problem 🤷🏻‍♂️

I agree. I support adopting Tinyglobby to fix our current situation, and then adjust later when it's supported natively. 👍