Skip to content

Harvest does not create LDD download temp files on Windows, causing all products in a bundle to fail #142

Description

@jordanpadams

Checked for duplicates

Yes - I've already checked

🐛 Describe the bug

When Harvest needs to download an LDD (e.g. because it isn't yet loaded in the target registry), it calls SchemaUpdater.createLddTempFile() (common/src/main/java/gov/nasa/pds/registry/common/es/service/SchemaUpdater.java). That method first tries Files.createTempFile(..., PosixFilePermissions...), and on UnsupportedOperationException/IOException falls back to plain File.createTempFile(...). Both calls rely on the same JVM default temp directory (java.io.tmpdir) with no override.

On at least one Windows harvest host, both attempts fail with:

IOException: The system cannot find the path specified

so createLddTempFile throws an LddException for every product that triggers an LDD update. Because no previously-loaded LDD exists for the namespace and force-load mode is off, this escalates to "No previously loaded LDD found for namespace '<prefix>'. Cannot load products with fields from this namespace," which propagates up and is caught in ProductProcessor.onFile(), logged as an ERROR, and counted as a failed file — for every single product in the bundle.

Net effect: the entire harvest run fails (0 products loaded) even though the actual referenced LDD schema itself is valid and reachable — the real problem (an unwritable/nonexistent default temp directory on the host) is masked by a generic OS error with no indication of which directory was attempted or how to work around it.

This appears to be a regression introduced by the recent registry-common#304 port that added the POSIX-permission-then-fallback logic in createLddTempFile() (commits 52b43786 and 9cae169d in this repo) — the fallback path doesn't actually provide a working alternative when the default temp directory itself is the problem.

🕵️ Expected behavior

  • Harvest should not fail 100% of products in a bundle because of a single temp-directory issue.
  • The error message should identify the actual temp directory path that was attempted, so operators can diagnose/fix the underlying environment problem.
  • Ideally, the temp directory used for LDD downloads should be configurable (e.g. via Harvest config or a system property), rather than being hardcoded to whatever java.io.tmpdir resolves to.

📜 To Reproduce

  1. On a Windows host where the resolved java.io.tmpdir path does not exist or is not writable, run Harvest against a bundle that requires updating/downloading an LDD not already loaded in the target registry.
  2. Observe repeated ERROR log lines such as:
    [ERROR] (ProductProcessor.java:onFile:173) Failed to create temp file for LDD download for namespace '<namespace>': The system cannot find the path specified
    
  3. Every product in the bundle fails to load. In the reported run: Skipped files: 7, Loaded files: 0, Failed files: 7834.

🖥 Environment Info

  • Version of this software: Harvest 5.1.3
  • Operating System: Windows Server (harvest installed under a local drive, data referenced from a mapped network share)
  • Registry backend: OpenSearch

📚 Version of Software Used

Harvest 5.1.3

🩺 Test Data / Additional context

  • Reported via email by a PDS Geosciences Node operator while harvesting bundle urn:nasa:pds:mer1_mtes_derived_emissivity, referencing LDD msn_surface v1 (PDS4_MSN_SURFACE_1A10.xsd).
  • Full harvest log (~31k lines, all repeating the same temp-file error) is available on request; a sanitized excerpt:
    2026-08-07 13:00:50 [INFO ] (SchemaUpdater.java:updateSchema:95) Updating LDDs.
    2026-08-07 13:00:50 [INFO ] (SchemaUpdater.java:updateLdd:166) Updating 'msn_surface' LDD. Schema location: https://pds.nasa.gov/pds4/mission/msn_surface/v1/PDS4_MSN_SURFACE_1A10.xsd
    2026-08-07 13:00:51 [ERROR] (ProductProcessor.java:onFile:173) Failed to create temp file for LDD download for namespace 'msn_surface': The system cannot find the path specified
    
  • Relevant code: common/src/main/java/gov/nasa/pds/registry/common/es/service/SchemaUpdater.java:332-352 (createLddTempFile).

🦄 Related requirements

N/A


For Internal Dev Team To Complete

⚙️ Engineering Details

🎉 Integration & Test

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

Type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions