Skip to content

Remove StringList & Hash* from python API - #1263

Open
eivindjahren wants to merge 19 commits into
mainfrom
fix_stringlist
Open

Remove StringList & Hash* from python API#1263
eivindjahren wants to merge 19 commits into
mainfrom
fix_stringlist

Conversation

@eivindjahren

@eivindjahren eivindjahren commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator

This fixes a number of buggy behaviors:

  • rd_strcmp_int was not a strict weak ordering which creates memory violations with std::sort and qsort.
  • case names without letter characters would be matched against lower case exensions which is rarely preferable.
  • rd::filename had a potential memory leak in case of fs::path throwing bad_alloc.
  • rd_get_file_pattern had a missing case for DATA files.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR removes the StringList type from the Python-facing API and migrates the underlying C++/pybind surfaces away from stringlist_type* toward standard containers (std::vector<std::string> / std::set<std::string>), updating tests and call sites accordingly.

Changes:

  • Removed Python StringList (and its dependent Hash wrappers) and deleted/updated tests that exercised those APIs.
  • Updated Summary-related APIs/bindings to return native Python lists (via STL containers in pybind) and adjusted key selection/export paths accordingly.
  • Refactored file-list selection utilities to return std::vector<std::string> and removed stringlist_select_* helpers from the C API surface.

Reviewed changes

Copilot reviewed 28 out of 28 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
tests/util_tests/test_string_list.py Removed tests for the deleted Python StringList API.
tests/util_tests/test_hash.py Removed tests for the deleted Python Hash wrappers.
tests/rd_tests/test_sum_equinor.py Updated imports/assertions to no longer reference StringList.
tests/rd_tests/test_rd_sum.py Updated expectations to treat Summary.keys()/wells()/groups() as lists, not StringList.
tests/rd_tests/test_npv.py Updated imports to no longer reference StringList.
python/resdata/util/util/stringlist.py Deleted the Python StringList wrapper.
python/resdata/util/util/hash.py Deleted Python Hash wrappers that depended on StringList.
python/resdata/util/util/init.py Stopped exporting StringList/Hash* from resdata.util.util.
python/resdata/summary/rd_sum.py Switched keys()/wells()/groups() plumbing to list/STL-based bindings; added natural sorting for keys.
lib/util/tests/ert_util_stringlist_test.cpp Removed predicate-matching test coverage tied to removed stringlist_select_files.
lib/util/stringlist.cpp Removed stringlist_select_files() and stringlist_select_matching_files() implementations.
lib/tests/test_rd_sum.cpp Updated tests to match new container-returning APIs (set/vector) instead of stringlist.
lib/resdata/tests/well_ts.cpp Updated restart file list handling to use std::vector<std::string> and STL sorting.
lib/resdata/tests/rd_filenames.cpp Updated file-list tests to use std::vector<std::string> return values.
lib/resdata/rd_util.cpp Implemented vector-returning file selection helpers; moved matching/dir scanning logic here.
lib/resdata/rd_sum.cpp Migrated summary read/export and key selection APIs to STL containers.
lib/resdata/rd_sum_vector.cpp Updated key expansion to use set-returning key selection and explicit sorting.
lib/resdata/rd_sum_pybind.cpp Updated pybind layer to pass/return STL containers instead of cwrap StringList.
lib/resdata/rd_sum_file_data.cpp Updated data-file reading to accept std::vector<std::string> and throw on invalid inputs.
lib/resdata/rd_sum_data.cpp Updated rd_sum_data_fread to accept std::vector<std::string>.
lib/resdata/rd_smspec.cpp Changed matching-key selection to return std::set<std::string> and map-list APIs to return vectors.
lib/resdata/cwrap_pybind.cpp Removed StringList cwrap conversion helpers.
lib/private-include/detail/resdata/rd_sum_file_data.hpp Updated interface to accept std::vector<std::string> instead of stringlist_type*.
lib/include/resdata/rd_util.hpp Updated rd_select_filelist signature and added select_matching_files declaration.
lib/include/resdata/rd_sum.hpp Updated public summary API signatures to use STL containers.
lib/include/resdata/rd_sum_data.hpp Updated rd_sum_data_fread signature to use STL containers.
lib/include/resdata/rd_smspec.hpp Updated selection/list APIs toward STL containers (but currently still declares a removed stringlist alloc API).
lib/include/ert/util/stringlist.hpp Removed declarations for stringlist_select_files and stringlist_select_matching_files.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread lib/resdata/rd_util.cpp Outdated
Comment thread lib/resdata/rd_util.cpp Outdated
Comment thread lib/include/resdata/rd_smspec.hpp Outdated
Comment thread lib/resdata/rd_sum_pybind.cpp Outdated
Comment thread lib/resdata/tests/well_ts.cpp Outdated
@eivindjahren
eivindjahren force-pushed the fix_stringlist branch 5 times, most recently from ae50ee7 to fea613e Compare August 13, 2026 09:17
@eivindjahren
eivindjahren requested a lite review from Copilot August 13, 2026 09:18

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 28 out of 28 changed files in this pull request and generated no new comments.

Suppressed comments (2)

lib/resdata/rd_util.cpp:594

  • On Windows, FindClose(file_handle) is called unconditionally even when FindFirstFile returns INVALID_HANDLE_VALUE. Calling FindClose on an invalid handle can fail and may crash in some environments. Only close the handle when it was successfully opened.
        file_handle = FindFirstFile(pattern, &file_data);
        if (file_handle != INVALID_HANDLE_VALUE) {
            do {
                char *full_path = util_alloc_filename(
                    path.c_str(), file_data.cFileName, NULL);
                names.emplace_back(full_path);
                free(full_path);
            } while (FindNextFile(file_handle, &file_data) != 0);
        }
        FindClose(file_handle);
        free(pattern);

lib/resdata/rd_util.cpp:649

  • The sort comparator takes its arguments by value (std::string a, std::string b), which copies each string during sorting. Use const std::string& parameters to avoid unnecessary allocations/copies.
    std::sort(filelist.begin(), filelist.end(),
              [](std::string a, std::string b) {
                  return rd_fname_report_cmp(a.c_str(), b.c_str()) < 0;
              });

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 44 out of 44 changed files in this pull request and generated no new comments.

Suppressed comments (2)

lib/resdata/tests/well_ts.cpp:44

  • The former second WellInfo block loaded the restart files in reverse order, so deleting it removes regression coverage that loading is order-independent. Replacing stringlist_reverse with std::reverse preserves that scenario without using StringList.
        for (const auto &file_name : file_list) {
            printf("Loading file:%s \n", file_name.c_str());
            well_info.load_rstfile(file_name, true);

python/resdata/util/util/init.py:35

  • Removing this export together with hash.py also deletes the public Hash, StringHash, IntegerHash, and DoubleHash APIs, although this change is scoped to removing StringList. Existing from resdata.util.util import StringHash consumers will now fail. Keep the hash wrappers and make Hash.keys() return a native Python list, or explicitly include this additional breaking API removal in the intended scope and migration plan.

@eivindjahren eivindjahren changed the title Remove StringList from python API Remove StringList & Hash* from python API Aug 13, 2026
@eivindjahren
eivindjahren requested a balanced review from Copilot August 13, 2026 09:41

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 44 out of 44 changed files in this pull request and generated 1 comment.

Comment thread lib/resdata/rd_util.cpp

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 44 out of 44 changed files in this pull request and generated no new comments.

Suppressed comments (1)

lib/resdata/rd_smspec.cpp:1228

  • The new API comment is grammatically incorrect: “Returns a all” should be “Returns all.”
/** Returns a all the (valid) well names.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 44 out of 44 changed files in this pull request and generated no new comments.

Suppressed comments (2)

lib/resdata/rd_util.cpp:538

  • This copied comment is now inaccurate: the helper returns a fresh std::vector and no longer clears a caller-provided string list. Please describe the returned matches instead so the implementation contract is not misleading.
/*
  This function uses the stdlib function glob() to select file/path
  names matching a pattern. The stringlist is cleared when the
  function starts.

lib/include/resdata/rd_util.hpp:159

  • select_matching_files is only used internally by rd_sum.cpp, but declaring this generic global helper in an installed public header unintentionally expands the library API and risks name collisions for consumers. Please place it in a private/detail header (and preferably the rd namespace) instead.
std::vector<std::string> select_matching_files(const std::string &path,
                                               const std::string &file_pattern);

@eivindjahren
eivindjahren force-pushed the fix_stringlist branch 6 times, most recently from c2e9c29 to f2d180d Compare August 14, 2026 09:21
Note that this removes the behavior of base == nullptr meaning
"*". This was never used and it didn't work as bas_has_upper
had UB for base==nullptr.
This also checks return values for windows
equivalent code.
These are redundant since util_abort is noreturn
this avoids manually handling the memory for the return value of
rd_alloc_filename_static.
In rd_select_filelist we want base="*" and file_type=FileType::EGRID
to match against "*.EGRID".

Also fixes a bug where summary files were matched with prefix of
basename rather than the full basename, and added unit tests for these.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 52 out of 52 changed files in this pull request and generated no new comments.

Suppressed comments (1)

lib/resdata/rd_util.cpp:507

  • is_directory(..., ec) reports permission and I/O failures through ec, but this branch treats every false result as a missing/non-directory path. That contradicts the contract above and makes inaccessible summary directories silently appear empty; check ec first and propagate it.
    std::error_code ec;
    if (!fs::is_directory(scan_dir, ec))
        return {};

@ajaust ajaust left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

Only found a tiny typo in one commit message.

Comment thread lib/resdata/rd_util.cpp Outdated
const char *base = strrchr(input_base, UTIL_PATH_SEP_CHAR);
if (base == NULL)
base = input_base;
static bool base_has_upper(std::string_view input_base) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small type in commit message where it says bas_has_upper instead of base_... (commit "Remove StringList from python API").

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants