Skip to content
This repository was archived by the owner on Jun 12, 2023. It is now read-only.
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
b94196d
Enhancing the array read modes to encompass the two sorted read modes
stavrospapadopoulos Aug 3, 2016
474f3e7
Substantial progress on sorted reads.
stavrospapadopoulos Sep 22, 2016
96fc123
More progress on sorted reads
stavrospapadopoulos Sep 26, 2016
1e0e86c
More changes on sorted reads.
stavrospapadopoulos Sep 27, 2016
fe15abb
More progress on dense sorted reads
stavrospapadopoulos Sep 28, 2016
e6cd375
More progress on dense sorted reads
stavrospapadopoulos Sep 29, 2016
8fc9845
Finished dense sorted reads. Not tested yet.
stavrospapadopoulos Sep 30, 2016
695b353
Finished preliminary testing and debugging of dense sorted reads
stavrospapadopoulos Oct 3, 2016
fe1ee25
Finished some more debugging.
stavrospapadopoulos Oct 4, 2016
5471039
Tested and debugged sorted dense reads for variable-sized attributes
stavrospapadopoulos Oct 5, 2016
3836625
Tested and debugged the 3D case for dense sorted reads.
stavrospapadopoulos Oct 5, 2016
c79d4af
Finished testing and debugging cases of overflow for dense sorted reads.
stavrospapadopoulos Oct 6, 2016
0fbbb66
Fixed bug in asynchornous AIO + sorted dense reads
stavrospapadopoulos Oct 7, 2016
f6e16f0
Finished sparse sorted reads. Not tested and debugged yet.
stavrospapadopoulos Oct 14, 2016
01a82ab
Debugged sorted sparse reads
stavrospapadopoulos Oct 17, 2016
48788f6
Fixed bug in sorted sparse reads and started sorted dense writes
stavrospapadopoulos Oct 17, 2016
69a9bae
Progress on dense sorted writes
stavrospapadopoulos Oct 19, 2016
b972a4a
Finished dense sorted writes (not tested yet).
stavrospapadopoulos Oct 21, 2016
497c824
Tested and debugged dense sorted writes
stavrospapadopoulos Oct 24, 2016
d8960b5
Minor fixes on some error messages
stavrospapadopoulos Oct 24, 2016
b6ec74b
Minor fix on error message
stavrospapadopoulos Oct 24, 2016
4a2e035
Fixed array iterator so that it can admit the various read modes (sor…
stavrospapadopoulos Oct 24, 2016
036b1a1
Updating the version
stavrospapadopoulos Oct 25, 2016
45f69a7
Merge pull request #2 from stavrospapadopoulos/sorted_reads_writes
stavrospapadopoulos Oct 25, 2016
1fd6c66
Moved file sync to fragment finalization. Also added C API functions …
stavrospapadopoulos Oct 26, 2016
269efd3
Merge pull request #3 from stavrospapadopoulos/modify_sync
stavrospapadopoulos Oct 26, 2016
2d28e0a
Minor fix in sync
stavrospapadopoulos Oct 26, 2016
840f768
Changed preprocessor flag OPENMP to HAVE_OPENMP, and fixed a minor bu…
stavrospapadopoulos Oct 26, 2016
ab4c99b
Made MPI optional. Now it is enabled only by setting MPI=1 as argumen…
stavrospapadopoulos Oct 26, 2016
b8fb322
Minor typo.
stavrospapadopoulos Oct 26, 2016
f2896c8
Merge pull request #4 from stavrospapadopoulos/make_mpi_optional
stavrospapadopoulos Oct 26, 2016
a9b1ed8
Finished reading in the presence of empty regions
stavrospapadopoulos Nov 1, 2016
768c6c7
Merge pull request #5 from stavrospapadopoulos/read_empty_dense_regions
stavrospapadopoulos Nov 1, 2016
faa1035
Bug fix in sorted reads with overflow
stavrospapadopoulos Nov 9, 2016
c3228ec
cleaning up dead code
stavrospapadopoulos Nov 9, 2016
f09f8ca
Bug fix in sorted reads
stavrospapadopoulos Nov 23, 2016
5e865a8
Fixed bug in sorted reads/writes
stavrospapadopoulos Dec 8, 2016
87b8cb7
Bug fix with dense fragments that cover part of the domain
stavrospapadopoulos Dec 16, 2016
9b19a98
Bug fix in array_read_state.cc: wrong memory allocation in sort_fragm…
stavrospapadopoulos Dec 22, 2016
c95fc1b
Added unit tests for sorted writes and reads for dense and sparse arrays
kdatta Dec 17, 2016
992746a
removed local paths from Makefile
kdatta Dec 17, 2016
ba54bf8
Bug fixes: (i) corrected calculation of search tile overlap in ReadSt…
stavrospapadopoulos Dec 23, 2016
53e9639
Modifying the creation of a fragment name to incorporate also the MAC…
stavrospapadopoulos Dec 26, 2016
dc1bbb5
Fixes on MAC address retrieval for MAC OSX.
stavrospapadopoulos Dec 26, 2016
d7d858f
Edited the array schema test spec
stavrospapadopoulos Dec 26, 2016
de5c316
Added progress bar in tests.
stavrospapadopoulos Dec 26, 2016
9b0498b
Edited c_api_dense_array_spec tests
stavrospapadopoulos Dec 27, 2016
9a60897
Edited c_api_sparse_array_spec tests.
stavrospapadopoulos Dec 27, 2016
e73e2aa
Fixing bug when creating the copy/aio thread in array_sorted_read/wri…
stavrospapadopoulos Dec 28, 2016
f4fc6ff
Minor fix in Makefile to properly handle VERBOSE error printouts
stavrospapadopoulos Dec 30, 2016
9180f21
Important bug fix with pthreads: we must ALWAYS pthread_join a termin…
stavrospapadopoulos Dec 30, 2016
b9021a7
Removing a minor compiler warning.
stavrospapadopoulos Dec 30, 2016
24cd9ae
Merge pull request #7 from stavrospapadopoulos/unit_tests
stavrospapadopoulos Dec 30, 2016
2a38dfe
Fixed bug with syncing.
stavrospapadopoulos Dec 30, 2016
0ec3a29
Bug fix in sorted reads for sparse arrays. We were calculating wrong …
stavrospapadopoulos Dec 31, 2016
2f0b2a4
Bug fix in sorted reads in the case of user buffer overflow
stavrospapadopoulos Jan 2, 2017
a6b168d
Adding new examples.
stavrospapadopoulos Jan 2, 2017
d5aac21
Minor documentation edit.
stavrospapadopoulos Jan 2, 2017
90b6c0f
Minor documentation edit.
stavrospapadopoulos Jan 2, 2017
189ca67
Renaming example files.
stavrospapadopoulos Jan 3, 2017
d34d81a
Minor typo that was creating a compilation error when MPI is activated
stavrospapadopoulos Jan 3, 2017
70d367f
Minor edits in the outputs of some examples
stavrospapadopoulos Jan 3, 2017
3ba5d4f
Merge pull request #53 from stavrospapadopoulos/dev
stavrospapadopoulos Jan 3, 2017
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 49 additions & 16 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -11,18 +11,21 @@ endif

# --- Configuration flags --- #
CPPFLAGS = -std=gnu++11 -fPIC -fvisibility=hidden \
-D_FILE_OFFSET_BITS=64
-D_FILE_OFFSET_BITS=64

# For the Travis integration
ifdef TRAVIS
CPPFLAGS += --coverage
endif

# --- Support for OpenMP --- #
OPENMP =
OPENMP_FLAG =
ifeq ($(COMPILER), gcc)
CPPFLAGS += -DOPENMP
OPENMP_FLAG = -fopenmp
ifeq ($(COMPILER), gcc)
ifeq ($(OPENMP), 1)
CPPFLAGS += -DHAVE_OPENMP
OPENMP_FLAG = -fopenmp
endif
endif

# --- Debug/Release mode handler --- #
Expand All @@ -42,12 +45,27 @@ endif
# --- Verbose mode handler --- #
VERBOSE =
ifeq ($(VERBOSE),1)
CPPFLAGS += -DNVERBOSE
CPPFLAGS += -DVERBOSE
endif

# --- Use parallel sort --- #
USE_PARALLEL_SORT =
ifeq ($(USE_PARALLEL_SORT),1)
CPPFLAGS += -DUSE_PARALLEL_SORT
endif

# --- Support for MPI --- #
MPI =
ifeq ($(MPI),1)
CPPFLAGS += -DHAVE_MPI
endif

# --- Compilers --- #
CXX = g++

# --- GTest Filters --- #
GTEST_FILTER='*'

# --- Directories --- #
CORE_INCLUDE_DIR = core/include
CORE_INCLUDE_SUBDIRS = $(wildcard core/include/*)
Expand Down Expand Up @@ -87,10 +105,22 @@ ifeq ($(BUILD),release)
EXAMPLES_OBJ_DIR = $(EXAMPLES_OBJ_REL_DIR)
EXAMPLES_BIN_DIR = $(EXAMPLES_BIN_REL_DIR)
endif
TEST_SRC_SUBDIRS = $(wildcard test/src/*)
TEST_INCLUDE_DIR = test/include
TEST_INCLUDE_SUBDIRS = $(wildcard test/include/*)
TEST_SRC_DIR = test/src
TEST_OBJ_DIR = test/obj
TEST_BIN_DIR = test/bin
TEST_SRC_SUBDIRS = $(wildcard test/src/*)
TEST_OBJ_DEB_DIR = test/obj/debug
TEST_BIN_DEB_DIR = test/bin/debug
ifeq ($(BUILD),debug)
TEST_OBJ_DIR = $(TEST_OBJ_DEB_DIR)
TEST_BIN_DIR = $(TEST_BIN_DEB_DIR)
endif
TEST_OBJ_REL_DIR = test/obj/release
TEST_BIN_REL_DIR = test/bin/release
ifeq ($(BUILD),release)
TEST_OBJ_DIR = $(TEST_OBJ_REL_DIR)
TEST_BIN_DIR = $(TEST_BIN_REL_DIR)
endif
DOXYGEN_DIR = doxygen
DOXYGEN_MAINPAGE = $(DOXYGEN_DIR)/mainpage.dox

Expand All @@ -99,7 +129,8 @@ INCLUDE_PATHS =
CORE_INCLUDE_PATHS = $(addprefix -I, $(CORE_INCLUDE_SUBDIRS))
EXAMPLES_INCLUDE_PATHS = -I$(EXAMPLES_INCLUDE_DIR)
TEST_INCLUDE_PATHS = $(addprefix -I, $(CORE_INCLUDE_SUBDIRS))
LIBRARY_PATHS =
TEST_INCLUDE_PATHS += $(addprefix -I, $(TEST_INCLUDE_SUBDIRS))
LIBRARY_PATHS =

ifdef TRAVIS
LIBRARY_PATHS += --coverage
Expand All @@ -110,6 +141,7 @@ ZLIB = -lz
OPENSSLLIB = -lcrypto
GTESTLIB = -lgtest -lgtest_main
MPILIB =
PTHREADLIB = -pthread

# --- For the TileDB dynamic library --- #
ifeq ($(OS), Darwin)
Expand All @@ -128,6 +160,7 @@ EXAMPLES_OBJ := $(patsubst $(EXAMPLES_SRC_DIR)/%.cc,\
$(EXAMPLES_OBJ_DIR)/%.o, $(EXAMPLES_SRC))
EXAMPLES_BIN := $(patsubst $(EXAMPLES_SRC_DIR)/%.cc,\
$(EXAMPLES_BIN_DIR)/%, $(EXAMPLES_SRC))
TEST_INCLUDE := $(foreach D,$(TEST_INCLUDE_SUBDIRS),$D/*.h)
TEST_SRC := $(wildcard $(foreach D,$(TEST_SRC_SUBDIRS),$D/*.cc))
TEST_OBJ := $(patsubst $(TEST_SRC_DIR)/%.cc, $(TEST_OBJ_DIR)/%.o, $(TEST_SRC))

Expand All @@ -151,7 +184,7 @@ doc: doxyfile.inc

test: libtiledb $(TEST_BIN_DIR)/tiledb_test
@echo "Running TileDB tests"
@$(TEST_BIN_DIR)/tiledb_test
@$(TEST_BIN_DIR)/tiledb_test --gtest_filter=$(GTEST_FILTER)

clean: clean_core clean_libtiledb \
clean_test clean_doc clean_examples
Expand Down Expand Up @@ -204,7 +237,7 @@ $(CORE_LIB_DIR)/libtiledb.$(SHLIB_EXT): $(CORE_OBJ)
@mkdir -p $(CORE_LIB_DIR)
@echo "Creating dynamic library libtiledb.$(SHLIB_EXT)"
@$(CXX) $(SHLIB_FLAGS) $(SONAME) -o $@ $^ $(LIBRARY_PATHS) $(MPILIB) \
$(ZLIB) $(OPENSSLLIB) $(OPENMP_FLAG)
$(PTHREADLIB) $(ZLIB) $(OPENSSLLIB) $(OPENMP_FLAG)

$(CORE_LIB_DIR)/libtiledb.a: $(CORE_OBJ)
@mkdir -p $(CORE_LIB_DIR)
Expand Down Expand Up @@ -243,7 +276,7 @@ $(EXAMPLES_BIN_DIR)/%: $(EXAMPLES_OBJ_DIR)/%.o $(CORE_LIB_DIR)/libtiledb.a
@mkdir -p $(EXAMPLES_BIN_DIR)
@echo "Creating $@"
@$(CXX) -std=gnu++11 -o $@ $^ $(LIBRARY_PATHS) $(MPILIB) $(ZLIB) \
$(OPENSSLLIB) $(OPENMP_FLAG)
$(PTHREADLIB) $(OPENSSLLIB) $(OPENMP_FLAG)

# --- Cleaning --- #

Expand All @@ -264,8 +297,8 @@ $(TEST_OBJ_DIR)/%.o: $(TEST_SRC_DIR)/%.cc
@mkdir -p $(dir $@)
@echo "Compiling $<"
@$(CXX) $(CPPFLAGS) $(OPENMP_FLAG) $(TEST_INCLUDE_PATHS) \
$(INCLUDE_PATHS) -c $< -o $@
@$(CXX) -MM $(TEST_INCLUDE_PATHS) \
$(INCLUDE_PATHS) $(PTHREADLIB) -c $< -o $@
@$(CXX) -MM $(TEST_INCLUDE_PATHS) $(PTHREADLIB) \
$(CORE_INCLUDE_PATHS) $(INCLUDE_PATHS) $< > $(@:.o=.d)
@mv -f $(@:.o=.d) $(@:.o=.d.tmp)
@sed 's|.*:|$@:|' < $(@:.o=.d.tmp) > $(@:.o=.d)
Expand All @@ -276,8 +309,8 @@ $(TEST_OBJ_DIR)/%.o: $(TEST_SRC_DIR)/%.cc
$(TEST_BIN_DIR)/tiledb_test: $(TEST_OBJ) $(CORE_LIB_DIR)/libtiledb.a
@mkdir -p $(TEST_BIN_DIR)
@echo "Creating test_cmd"
@$(CXX) -std=gnu++11 -o $@ $^ $(LIBRARY_PATHS) $(MPILIB) $(ZLIB) \
$(OPENSSLLIB) $(GTESTLIB) $(OPENMP_FLAG)
@$(CXX) -o $@ $^ $(LIBRARY_PATHS) $(MPILIB) $(ZLIB) \
$(PTHREADLIB) $(OPENSSLLIB) $(GTESTLIB) $(OPENMP_FLAG)

# --- Cleaning --- #

Expand Down
9 changes: 9 additions & 0 deletions core/include/array/aio_request.h
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,15 @@ struct AIO_Request {
void* completion_data_;
/** A unique request id. */
size_t id_;
/**
* It can be one of the following:
* - TILEDB_ARRAY_READ
* - TILEDB_ARRAY_READ_SORTED_COL
* - TILEDB_ARRAY_READ_SORTED_ROW
* - TILEDB_ARRAY_WRITE
* - TILEDB_ARRAY_WRITE_UNSORTED
*/
int mode_;
/**
* Applicable only to read requests.
* Indicates whether a buffer has overflowed during a read request.
Expand Down
Loading