Add TODO list for LIB_DATA - #587

* First draft of new Naksha Data Model with modified/optimized JBON.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Updated JBON and LIB_DATA, ensure downward compatiblity with current implementation as far as possible.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Update documentation.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Improved documentation about JBON2, LIB_DATA

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix bitmask in partitioning

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Added back JBON1

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Add ai directory and plan remove_uid

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Remove uid, encode action in the lower two bit of the version/txn.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix broken tests.

Root cause #1: Guid.ALL_PARTS was still 13 (expecting old uid field)
- Guid.kt — removed UID constant, changed ALL_PARTS from 13 to 12. The
Guid.fromString() was rejecting valid GUID strings because it expected
13 colon-separated parts (the old format with uid) but the new format
only has 12. This caused XyzNs.guid to silently return null, breaking 23
tests.
Root cause #2: next_tn/prev_tn/base_tn were B64 (8 bytes = txn only),
not unique per feature
- All features in the same transaction share the same txn value, so B64
next_tn was identical for all features written in one batch — making
history queries by next_tn return all features from that transaction
instead of only the intended ones.
- Fixed by changing next_tn/prev_tn/base_tn to B128 (16 bytes =
featureNumber + txn), which is unique per feature.
Files changed:
- Guid.kt — removed UID = 12, changed ALL_PARTS to 12
- PgColumnRows.kt — reads/writes prev_tn/next_tn/base_tn with B128
instead of B64
- PgWriterUpdate.kt — history next_tn set to new_row.tn (full B128)
- PgWriterUpsert.kt — history next_tn set to naksha_tn_128(...) with
action bits; prev_tn set to full head_row.tn; reads prev_tn via getB128
- PgWriterDelete.kt — tombstone next_tn set to naksha_tn_128(...);
prev_tn set to full head_row.tn; history next_tn set to tombstone.tn
- PgTable.kt — tombstone constraint changed from next_tn =
naksha_tn_64(tn) to next_tn = tn
- PgColumn.kt — KDocs updated for next_tn/prev_tn/base_tn
- ReadFeaturesByOtherTns.kt — test updated to serialize TupleNumbers as
B128 for next_tn queries

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Add user-defined members and indices to NakshaCollection

Introduces `NakshaCollection.members` (typed user-defined columns) and
`NakshaCollection.indices` (replaces the legacy StringList opt-in list of
built-in indexes). At write time, the storage walks each feature using the
member's JsonPath (defaults to ["properties", name]), coerces the value to
the declared CustomMemberType, and stores it in a real Postgres column
namespaced with `$` (`$age`, `$price`, ...) to avoid collision with built-ins.
The value also remains in the encoded feature blob.

CustomMemberType (model-only, no SQL flavor):
  BOOLEAN, INT8/16/32/64, FLOAT32/64, STRING, BYTE_ARRAY,
  FLAT_MAP (object of primitives), TAGS (string-array expanded to flat map).

CustomIndexType: BTREE, SPATIAL, FLAT_MAP.

Schema mutability runs on UPSERT/UPDATE of an existing collection:
  - same name + same dataType -> no-op
  - same name + different dataType -> ILLEGAL_ARGUMENT (no type change)
  - new member -> ALTER TABLE ADD COLUMN on HEAD/HISTORY/DELETED/META roots
                  (nullable, no default - metadata-only on PG 11+)
  - removed member -> requires Write.force=true; ALTER TABLE DROP COLUMN
  - index identity = (name, type, on, include, unique); any diff -> drop+create

PgIndex.DEFAULT_INDICES is now always applied; the closed-enum opt-in via
`indices: StringList` is removed.

Naming:
  - Member/index names use the standard Naksha.verifyId rules (no new regex
    or MAX_LENGTH).
  - Physical PG column is `$<name>` so user names cannot collide with the
    built-in column namespace.
Signed-off-by: kkin-here <[email protected]>

* Added proxy skill

Signed-off-by: kkin-here <[email protected]>

* Improve JBON2, more to come.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Finish JBON2 specification, to be reviewed.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Adjust JBON2 and LIB_DATA documentation.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Cleanup of prev_tn field (#584)

Signed-off-by: kkin-here <[email protected]>

* Minor changes to JBON2, improve LIB_DATA

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Improve JBON2 and LIB_DATA, adding object support, maps are not enough.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Lib data next tn (#585)

* Switch to next_version

Signed-off-by: kkin-here <[email protected]>

* Refactor callers after prev_tn cleanup

Signed-off-by: kkin-here <[email protected]>

---------

Signed-off-by: kkin-here <[email protected]>

* Switch to using version, fn. Drop tn

Signed-off-by: kkin-here <[email protected]>

* Add TODO list for LIB_DATA (#587)

Signed-off-by: kkin-here <[email protected]>

* Remove Operation, we do not need it any longer, it is replaced replacement groups, it was anyway never used. (#588)

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Remove deletion table. (#589)

* Add binary encodings to structs in JBON1 documentation.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Integrate a v1 compatibility mode, bring v2 and v1 closer together.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Remove deletion table, ensure that queryHistory and queryDeleted are orthogonal.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

---------

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Remove `flags` column (#591)

* Drop geo/tags encoding from flags; collapse to TWKB and JBON_GZIP

Signed-off-by: kkin-here <[email protected]>

* Move action from flags into the version's low two bits

Signed-off-by: kkin-here <[email protected]>

* Drop the per-row flags column; encoding lives on the collection

Signed-off-by: kkin-here <[email protected]>

* drop misnamed Tuple.uid

Signed-off-by: kkin-here <[email protected]>

* Replace Flags with a DataEncoding enum

Signed-off-by: kkin-here <[email protected]>

---------

Signed-off-by: kkin-here <[email protected]>

* Replace two array iterations in query builder with a single pass (#594)

Signed-off-by: kkin-here <[email protected]>

* Clean up tuples and next version stripping

Signed-off-by: kkin-here <[email protected]>

* Update TODO_LIB_DATA

Signed-off-by: kkin-here <[email protected]>

* Improve JBON2 spec. Needs review and more work, but getting close.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Polish and improve JBON2 specification; except for the Java code part, we should now have a final state.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Store tags as raw JSONB (#597)

Signed-off-by: kkin-here <[email protected]>

* V3 lib data jbon2 (#598)

* Implement JBON2 encoder/decoder.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Update docker documentation.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Use JBON2 as default encoding.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Add workaround to decode JBON2 features to filter.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fixed that most columns are not mandatory, fix that positive decimal identifiers are handled correctly.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

---------

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Update TODO_LIB_DATA 1/06/2026

Signed-off-by: kkin-here <[email protected]>

* Ensure that all collections treated the same, Add TAGS, TAGS_FROM_ARRAY, and SPATIAL member types. Fix issues.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix the broken tests.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix version.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Improve JBON2 encoder, add support for member replacement.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Minor fixes, remove undefined from primitives, make set to be an ordered set.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Improve JbonEncoder2 to support geometry, remove special handlings, add support for member extractor.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Implement support for TWKB and member references.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Extend the definition of the members book, so that member names are now contained.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Adjust to new JBON2 specification.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix a bug for structs greater than 65535 byte in JBON2 encoder.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix random GUID bits, requires higher 8 bit being 0.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix tests by adjusting expected order. However, eventually we need to fix again, ones we fixed tag handling.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix that version is now only a decimal number, it no longer is guratneed to be year, month, day, and seq.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Add helper getByName

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Autodetect encoding in naksha_feature (#600)

Signed-off-by: kkin-here <[email protected]>

* Update JBON2 specification, introduce support for a simple byte-array

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Rename IDict into IBook, add a new default implementation HeapBook.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Update Tuple and members, so that a tuple is only the mandatory, the rest must be delivered by the storage.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* fix tests

Signed-off-by: phmai <[email protected]>

* fix test

Signed-off-by: phmai <[email protected]>

* SET implementation for tags

Signed-off-by: kkin-here <[email protected]>

* fix test

Signed-off-by: phmai <[email protected]>

* Auto-detection for Naksha decoding

Signed-off-by: kkin-here <[email protected]>

* fix test

Signed-off-by: phmai <[email protected]>

* fix test

Signed-off-by: phmai <[email protected]>

* Add member path test (#605)

Signed-off-by: kkin-here <[email protected]>

* Add skill for AI to run tests.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Minor fix in delete and update feature tests.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Add JSON path to standard members.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Replaced hardcoded strings with StandardMembers references.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Import StandardMembers

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Renamed map into path in members, because thats what it actually is.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Improve the member handling, partially removed the hardcoded workarounds.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Little architectural overview generated by AI.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix errors left over from conflicts after rebasing.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Improve member handling, so that the path is always used and features can be put together as wanted.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Add support for BookType, delete metadata and repalce with members. Fix tuple encoding and decoding.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix compilation errors that were the result of the previous modifications, still some left.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Latest state, work in progress.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix minor issue in JBON2 spec.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Next bunch of fixes about the members not being hardcoded.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Handle Xyz members (#606)

Signed-off-by: kkin-here <[email protected]>

* Next round of AI code cleanup, now members are as they should be, still much code is not up-to-date.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix more issues in PgRows and related.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Some more fixes.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix catalog, add some comments where needed, deprecate usage of encoding.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* More fixes in PgCatalog, session, transaction, aso.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix more compiler issues.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* More minor fixes, mainly naming and comments.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Rename more maps into catalogs

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Start fixing query builder.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* fix Write classes

Signed-off-by: phmai <[email protected]>

* Add dedicated members as helper, specifically for query convertion.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Add support for member queries.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Ensure that we consistently talk about TagList and TagMap.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Add missing operations.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Update JBON2 examples.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Add queryMembers to ReadFeatures as new member query, add converter from old query to new.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Moved code to correct placed.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fixed PgQueryBuilder

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix PgQueryBuilder and lots of small issues, like JS annotations.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix more issues.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix PgWriteDelete.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Improve ID verification.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix PgWriter.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Add query converter methods for new members (#607)

* Add query converter methods for new members

Signed-off-by: kkin-here <[email protected]>

* Add TagMatches op

Signed-off-by: kkin-here <[email protected]>

---------

Signed-off-by: kkin-here <[email protected]>

* implement TagList ops

Signed-off-by: phmai <[email protected]>

* Fix id usage in delete, fix insert

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix parts of upsert, minor improvements to members and heap book.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* implement TagList ops

Signed-off-by: phmai <[email protected]>

* Fixed WriterUpsert

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix issues in WriterUpdate

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Final fixes for update.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Add missing documentation to members.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Add equals infix operator to member and allow TupleNumber.fromByteArray to auto-detect by array length.

* Make default XYZ member being typed members.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix compilation errors of test.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Update TagList to be a string-list.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* AIs fixes of tests compilation.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix indices

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix compilation errors.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Add support in the get methods to read from Tuple, next to read from feature.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* fix some activitylog tests

Signed-off-by: phmai <[email protected]>

* Cleanup member queries

Signed-off-by: kkin-here <[email protected]>

* Fix test targets

Signed-off-by: kkin-here <[email protected]>

* Address review comments

Signed-off-by: kkin-here <[email protected]>

* temp guid ReadFeatures support and small bugfix

Signed-off-by: phmai <[email protected]>

* bugfix

Signed-off-by: phmai <[email protected]>

* make ActivityLogHandler move away from ReadFeatures guids query and use new Op membersQuery instead (#609)

Signed-off-by: phmai <[email protected]>

* use backward guid in ReadFeatures for ActivityLogHandler, fix second fetch of missing

Signed-off-by: phmai <[email protected]>

* fix old Property of PQuery

Signed-off-by: phmai <[email protected]>

* fix columns name in postgres indices

Signed-off-by: phmai <[email protected]>

* temp bypass closed loop error

Signed-off-by: phmai <[email protected]>

* comment correction

Signed-off-by: phmai <[email protected]>

* Lib data fix activity log and standard indices (#617)

* move away from old guids read request model

Signed-off-by: phmai <[email protected]>

* column names to predefined val in StandardIndices

Signed-off-by: phmai <[email protected]>

* inject StandardIndices MANDATORY when creating collections

Signed-off-by: phmai <[email protected]>

---------

Signed-off-by: phmai <[email protected]>

* Lib data members fix1 libsql fix (#615)

* Fix query layer

Signed-off-by: kkin-here <[email protected]>

* Fix psql write executor, delete dispatch, column typing and index DDL

Signed-off-by: kkin-here <[email protected]>

* Handle spatial index

Signed-off-by: kkin-here <[email protected]>

* Fix psql commit/read path, tag jsonb storage, and member round-trip

Signed-off-by: kkin-here <[email protected]>

* Tags array from jsonb to an array, fixes to queries

Signed-off-by: kkin-here <[email protected]>

* Second batch of psql fixes

Signed-off-by: kkin-here <[email protected]>

* Third batch of psql fixes

Signed-off-by: kkin-here <[email protected]>

* Address lib-model issues

Signed-off-by: kkin-here <[email protected]>

* Add a TODO note

Signed-off-by: kkin-here <[email protected]>

* fix some lib view tests

Signed-off-by: phmai <[email protected]>

---------

Signed-off-by: kkin-here <[email protected]>
Signed-off-by: phmai <[email protected]>
Co-authored-by: phmai <[email protected]>

* Update PartitioningTest TODO comment

Signed-off-by: kkin-here <[email protected]>

* fix tuple-number tests, seed admin-catalog history partitions (#619)

Signed-off-by: kkin-here <[email protected]>

* Upgrade libraries, JVM and Kotlin. (#620)

* Merged JVM target code so we can change in properties, upgraded gradle to 9.6.1, upgraded libraries, upgraded to Kotlin 2.4.0, upgraded to JVM 25.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix JavaScript compilation, allow compiler to convert Long into BigInt and vice versa.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix relocated datetime classes.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

---------

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Cleanup build files, remove long deprecated stuff, ensure all versions are in libs.versions.toml. (#622)

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* missing list type handling and TODO

Signed-off-by: phmai <[email protected]>

* Fix missing static annotation in getTestStorageId implementation.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Add AI generated debug information for app-service.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Add explicit type to avoid error, previous in kotlin 2.2 this was only a warning.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Adjust test, for some reason creation of internal classes work now.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix imports in lib-psql tests.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix lib-view tests for removed CommonTestConstants.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Add support into ApiTest and CommonApiTestSetup to gather the database, catalog, and collection identifiers and numbers.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix schema/catalog and collection detection, accept that sometimes we simply do not know.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix cli tests (#623)

* Fix cli tests

Signed-off-by: kkin-here <[email protected]>

* Adjust forking, refs, fix tests

Signed-off-by: kkin-here <[email protected]>

---------

Signed-off-by: kkin-here <[email protected]>

* Add TODO to PgSession for cache race condition

Signed-off-by: kkin-here <[email protected]>

* Unit tests activity log (#624)

* fix tests 1 left

Signed-off-by: phmai <[email protected]>

* last activity log test

Signed-off-by: phmai <[email protected]>

---------

Signed-off-by: phmai <[email protected]>

* allow features in response to have different order

Signed-off-by: phmai <[email protected]>

* Lib data fix view union (#625)

* Added missing documentation.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Just modify the code so I can set breakpoint.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix code, it was using wrong timeout, remove streaming API usage to be able to step through code in debug mode, simplify code avoid extra loops.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Create helper class to generate random new UUIDs.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Finally fix the test, it was just missing to inject the UUID into the mocked response.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

---------

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Refactor Action, Guid, NakshaError, NakshaException, TupleNumber, TuleNUmberVariant, and Version into lib-base. (#626)

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Added support for TupleNumber into JbDecoder2 and JbEncoder2, fixing SpaceApiTest. (#627)

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Improve JVM target configuration for projects without JavaScript usage.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* fix patch feature test (#629)

* fix PatchFeatureTest

Signed-off-by: phmai <[email protected]>

* clarify TODO

Signed-off-by: phmai <[email protected]>

---------

Signed-off-by: phmai <[email protected]>

* Clean up prepareWrite, session-scoped cache (#628)

Signed-off-by: kkin-here <[email protected]>

* Updated ReadFeaturesByTileTest to ignore feature order. (#630)

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Add valid UUIDs refering to HEAD to fix UUID parsing. (#632)

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Add missing UUID to result file.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* fix read by bbox and TODO clarifications for PatchFeature REST (#631)

* fix read by bboy and TODO clarifications for PatchFeature REST

Signed-off-by: phmai <[email protected]>

* todo

Signed-off-by: phmai <[email protected]>

---------

Signed-off-by: phmai <[email protected]>

* Fix Tuple.encodeFeature, fix PgWriter.prepareWrite, rename TagsMember to TagMapMember, minor changes ...

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Update test.md

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* update 3.0.0-beta.41

Signed-off-by: phmai <[email protected]>

* Fix Member.isVirtual to return the correct value.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Lib data fix psql (#633)

* Rename deprecated maps to catalog.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix atomic expectations, fix fake UUID in test.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

---------

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix DELETE for forked and new objects, fix CREATE for modified objects.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Update version to 3.0.0-beta.42

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix some merge issues, increased version to 3.0.0-beta.43.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Disable very expensive tests with env var CICD=true

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Add CICD=true environment variable to build to improve build speed, just avoids tests with big data.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Add missing parts of graddle wrapper.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Switch back to JVM 11 target with exceptions for some modules.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Remove illegal description next to $ref in openapi.yml

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* upgraded version for deployment

* updated version for deployment

* downgrade junit to accomodate jvm 11

Signed-off-by: phmai <[email protected]>

* Fix naksha.base warnings for unexpected NakshaFeature (#635)

Signed-off-by: kkin-here <[email protected]>

* add tests and optimize TagListContains underlying SQL (#636)

* add tests and optimize TagListContains underlying SQL

Signed-off-by: phmai <[email protected]>

* remove TODO, we have dedicated Op for those already

Signed-off-by: phmai <[email protected]>

---------

Signed-off-by: phmai <[email protected]>

* Remove virtual members from members book on encode/decode (#637)

Signed-off-by: kkin-here <[email protected]>

* minor delayed object creations

Signed-off-by: phmai <[email protected]>

* Bugfix PgWriterDelete (#641)

* bugfix

Signed-off-by: phmai <[email protected]>

* change env var to something not generic that can be conflicting on gitlab cicd

Signed-off-by: phmai <[email protected]>

* add test, more offset bugs

Signed-off-by: phmai <[email protected]>

---------

Signed-off-by: phmai <[email protected]>

* bump v3 version

Signed-off-by: phmai <[email protected]>

* Fix so that WriteRequest execution does not mutate input NakshaFeature (#642)

* fix so that WriteRequest execution does not mutate input NakshaFeature

Signed-off-by: phmai <[email protected]>

* null handling

Signed-off-by: phmai <[email protected]>

* bump v3 version

Signed-off-by: phmai <[email protected]>

---------

Signed-off-by: phmai <[email protected]>

* Tone down members and indices creation for admin collections (#643)

* use only few members and indicies for admin collections

Signed-off-by: phmai <[email protected]>

* also for book collection

Signed-off-by: phmai <[email protected]>

---------

Signed-off-by: phmai <[email protected]>

* bump to beta.48

Signed-off-by: phmai <[email protected]>

* minor fix redundant uuid override because lib-psql handles it correctly

Signed-off-by: phmai <[email protected]>

* Support foreign UUID like from V2 (#647)

* Revert "minor fix redundant uuid override because lib-psql handles it correctly"

This reverts commit f3f79b7.

* support foreign uuid like from v2

Signed-off-by: phmai <[email protected]>

---------

Signed-off-by: phmai <[email protected]>

* Address review comments regarding custom members tests (#644)

* address review comments

Signed-off-by: phmai <[email protected]>

* address review comments

Signed-off-by: phmai <[email protected]>

---------

Signed-off-by: phmai <[email protected]>

* remove TODO_LIB_DATA.md

Signed-off-by: kkin-here <[email protected]>

* Lib data r49 (#650)

* Add support for versioned layers to view.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix id property of view, add asVersion to Version setting the low two bit

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Remove asVersion, it would be confusion with the static function.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

---------

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Lib data tombstones (#651)

* Suppress tombstones for catalogs and collections by default.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Add test to new allowTombstone feature.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix implementation which was not loading tombstones.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Added changes for classes that implements ISession (#652)

Signed-off-by: Marcin-Here <[email protected]>

* Improve the test to re-create a collection.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix cache updating, so when catalogs or collections are modified, the cache is invalidated.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix delete of already deleted features.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix isCREATE,UPDATE,DELETE and VERSION, fix cache loop.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

---------

Signed-off-by: Alexander Lowey-Weber <[email protected]>
Signed-off-by: Marcin-Here <[email protected]>
Co-authored-by: Marcin-Here <[email protected]>

* Lib data history partitions 3 years (#653)

* fix casting for some tag queries

Signed-off-by: phmai <[email protected]>

* rollback history partitions creation during write sessions

Signed-off-by: phmai <[email protected]>

* bugs that got bypassed

Signed-off-by: phmai <[email protected]>

* review comments

Signed-off-by: phmai <[email protected]>

---------

Signed-off-by: phmai <[email protected]>

* Cleanup mandatory indices (#646)

* Cleanup mandatory indices

Signed-off-by: kkin-here <[email protected]>

* Clean up minimal indices set

Signed-off-by: kkin-here <[email protected]>

* Review addressed

Signed-off-by: kkin-here <[email protected]>

* Address feedback

Signed-off-by: kkin-here <[email protected]>

---------

Signed-off-by: kkin-here <[email protected]>

* Add support for virtual versions and tuple-numbers. (#654)

* Add support for virtual versions and tuple-numbers.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Lib data virt tn http fix (#655)

* Added virtual TN to features from HTTPStorage

Signed-off-by: Marcin-Here <[email protected]>

* added test

Signed-off-by: Marcin-Here <[email protected]>

* clean up

Signed-off-by: Marcin-Here <[email protected]>

---------

Signed-off-by: Marcin-Here <[email protected]>

* Added helper for storages to generate virtual tuple-numbers.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Added a test for the new virtual version code.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Update AI instrutions.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Add new method to test if a version lags behind a specific date, use it in storage and lib-psql.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* When env var NAKSHA_DEBUG=true is given, redirect debug logs to info.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Initialize storage with a valid sequence counter that is no manual version.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Improve the AbstractStorageTest so that rollover concurrency is verified.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix minor issue in assertion order.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix inner secondary query to be the same as the tested first one.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Update ai/test.md

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Restore old transaction start time (which can differ from real clock time), fix second log msg for rollover.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix reading from wrong cursor in case of sequence rollover

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix start version of `nextVirtualVersion` to be a valid date.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Adjust deprecated property usage.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Add a test that verifies that versions are dated.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Explicitly disable sequence caching by setting it to 1 to prevent stale caches causing rollback of rollover.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Minor fix, close a hole in the sequence counter.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* adjust HttpStorage for new Virtual Tuple Number changes. (#657)

Signed-off-by: Marcin-Here <[email protected]>

---------

Signed-off-by: Alexander Lowey-Weber <[email protected]>
Signed-off-by: Marcin-Here <[email protected]>
Co-authored-by: Marcin-Here <[email protected]>

* Lib data review comments phuc (#658)

* wrong WHERE clauses bracketing for some tag map ops

Signed-off-by: phmai <[email protected]>

* review comments

Signed-off-by: phmai <[email protected]>

* not supporting custom TupleNumberMember as PgColumn

Signed-off-by: phmai <[email protected]>

* geometry to and from TWKB now handled by Pg classes, not by JBON encoder and decoder

Signed-off-by: phmai <[email protected]>

* review comments

Signed-off-by: phmai <[email protected]>

* review comments

Signed-off-by: phmai <[email protected]>

---------

Signed-off-by: phmai <[email protected]>

* Lib data only write path cloning, and review comments (#659)

* copy on write path only

Signed-off-by: phmai <[email protected]>

* review comments

Signed-off-by: phmai <[email protected]>

* only deep clone xyz namespace

Signed-off-by: phmai <[email protected]>

---------

Signed-off-by: phmai <[email protected]>

* fix for TagMap and bump to beta 50

Signed-off-by: phmai <[email protected]>

* fix for review comments

Signed-off-by: phmai <[email protected]>

* revert big file

Signed-off-by: phmai <[email protected]>

---------

Signed-off-by: Alexander Lowey-Weber <[email protected]>
Signed-off-by: kkin-here <[email protected]>
Signed-off-by: phmai <[email protected]>
Signed-off-by: Marcin-Here <[email protected]>
Co-authored-by: kkin-here <[email protected]>
Co-authored-by: phmai <[email protected]>
Co-authored-by: Phuc Mai <[email protected]>
Co-authored-by: Hiren Patel <[email protected]>
Co-authored-by: Marcin-Here <[email protected]>