Add feature collapse proposal - #640

…implementation as far as possible.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

Signed-off-by: Alexander Lowey-Weber <[email protected]>

Signed-off-by: Alexander Lowey-Weber <[email protected]>

Signed-off-by: Alexander Lowey-Weber <[email protected]>

Signed-off-by: Alexander Lowey-Weber <[email protected]>

Signed-off-by: Alexander Lowey-Weber <[email protected]>

Signed-off-by: Alexander Lowey-Weber <[email protected]>

Root cause #1: Guid.ALL_PARTS was still 13 (expecting old uid field)
- Guid.kt — removed UID constant, changed ALL_PARTS from 13 to 12. The
Guid.fromString() was rejecting valid GUID strings because it expected
13 colon-separated parts (the old format with uid) but the new format
only has 12. This caused XyzNs.guid to silently return null, breaking 23
tests.
Root cause #2: next_tn/prev_tn/base_tn were B64 (8 bytes = txn only),
not unique per feature
- All features in the same transaction share the same txn value, so B64
next_tn was identical for all features written in one batch — making
history queries by next_tn return all features from that transaction
instead of only the intended ones.
- Fixed by changing next_tn/prev_tn/base_tn to B128 (16 bytes =
featureNumber + txn), which is unique per feature.
Files changed:
- Guid.kt — removed UID = 12, changed ALL_PARTS to 12
- PgColumnRows.kt — reads/writes prev_tn/next_tn/base_tn with B128
instead of B64
- PgWriterUpdate.kt — history next_tn set to new_row.tn (full B128)
- PgWriterUpsert.kt — history next_tn set to naksha_tn_128(...) with
action bits; prev_tn set to full head_row.tn; reads prev_tn via getB128
- PgWriterDelete.kt — tombstone next_tn set to naksha_tn_128(...);
prev_tn set to full head_row.tn; history next_tn set to tombstone.tn
- PgTable.kt — tombstone constraint changed from next_tn =
naksha_tn_64(tn) to next_tn = tn
- PgColumn.kt — KDocs updated for next_tn/prev_tn/base_tn
- ReadFeaturesByOtherTns.kt — test updated to serialize TupleNumbers as
B128 for next_tn queries

Signed-off-by: Alexander Lowey-Weber <[email protected]>

Introduces `NakshaCollection.members` (typed user-defined columns) and
`NakshaCollection.indices` (replaces the legacy StringList opt-in list of
built-in indexes). At write time, the storage walks each feature using the
member's JsonPath (defaults to ["properties", name]), coerces the value to
the declared CustomMemberType, and stores it in a real Postgres column
namespaced with `$` (`$age`, `$price`, ...) to avoid collision with built-ins.
The value also remains in the encoded feature blob.

CustomMemberType (model-only, no SQL flavor):
  BOOLEAN, INT8/16/32/64, FLOAT32/64, STRING, BYTE_ARRAY,
  FLAT_MAP (object of primitives), TAGS (string-array expanded to flat map).

CustomIndexType: BTREE, SPATIAL, FLAT_MAP.

Schema mutability runs on UPSERT/UPDATE of an existing collection:
  - same name + same dataType -> no-op
  - same name + different dataType -> ILLEGAL_ARGUMENT (no type change)
  - new member -> ALTER TABLE ADD COLUMN on HEAD/HISTORY/DELETED/META roots
                  (nullable, no default - metadata-only on PG 11+)
  - removed member -> requires Write.force=true; ALTER TABLE DROP COLUMN
  - index identity = (name, type, on, include, unique); any diff -> drop+create

PgIndex.DEFAULT_INDICES is now always applied; the closed-enum opt-in via
`indices: StringList` is removed.

Naming:
  - Member/index names use the standard Naksha.verifyId rules (no new regex
    or MAX_LENGTH).
  - Physical PG column is `$<name>` so user names cannot collide with the
    built-in column namespace.
Signed-off-by: kkin-here <[email protected]>

Signed-off-by: kkin-here <[email protected]>

Signed-off-by: Alexander Lowey-Weber <[email protected]>

Signed-off-by: Alexander Lowey-Weber <[email protected]>

Signed-off-by: Alexander Lowey-Weber <[email protected]>

Signed-off-by: kkin-here <[email protected]>

Signed-off-by: Alexander Lowey-Weber <[email protected]>

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Switch to next_version

Signed-off-by: kkin-here <[email protected]>

* Refactor callers after prev_tn cleanup

Signed-off-by: kkin-here <[email protected]>

---------

Signed-off-by: kkin-here <[email protected]>

Signed-off-by: kkin-here <[email protected]>

Switch to using version, fn. Drop tn

Signed-off-by: kkin-here <[email protected]>

…ement groups, it was anyway never used. (#588)

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Add binary encodings to structs in JBON1 documentation.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Integrate a v1 compatibility mode, bring v2 and v1 closer together.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Remove deletion table, ensure that queryHistory and queryDeleted are orthogonal.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

---------

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Drop geo/tags encoding from flags; collapse to TWKB and JBON_GZIP

Signed-off-by: kkin-here <[email protected]>

* Move action from flags into the version's low two bits

Signed-off-by: kkin-here <[email protected]>

* Drop the per-row flags column; encoding lives on the collection

Signed-off-by: kkin-here <[email protected]>

* drop misnamed Tuple.uid

Signed-off-by: kkin-here <[email protected]>

* Replace Flags with a DataEncoding enum

Signed-off-by: kkin-here <[email protected]>

---------

Signed-off-by: kkin-here <[email protected]>

Signed-off-by: kkin-here <[email protected]>

Signed-off-by: kkin-here <[email protected]>

Clean up tuples and next version stripping

Signed-off-by: kkin-here <[email protected]>

Signed-off-by: Alexander Lowey-Weber <[email protected]>

Signed-off-by: Alexander Lowey-Weber <[email protected]>

Signed-off-by: phmai <[email protected]>

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Rename deprecated maps to catalog.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

* Fix atomic expectations, fix fake UUID in test.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

---------

Signed-off-by: Alexander Lowey-Weber <[email protected]>

Signed-off-by: Alexander Lowey-Weber <[email protected]>

Signed-off-by: Alexander Lowey-Weber <[email protected]>

Signed-off-by: Alexander Lowey-Weber <[email protected]>

Signed-off-by: Alexander Lowey-Weber <[email protected]>

…ust avoids tests with big data.

Signed-off-by: Alexander Lowey-Weber <[email protected]>

Signed-off-by: Alexander Lowey-Weber <[email protected]>

Signed-off-by: Alexander Lowey-Weber <[email protected]>

Signed-off-by: Alexander Lowey-Weber <[email protected]>

Signed-off-by: phmai <[email protected]>

Signed-off-by: kkin-here <[email protected]>

* add tests and optimize TagListContains underlying SQL

Signed-off-by: phmai <[email protected]>

* remove TODO, we have dedicated Op for those already

Signed-off-by: phmai <[email protected]>

---------

Signed-off-by: phmai <[email protected]>

Signed-off-by: kkin-here <[email protected]>

Signed-off-by: phmai <[email protected]>

* bugfix

Signed-off-by: phmai <[email protected]>

* change env var to something not generic that can be conflicting on gitlab cicd

Signed-off-by: phmai <[email protected]>

* add test, more offset bugs

Signed-off-by: phmai <[email protected]>

---------

Signed-off-by: phmai <[email protected]>

#642)

* fix so that WriteRequest execution does not mutate input NakshaFeature

Signed-off-by: phmai <[email protected]>

* null handling

Signed-off-by: phmai <[email protected]>

* bump v3 version

Signed-off-by: phmai <[email protected]>

---------

Signed-off-by: phmai <[email protected]>

Signed-off-by: kkin-here <[email protected]>