Skip to content

Fix parent detachment on model copy and reference resolution in list fields - #54

Open
whawk46 wants to merge 3 commits into
finos:mainfrom
whawk46:fix/parent-copy-and-list-references
Open

whawk46 wants to merge 3 commits into
finos:mainfrom
whawk46:fix/parent-copy-and-list-references

Conversation

@whawk46

@whawk46 whawk46 commented Oct 30, 2026

Copy link
Copy Markdown

Summary

  • Detach __rune_parent on copy while preserving in-subtree parent links (Fixes model_copy(deep=True) follows __rune_parent links, producing an attached copy instead of a detached subtree. #49):
    • Override __copy__ and __deepcopy__ on BaseDataClass, basic-type metadata wrappers (BasicTypeMetaDataMixin / _DateTimeCopyMixin / NumberWithMeta), and _EnumWrapper so model_copy(deep=True), _clone_for_write, copy.copy, and copy.deepcopy detach the copied root from uncopied ancestors without traversing __rune_parent up the ancestor chain, while re-parenting cloned descendants inside the copied subtree.
    • Guard self.__dict__.get(name) in BaseDataClass.__setattr__ so model_copy(deep=True, update=...) does not raise KeyError when copying __pydantic_private__.
  • Resolve and serialize references in list-typed fields (Fixes Bug: resolve_references does not resolve references held in list fields #43, addresses Multiple rune_deserialize / rune_serialize failures block CDM TradeState roundtrip #42 Bug 1 & Bug 4):
    • Extend BaseDataClass.resolve_references, _bind_property_to, _serialize_refs, _init_rune_parent, and resolve_ref_key to resolve UnresolvedReference / Reference elements held inside sequence fields and serialize bound reference indices back as {@ref*: key} dicts.
    • Fall back to extracting allowed @ref* tags from field validator metadata when _KEY_REF_CONSTRAINTS is not explicitly defined on the model class.
    • Guard ComplexTypeMetaDataMixin.serialise, BasicTypeMetaDataMixin.serialise, and EnumWithMetaMixin.serialise for None, list, and reference values when serializers wrap list-cardinality or optional fields.

Testing

  • Added regression tests in test/test_rune_parent.py, test/test_keys_and_references.py, and test/test_list_cardinality_with_meta.py.
  • pytest --cov=rune.runtime (369 passed).

Override __copy__ and __deepcopy__ on BaseDataClass, basic-type metadata
wrappers, and _EnumWrapper so copying a node detaches it from uncopied
ancestors without walking up __rune_parent, while preserving parent
links for descendants within the copied subtree.

Add regression coverage for _clone_for_write, model_copy, copy/deepcopy,
deep parent chains, and nested COW writes.

Fixes finos#49

Signed-off-by: Walter Hawkins <wdhawkins46@gmail.com>
Extend BaseDataClass.resolve_references, _bind_property_to,
_serialize_refs, _init_rune_parent, and resolve_ref_key to support
multi-cardinality reference fields, while inferring allowed @ref tags
from field validator metadata when _KEY_REF_CONSTRAINTS is absent.

Update ComplexTypeMetaDataMixin.serialise, BasicTypeMetaDataMixin.serialise,
and EnumWithMetaMixin.serialise to handle list, None, and reference values
when serializers wrap list-typed or optional fields.

Add regression coverage for list reference resolution, mixed inline and
reference lists, assignment, dangling list references, and serialization
roundtrips.

Fixes finos#43

Signed-off-by: Walter Hawkins <wdhawkins46@gmail.com>
@plamen-neykov

plamen-neykov commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

This PR has some substantial issues:

  1. List mutations silently serialize the wrong references. Bindings are stored by index, independently of the mutable list. Replacing observations[0] with a new inline object still serializes the old {"@ref": "obs-1"}, discarding the replacement. reverse() and pop() also corrupt the mapping. Neither resolve_references() nor validation repairs it.

  2. += and copy-on-write list updates discard existing bindings. Once resolved, list elements are target objects rather than Reference instances. Assignment therefore enters the branch that removes the reference metadata. Serialization emits repeated inline definitions; deserializing that output fails with duplicate keys.

  3. Rejected list assignments leave partial updates. The setter removes existing bindings and installs the new list before checking every reference. Assigning [valid_ref, wrong_type_ref] raises, but leaves the first item bound and the invalid second reference installed. Validation should complete before committing the replacement.

  4. Detached subtree copies lose key lookup. When a nested model’s key registry lives in an ancestor, the copied subtree loses access to it. Existing resolved aliases survive, but Reference(key, parent=clone) raises KeyError. Detached copies need a registry built from their copied subtree.

…tion, and detached subtree key registries

Addresses review feedback on finos#54:
1. Introduce _ReferenceList to keep per-element reference bindings in sync across in-place mutations (__setitem__, reverse, pop, insert, del, extend, sort) and reconcile list references on resolve_references(), validation, and serialization.
2. Preserve existing resolved reference bindings across +=, list concatenation, plain-list copy-on-write assignments, and rune_cow / rune_unwrap list updates without emitting duplicate inline keys.
3. Validate all Reference items in sequence assignments atomically before mutating existing list items or reference bindings.
4. Rebuild RUNE_OBJ_MAPS key registries on detached deep and shallow subtree copies so Reference(key, parent=clone) and get_object_by_key() resolve against the copied subtree.

Signed-off-by: Walter Hawkins <wdhawkins46@gmail.com>
@whawk46

whawk46 commented Oct 30, 2026

Copy link
Copy Markdown
Author

Thanks for the thorough review @plamen-neykov. All four points are addressed in commit c69f193:

  1. In-place list mutations & reconciliation: Added _ReferenceList in src/rune/runtime/metadata.py and _reconcile_list_refs() in src/rune/runtime/base_data_class.py so index assignments (observations[0] = ...), slices, reverse(), pop(), insert(), del, and sort() update reference bindings in lockstep, with reconciliation on resolve_references(), validate_model(), and serialization (test_list_in_place_mutations_keep_reference_bindings_in_sync).
  2. += and copy-on-write list updates: Preserved existing reference bindings across __iadd__ (+=), list concatenation (+), plain-list copy-on-write reassignments, and _COWList / rune_unwrap() in src/rune/runtime/cow.py (test_list_iadd_concat_and_cow_preserve_reference_bindings).
  3. Atomic list assignment validation: _assign_sequence_property() now validates all Reference elements via _validate_ref_for_property() before mutating the list or __rune_references (test_rejected_list_reference_assignment_is_atomic).
  4. Detached subtree key registries: BaseDataClass.__deepcopy__, __copy__, and model_copy now rebuild RUNE_OBJ_MAPS from the copied subtree via _rebuild_subtree_object_maps() / _rebuild_shallow_object_maps() so Reference(key, parent=clone) and get_object_by_key() resolve against detached copies (test_detached_subtree_copies_rebuild_key_registry).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants