Skip to content

feat: Virtual leaves should be hashed as valid block items #20401

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

thenswan
Copy link
Contributor

Description:
This PR:

  • Renames VirtualLeaf to StateItem in the virtual_map_state.proto;
  • Renames VirtualMapKey and VirtualMapValue to StateKey and StateValue, respectively;
  • Changes VirtualLeafBytes hashing bytes to wrap key+value into a OneOf.

Related issue(s):
Fixes: #20371

Checklist

  • Documented (Code comments, README, etc.)
  • Tested (unit, integration, migration, state validation, etc.)

Signed-off-by: Nikita Lebedev <nikita.lebedev@limechain.tech>
@lfdt-bot
Copy link

lfdt-bot commented Jul 29, 2025

🎉 Snyk checks have passed. No issues have been found so far.

security/snyk check is complete. No issues have been found. (View Details)

@thenswan thenswan self-assigned this Jul 29, 2025
@thenswan thenswan added the Platform Tickets pertaining to the platform label Jul 29, 2025
@thenswan thenswan added this to the v0.65 milestone Jul 29, 2025
@thenswan thenswan marked this pull request as ready for review July 29, 2025 17:49
@thenswan thenswan requested review from a team as code owners July 29, 2025 17:50
imalygin
imalygin previously approved these changes Jul 29, 2025
Copy link

codecov bot commented Jul 29, 2025

Codecov Report

❌ Patch coverage is 71.76471% with 24 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...java/com/swirlds/state/merkle/MerkleStateRoot.java 0.00% 14 Missing ⚠️
...wirlds/virtualmap/datasource/VirtualLeafBytes.java 78.94% 0 Missing and 4 partials ⚠️
...tate/merkle/disk/OnDiskWritableSingletonState.java 50.00% 3 Missing ⚠️
...main/java/com/swirlds/state/merkle/StateUtils.java 88.88% 1 Missing ⚠️
...m/swirlds/state/merkle/disk/OnDiskQueueHelper.java 88.88% 0 Missing and 1 partial ⚠️
...ds/state/merkle/disk/OnDiskWritableQueueState.java 83.33% 0 Missing and 1 partial ⚠️

Impacted file tree graph

@@             Coverage Diff              @@
##               main   #20401      +/-   ##
============================================
- Coverage     71.22%   71.19%   -0.04%     
+ Complexity    24137    24136       -1     
============================================
  Files          2652     2652              
  Lines        102279   102287       +8     
  Branches      10629    10633       +4     
============================================
- Hits          72853    72827      -26     
- Misses        25452    25476      +24     
- Partials       3974     3984      +10     
Files with missing lines Coverage Δ Complexity Δ
...java/com/swirlds/state/merkle/VirtualMapState.java 81.78% <100.00%> (-0.13%) 32.00 <0.00> (ø)
.../com/swirlds/state/merkle/disk/OnDiskIterator.java 84.00% <100.00%> (ø) 7.00 <0.00> (ø)
...irlds/state/merkle/disk/OnDiskReadableKVState.java 80.00% <100.00%> (-1.25%) 5.00 <3.00> (ø)
...irlds/state/merkle/disk/OnDiskSingletonHelper.java 100.00% <100.00%> (ø) 2.00 <2.00> (ø)
...irlds/state/merkle/disk/OnDiskWritableKVState.java 84.00% <100.00%> (-0.62%) 8.00 <4.00> (ø)
...main/java/com/swirlds/state/merkle/StateUtils.java 80.18% <88.88%> (ø) 86.00 <4.00> (ø)
...m/swirlds/state/merkle/disk/OnDiskQueueHelper.java 75.00% <88.88%> (ø) 7.00 <4.00> (ø)
...ds/state/merkle/disk/OnDiskWritableQueueState.java 87.50% <83.33%> (ø) 5.00 <0.00> (ø)
...tate/merkle/disk/OnDiskWritableSingletonState.java 68.75% <50.00%> (ø) 3.00 <1.00> (ø)
...wirlds/virtualmap/datasource/VirtualLeafBytes.java 77.77% <78.94%> (-1.09%) 36.00 <2.00> (+1.00) ⬇️
... and 1 more

... and 12 files with indirect coverage changes

Impacted file tree graph

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link

codacy-production bot commented Jul 29, 2025

Coverage summary from Codacy

See diff coverage on Codacy

Coverage variation Diff coverage
-0.02% (target: -1.00%) 78.82%
Coverage variation details
Coverable lines Covered lines Coverage
Common ancestor commit (0aff89d) 102184 76781 75.14%
Head commit (a19cbb1) 102192 (+8) 76765 (-16) 75.12% (-0.02%)

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details
Coverable lines Covered lines Diff coverage
Pull request (#20401) 85 67 78.82%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

See your quality gate settings    Change summary preferences

final Bytes vb = valueBytes();
if (vb != null) {
len += ProtoWriterTools.sizeOfDelimited(FIELD_LEAFRECORD_VALUE, Math.toIntExact(vb.length()));
final int valueLen = (vb != null) ? Math.toIntExact(vb.length()) : -1;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If valueBytes is not null, but empty, it won't be serialized. What about

final Bytes vb = valueBytes();
if (vb != null) {
    final int valueLen = Math.toIntExact(vb.length());
    if (valueLen > 0) {
        innerLen += ProtoWriterTools.sizeOfDelimited(FIELD_LEAFRECORD_VALUE, valueLen);
    }
}

?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good, done

final Bytes vb = valueBytes();
if (vb != null) {
ProtoWriterTools.writeDelimited(out, FIELD_LEAFRECORD_VALUE, Math.toIntExact(vb.length()), vb::writeTo);
final int valueLen = (vb != null) ? Math.toIntExact(vb.length()) : -1;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as my comment above in getSizeInBytesForHashing()

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

thenswan added 2 commits July 30, 2025 15:32
Signed-off-by: Nikita Lebedev <nikita.lebedev@limechain.tech>
# Conflicts:
#	hedera-state-validator/src/main/java/com/hedera/statevalidation/validators/merkledb/ValidateLeafIndexHalfDiskHashMap.java
final Bytes vb = valueBytes();
if (vb != null) {
ProtoWriterTools.writeDelimited(out, FIELD_LEAFRECORD_VALUE, Math.toIntExact(vb.length()), vb::writeTo);
final int valueLen = Math.toIntExact(vb.length());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this the right handling of null? we do we support null as a valid return from valueBytes() should it just return empty array? how does VirtualMap handle the difference between null values and default values as a zero length byte[] in protobuf is a valid object with all default values.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we testing all the edge cases like null, default etc? Also do we test that written objects are readable with PBJ generated code from schema?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Platform Tickets pertaining to the platform
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Virtual leaves should be hashed as valid block items
5 participants