Skip to content

[NOT-FOR-MERGE] Test simdjson and broken stream #21453

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

yumkam
Copy link
Collaborator

@yumkam yumkam commented Jul 21, 2025

Changelog entry

...

Changelog category

  • New feature
  • Experimental feature
  • Improvement
  • Performance improvement
  • User Interface
  • Bugfix
  • Backward incompatible change
  • Documentation (changelog entry is not required)
  • Not for changelog (changelog entry is not required)

Description for reviewers

This code demonstrate lack of validation in json parsing in row dispatcher. We feed intentionally incorrect events (split by random boundary instead of json boundary), still rd parser accept them as correct.
I don't think this needs to be fixed (besides, simdjson parser is designed to accept specially crafted invalid json), but still.

Copy link

🟢 2025-07-21 18:14:32 UTC The validation of the Pull Request description is successful.

…ken-stream

Conflicts:
	ydb/tests/fq/yds/test_row_dispatcher.py
Copy link

github-actions bot commented Jul 22, 2025

2025-07-22 14:54:41 UTC Pre-commit check linux-x86_64-relwithdebinfo for 2169a12 has started.
2025-07-22 14:54:45 UTC Artifacts will be uploaded here
2025-07-22 14:57:12 UTC ya make is running...
🟢 2025-07-22 15:21:18 UTC Tests successful.

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
189 181 0 0 8 0

🟢 2025-07-22 15:21:26 UTC Build successful.
🟢 2025-07-22 15:21:36 UTC ydbd size 2.2 GiB changed* by +27.0 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 4542c8b merge: 2169a12 diff diff %
ydbd size 2 397 969 328 Bytes 2 397 996 952 Bytes +27.0 KiB +0.001%
ydbd stripped size 501 472 680 Bytes 501 475 752 Bytes +3.0 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Jul 22, 2025

2025-07-22 14:54:42 UTC Pre-commit check linux-x86_64-release-asan for 2169a12 has started.
2025-07-22 14:54:46 UTC Artifacts will be uploaded here
2025-07-22 14:57:17 UTC ya make is running...
🟢 2025-07-22 15:19:14 UTC Tests successful.

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
31 31 0 0 0 0

🟢 2025-07-22 15:19:21 UTC Build successful.
🟢 2025-07-22 15:19:37 UTC ydbd size 3.9 GiB changed* by +26.2 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 4542c8b merge: 2169a12 diff diff %
ydbd size 4 215 984 632 Bytes 4 216 011 504 Bytes +26.2 KiB +0.001%
ydbd stripped size 1 461 077 976 Bytes 1 461 098 328 Bytes +19.9 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant