Skip to content

fix(plugin-import-export): json preview and downloads preserve nesting and exclude disabled fields #13210

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 18 commits into from
Jul 24, 2025

Conversation

PatrikKozak
Copy link
Contributor

@PatrikKozak PatrikKozak commented Jul 17, 2025

What?

Improves both the JSON preview and export functionality in the import-export plugin:

  • Preserves proper nesting of object and array fields (e.g., groups, tabs, arrays)
  • Excludes any fields explicitly marked as disabled via custom.plugin-import-export
  • Ensures downloaded files use proper JSON formatting when format is json (no CSV-style flattening)

Why?

Previously:

  • The JSON preview flattened all fields to a single level and included disabled fields.
  • Exported files with format: json were still CSV-style data encoded as .json, rather than real JSON.

How?

  • Refactored /preview-data JSON handling to preserve original document shape.
  • Applied removeDisabledFields to clean nested fields using dot-notation paths.
  • Updated createExport to skip flattenObject for JSON formats, using a nested JSON filter instead.
  • Fixed streaming and buffered export paths to output valid JSON arrays when format is json.

Copy link
Contributor

github-actions bot commented Jul 17, 2025

📦 esbuild Bundle Analysis for payload

This analysis was generated by esbuild-bundle-analyzer. 🤖

Meta File Out File Size (raw) Note
packages/next/meta_index.json esbuild/index.js 748.21 KB 🆕 Added
packages/payload/meta_index.json esbuild/index.js 1.20 MB 🆕 Added
packages/payload/meta_shared.json esbuild/exports/shared.js 160.56 KB 🆕 Added
packages/richtext-lexical/meta_client.json esbuild/exports/client_optimized/index.js 259.22 KB 🆕 Added
packages/ui/meta_client.json esbuild/exports/client_optimized/index.js 1.13 MB 🆕 Added
packages/ui/meta_shared.json esbuild/exports/shared_optimized/index.js 14.10 KB 🆕 Added
Largest paths These visualization shows top 20 largest paths in the bundle.

Meta file: packages/next/meta_index.json, Out file: esbuild/index.js

Path Size
../../node_modules ${{\color{Goldenrod}{ ████████████████████▎ }}}$ 81.0%, 601.90 KB
dist/views/Version ${{\color{Goldenrod}{ █▋ }}}$ 6.6%, 49.36 KB
dist/views/Document ${{\color{Goldenrod}{ ▍ }}}$ 1.9%, 14.20 KB
dist/views/Root ${{\color{Goldenrod}{ ▏ }}}$ 0.9%, 6.67 KB
dist/views/Versions ${{\color{Goldenrod}{ ▏ }}}$ 0.8%, 6.16 KB
dist/views/List ${{\color{Goldenrod}{ ▏ }}}$ 0.8%, 6.05 KB
dist/views/API ${{\color{Goldenrod}{ ▏ }}}$ 0.8%, 5.90 KB
dist/views/Account ${{\color{Goldenrod}{ ▏ }}}$ 0.7%, 5.33 KB
dist/elements/Nav ${{\color{Goldenrod}{ ▏ }}}$ 0.7%, 4.84 KB
dist/elements/DocumentHeader ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 4.63 KB
dist/views/Login ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 4.38 KB
dist/views/Dashboard ${{\color{Goldenrod}{ ▏ }}}$ 0.5%, 3.68 KB
dist/views/ForgotPassword ${{\color{Goldenrod}{ }}}$ 0.4%, 3.09 KB
dist/layouts/Root ${{\color{Goldenrod}{ }}}$ 0.4%, 3.09 KB
dist/utilities/initPage ${{\color{Goldenrod}{ }}}$ 0.4%, 3.01 KB
dist/templates/Default ${{\color{Goldenrod}{ }}}$ 0.4%, 2.83 KB
dist/views/BrowseByFolder ${{\color{Goldenrod}{ }}}$ 0.3%, 2.59 KB
dist/views/CreateFirstUser ${{\color{Goldenrod}{ }}}$ 0.3%, 2.48 KB
dist/views/CollectionFolders ${{\color{Goldenrod}{ }}}$ 0.3%, 2.46 KB
dist/views/ResetPassword ${{\color{Goldenrod}{ }}}$ 0.3%, 2.41 KB
(other) ${{\color{Goldenrod}{ ████▊ }}}$ 19.0%, 141.64 KB

Meta file: packages/payload/meta_index.json, Out file: esbuild/index.js

Path Size
../../node_modules ${{\color{Goldenrod}{ █████████████████▌ }}}$ 70.2%, 841.01 KB
dist/fields/hooks ${{\color{Goldenrod}{ ▊ }}}$ 3.3%, 40.09 KB
dist/collections/operations ${{\color{Goldenrod}{ ▋ }}}$ 2.8%, 33.43 KB
dist/auth/operations ${{\color{Goldenrod}{ ▎ }}}$ 1.2%, 14.58 KB
dist/queues/operations ${{\color{Goldenrod}{ ▎ }}}$ 1.0%, 11.92 KB
dist/utilities/configToJSONSchema.js ${{\color{Goldenrod}{ ▎ }}}$ 1.0%, 11.54 KB
dist/globals/operations ${{\color{Goldenrod}{ ▏ }}}$ 0.9%, 11.03 KB
dist/fields/config ${{\color{Goldenrod}{ ▏ }}}$ 0.9%, 10.82 KB
dist/fields/validations.js ${{\color{Goldenrod}{ ▏ }}}$ 0.8%, 9.35 KB
dist/bin/generateImportMap ${{\color{Goldenrod}{ ▏ }}}$ 0.7%, 8.27 KB
dist/database/migrations ${{\color{Goldenrod}{ ▏ }}}$ 0.7%, 7.79 KB
dist/uploads/fetchAPI-multipart ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 7.74 KB
dist/collections/config ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 7.58 KB
dist/index.js ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 7.07 KB
dist/collections/endpoints ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 6.92 KB
dist/config/orderable ${{\color{Goldenrod}{ ▏ }}}$ 0.5%, 6.25 KB
dist/config/sanitize.js ${{\color{Goldenrod}{ ▏ }}}$ 0.5%, 5.44 KB
dist/auth/endpoints ${{\color{Goldenrod}{ ▏ }}}$ 0.5%, 5.41 KB
dist/utilities/telemetry ${{\color{Goldenrod}{ }}}$ 0.4%, 5.31 KB
dist/auth/strategies ${{\color{Goldenrod}{ }}}$ 0.4%, 4.68 KB
(other) ${{\color{Goldenrod}{ ███████▍ }}}$ 29.8%, 356.85 KB

Meta file: packages/payload/meta_shared.json, Out file: esbuild/exports/shared.js

Path Size
../../node_modules ${{\color{Goldenrod}{ ████████████████████ }}}$ 80.2%, 126.19 KB
dist/fields/validations.js ${{\color{Goldenrod}{ █▍ }}}$ 5.9%, 9.36 KB
dist/fields/baseFields ${{\color{Goldenrod}{ ▍ }}}$ 1.8%, 2.79 KB
dist/utilities/deepCopyObject.js ${{\color{Goldenrod}{ ▍ }}}$ 1.6%, 2.48 KB
dist/auth/cookies.js ${{\color{Goldenrod}{ ▎ }}}$ 1.0%, 1.55 KB
dist/utilities/flattenTopLevelFields.js ${{\color{Goldenrod}{ ▏ }}}$ 0.9%, 1.41 KB
dist/fields/config ${{\color{Goldenrod}{ ▏ }}}$ 0.8%, 1.28 KB
dist/utilities/flattenAllFields.js ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 943 B
dist/utilities/fieldSchemaToJSON.js ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 917 B
dist/folders/utils ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 881 B
dist/utilities/unflatten.js ${{\color{Goldenrod}{ ▏ }}}$ 0.5%, 779 B
dist/utilities/sanitizeUserDataForEmail.js ${{\color{Goldenrod}{ ▏ }}}$ 0.5%, 713 B
dist/collections/config ${{\color{Goldenrod}{ }}}$ 0.4%, 570 B
dist/bin/generateImportMap ${{\color{Goldenrod}{ }}}$ 0.4%, 559 B
dist/utilities/getSafeRedirect.js ${{\color{Goldenrod}{ }}}$ 0.3%, 423 B
dist/utilities/deepMerge.js ${{\color{Goldenrod}{ }}}$ 0.3%, 413 B
dist/utilities/getFieldPermissions.js ${{\color{Goldenrod}{ }}}$ 0.2%, 391 B
dist/utilities/formatLabels.js ${{\color{Goldenrod}{ }}}$ 0.2%, 380 B
dist/utilities/mergeListSearchAndWhere.js ${{\color{Goldenrod}{ }}}$ 0.2%, 344 B
dist/utilities/getBestFitFromSizes.js ${{\color{Goldenrod}{ }}}$ 0.2%, 340 B
(other) ${{\color{Goldenrod}{ ████▉ }}}$ 19.8%, 31.18 KB

Meta file: packages/richtext-lexical/meta_client.json, Out file: esbuild/exports/client_optimized/index.js

Path Size
dist/lexical/plugins ${{\color{Goldenrod}{ ███ }}}$ 12.1%, 30.91 KB
dist/features/experimental_table ${{\color{Goldenrod}{ ██▍ }}}$ 9.6%, 24.60 KB
dist/lexical/ui ${{\color{Goldenrod}{ ██▎ }}}$ 9.2%, 23.47 KB
dist/features/blocks ${{\color{Goldenrod}{ ██▏ }}}$ 8.8%, 22.43 KB
dist/packages/@lexical ${{\color{Goldenrod}{ █▊ }}}$ 7.4%, 18.99 KB
dist/features/link ${{\color{Goldenrod}{ █▊ }}}$ 7.0%, 17.96 KB
dist/features/toolbars ${{\color{Goldenrod}{ █▋ }}}$ 6.8%, 17.44 KB
dist/features/textState ${{\color{Goldenrod}{ █ }}}$ 4.3%, 10.98 KB
dist/features/upload ${{\color{Goldenrod}{ ▉ }}}$ 3.7%, 9.60 KB
dist/features/relationship ${{\color{Goldenrod}{ ▉ }}}$ 3.7%, 9.41 KB
dist/lexical/utils ${{\color{Goldenrod}{ ▊ }}}$ 3.2%, 8.08 KB
dist/features/debug ${{\color{Goldenrod}{ ▋ }}}$ 2.9%, 7.34 KB
dist/features/converters ${{\color{Goldenrod}{ ▋ }}}$ 2.7%, 7.04 KB
dist/utilities/fieldsDrawer ${{\color{Goldenrod}{ ▋ }}}$ 2.7%, 7.01 KB
dist/lexical/config ${{\color{Goldenrod}{ ▌ }}}$ 2.0%, 5.10 KB
dist/features/lists ${{\color{Goldenrod}{ ▍ }}}$ 1.9%, 4.95 KB
dist/lexical/theme ${{\color{Goldenrod}{ ▍ }}}$ 1.6%, 4.01 KB
dist/features/format ${{\color{Goldenrod}{ ▎ }}}$ 1.4%, 3.46 KB
dist/lexical/LexicalEditor.js ${{\color{Goldenrod}{ ▎ }}}$ 1.3%, 3.23 KB
dist/features/indent ${{\color{Goldenrod}{ ▎ }}}$ 1.0%, 2.50 KB
(other) ${{\color{Goldenrod}{ █████████████████████▉ }}}$ 87.9%, 225.29 KB

Meta file: packages/ui/meta_client.json, Out file: esbuild/exports/client_optimized/index.js

Path Size
../../node_modules ${{\color{Goldenrod}{ ████████████▊ }}}$ 51.3%, 572.92 KB
dist/elements/FolderView ${{\color{Goldenrod}{ ▋ }}}$ 2.7%, 30.36 KB
dist/elements/BulkUpload ${{\color{Goldenrod}{ ▌ }}}$ 2.2%, 24.52 KB
dist/elements/WhereBuilder ${{\color{Goldenrod}{ ▍ }}}$ 1.6%, 17.94 KB
dist/fields/Relationship ${{\color{Goldenrod}{ ▎ }}}$ 1.4%, 15.93 KB
dist/elements/Table ${{\color{Goldenrod}{ ▎ }}}$ 1.4%, 15.16 KB
dist/views/Edit ${{\color{Goldenrod}{ ▎ }}}$ 1.3%, 14.78 KB
dist/forms/Form ${{\color{Goldenrod}{ ▎ }}}$ 1.3%, 14.21 KB
dist/fields/Blocks ${{\color{Goldenrod}{ ▎ }}}$ 1.1%, 12.37 KB
dist/fields/Upload ${{\color{Goldenrod}{ ▎ }}}$ 1.1%, 11.94 KB
dist/fields/Array ${{\color{Goldenrod}{ ▏ }}}$ 0.8%, 9.29 KB
dist/elements/ListControls ${{\color{Goldenrod}{ ▏ }}}$ 0.8%, 9.03 KB
dist/elements/PublishButton ${{\color{Goldenrod}{ ▏ }}}$ 0.8%, 8.57 KB
dist/providers/Folders ${{\color{Goldenrod}{ ▏ }}}$ 0.7%, 8.38 KB
dist/elements/LivePreview ${{\color{Goldenrod}{ ▏ }}}$ 0.7%, 8.18 KB
dist/elements/HTMLDiff ${{\color{Goldenrod}{ ▏ }}}$ 0.7%, 7.81 KB
dist/views/CollectionFolder ${{\color{Goldenrod}{ ▏ }}}$ 0.7%, 7.66 KB
dist/elements/ReactSelect ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 6.91 KB
dist/fields/Tabs ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 6.90 KB
dist/views/List ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 6.70 KB
(other) ${{\color{Goldenrod}{ ████████████▏ }}}$ 48.7%, 544.96 KB

Meta file: packages/ui/meta_shared.json, Out file: esbuild/exports/shared_optimized/index.js

Path Size
dist/graphics/Logo ${{\color{Goldenrod}{ █████▊ }}}$ 23.1%, 3.12 KB
../../node_modules ${{\color{Goldenrod}{ ████▉ }}}$ 19.6%, 2.65 KB
dist/graphics/Icon ${{\color{Goldenrod}{ ██▊ }}}$ 11.3%, 1.52 KB
dist/utilities/formatDocTitle ${{\color{Goldenrod}{ ██▍ }}}$ 9.8%, 1.32 KB
dist/utilities/groupNavItems.js ${{\color{Goldenrod}{ █▌ }}}$ 6.0%, 814 B
dist/providers/TableColumns ${{\color{Goldenrod}{ █▍ }}}$ 5.6%, 757 B
dist/utilities/api.js ${{\color{Goldenrod}{ █▍ }}}$ 5.6%, 756 B
dist/elements/Translation ${{\color{Goldenrod}{ ▉ }}}$ 3.7%, 493 B
dist/elements/withMergedProps ${{\color{Goldenrod}{ ▋ }}}$ 2.5%, 339 B
dist/utilities/handleTakeOver.js ${{\color{Goldenrod}{ ▍ }}}$ 1.9%, 251 B
dist/elements/WithServerSideProps ${{\color{Goldenrod}{ ▍ }}}$ 1.7%, 232 B
dist/utilities/handleGoBack.js ${{\color{Goldenrod}{ ▎ }}}$ 1.2%, 168 B
dist/fields/mergeFieldStyles.js ${{\color{Goldenrod}{ ▎ }}}$ 1.2%, 159 B
dist/forms/Form ${{\color{Goldenrod}{ ▎ }}}$ 1.1%, 147 B
dist/utilities/abortAndIgnore.js ${{\color{Goldenrod}{ ▎ }}}$ 1.1%, 146 B
dist/utilities/hasSavePermission.js ${{\color{Goldenrod}{ ▎ }}}$ 1.0%, 136 B
dist/utilities/handleBackToDashboard.js ${{\color{Goldenrod}{ ▎ }}}$ 1.0%, 129 B
dist/utilities/findLocaleFromCode.js ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 84 B
dist/utilities/sanitizeID.js ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 77 B
dist/utilities/isEditing.js ${{\color{Goldenrod}{ }}}$ 0.4%, 59 B
(other) ${{\color{Goldenrod}{ ███████████████████▏ }}}$ 76.9%, 10.38 KB
Details

Next to the size is how much the size has increased or decreased compared with the base branch of this PR.

  • ‼️: Size increased by 20% or more. Special attention should be given to this.
  • ⚠️: Size increased in acceptable range (lower than 20%).
  • ✅: No change or even downsized.
  • 🗑️: The out file is deleted: not found in base branch.
  • 🆕: The out file is newly found: will be added to base branch.

@PatrikKozak PatrikKozak changed the title fix(plugin-import-export): json preview to preserve nesting and exclude disabled fields fix(plugin-import-export): json preview and downloads preserve nesting and exclude disabled fields Jul 18, 2025
Copy link
Contributor

@DanRibbens DanRibbens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like to add some int test coverage for json exports. We have some already so this should be very quick to add some nested field assertions.

Copy link
Contributor

@DanRibbens DanRibbens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should be able to write an assertion of the downloaded data. Here is an example from gippity:

describe('GET /api/download', () => {
  it('should download a JSON file with correct data', async () => {
    const res = await request(app)
      .get('/api/download')
      .expect(200)
      .expect('Content-Type', /application\/json|octet-stream/)

    // Optional: assert it's served as a downloadable file
    expect(res.headers['content-disposition']).toMatch(/attachment; filename=.*\.json/)

    // Parse the file content as JSON
    const data = JSON.parse(res.text)

    // Write assertions on the JSON structure
    expect(data).toHaveProperty('items')
    expect(Array.isArray(data.items)).toBe(true)
    expect(data.items[0]).toMatchObject({ id: expect.any(Number), name: expect.any(String) })
  })
})


expect(Array.isArray(data)).toBe(true)
expect(['string', 'number']).toContain(typeof data[0].id)
expect(typeof data[0].title).toBe('string')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should include a nested field as another assertion to make sure the json structure isn't being flattened.

Copy link
Contributor

@DanRibbens DanRibbens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving despite wanting a little more from the test case added. We can always add when we get around to refactoring the createExport function which needs to have some logic deduplicated.

@DanRibbens DanRibbens merged commit 8f85da8 into main Jul 24, 2025
81 checks passed
@DanRibbens DanRibbens deleted the fix/disabled-nested-with-json-preview branch July 24, 2025 15:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants