Use temp allocator for kernel registry #13012

JakeStevens · 2025-07-30T20:10:30Z

Summary: When indexing to the registry to get the op, memory is allocated from the method allocator to instantiate some TensorMeta and included objects. This memory is only used for that purpose and is not needed for the entire lifetime of the Method. Thus, we can instead use temp allocator which can later be reset and free up memory as needed.

Differential Revision: D79285675

cc @larryliu0820 @JacobSzwejbka @lucylq

pytorch-bot · 2025-07-30T20:10:34Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/13012

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

ghstack-mergeability-check and Check labels failing with 'Resource not accessible by integration'

❌ 2 New Failures, 1 Unrelated Failure

As of commit c831756 with merge base 90ff059 ():

NEW FAILURES - The following jobs have failed:

Build documentation / build (buck2) / Build doc (gh)
The process '/usr/bin/git' failed with exit code 128
pull / test-moshi-linux / linux-job (gh)
RuntimeError: Command docker exec -t 88ad12c1424c6ee4fe83f012a1b27a369d0ea421d46a8595edd0ef1640240aa1 /exec failed with exit code 1

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

pull / test-openvino-linux / linux-job (gh) (trunk failure)
AttributeError: '_OpNamespace' 'quantized_decomposed' object has no attribute 'convert_element_type'

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2025-07-30T20:10:38Z

This pull request was exported from Phabricator. Differential Revision: D79285675

github-actions · 2025-07-30T20:11:18Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

Summary: When indexing to the registry to get the op, memory is allocated from the method allocator to instantiate some TensorMeta and included objects. This memory is only used for that purpose and is not needed for the entire lifetime of the Method. Thus, we can instead use temp allocator which can later be reset and free up memory as needed. Differential Revision: D79285675

facebook-github-bot · 2025-07-30T20:56:12Z

This pull request was exported from Phabricator. Differential Revision: D79285675

Summary: Pull Request resolved: pytorch#13012 When indexing to the registry to get the op, memory is allocated from the method allocator to instantiate some TensorMeta and included objects. This memory is only used for that purpose and is not needed for the entire lifetime of the Method. Thus, we can instead use temp allocator which can later be reset and free up memory as needed. Differential Revision: D79285675

larryliu0820

Thank you for doing this.

Summary: When indexing to the registry to get the op, memory is allocated from the method allocator to instantiate some TensorMeta and included objects. This memory is only used for that purpose and is not needed for the entire lifetime of the Method. Thus, we can instead use temp allocator which can later be reset and free up memory as needed. Reviewed By: larryliu0820 Differential Revision: D79285675

facebook-github-bot · 2025-07-31T19:46:30Z

This pull request was exported from Phabricator. Differential Revision: D79285675

Summary: Pull Request resolved: pytorch#13012 When indexing to the registry to get the op, memory is allocated from the method allocator to instantiate some TensorMeta and included objects. This memory is only used for that purpose and is not needed for the entire lifetime of the Method. Thus, we can instead use temp allocator which can later be reset and free up memory as needed. Reviewed By: larryliu0820 Differential Revision: D79285675

Summary: When indexing to the registry to get the op, memory is allocated from the method allocator to instantiate some TensorMeta and included objects. This memory is only used for that purpose and is not needed for the entire lifetime of the Method. Thus, we can instead use temp allocator which can later be reset and free up memory as needed. Reviewed By: larryliu0820 Differential Revision: D79285675

facebook-github-bot · 2025-08-01T01:09:24Z

This pull request was exported from Phabricator. Differential Revision: D79285675

Summary: Pull Request resolved: pytorch#13012 When indexing to the registry to get the op, memory is allocated from the method allocator to instantiate some TensorMeta and included objects. This memory is only used for that purpose and is not needed for the entire lifetime of the Method. Thus, we can instead use temp allocator which can later be reset and free up memory as needed. Reviewed By: larryliu0820 Differential Revision: D79285675

Summary: When indexing to the registry to get the op, memory is allocated from the method allocator to instantiate some TensorMeta and included objects. This memory is only used for that purpose and is not needed for the entire lifetime of the Method. Thus, we can instead use temp allocator which can later be reset and free up memory as needed. Reviewed By: larryliu0820 Differential Revision: D79285675

facebook-github-bot · 2025-08-01T14:19:06Z

This pull request was exported from Phabricator. Differential Revision: D79285675

Summary: Pull Request resolved: pytorch#13012 When indexing to the registry to get the op, memory is allocated from the method allocator to instantiate some TensorMeta and included objects. This memory is only used for that purpose and is not needed for the entire lifetime of the Method. Thus, we can instead use temp allocator which can later be reset and free up memory as needed. Reviewed By: larryliu0820 Differential Revision: D79285675

Summary: When indexing to the registry to get the op, memory is allocated from the method allocator to instantiate some TensorMeta and included objects. This memory is only used for that purpose and is not needed for the entire lifetime of the Method. Thus, we can instead use temp allocator which can later be reset and free up memory as needed. Reviewed By: larryliu0820 Differential Revision: D79285675

Summary: Pull Request resolved: pytorch#13012 When indexing to the registry to get the op, memory is allocated from the method allocator to instantiate some TensorMeta and included objects. This memory is only used for that purpose and is not needed for the entire lifetime of the Method. Thus, we can instead use temp allocator which can later be reset and free up memory as needed. Reviewed By: larryliu0820 Differential Revision: D79285675

facebook-github-bot · 2025-08-05T13:15:55Z

This pull request was exported from Phabricator. Differential Revision: D79285675

JakeStevens · 2025-08-06T13:20:01Z

Failing tests pre-existing. moshi from undefined libtorchaudio symbol, openvino from backend import. Landing.

JakeStevens requested review from JacobSzwejbka, lucylq and swolchok as code owners July 30, 2025 20:10

meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 30, 2025

facebook-github-bot added the fb-exported label Jul 30, 2025

JakeStevens added the module: runtime Issues related to the core runtime and code under runtime/ label Jul 30, 2025

JakeStevens force-pushed the export-D79285675 branch from 024df78 to 7128e81 Compare July 30, 2025 20:52

JakeStevens force-pushed the export-D79285675 branch from 7128e81 to 07a7687 Compare July 30, 2025 20:56

larryliu0820 approved these changes Jul 30, 2025

View reviewed changes

JakeStevens force-pushed the export-D79285675 branch from 07a7687 to c5ce955 Compare July 31, 2025 19:42

JakeStevens force-pushed the export-D79285675 branch 2 times, most recently from 23ddf27 to 8f73ee3 Compare August 1, 2025 01:05

JakeStevens force-pushed the export-D79285675 branch from 8f73ee3 to 1f41298 Compare August 1, 2025 01:09

JakeStevens force-pushed the export-D79285675 branch from 1f41298 to 433e6af Compare August 1, 2025 14:15

JakeStevens force-pushed the export-D79285675 branch from 433e6af to 4e159e8 Compare August 1, 2025 14:19

JakeStevens force-pushed the export-D79285675 branch from 4e159e8 to c8e360d Compare August 5, 2025 13:12

JakeStevens force-pushed the export-D79285675 branch from c8e360d to c831756 Compare August 5, 2025 13:16

JakeStevens merged commit 414fc32 into pytorch:main Aug 6, 2025
101 of 105 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Use temp allocator for kernel registry #13012

Use temp allocator for kernel registry #13012

Uh oh!

JakeStevens commented Jul 30, 2025 •

edited by pytorch-bot bot

Loading

Uh oh!

pytorch-bot bot commented Jul 30, 2025 •

edited

Loading

Uh oh!

facebook-github-bot commented Jul 30, 2025

Uh oh!

github-actions bot commented Jul 30, 2025

Uh oh!

facebook-github-bot commented Jul 30, 2025

Uh oh!

larryliu0820 left a comment

Uh oh!

facebook-github-bot commented Jul 31, 2025

Uh oh!

facebook-github-bot commented Aug 1, 2025

Uh oh!

facebook-github-bot commented Aug 1, 2025

Uh oh!

facebook-github-bot commented Aug 5, 2025

Uh oh!

JakeStevens commented Aug 6, 2025

Uh oh!

Uh oh!

Uh oh!

Use temp allocator for kernel registry #13012

Use temp allocator for kernel registry #13012

Uh oh!

Conversation

JakeStevens commented Jul 30, 2025 • edited by pytorch-bot bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Jul 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/13012

❗ 1 Active SEVs

❌ 2 New Failures, 1 Unrelated Failure

Uh oh!

facebook-github-bot commented Jul 30, 2025

Uh oh!

github-actions bot commented Jul 30, 2025

This PR needs a release notes: label

Uh oh!

facebook-github-bot commented Jul 30, 2025

Uh oh!

larryliu0820 left a comment

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Jul 31, 2025

Uh oh!

facebook-github-bot commented Aug 1, 2025

Uh oh!

facebook-github-bot commented Aug 1, 2025

Uh oh!

facebook-github-bot commented Aug 5, 2025

Uh oh!

JakeStevens commented Aug 6, 2025

Uh oh!

Uh oh!

Uh oh!

JakeStevens commented Jul 30, 2025 •

edited by pytorch-bot bot

Loading

pytorch-bot bot commented Jul 30, 2025 •

edited

Loading

This PR needs a `release notes:` label