llext: Split linking and loading

[teburd]

Summary

LLEXT in its llext_load function helpfully sets up memory, copies sections, and links (applies relocations) for the ELF assuming the memory it setup is where the extension will be. There's some loader options that make it do slightly different things with branching internally but this I think makes the code more complex than it needs to be.

It can be useful to store the extension in one location (flash or slower ram) but link it as if it were placed in a different location (sram/tcm). This is what SOF does, and its what XIP/flash stored extensions would also want to do.

Not all sections need to work this way... its unlikely .rodata would be moved, but .text might be, and .data/.bss almost certainly have to be in the XIP from flash scenario.

Describe the solution you'd like

A way to link and copy an extension to a storage location (flash, slow ram, etc) while linking it for a different location where some sections will be copied/allocated to in the future (sram/tcm).

Another function to load it into the different location (sram) with either real or virtualized addressing when an MMU exists.

All mpu/mmu region setup, icache/dcache invalidation would happen at load time, not copy and link time.

Alternatives

Continue leaving SOF to work around this, not having XIP support for others in Zephyr

Additional Context

No response

[teburd]

@lgirdwood @lyakh @pillo79 Please do add your thoughts here, but we seem to keep running into some friction around this.

[pillo79]

Some ideas on the subject

• first on the drawing board: cleanly separate those cold storage / standby / runtime mappings. They should be explicitly named and used in LLEXT, at the moment it's all the same kind of pointers.
  E.g. ELF loading is from cold storage, linking on standby areas (using runtime values), mem partitions on runtime. Some may coincide, but that is not exploited directly.

• we need to come up with a flexible enough way to give the user the option of editing (or specifying!) those mappings. The current behaviors of a) allocating everything and b) reusing the ELF buffer should be implemented as examples for a more complex scheme (as e.g. needed by SOF).

• LLEXT is currently not fully handling VMAs specified in the ELF. This is very helpful in this context - ELF already describes independent locations, so it should be possible to use this information at some of the above stages.

• in the context of all this, do we still need to support the current stream-oriented API? What about minimizing memory at the cost of flexibility (e.g. not having what is now inspection data always available)?
  From our side it's a no-brainer: xip support >>> saving a few bytes of memory.

Wild theories for an LLEXT 2.0 (or 1.0?)

• struct llext fully describes the extension (no data in loader apart from loader-specific stuff)
• have a standardized way to describe a section address map and different instances for standby and runtime
• need to expose some internal elf structures to the user (something like the current inspection api)
• proposal for a possible API / flow:
  • llext_parse_elf -> gets all ELF records, section map etc from loader, returns an "unloaded" struct llext
  • user sets the standby and runtime addresses or leaves them NULL for heap allocation or direct reuse (as today)
  • llext_load_link -> copies data to standby areas, links using runtime addresses
  • llext_resume / llext_standby copy to/from runtime and standby areas and adjust memory permissions .

Please excuse my horrible naming 😉 but floor is open for comments! 🎤

[teburd]

I like the idea of breaking up the internal flow, and providing function hooks for decision making rather than flags. SOF is a prime example of where hooks I think would help more than flags. Default decision making functions that provide our current flow can be used.m but functions allow for sof to allocate the destination to copy itself. Decide what address to link for etc.

I like the idea of parsing the elf, copy+link which returns a handle, and a load+unload from this handle.

That matches I think the sof flow and a theoretical xip flow.

With hooks we can provide both direct sram, and cold storage decision paths cleaner than flags imo.