WARP Signature Matcher / Custom Types Interaction Incomplete

[utkonos]

Version and Platform (required):

• Binary Ninja Version: 4.2.6455
• OS: macOS
• OS Version: 15.3
• CPU Architecture: x64

Bug Description:
Given one database with full annotation, custom types, correct function names, correct variable names, and a new file getting a fresh analysis, functions with a custom type in the function's type cause WARP to only partially work. The type name of the argument changes, but the variable is still arg1. Also, I can tell by the return type still being grey that there was something that failed during the setting of the new function type and it only partially completed. The name of the function still starts with sub_.

Steps To Reproduce:

1. Open a database with full custom types, connected to a type library, funtion names, everything that can be annotated is done.
2. Move all the old SigKit signatures out of the signatures directory for the duration of this process so they don't have a chance to add any interactions with any of this.
3. Make a signature file from the database using WARP
4. Disable WARP sig matching in binja config. If you don't have the type archive attached before WARP runs, this function gets nothing at all. Not even the partial change here.
5. Open a new file that doesn't have a database yet.
6. Notice that if a function has a custom type in its type that type name appears, but the function name doesn't change nor do the variable names.
7. **Expected Behavior:**The function should have the name changed, it should be WARP tagged, parameters renamed, all the usual WARPy stuff.

Screenshots/Video Recording:
z_stream is a linked custom type that is found in NSIS installers. this isn't the whole type. Just the top piece that I needed, but that shouldn't have an effect.

Here is the source database:

Here is the result in the new file:

Binary:
You can use the same zlib nsis sample I already sent you.

Additional Information:
This part is probably a feature request in addition to the bug above. Or maybe I am missing a step or feature: Having a type archive connected to a database and specific types pulled in seems to help with WARP. It would be good if there were a way to do the type library connection and pull before WARP does its thing without having to just turn off the WARP matcher and then run it manually. Maybe WARP can be linked to a type archive? Maybe I can do something in open with options? Maybe there's already some way I'm missing?

[emesare]

This was caused by the way the matching applied information to the binary. This should be resolved already in https://github.com/Vector35/binaryninja-api/tree/test_warp_next however that is currently a very WIP branch so this specific scenerio should be tested to verify.

As an aside, the type archive stuff not being pulled automatically is not done, and as such this issue also can't be fully resolved until that is done as well. The type archive stuff shouldn't matter assuming the WARP signature file actually can collect the types correctly when generating the WARP file.

To recap: I need to validate that the signature application works correctly on the next branch, then we need to figure out what was causing function signatures to have incomplete type information, this is likely a result of the create signature action not correctly pulling in named type references.

[utkonos]

The step where I move the SigKit signatures temporarily to my desktop: am I being paranoid? They run before WARP. So, my thinking there is to have them elsewhere while WARP does its thing. Then move them back and run them to fill in gaps that WARP missed.

[emesare]

The step where I move the SigKit signatures temporarily to my desktop: am I being paranoid? They run before WARP.

They should not override WARP, the WARP matcher runs after SigKit and will overwrite SigKit, the only thing that I can think of that could impact WARP would be when SigKit applies incorrect function symbols which are then picked up by WARP for function matching. However the chances of that happening are slim.

If SigKit does actually override WARP than that should be documented, with the plan to fully replace SigKit this next release we don't really have a priority to fix such an issue (if it even occurs at all).

[utkonos]

I woudn't even bother chasing that one around since it's going away completely.

One more thing to add:

I still have one more step after all this: for any function that was not WARPed and tagged, it gets a name change by the SigKit cleanup. That then sets me up for this last step: importing just those specific functions from the database that the WARP sigs and sigkit sigs were made from that is the master or authoritative database. I mean the oldest of the old "Analysis" Menu -> "Import from BNDB..." And then unchecking everything but the set of functions that were named by SigKit.

I'm thinking about writing a quickie script that will tell me which ones this needs to be done to and then looking at the python API to see if there is a way that I can perform this import on the relavant components automagically at least with a click on a plugins menu entry that i wrote myself.

[utkonos]

I'm so looking forward to the next release with all the WARP stuff! You should have called it "Engage" rather than "Gallifrey" ;)