Skip to content

Excessive memory consumption when scanning large MS-CAB files #370

@Zopolis4

Description

@Zopolis4

When attempting to scan large MS-CAB files, an excessive amount of memory is consumed, appearing to be more than double the size of the file.

To demonstrate, this is what happens when I try and run ProtectionScan on Content1.cab and providing it with 2GB of memory. (If I do not cap it like this, the high memory consumption results in my computer being unresponsive and I have to reboot)

$ sudo systemd-run --scope -p MemoryMax=2000M /tmp/ProtectionScan -d Content1.cab
Running as unit: run-ree8d5917c911412094017ea6a37891d7.scope; invocation ID: 30aa8b43f21d442db81b2cbb4a8167a5
0.00%:  - 
0.00%: Content1.cab - Checking file
System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
   at System.IO.MemoryStream.set_Capacity(Int32 value)
   at System.IO.MemoryStream.EnsureCapacity(Int32 value)
   at SabreTools.Compression.MSZIP.Decompressor.CopyTo(Stream source, Stream dest)
   at BinaryObjectScanner.FileType.MicrosoftCAB.Extract(Stream stream, String file, String outDir, Boolean includeDebug)
Time elapsed: 00:00:11.3040508
100.00%: Content1.cab - 

For context, Content1.cab is 878 MB.

This issue has resulted in me being unable to dump multiple discs (http://redump.org/disc/49677/, for example) with protection scanning enabled, as it consumes all the available memory and then everything becomes unresponsive and the scan never finishes.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions