Integrate Bria 3.1/3.2 Models and ControlNet Pipelines into InvokeAI

[ilanbria]

Summary

This feature PR integrates Bria 3.1/3.2 text-to-image models and their ControlNet pipelines into InvokeAI.

How

Area	Key Changes
Backend	Added invokeai/backend/bria/** including:- pipeline_bria.py, pipeline_bria_controlnet.py (diffusers workflows)- transformer_bria.py, controlnet_bria.py, helper & utility scripts- controlnet_aux detectors (Canny/OpenPose)
Model Manager	Added model_loaders/bria.py and updated probe/taxonomy for auto-discovery of Bria Main & ControlNet models.
Invocations / Nodes	New nodes under invokeai/nodes/bria_nodes/* including model loader, encoder, decoder, denoiser, latent sampler, and ControlNet handlers.
UI / API	- New field components (BriaMainModelFieldInputComponent.tsx, BriaControlNetModelFieldInputComponent.tsx)- Extended UIType enum for Node Editor integration- Updated Model Manager badges and schemas
Dependencies	Added scikit-image (required for OpenPose detector), updated uv.lock.
Scope	46 files changed, ~5,500 new LOC, 3 LOC removed

Representative commits:

• Add Bria text to image model and controlnet support

• Setup Probe and UI to accept bria main/controlnet models

• Added scikit-image required for Bria's OpenposeDetector model

---

Related Issues / Discussions

• Follow-up to exploratory PR Setup Probe and UI to accept bria main models #7973 ("Setup Probe and UI to accept Bria main models").

---

QA Instructions

Requires Bria model weights and a CUDA-11+ GPU (CPU supported, slower).

Steps:

1. Install Models

   • Set HF token with access to Bria's models.

   • Install briaai/BRIA-3.2.

   • Install briaai/BRIA-3.2-ControlNet-Union.

2. Load Workflow

   • Load the workflow from the attached file:

     [Bria's ControlNet workflow.json](https://github.com/user-attachments/files/21248912/Bria.s.ControlNet.workflow.json) [Bria's text-to-image workflow.json](https://github.com/user-attachments/files/21248924/Bria.s.text-to-image.workflow.json)

3. Run Workflow

   • Run workflow without controlnet.

   • Run workflow with controlnet.

---

Merge Plan

• Rebase onto main after current release.

---

Checklist

• The PR has a short but descriptive title, suitable for a changelog
• Tests added / updated (if applicable)
• Documentation added / updated (if applicable)
• Updated What's New copy (if doing a release after this PR)

[hipsterusername]

Hey! Confirming, have you tested that this works with the new location? I ran into some issues and may need to poke around in my local install to see what's going on if you were able to run these without issue

[ilanbria]

Hey! Confirming, have you tested that this works with the new location? I ran into some issues and may need to poke around in my local install to see what's going on if you were able to run these without issue

Hey! yeah had an import error, fixed it. Should work now

[hipsterusername]

Thanks - Trying to get this to run in between a lot of other stuff. I'm getting an error

Device-side assertions were explicitly omitted for this error check; the error probably arose while initializing the DSA handlers.
[2025-07-17 11:54:32,972]::[InvokeAI]::ERROR --> Traceback (most recent call last):
File "/home/kent/InvokeAI/invokeai/app/services/session_processor/session_processor_default.py", line 130, in run_node
output = invocation.invoke_internal(context=context, services=self._services)
File "/home/kent/InvokeAI/invokeai/app/invocations/baseinvocation.py", line 241, in invoke_internal
output = self.invoke(context)
File "/home/kent/InvokeAI/invokeai/app/invocations/bria_latent_sampler.py", line 54, in invoke
generator = torch.Generator(device=device).manual_seed(self.seed)
RuntimeError: CUDA error: unknown error
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Device-side assertions were explicitly omitted for this error check; the error probably arose while initializing the DSA handlers.

---

Looking at the above, it may be wise to reference some of the device references elsewhere -- Is the noise generation something that needs to happen on GPU?

[ilanbria]

Thanks - Trying to get this to run in between a lot of other stuff. I'm getting an error

Device-side assertions were explicitly omitted for this error check; the error probably arose while initializing the DSA handlers. [2025-07-17 11:54:32,972]::[InvokeAI]::ERROR --> Traceback (most recent call last): File "/home/kent/InvokeAI/invokeai/app/services/session_processor/session_processor_default.py", line 130, in run_node output = invocation.invoke_internal(context=context, services=self._services) File "/home/kent/InvokeAI/invokeai/app/invocations/baseinvocation.py", line 241, in invoke_internal output = self.invoke(context) File "/home/kent/InvokeAI/invokeai/app/invocations/bria_latent_sampler.py", line 54, in invoke generator = torch.Generator(device=device).manual_seed(self.seed) RuntimeError: CUDA error: unknown error CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1 Device-side assertions were explicitly omitted for this error check; the error probably arose while initializing the DSA handlers.

Looking at the above, it may be wise to reference some of the device references elsewhere -- Is the noise generation something that needs to happen on GPU?

It would be significantly faster if it runs on the gpu, but I suspect that if it fails here it will also fail in the denoiser node.
I'm not getting it on my g6e.xlarge instance (uses L40S) though, what machine are you running it on?
I've set the device to be as the transformer device, can you check again?

[hipsterusername]

I am able to run the node now (progress) although it's extremely slow - I suspect that there are a number of missing hooks into existing device references, tools for memory management, etc.

Did you reference other inference nodes for conventions, or is this primarily a simple code port?

(Re: Device I'm on - testing on a box with 4 ADA 6000s)

[ilanbria]

I am able to run the node now (progress) although it's extremely slow - I suspect that there are a number of missing hooks into existing device references, tools for memory management, etc.

Did you reference other inference nodes for conventions, or is this primarily a simple code port?

I referenced flux and sd3 inference nodes, but I didn't found anything special other then the use of model_on_device() that didn't improve vram usage or loading time when testing.
Aside from the initial model load slowdown, once cached, the denoiser node completes a 1024 × 1024 image at 50 steps in about ~20 s, and the text-encoder and noise nodes finish within a few hundred milliseconds on my l40s.
If there are other hooks you rely on or further optimizations I should include, just let me know and I'll wire them in.

[hipsterusername]

@ilanbria - We're planning on merging this to begin work on the UI, however it's running into some testing failures now that our standard workflows are running. Let us know if any you need clarity on any of these

[ilanbria]

@ilanbria - We're planning on merging this to begin work on the UI, however it's running into some testing failures now that our standard workflows are running. Let us know if any you need clarity on any of these

Fixed the ruff and the schema, should work now

[psychedelicious]

Massive effort on this, well done!

It does need some changes to fit in with Invoke's conventions and patterns.

• What is the latent scale factor for Bria 3.1/3.2? We will use this constant value in the Canvas to constraint the generation sizes.
• The ControlNet nodes are not compatible with Invoke's Canvas. Invoke will handle all "preprocessing" up-front and provide the control image directly to the ControlNet node. Canvas will resize the image to match the input noise/latents.
• Invoke has nodes to handle Canny edge detection, "color tile", depth detection, etc. We don't need to vendor in the code for that functionality.
  • Also, as mentioned in a comment in the openpose code, Invoke is Apache-2.0 and not compatible with the explicitly non-commercial license in that openpose code.
• Does Bria support image-to-image workflows? If so, we'll need a VAE encoder node.

[psychedelicious]

The logic in this file is the legacy implementation. We are not adding to it.

Please define the Bria model config in invokeai/backend/model_manager/config.py. Write the classification logic in that config class, implemented with the matches() and parse() methods.

[ilanbria]

Massive effort on this, well done!

It does need some changes to fit in with Invoke's conventions and patterns.

• What is the latent scale factor for Bria 3.1/3.2? We will use this constant value in the Canvas to constraint the generation sizes.

• The ControlNet nodes are not compatible with Invoke's Canvas. Invoke will handle all "preprocessing" up-front and provide the control image directly to the ControlNet node. Canvas will resize the image to match the input noise/latents.

• Invoke has nodes to handle Canny edge detection, "color tile", depth detection, etc. We don't need to vendor in the code for that functionality.

  • Also, as mentioned in a comment in the openpose code, Invoke is Apache-2.0 and not compatible with the explicitly non-commercial license in that openpose code.

• Does Bria support image-to-image workflows? If so, we'll need a VAE encoder node.

• latent scale factor is 0.13025
• yes, we support image to image with controlnet, why do we need vae encoder?

[psychedelicious]

Here's how to implement the Bria model probing logic in config.py:

diff --git a/invokeai/backend/model_manager/config.py b/invokeai/backend/model_manager/config.py
index e90d64509e..7c35cea3d5 100644
--- a/invokeai/backend/model_manager/config.py
+++ b/invokeai/backend/model_manager/config.py
@@ -442,26 +442,6 @@ class LoRADiffusersConfig(LoRAConfigBase, ModelConfigBase):
             "base": cls.base_model(mod),
         }
 
-class BriaDiffusersConfig(LoRAConfigBase, ModelConfigBase):
-    """Model config for Bria/Diffusers models."""
-
-    format: Literal[ModelFormat.Diffusers] = ModelFormat.Diffusers
-
-    @classmethod
-    def matches(cls, mod: ModelOnDisk) -> bool:
-        if mod.path.is_file():
-            return cls.flux_lora_format(mod) == FluxLoRAFormat.Diffusers
-
-        suffixes = ["bin", "safetensors"]
-        weight_files = [mod.path / f"pytorch_lora_weights.{sfx}" for sfx in suffixes]
-        return any(wf.exists() for wf in weight_files)
-
-    @classmethod
-    def parse(cls, mod: ModelOnDisk) -> dict[str, Any]:
-        return {
-            "base": cls.base_model(mod),
-        }
-
 
 class VAECheckpointConfig(CheckpointConfigBase, LegacyProbeMixin, ModelConfigBase):
     """Model config for standalone VAE models."""
@@ -541,6 +521,35 @@ class MainDiffusersConfig(DiffusersConfigBase, MainConfigBase, LegacyProbeMixin,
     pass
 
 
+class BriaDiffusersConfig(DiffusersConfigBase, MainConfigBase):
+    """Model config for Bria/Diffusers models."""
+
+    format: Literal[ModelFormat.Diffusers] = ModelFormat.Diffusers
+    base: Literal[BaseModelType.Bria] = BaseModelType.Bria
+
+    @classmethod
+    def matches(cls, mod: ModelOnDisk) -> bool:
+        if mod.path.is_file():
+            return False
+
+        config_path = mod.path / "transformer" / "config.json"
+        if config_path.exists():
+            with open(config_path) as file:
+                transformer_conf = json.load(file)
+            if transformer_conf["_class_name"] == "BriaTransformer2DModel":
+                return True
+
+        return False
+
+    @classmethod
+    def parse(cls, mod: ModelOnDisk) -> dict[str, Any]:
+        return {}
+
+    @classmethod
+    def get_tag(cls) -> Tag:
+        return Tag(f"{ModelType.Main.value}.{ModelFormat.Diffusers.value}.{BaseModelType.Bria.value}")
+
+
 class IPAdapterConfigBase(ABC, BaseModel):
     type: Literal[ModelType.IPAdapter] = ModelType.IPAdapter

[ilanbria]

doesn't work, if i delete

i get KeyError: 'cross_attention_dim'

if i delete

I get no config type error

[psychedelicious]

• Does Bria support image-to-image workflows? If so, we'll need a VAE encoder node.

• yes, we support image to image with controlnet, why do we need vae encoder?

For Invoke, "image-to-image" means partial denoising. We provide partially noised latents to the transformer or UNet and then only do some of the denoising steps.

For example, our existing denoise nodes have fields for:

• Denoising start (percentage)
• Denoising end (percentage)
• Step count (int > 0)
• Input latents (VAE-encoded image)

We use these parameters to add the appropriate amount of noise to the input latents and do partial denoising.

ControlNet is handled separately, and can be used with text-to-image or image-to-image.