|
| 1 | +--- |
| 2 | +title: OpenObserve Streaming Search |
| 3 | +description: Learn how OpenObserve's Streaming Search delivers incremental query results using HTTP/2 partitioning for faster log analysis and real-time data processing. |
| 4 | +--- |
| 5 | +This user guide provides details on how to configure, and use OpenObserve's **Streaming Search** feature to improve query performance and responsiveness. |
| 6 | + |
| 7 | +## What is Streaming Search? |
| 8 | + |
| 9 | +Streaming Search is a feature that delivers query results incrementally to the user interface using an **HTTP/2** streaming connection. Instead of waiting for the entire query to complete, OpenObserve breaks large queries into smaller partitions and streams the results as they are processed. This allows matching log entries to appear more quickly, even while the query continues to run in the background. |
| 10 | + |
| 11 | +!!! note "Where to Find" |
| 12 | + The **Streaming Search** toggle is located under **Management > General Settings**. |
| 13 | + |
| 14 | +!!! note "Who Can Access" |
| 15 | + The `Root` user and any other user with permission to **update** the **Settings** module can modify the **Streaming Search** setting. Access is controlled through role-based access control (RBAC). |
| 16 | +  |
| 17 | + |
| 18 | + |
| 19 | +## Enable or Disable Streaming Search |
| 20 | + |
| 21 | +1. Go to **Management**. |
| 22 | +2. Select **General Settings**. |
| 23 | + |
| 24 | +3. Locate the **Enable Streaming Search** option. |
| 25 | +4. Toggle this switch to **On** to enable streaming mode, or **Off** to disable it. |
| 26 | +5. Click **Save** to save the changes. |
| 27 | + |
| 28 | +## How does it work? |
| 29 | + |
| 30 | +### 1. Partitioning |
| 31 | + |
| 32 | +OpenObserve splits the total time range into partitions, each covering a segment of time or data volume, called **partitions**. For example, an 8-hour query with 8 GB of data may be split into 8 partitions, each processing 1 GB. Each partition is executed independently and contributes partial results to the overall output. |
| 33 | + |
| 34 | +Streaming begins as soon as the first partition completes, improving responsiveness. |
| 35 | + |
| 36 | +### 2. Mini-Partitioning |
| 37 | + |
| 38 | +Mini-partitioning further breaks the first partition into small slices based strictly on time. Controlled by: `ZO_MINI_SEARCH_PARTITION_DURATION_SECS`=60 (default is 60 seconds) |
| 39 | + |
| 40 | +These short-duration partitions are created only at the beginning of the query time range. Enables the result to appear within milliseconds, even if the full partition takes longer. Once mini-partitions complete, the system switches back to processing full-size partitions. |
| 41 | + |
| 42 | +### 3. Streaming via HTTP/2 |
| 43 | + |
| 44 | +Results from each partition are sent incrementally to the browser using a single persistent HTTP/2 connection. |
| 45 | + |
| 46 | +### 4. Real-time delivery |
| 47 | + |
| 48 | +As each partition is processed, its results are immediately streamed to the user interface. |
| 49 | + |
| 50 | +This mechanism avoids delays caused by waiting for the entire result set and is especially useful for long-running or high-volume searches. |
| 51 | + |
| 52 | + |
| 53 | +## Considerations |
| 54 | + |
| 55 | +- Requires HTTP/2 support in the network stack. |
| 56 | +- Fallbacks to standard query mode if disabled. |
| 57 | +- Partitioning behavior is automatic. Mini-partitioning improves the time-to-first-result without affecting the accuracy of final results. |
0 commit comments