Skip to content

Devops integrations #111

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Jul 31, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion docs/integration/.pages
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,6 @@ nav:
- AWS: aws
- Cloudflare: cloudflare.md
- PostgreSQL: postgresql.md
- Snowflake: snowflake.md
- Snowflake: snowflake.md
- Servers: servers
- DevOps : devops
7 changes: 7 additions & 0 deletions docs/integration/devops/.pages
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
nav:

- Jenkins: jenkins.md
- Ansible: ansible.md
- Terraform: terraform.md
- Github Actions: github-actions.md

159 changes: 159 additions & 0 deletions docs/integration/devops/ansible.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,159 @@
---
title: Ansible Logs Integration Guide
description: Forward Ansible execution logs to OpenObserve using Fluent Bit for real-time search and observability.

---

# Integration with Ansible

This guide walks you through forwarding **Ansible logs** from an Ubuntu machine to OpenObserve using Fluent Bit.

## Overview

Ansible doesn’t send logs to external systems by default. This integration configures a local playbook to write logs to a file, and uses Fluent Bit to tail that file and send logs to OpenObserve over HTTP.

## Steps to Integrate

??? "Prerequisites"

- Ubuntu machine (local, virtual machine, or cloud VM)
- User with `sudo` privileges
- Basic familiarity with Ansible and Fluent Bit
- OpenObserve account ([Cloud](https://cloud.openobserve.ai/web/) or [Self-Hosted](../../../quickstart/#self-hosted-installation))
- Ansible Installed

??? "Step 1: Configure Ansible log File"

1. **Configure log output path in `ansible.cfg`**:
```ini
[defaults]
log_path = /tmp/ansible_log_demo.txt
```

??? "Step 2: Create a Local Ansible Task (Optional)"

1. **Create a sample playbook**:
```bash
vi local_task.yml
```
2. Add the following contents:
```yaml
---
- name: Ansible Local Task Demo
hosts: localhost
connection: local
tasks:
- name: Create a sample file
ansible.builtin.file:
path: /tmp/ansible_log_demo.txt
state: touch

- name: Write a message to the sample file
ansible.builtin.copy:
content: "This is a demo log entry from Ansible."
dest: /tmp/ansible_log_demo.txt
```

3. **Run the playbook**:
```bash
ansible-playbook local_task.yml
```

After running, check `/tmp/ansible_log_demo.txt` to confirm logs are generated.

??? "Step 3: Install Fluent Bit on Ubuntu"

1. **Install Fluent Bit using the official script**:
```bash
curl https://raw.githubusercontent.com/fluent/fluent-bit/master/install.sh | sh
```
> For manual install or using package managers, refer to [Fluent Bit Docs](https://docs.fluentbit.io/manual/installation/linux/ubuntu)

2. **Verify installation**:
```bash
fluent-bit --version
```

??? "Step 4: Configure Fluent Bit for Ansible Logs"

1. **Edit Fluent Bit config file**:
```bash
sudo vi /etc/fluent-bit/fluent-bit.conf
```

2. **Add the following configuration**:
```ini
[SERVICE]
Flush 5
Daemon Off
Log_Level info

[INPUT]
Name tail
Path /tmp/ansible_log_demo.txt
Tag ansible.demo
Refresh_Interval 5

[OUTPUT]
Name http
Match *
URI /api/<O2_ORG_NAME>/<O2_STREAM_NAME>/_json
Host <O2_HOST>
Port 443
tls On
Format json
Json_date_key _timestamp
Json_date_format iso8601
HTTP_User <O2_USER>
HTTP_Passwd <O2_PASSWORD>
compress gzip
```

> Note
- Replace `<O2_ORG_NAME>`, `<O2_STREAM_NAME>`, `<O2_HOST>`, `<O2_USER>`, and `<O2_PASSWORD>` with your OpenObserve values.
- Example URI: `/api/default/ansible/_json` for the `default` org and `ansible` stream.

??? "Step 5: Start Fluent Bit"

1. **Start the Fluent Bit service**:
```bash
sudo systemctl start fluent-bit
sudo systemctl enable fluent-bit
```
2. **Check service status**:
```bash
sudo systemctl status fluent-bit
```
> Ensure there are no startup errors. Logs will now be tailed and sent to OpenObserve.

??? "Step 6: Verify Logs in OpenObserve"

1. In your Openobserve instance, Go to **Logs** → select your log stream → Set time range → Click **Run Query**

![Verify Ansible Logs in OpenObserve](https://openobserve.ai/assets%2Fansible_logs_o2_2fa50f03c6.gif)
## Troubleshooting

??? "No logs in OpenObserve?"

- Ensure Fluent Bit is running: `sudo systemctl status fluent-bit`
- Double-check the HTTP config and authentication
- Use Fluent Bit in debug mode:
```bash
sudo fluent-bit -c /etc/fluent-bit/fluent-bit.conf -vv
```

??? "Ansible log file not updating?"

- Confirm `log_path` is correctly set in `ansible.cfg`
- Use verbose flags (`-v`, `-vv`, `-vvv`) to produce more logs
- Confirm Ansible has permission to write to `/tmp`

??? "Fluent Bit not picking up logs?"

- Make sure the path `/tmp/ansible_log_demo.txt` exists
- Tail the file manually to confirm updates:
```bash
tail -f /tmp/ansible_log_demo.txt
```

- Adjust `Refresh_Interval` or restart Fluent Bit if needed
101 changes: 101 additions & 0 deletions docs/integration/devops/github-actions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
---

title: GitHub Actions Integration Guide
description: Stream GitHub Actions workflow logs to OpenObserve using the `mdp/openobserve_github_action_logs` action.

---

# Integration with GitHub Actions

This guide explains how to stream **GitHub Actions workflow logs** to OpenObserve using a community-supported GitHub Action. Once integrated, logs from your CI/CD workflows are automatically sent to OpenObserve for monitoring and analysis.

## Overview

This integration captures metadata and logs from GitHub Actions jobs and pushes them directly to OpenObserve using the [mdp/openobserve_github_action_logs](https://github.com/mdp/openobserve_github_action_logs) Action. It requires minimal configuration and enables observability for CI pipelines.

## Steps to Integrate

??? "Prerequisites"

* OpenObserve account ([Cloud](https://cloud.openobserve.ai/web/) or [Self-Hosted](../../../quickstart/#self-hosted-installation))
* GitHub repository

??? "Step 1: Get OpenObserve Credentials"

1. Log in to your OpenObserve instance
2. Navigate to **Data Sources → custom → logs → curl**
3. Copy the following values from the generated `curl` command:

From:

```bash
curl -u you@yourdomain.com:18qlg4b673Rgdgd2 -k https://api.openobserve.ai/api/your_organization/<stream_name>/_json -d [JSON-DATA]
```

Extract values:

```
OPENOBSERVE_ENDPOINT : https://api.openobserve.ai/api/your_organization/<stream_name>/_json
OPENOBSERVE_USERNAME : you@yourdomain.com
OPENOBSERVE_KEY : 18qlg4b673Rgdgd2
```

??? "Step 2: Add Secrets to GitHub Environment"

1. In your GitHub repository, go to **Settings → Environments**
2. Click **New environment** and name it `dev`

![Create GitHub Environment](../images/devops/github-actions/dev-env.png){: style="height:500px"}
3. Under the `dev` environment, click **Environment secrets → Add environment secret** and add the following: `OPENOBSERVE_USERNAME`, `OPENOBSERVE_KEY`, and `OPENOBSERVE_ENDPOINT` using the values from Step 1.

![Add Secrets to GitHub Environment](../images/devops/github-actions/add-secret.png){: style="height:300px"}

??? "Step 3: Add GitHub Action to Your Workflow"

Add the following to your GitHub Actions YAML file:

```yaml
jobs:
build-and-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- run: |
npm ci
- run: |
npm run all
- run: npm test

send_logs_to_openobserve:
runs-on: ubuntu-latest
needs: build-and-test # ensures logs are sent after main job
environment: dev
steps:
- uses: actions/checkout@v3
- uses: mdp/openobserve_github_action_logs@main
with:
openobserve_endpoint: ${{ secrets.OPENOBSERVE_ENDPOINT }}
openobserve_username: ${{ secrets.OPENOBSERVE_USERNAME }}
openobserve_key: ${{ secrets.OPENOBSERVE_KEY }}
github_token: ${{ secrets.GITHUB_TOKEN }}
```

Once this is in place, all logs from your GitHub Actions jobs will be automatically piped into OpenObserve.

> Note: The example uses Node.js-specific commands (npm ci, npm run all).
If your project is not Node.js based, replace these steps with your own project's build/test commands (e.g., make, pytest, go test, etc.).

??? "Step 4: Verify Logs in OpenObserve"

1. In your Openobserve instance, Go to **Logs** → select your log stream → Set time range → Click **Run Query**

![Verify Logs](../images/devops/github-actions/verify-logs.png)

## Troubleshooting

??? "No logs appearing?"

* Double-check that the GitHub secrets were added under the correct **environment (`dev`)**
* Ensure the `openobserve_endpoint` is correct and matches the `curl` example
* Confirm the `send_logs_to_openobserve` job ran successfully in GitHub Actions

Loading