Skip to content

Add Azure Log Analytics Ingestion API plugin #10306

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 10 commits into
base: main
Choose a base branch
from
Open

Add Azure Log Analytics Ingestion API plugin #10306

wants to merge 10 commits into from

Conversation

wtcline-intc
Copy link

The Log Ingestion API allows sending data to a Log Analytics workspace in Azure Monitor.

SUMMARY

Add a Log Analytics plugin for the Log Ingestion API. The module is based on the existing Log Analytics plugin. It is a callback module so data from each task can be transparently logged.

ISSUE TYPE
  • New Module/Plugin Pull Request
COMPONENT NAME

loganalytics_ingestion

ADDITIONAL INFORMATION

Will gladly rename the module to something less lengthy, pending your input.

@wtcline-intc
Add Azure Log Analytics Ingestion API plugin
a8cd72f 
The Ingestion API allows sending data to a Log Analytics workspace in
Azure Monitor.
@ansibullbot ansibullbot added callback callback plugin new_contributor Help guide this first time contributor new_plugin New plugin plugins plugin (any type) tests tests unit tests/unit labels Jun 26, 2025
@ansibullbot

This comment was marked as outdated.

Sign in to view
@ansibullbot ansibullbot added ci_verified Push fixes to PR branch to re-run CI needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR labels Jun 26, 2025
@felixfontein
Copy link
Collaborator

Thanks for your contribution! Since this is for Azure, wouldn't it make more sense to have this in azure.azcollection (https://github.com/ansible-collections/azure)?

@felixfontein felixfontein added check-before-release PR will be looked at again shortly before release and merged if possible. backport-11 Automatically create a backport for the stable-10 branch labels Jun 26, 2025
@wtcline-intc
Copy link
Author

Thanks for your contribution! Since this is for Azure, wouldn't it make more sense to have this in azure.azcollection (https://github.com/ansible-collections/azure)?

Thanks for the quick response! It might indeed make more sense for this module to be in the Azure collection, but the Log Analytics callback plugin is in this collection despite being Azure-specific, so, based on precedent, it might make more sense for this module to also be in this collection. I'm okay with either.

@wtcline-intc
Copy link
Author

That should fix CI issues. I'm also open to refactoring to using the ansible.builtin.uri module instead of requests.

@ansibullbot
Copy link
Collaborator

cc @zhcli
click here for bot help

@ansibullbot ansibullbot removed the ci_verified Push fixes to PR branch to re-run CI label Jun 26, 2025
@ansibullbot

This comment was marked as outdated.

Sign in to view
@ansibullbot

This comment was marked as outdated.

Sign in to view
@wtcline-intc wtcline-intc force-pushed the loganalytics_ingestion branch from 6a7f46a to eb7d846 Compare June 27, 2025 02:20
@ansibullbot ansibullbot removed the needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR label Jun 27, 2025
@felixfontein
Copy link
Collaborator

Thanks for your contribution! Since this is for Azure, wouldn't it make more sense to have this in azure.azcollection (https://github.com/ansible-collections/azure)?

Thanks for the quick response! It might indeed make more sense for this module to be in the Azure collection, but the Log Analytics callback plugin is in this collection despite being Azure-specific, so, based on precedent, it might make more sense for this module to also be in this collection. I'm okay with either.

Hmm, I wasn't aware / didn't notice before that community.general.loganalytics is also Azure specific.

@Fred-sun, since you maintain azure.azcollection, do you think this plugin (and maybe also community.general.loganalytics) should be placed in azure.azcollection? Or is there another collection you are aware of that this callback would be well suited for?

@Fred-sun
Copy link

Yes, The azure.azcollection already supports the management of log analytics workspace, and some functions are still under development.

@pboushy
Copy link

pboushy commented Jun 30, 2025

I can see a reason to place it in azure.azcollection, but within our team, I argued for keeping it in community.general because:

  • The Azure.azcollection is focused on managing Azure assets and environments, and doesn't have any callback plugins already.
  • This doesn't create or manage Azure. It sends Ansible task results to Azure.
  • community.general contains several other related plugins that send Ansible task results to other log collection tools including logstash (related repo: community.elastic), and splunk (related repo: splunk.es)

It appears that the individual collections are all focused on managing the respective service (azure, elastic, splunk) where as these items that have been placed in community.general are about shipping data into one of the services.

Considering this, do you still want this moved to azcollection?

@wtcline-intc
Add Log Analytics Ingestion auth timeout
1df54c9 
Previous behavior was to use the 'request' module's default timeout;
this makes auth timeout value consistent with the task submission
timeout value.
@ansibullbot ansibullbot added the stale_ci CI is older than 7 days, rerun before merging label Jul 8, 2025
@wtcline-intc
Display Log Analytics Ingestion event data as JSON
ca43400 
Previous behavior was to display the data as a Python dictionary.
The new behavior makes it easier to generate a sample JSON file in order
to import into Azure when creating the table.
@wtcline-intc
Add Azure Log Analytics Ingestion timeout param
8324b3c 
This parameter controls how long the plugin will wait for an HTTP response
from the Azure Log Analytics API before considering the request a failure.
Previous behavior was hardcoded to 2 seconds.
@wtcline-intc
Copy link
Author

Made a few changes that we found helpful.

  • Added the request timeout value to the initial auth request in addition to the task data callback
  • Output debug event data as JSON instead of a Python dictionary; this makes it easier to create the Azure table schema during set-up
  • Allow configuring the request timeout value

@ansibullbot ansibullbot added needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR and removed stale_ci CI is older than 7 days, rerun before merging labels Jul 11, 2025
@wtcline-intc
Fix Azure Log Ingestion unit test
761e55c 
The class instantiation was missing an additional argument that was added
in a previous patch; add it.  Converting to JSON also caused the Mock
TaskResult object to throw a serialization error; override the function
for JSON conversion to just return bogus data instead.
@ansibullbot ansibullbot removed the needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR label Jul 11, 2025
@ansibullbot ansibullbot added the stale_ci CI is older than 7 days, rerun before merging label Jul 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
backport-11 Automatically create a backport for the stable-10 branch callback callback plugin check-before-release PR will be looked at again shortly before release and merged if possible. new_contributor Help guide this first time contributor new_plugin New plugin plugins plugin (any type) stale_ci CI is older than 7 days, rerun before merging tests tests unit tests/unit
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@wtcline-intc @ansibullbot @felixfontein @Fred-sun @pboushy