Skip to content

PMM-14072 Automate Grafana plugin upgrade #1687

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 9 commits into
base: v3
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -30,3 +30,12 @@ docker_clean:
docker-compose stop \
&& docker-compose rm -f -v \
&& docker system prune --volumes -f

.PHONY: upgrade_plugins
upgrade_plugins:
@echo "Upgrading PMM plugins..."
@echo "To override the platform, set the PLATFORM environment variable."
@echo "Example: PLATFORM=linux/arm64 make upgrade_plugins"
@echo
@cd misc && \
./upgrade-plugins.sh
9 changes: 9 additions & 0 deletions misc/Dockerfile.upgrade
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
ARG IMAGE_VERSION=grafana/grafana:latest
FROM ${IMAGE_VERSION}

USER root

# No need to run the server, we want to exit after the upgrade
RUN sed -i '/exec grafana server/,$d' /run.sh

USER grafana
23 changes: 23 additions & 0 deletions misc/upgrade-plugins.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
#!/bin/bash -ex

declare PLATFORM=${PLATFORM:-linux/amd64}
declare DIR
DIR=$(realpath ../panels)

# Note: the PLATFORM matters because some plugins are built for a specific architecture.
docker buildx build --platform="$PLATFORM" --progress=plain -t local/grafana -f Dockerfile.upgrade .

rm -rf "${DIR:?}"/*

docker run \
--rm -t \
--name grafana \
--platform="$PLATFORM" \
-e GF_INSTALL_PLUGINS=grafana-clickhouse-datasource,grafana-polystat-panel,jdbranham-diagram-panel \
-v "${DIR}":/var/lib/grafana/plugins \
local/grafana

echo "Listing upgraded Grafana plugins..."
ls -l "$DIR"

docker rmi local/grafana
107 changes: 106 additions & 1 deletion panels/grafana-clickhouse-datasource/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,108 @@
# Changelog

# 4.10.1

### Fixes

- Bump grafana/plugin-actions from ff169fa386880e34ca85a49414e5a0ff84c3f7ad to b788be6746403ff9bae26d5e800794f2a5620b4c (#1286)
- Bump cspell from 9.0.2 to 9.1.1 (#1278)

# 4.10.0

### Features

- Ad-hoc queries: Allow to filter by values inside the map (#1265)

### Fixes

- Fix ad-hoc filter application with templated target tables (#1241)
- Dependency updates

# 4.9.1

### Fixes

- Error logging fix

# 4.9.0

### Features

- Add support for the Grafana `row_limit` [configuration setting](https://grafana.com/docs/grafana/latest/setup-grafana/configure-grafana/#row_limit).
- Add support for kind, status, instrumentation library, links, events and state data for traces (#1043, #1208)
- Cancel JSON paths query after 10s (#1206)
- SQL Editor now suggests database, table, column, and function names while typing (#1204)
- Add SQL Formatter button + shortcut for making long queries more readable in the editor (#1205)

### Fixes

- Fixed "run query" shortcut from running stale query (#1205)
- Dependency updates

## 4.8.2

### Fixes

- Dependency updates

## 4.8.1

### Fixes

- Dependency updates

## 4.8.0

### Features

- Enable CtrlCmd + Enter keyboard shortcut to Run Query (#1158)

### Fixes

- Refactor `MutateResponse` function and PDC dialler creation (#1155)
- Refactor `clickhouse.Connect` to improve context cancellation handling (#1154)
- Prevent usage of failed connections and improve connection management (#1156). Please note that following this change, the following limits will be set. Although we believe these limits are reasonable, you can adjust them in the datasource settings if needed:
- `MaxOpenConns` to 50.
- `MaxIdleConns` to 25.
- `ConnMaxLifetime` to 5 minutes.
- Dependency updates

## 4.7.0

### Features

- Add JSON column sub-paths to column selection in query builder
- Added events support in trace detail view.(https://github.com/grafana/clickhouse-datasource/pull/1128)

## 4.6.0

### Features

- Add support for new Variant, Dynamic, and JSON types (https://github.com/grafana/clickhouse-datasource/pull/1108)

### Fixes

- Optimized performance for log volumes processing using ClickHouse `multiSearchAny`

## 4.5.1

### Fixes

- Dependency updates

## 4.5.0

### Features

- Implemented log context for log queries
- Added configuration options for log context columns
- Queries parsed from the SQL editor will now attempt to re-map columns into their correct fields for Log and Trace queries.
- Added support for IN operator in adhoc filters

### Fixes

- Fixed and enhanced the logic for parsing a query back into the query builder.

## 4.4.0

### Features
Expand Down Expand Up @@ -105,7 +208,7 @@
- Fixed empty builder options when switching from SQL Editor back to Query Editor
- Fix SQL Generator including "undefined" in `FROM` when database isn't defined
- Allow adding spaces in multi filters (such as `WHERE .. IN`)
- Fixed missing `AND` keyword when adding a filter to a Trace ID query
- Fixed missing `AND` keyword when adding a filter to a Trace ID query

## 4.0.2

Expand Down Expand Up @@ -155,10 +258,12 @@ Version 4.0.0 contains major revisions to the query builder and datasource confi
- Added column hints, which offers better linking across query components when working with columns and filters. For example, a filter can be added for the `Time` column, even without knowing what the time column name is yet. This enables better SQL generation that is "aware" of a column's intended use.

### Plugin Backend

- Added migration logic for `v3` configs going to `v4+`. This is applied when the config is loaded when building a database connection.
- `$__timeFilter`, `$__fromTime`, and `$__toTime` macros now convert to `DateTime64(3)` for better server-side type conversion. Also enables millisecond precision time range filtering.

#### Datasource Configuration

- Added migration script for `v3.x` configurations to `v4+`. This runs automatically when opening/saving the datasource configuration.
- Renamed config value `server` to `host`.
- Renamed config value `timeout` to the more specific `dial_timeout`.
Expand Down
82 changes: 41 additions & 41 deletions panels/grafana-clickhouse-datasource/MANIFEST.txt
Original file line number Diff line number Diff line change
@@ -1,41 +1,41 @@

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

{
"manifestVersion": "2.0.0",
"signatureType": "grafana",
"signedByOrg": "grafana",
"signedByOrgName": "Grafana Labs",
"plugin": "grafana-clickhouse-datasource",
"version": "4.4.0",
"time": 1726245931902,
"keyId": "7e4d0c6a708866e7",
"files": {
"CHANGELOG.md": "580cccb7707725e8c059fa60952bb7ea40d36b3153ea56f8418afb9b24b84a02",
"LICENSE": "cdd15e614b50e88443fe574ad56bde5ba697d958a45376431638eea816e3bfc3",
"README.md": "01c2a56425fa4fb836dcb30b7575b0ed031be483567704025e160497e5002e24",
"dashboards/cluster-analysis.json": "7f83d4d09cc6f045768f5bf47485b864dd6da03094f8607808dec77cb96902e5",
"dashboards/data-analysis.json": "71695f08dfad47f3d4da4c2d331a09eb8fe5789520d5566fffeecd626079caa1",
"dashboards/opentelemetry-clickhouse.json": "7000bb0d91bf0474eb5da966ee65fece8a591785e4a06f8dc7c4431fef35314a",
"dashboards/query-analysis.json": "1b2006a3f4142e512e50156a7d2fd8cf03a178019aaccc964c0283c8559298b3",
"dashboards/system-dashboards.json": "a47eb47b9cd0bea82a7276fc805bba214164e677f68fb1e821b0278b17bee7f0",
"go_plugin_build_manifest": "7980a82c9b5646c237a4e240e3879ba6a836566b6dbe746cb68c907921188732",
"gpx_clickhouse_linux_amd64": "1d34927dde5dfc3ee4e44e7d3060cdb0d4132b8770515d4a038a4cb77e991df3",
"img/logo.svg": "838199055d86584ff105e5e91013203a6acb7d3f2061ae27678786125ab11f09",
"module.js": "5479144fa6aa0d4254153c2e25568cd1bf11af7f91c4cbd308203a59f94e226c",
"module.js.map": "545a409ec85976c4ab66dc479eecf3de821d606eaf6883cda43f932a9a95f024",
"plugin.json": "c04b488966776c58a8deafbabc113acd44dc245035c9eaaf8776102019fca131"
}
}
-----BEGIN PGP SIGNATURE-----
Version: OpenPGP.js v4.10.11
Comment: https://openpgpjs.org

wrkEARMKAAYFAmbkbCwAIQkQfk0ManCIZucWIQTzOyW2kQdOhGNlcPN+TQxq
cIhm55bwAgkAJRo3J8E0fgJhZDy4iWPdgdra/6ErAdqYc3YDTHEFCVWtU2j1
WNGq5SMAvakm59R7aJmCP5I+9zowsEAi93NkKi0CCQCuNfkaaYqDZoP71gSd
RkeAtao81WyrsKRx0SVho38UpQ+B2rEBfCpEtjinpgxuosQOcCaImJPGatnX
bcT6fbtWZg==
=9Dsz
-----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512
{
"manifestVersion": "2.0.0",
"signatureType": "grafana",
"signedByOrg": "grafana",
"signedByOrgName": "Grafana Labs",
"plugin": "grafana-clickhouse-datasource",
"version": "4.10.1",
"time": 1750781693061,
"keyId": "7e4d0c6a708866e7",
"files": {
"dashboards/system-dashboards.json": "a47eb47b9cd0bea82a7276fc805bba214164e677f68fb1e821b0278b17bee7f0",
"dashboards/data-analysis.json": "71695f08dfad47f3d4da4c2d331a09eb8fe5789520d5566fffeecd626079caa1",
"dashboards/query-analysis.json": "1b2006a3f4142e512e50156a7d2fd8cf03a178019aaccc964c0283c8559298b3",
"dashboards/opentelemetry-clickhouse.json": "1a710a750988fdcd7d788e8aef001c1fb106e00253ea0208c2963d9a71d717e8",
"dashboards/cluster-analysis.json": "7f83d4d09cc6f045768f5bf47485b864dd6da03094f8607808dec77cb96902e5",
"go_plugin_build_manifest": "a551ee5b15a305f56c583d1bcc4f6dd6ec00a04ade2b5cb3abd244a3f52908a4",
"plugin.json": "2ca01d993b8db792fae21bc99ce36e83010944847421188d8c663c418ae2d546",
"README.md": "01c2a56425fa4fb836dcb30b7575b0ed031be483567704025e160497e5002e24",
"img/logo.svg": "838199055d86584ff105e5e91013203a6acb7d3f2061ae27678786125ab11f09",
"LICENSE": "cdd15e614b50e88443fe574ad56bde5ba697d958a45376431638eea816e3bfc3",
"module.js.map": "758221726746a2aaa0537ca44f884bfc2c38af8b13fb84a0ffddbff960d1b16f",
"gpx_clickhouse_linux_amd64": "ce9a81fc971a58fa4a96cc8589dab19cc6d81a1637ed86165342b49d7e7b7ddf",
"module.js": "1398a55022469ac6afaeac3e478b64c8cab70ceb314de0fefc105bb12a86a939",
"CHANGELOG.md": "b757adad32dac0b0486e9587929928aa7a071d15cffe49388d88d9e837fb45a3"
}
}
-----BEGIN PGP SIGNATURE-----
Version: OpenPGP.js v4.10.11
Comment: https://openpgpjs.org
wrkEARMKAAYFAmhazv0AIQkQfk0ManCIZucWIQTzOyW2kQdOhGNlcPN+TQxq
cIhm5xq5AgkBuoiMW6qyI+M/lAroVVzYLOSUF6wVa8IDuhKLKwrnKVa2YMYA
l7UKqKDSiOUFJTko9g4+KQEbYmUm8JCn56ddgakCCQBcHqAUE9iWuo4g+pTw
9rrQYoIiGCmXoUPD8TuJ5Yu5SllVmS+x4DcKqPDn+IE9mBEVVCyWxA4Lz2fv
AOqbPc23NA==
=dFga
-----END PGP SIGNATURE-----
Original file line number Diff line number Diff line change
Expand Up @@ -806,7 +806,7 @@
},
"pluginVersion": "4.0.6",
"queryType": "traces",
"rawSql": "WITH '${trace_id}' as trace_id, (SELECT min(Start) FROM \"default\".\"otel_traces_trace_id_ts\" WHERE TraceId = trace_id) as trace_start, (SELECT max(End) + 1 FROM \"default\".\"otel_traces_trace_id_ts\" WHERE TraceId = trace_id) as trace_end SELECT \"TraceId\" as traceID, \"SpanId\" as spanID, \"ParentSpanId\" as parentSpanID, \"ServiceName\" as serviceName, \"SpanName\" as operationName, \"Timestamp\" as startTime, multiply(\"Duration\", 0.000001) as duration, arrayMap(key -> map('key', key, 'value',\"SpanAttributes\"[key]), mapKeys(\"SpanAttributes\")) as tags, arrayMap(key -> map('key', key, 'value',\"ResourceAttributes\"[key]), mapKeys(\"ResourceAttributes\")) as serviceTags FROM \"default\".\"otel_traces\" WHERE traceID = trace_id AND startTime >= trace_start AND startTime <= trace_end AND ( Timestamp >= $__fromTime AND Timestamp <= $__toTime ) AND ( Duration > 0 ) ORDER BY Timestamp DESC, Duration DESC LIMIT 1000",
"rawSql": "WITH '${trace_id}' as trace_id, (SELECT min(Start) FROM \"otel_traces_trace_id_ts\" WHERE TraceId = trace_id) as trace_start, (SELECT max(End) + 1 FROM \"otel_traces_trace_id_ts\" WHERE TraceId = trace_id) as trace_end SELECT \"TraceId\" as traceID, \"SpanId\" as spanID, \"ParentSpanId\" as parentSpanID, \"ServiceName\" as serviceName, \"SpanName\" as operationName, \"Timestamp\" as startTime, multiply(\"Duration\", 0.000001) as duration, arrayMap(key -> map('key', key, 'value',\"SpanAttributes\"[key]), mapKeys(\"SpanAttributes\")) as tags, arrayMap(key -> map('key', key, 'value',\"ResourceAttributes\"[key]), mapKeys(\"ResourceAttributes\")) as serviceTags FROM \"otel_traces\" WHERE traceID = trace_id AND startTime >= trace_start AND startTime <= trace_end AND ( Timestamp >= $__fromTime AND Timestamp <= $__toTime ) AND ( Duration > 0 ) ORDER BY Timestamp DESC, Duration DESC LIMIT 1000",
"refId": "A"
}
],
Expand Down Expand Up @@ -887,7 +887,7 @@
"editorType": "builder",
"format": 2,
"pluginVersion": "4.0.6",
"rawSql": "SELECT Timestamp as timestamp, Body as body, SeverityText as level FROM \"default\".\"otel_logs\" LIMIT 1000",
"rawSql": "SELECT Timestamp as timestamp, Body as body, SeverityText as level FROM \"otel_logs\" LIMIT 1000",
"refId": "A"
}
],
Expand Down
16 changes: 8 additions & 8 deletions panels/grafana-clickhouse-datasource/go_plugin_build_manifest
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
04707ad0a7f03cb6ab28466b82d791e7578c8d11274c4855d226acf11d5ecafb:Magefile.go
633cead7326f37a33d950c6ed57bbc517125664069b6bd7679b56c524218f0a2:pkg/converters/converters.go
7a274598c05dc66b69e58ca594882ece9aa6f33f8f00251599132ce8911dfc0b:pkg/converters/converters.go
a1c6a6171521d93dfc24353416a2bf90f3db3f51301888a18c98f950189f3e67:pkg/converters/converters_test.go
847ff0c9fc6ff85d85c3997e1f48fbf922cae7143c56f2f6b8a42559f2919b5b:pkg/macros/macros.go
3a39042ce625ec33b2003a162d57d5a86a21ae6445e66ffc33d518ac88fe0b51:pkg/macros/macros_test.go
f18be511b4ee3b0decd135232e16a6c8960a387eb36622baf4a160fe8cc601b0:pkg/macros/macros.go
f2df2e47c293763876937dd5ea1775dd00696eecb68d0b87769dc9f3290a6715:pkg/macros/macros_test.go
67ffdde8c54a45348f13695d4c8312f990549fd375665f3506361f14fb68c468:pkg/main.go
ce3bd56371c3918a2f81e9f1fbb7357e3d8b9babc1691897e1912630852f263f:pkg/plugin/datasource.go
465f9c7963d197f6f864fe107c33beef49d50cd41720c093a55805e19402d4a6:pkg/plugin/driver.go
4e91d18deaf83fd5bc3617a185aa2fdecabb5b5427ff47fbd69098ebc24cdbe8:pkg/plugin/driver_test.go
148912315b55ed9f2cdbd688d476d79c9dd23621cb92183674d5aadedfde616d:pkg/plugin/datasource.go
5d9ad68187ce00707cf82a4d01f5264e18d6c8d96f842130a12fad5ed542b25b:pkg/plugin/driver.go
fe25f7ccd93bfcd1d4a49e717d97a4fd5bbcc343673f2411f9b9d5116092fb90:pkg/plugin/driver_test.go
51ee8486dde6fea63e682154446cf3b47376d63b0d60eaadeb5823f792a07ce7:pkg/plugin/errors.go
d0d5ece8062b1d1debeacae64fd68b69fbc9add3dd975f5f84db8ebfa0134613:pkg/plugin/settings.go
699e8c9e73b4293d9b80be7013f336c855400c7b795cfb2d9f6c4c0804427b78:pkg/plugin/settings_test.go
2d5c2b0e5d1f4a0b49133d27f30fd13742128e90d993ce93133689fb1cd12f59:pkg/plugin/settings.go
464da1dc843ce2f11f7f6ced666ca2d06cfa004b69d8f50d5d9dfa3f38d9fe0b:pkg/plugin/settings_test.go
Binary file not shown.
69 changes: 55 additions & 14 deletions panels/grafana-clickhouse-datasource/module.js

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion panels/grafana-clickhouse-datasource/module.js.map

Large diffs are not rendered by default.

Loading
Loading