Update to v0.2 (#2)

* update go version and dependencies

* bump version

* get full data for threshold based policies without aggregation to avoid internal server error

* Specify supported condition types in README

* fix typo

* update deps

* update dependencies

* clarify execution period in calculation comments

* move link in README

* update deps

* update Go version and GitHub action

* update deps

* update actions

* update go version

* update default duration to 12h

* update readme

* update deps

* add summary flag and update readme

* update goreleaser

* update readme

Author: hanneshayashi

.github/workflows/release.yml

@@ -9,21 +9,21 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Checkout
-        uses: actions/checkout@v4.1.7
+        uses: actions/checkout@v4.2.2
       - name: Unshallow
         run: git fetch --prune --unshallow
       - name: Set up Go
-        uses: actions/setup-go@v5.0.2
+        uses: actions/setup-go@v5.1.0
         with:
-          go-version: 1.23
+          go-version: 1.23.3
       - name: Import GPG key
         id: import_gpg
-        uses: crazy-max/ghaction-import-gpg@v6.1.0
+        uses: crazy-max/ghaction-import-gpg@v6.2.0
         with:
           gpg_private_key: ${{ secrets.GPG_PRIVATE_KEY }}
           passphrase: ${{ secrets.PASSPHRASE }}
       - name: Run GoReleaser
-        uses: goreleaser/goreleaser-action@v6.0.0
+        uses: goreleaser/goreleaser-action@v6.1.0
         with:
           version: latest
           args: release --clean

README.md

@@ -1,7 +1,7 @@
 # IMPORTANT NOTE
 
-Google has postponed the launch to April 2026 and has implemented a preview function of the estimated cost when editing an alerting policy. This estimate is considered much more accurate because `appe` can currently only use the total number of time series in the estimation period (30 days). In reality, this may fluctuate causing `appe` to produce an over-estimation.
-As of this time, Google has not made the underlying data for their calculation available via any metric or API. If this changes in the future, `appe` will be updated to produce more accurate results.
+Google has postponed the launch to April 2026 and has implemented a preview function of the estimated cost when editing an alerting policy. This estimate is uses a very similar calculation to `appe` but only looks at the last few minutes / hours (the exact value is not known and seems to vary slightly). This will generally result in lower estimates than `appe` but you can change the length of the window `appe` goes back using the `-d` flag (eg. `-d 5m` or `-d 4h`) to get the same or similar results. In order to more closely match the results of the preview functionality, `appe`'s default has been changed to `12h` for now. This will likely change again if or when Google updates the built-in preview functionality again.
+Please note that both the preview function and `appe`'s default are not 100% accurate.
 
 # appe - Alerting Policy Price Estimator
 
@@ -36,8 +36,6 @@ We recommend that you assign the following two roles for full compatibility:
 ## Installation
 The easiest way to get and install `appe` is to download one of the pre-compiled binaries from the [releases](https://github.com/doitintl/gcp-tool-appe/releases). `appe` is a self-contained binary without any dependencies and can be run from anywhere. You do not need to download any runtime and there is no need for an installer.
 
-See also https://cloud.google.com/docs/authentication/provide-credentials-adc#local-dev.
-
 ### macOS
 Since the binary is not signed with an Apple Developer Certificate, your Mac will likely report it as untrustworthy.
 There are two ways to deal with this:
@@ -62,6 +60,8 @@ If you are running `appe` locally, the easiest way to set up ADC is to use [gclo
 gcloud auth application-default login
 ```
 
+See also https://cloud.google.com/docs/authentication/provide-credentials-adc#local-dev.
+
 ## Usage
 Using `appe` is fairly straightforward
 
@@ -107,10 +107,16 @@ You can also specify multiple organizations:
 ```
 Note that you will need to specify the `--recursive` or `-r` flag to also scan subfolders.
 
+### Supported Condition Types
+The following condition types are supported by `appe`:
+- Monitoring Query Language (MQL) via [projects.timeSeries.query](https://cloud.google.com/monitoring/api/ref_v3/rest/v3/projects.timeSeries/query)
+- Threshold and Absence via [projects.timeSeries.list](https://cloud.google.com/monitoring/api/ref_v3/rest/v3/projects.timeSeries/list)
+- Prometheus Query Language (PromQL / PQL) via [projects.location.prometheus.api.v1.query_range](https://cloud.google.com/monitoring/api/ref_v3/rest/v1/projects.location.prometheus.api.v1/query_range)
+
 ### All Flags
 ```
   -c, --csvOut string           Path to a CSV file to redirect output to. If this is not set, human-readable output will be given on stdout.
-  -d, --duration duration       The delta from now to go back in time for query. Default is 30 days. (default 720h0m0s)
+  -d, --duration duration       The delta from now to go back in time for query. Default is 12 hours. (default 12h0m0s)
   -e, --excludeFolder strings   One or more folders to exclude. Separated by  ",".
   -f, --folder strings          One or more folders to scan. Use the "-r" flag to scan recursively. Separated by ",".
   -h, --help                    help for appe
@@ -120,6 +126,7 @@ Note that you will need to specify the `--recursive` or `-r` flag to also scan s
   -p, --project strings         One or more projects to scan. Separated by ",".
   -q, --quotaProject string     A quota or billing project. Useful if you don't have the serviceusage.services.use permission in the target project.
   -r, --recursive               If parent should be scanned recursively. If this is not set, only projects at the root of the folder or organization will be scanned. (default false)
+  -s, --summary                 Whether the output should just be a summary (sum of all scanned policies) (default false)
   -t, --testPermissions         If the application should verify that the user has the necessary permissions before processing a project. (default false)
       --threads int             Number of threads to use to process folders, projects and policies in parallel. (default 4)
   -v, --version                 version for appe

cmd/monitoring.go

@@ -158,7 +158,7 @@ func processAlertPolicy(
 					policyOut.Error = err.Error()
 					break
 				}
-				// 60 (seconds) * 60 (minutes) * 24 (hours) * 30 (days) / 30 (step) * 0.35 (price) / 1000000 (per 1M) = 0.03024 (price per time series)
+				// 60 (seconds) * 60 (minutes) * 24 (hours) * 30 (days) / 30 (execution period) * 0.35 (price) / 1000000 (per 1M) = 0.03024 (price per time series)
 				policyOut.Price += 0.03024
 				policyOut.TimeSeries++
 			}
@@ -188,7 +188,7 @@ func processAlertPolicy(
 			}
 			// 60 (seconds) * 60 (minutes) * 24 (hours) * 30 (days) = 2592000
 			// 2592000 * 0.35 (price) / 1000000 (per 1M) =
-			// 0.9072 / step * time series = price for all time series with this condition
+			// 0.9072 / execution period * time series = price for all time series with this condition
 			policyOut.Price += 0.9072 / float64(seconds) * float64(len(pqlResp.Data.Result))
 			policyOut.TimeSeries += len(pqlResp.Data.Result)
 		}
@@ -216,7 +216,7 @@ func processAlertPolicy(
 			if len(aggregations) > 1 {
 				tsReq.SecondaryAggregation = aggregations[1]
 			}
-			if tsReq.Aggregation.GetCrossSeriesReducer().String() == "REDUCE_COUNT_FALSE" || tsReq.SecondaryAggregation.GetCrossSeriesReducer().String() == "REDUCE_COUNT_FALSE" {
+			if tsReq.Aggregation == nil || tsReq.Aggregation.GetCrossSeriesReducer().String() == "REDUCE_COUNT_FALSE" || tsReq.SecondaryAggregation.GetCrossSeriesReducer().String() == "REDUCE_COUNT_FALSE" {
 				tsReq.View = monitoringpb.ListTimeSeriesRequest_FULL
 			}
 			tsIt := metricClient.ListTimeSeries(ctx, tsReq)
@@ -229,7 +229,7 @@ func processAlertPolicy(
 					policyOut.Error = err.Error()
 					break
 				}
-				// 60 (seconds) * 60 (minutes) * 24 (hours) * 30 (days) / 30 (step) * 0.35 (price) / 1000000 (per 1M) = 0.03024 (price per time series)
+				// 60 (seconds) * 60 (minutes) * 24 (hours) * 30 (days) / 30 (execution period) * 0.35 (price) / 1000000 (per 1M) = 0.03024 (price per time series)
 				policyOut.Price += 0.03024
 				policyOut.TimeSeries++
 			}

cmd/root.go

@@ -22,7 +22,7 @@ import (
 
 // rootCmd represents the base command when called without any subcommands
 var rootCmd = &cobra.Command{
-	Version: "0.1",
+	Version: "0.2",
 	Use:     "appe",
 	Short:   "Alerting Policy Price Estimator",
 	Long:    `Scans for alerting policies in the specified projects, folder or orgs and approximates their cost by executing the queries defined in them against the monitoring API`,
@@ -90,6 +90,10 @@ func Execute() {
 	if err != nil {
 		log.Fatalln(err)
 	}
+	summary, err := rootCmd.Flags().GetBool("summary")
+	if err != nil {
+		log.Fatalln(err)
+	}
 	quotaProject, err := rootCmd.Flags().GetString("quotaProject")
 	if err != nil {
 		log.Fatalln(err)
@@ -182,7 +186,7 @@ func Execute() {
 
 	// If one or more individual policies should be analyzed, we need to first get them from the API.
 	// We then put them directly on the policiesIn channel, which will be processes by threads that are spawned below.
-	// Finally, we will close teh projectsIn channel once done, because the policiesIn channel will be closed automatically.
+	// Finally, we will close the projectsIn channel once done, because the policiesIn channel will be closed automatically.
 	if lenPol > 0 {
 		if lenPol > int(threads) {
 			threads = int64(lenPol)
@@ -274,6 +278,18 @@ func Execute() {
 			csvWriter.Flush()
 		}
 		// Otherwise, the application will just output to stdout
+	} else if summary {
+		policiesSum := 0
+		conditionsSum := 0
+		timeSeriesSum := 0
+		priceSum := 0.0
+		for policy := range policiesOut {
+			policiesSum++
+			conditionsSum += policy.Conditions
+			timeSeriesSum += policy.TimeSeries
+			priceSum += policy.Price
+		}
+		log.Printf("Summary: You have %d policies with a combined total of %d conditions and %d time series. It will cost approximately $%f\n", policiesSum, conditionsSum, timeSeriesSum, priceSum)
 	} else {
 		for policy := range policiesOut {
 			log.Printf("Alerting Policy %s (%s) has %d condition(s) and %d time series. It will cost approximately $%f\n", policy.DisplayName, policy.Name, policy.Conditions, policy.TimeSeries, policy.Price)
@@ -291,13 +307,15 @@ func init() {
 	rootCmd.Flags().StringSliceP("excludeFolder", "e", nil, "One or more folders to exclude. Separated by  \",\".")
 	rootCmd.Flags().BoolP("testPermissions", "t", false, "If the application should verify that the user has the necessary permissions before processing a project. (default false)")
 	rootCmd.Flags().BoolP("includeDisabled", "i", false, "If the application should also include disabled policies. (default false)")
+	rootCmd.Flags().BoolP("summary", "s", false, "Whether the output should just be a summary (sum of all scanned policies) (default false)")
 	rootCmd.Flags().BoolP("recursive", "r", false, "If parent should be scanned recursively. If this is not set, only projects at the root of the folder or organization will be scanned. (default false)")
 	rootCmd.Flags().Int64("threads", 4, "Number of threads to use to process folders, projects and policies in parallel.")
-	rootCmd.Flags().DurationP("duration", "d", 24*time.Hour*30, "The delta from now to go back in time for query. Default is 30 days.")
+	rootCmd.Flags().DurationP("duration", "d", 12*time.Hour, "The delta from now to go back in time for query. Default is 12 hours.")
 	rootCmd.MarkFlagsOneRequired("policy", "project", "folder", "organization")
 	rootCmd.MarkFlagsMutuallyExclusive("policy", "project", "recursive")
 	rootCmd.MarkFlagsMutuallyExclusive("policy", "testPermissions")
 	rootCmd.MarkFlagsMutuallyExclusive("policy", "includeDisabled")
 	rootCmd.MarkFlagsMutuallyExclusive("policy", "project", "excludeFolder")
 	rootCmd.MarkFlagsMutuallyExclusive("policy", "project", "folder", "organization")
+	rootCmd.MarkFlagsMutuallyExclusive("csvOut", "summary")
 }

go.mod

@@ -3,47 +3,47 @@ module github.com/doitintl/gcp-tool-appe
 go 1.23
 
 require (
-	cloud.google.com/go/iam v1.2.0
-	cloud.google.com/go/monitoring v1.21.0
-	cloud.google.com/go/resourcemanager v1.10.0
+	cloud.google.com/go/iam v1.2.2
+	cloud.google.com/go/monitoring v1.21.2
+	cloud.google.com/go/resourcemanager v1.10.2
 	github.com/spf13/cobra v1.8.1
-	google.golang.org/api v0.194.0
-	google.golang.org/protobuf v1.34.2
+	google.golang.org/api v0.209.0
+	google.golang.org/protobuf v1.35.2
 )
 
 require (
-	cloud.google.com/go v0.115.1 // indirect
-	cloud.google.com/go/auth v0.9.1 // indirect
-	cloud.google.com/go/auth/oauth2adapt v0.2.4 // indirect
-	cloud.google.com/go/compute/metadata v0.5.0 // indirect
-	cloud.google.com/go/longrunning v0.6.0 // indirect
+	cloud.google.com/go v0.116.0 // indirect
+	cloud.google.com/go/auth v0.11.0 // indirect
+	cloud.google.com/go/auth/oauth2adapt v0.2.6 // indirect
+	cloud.google.com/go/compute/metadata v0.5.2 // indirect
+	cloud.google.com/go/longrunning v0.6.3 // indirect
 	github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
 	github.com/felixge/httpsnoop v1.0.4 // indirect
 	github.com/go-logr/logr v1.4.2 // indirect
 	github.com/go-logr/stdr v1.2.2 // indirect
 	github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
 	github.com/google/s2a-go v0.1.8 // indirect
 	github.com/google/uuid v1.6.0 // indirect
-	github.com/googleapis/enterprise-certificate-proxy v0.3.2 // indirect
-	github.com/googleapis/gax-go/v2 v2.13.0 // indirect
+	github.com/googleapis/enterprise-certificate-proxy v0.3.4 // indirect
+	github.com/googleapis/gax-go/v2 v2.14.0 // indirect
 	github.com/inconshreveable/mousetrap v1.1.0 // indirect
 	github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
 	github.com/spf13/pflag v1.0.5 // indirect
 	go.opencensus.io v0.24.0 // indirect
-	go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.53.0 // indirect
-	go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.53.0 // indirect
-	go.opentelemetry.io/otel v1.28.0 // indirect
-	go.opentelemetry.io/otel/metric v1.28.0 // indirect
-	go.opentelemetry.io/otel/trace v1.28.0 // indirect
-	golang.org/x/crypto v0.26.0 // indirect
-	golang.org/x/net v0.28.0 // indirect
-	golang.org/x/oauth2 v0.22.0 // indirect
-	golang.org/x/sync v0.8.0 // indirect
-	golang.org/x/sys v0.24.0 // indirect
-	golang.org/x/text v0.17.0 // indirect
-	golang.org/x/time v0.6.0 // indirect
-	google.golang.org/genproto v0.0.0-20240822170219-fc7c04adadcd // indirect
-	google.golang.org/genproto/googleapis/api v0.0.0-20240822170219-fc7c04adadcd // indirect
-	google.golang.org/genproto/googleapis/rpc v0.0.0-20240822170219-fc7c04adadcd // indirect
-	google.golang.org/grpc v1.65.0 // indirect
+	go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.57.0 // indirect
+	go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.57.0 // indirect
+	go.opentelemetry.io/otel v1.32.0 // indirect
+	go.opentelemetry.io/otel/metric v1.32.0 // indirect
+	go.opentelemetry.io/otel/trace v1.32.0 // indirect
+	golang.org/x/crypto v0.29.0 // indirect
+	golang.org/x/net v0.31.0 // indirect
+	golang.org/x/oauth2 v0.24.0 // indirect
+	golang.org/x/sync v0.9.0 // indirect
+	golang.org/x/sys v0.27.0 // indirect
+	golang.org/x/text v0.20.0 // indirect
+	golang.org/x/time v0.8.0 // indirect
+	google.golang.org/genproto v0.0.0-20241118233622-e639e219e697 // indirect
+	google.golang.org/genproto/googleapis/api v0.0.0-20241118233622-e639e219e697 // indirect
+	google.golang.org/genproto/googleapis/rpc v0.0.0-20241118233622-e639e219e697 // indirect
+	google.golang.org/grpc v1.68.0 // indirect
 )