summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorJerop Kipruto <[email protected]>2025-06-13 10:27:22 -0700
committerGitHub <[email protected]>2025-06-13 10:27:22 -0700
commit1f63f3331f1d19e2157d87e8316828dc1c4c8b6b (patch)
tree7310f3eb29db6a862f46be9afccee083788e8e3a
parentdaceb9963f8962051628127c661297fbd95b4a1d (diff)
Revert "Telemetry: Auto-export to GCP if GOOGLE_CLOUD_PROJECT is set" (#1011)
-rw-r--r--docs/core/telemetry.md238
-rw-r--r--packages/core/src/config/config.ts3
-rw-r--r--packages/core/src/telemetry/sdk.ts73
3 files changed, 88 insertions, 226 deletions
diff --git a/docs/core/telemetry.md b/docs/core/telemetry.md
index 538150cf..2e199465 100644
--- a/docs/core/telemetry.md
+++ b/docs/core/telemetry.md
@@ -6,45 +6,6 @@ This entire system is built on the **[OpenTelemetry] (OTEL)** standard, allowing
[OpenTelemetry]: https://opentelemetry.io/
-## Quick Start: Google Cloud Telemetry
-
-This quick start guide helps you send Gemini CLI telemetry data directly to Google Cloud (Cloud Trace, Cloud Monitoring, Cloud Logging).
-
-**Prerequisites:**
-
-1. **Google Cloud Project:** You need an active Google Cloud Project.
-2. **APIs Enabled:** Ensure the following APIs are enabled in your project:
- - Cloud Trace API
- - Cloud Monitoring API
- - Cloud Logging API
-
-**Steps:**
-
-1. **Set Environment Variable:**
- Set the `GOOGLE_CLOUD_PROJECT` environment variable to your project ID.
-
- ```bash
- export GOOGLE_CLOUD_PROJECT="your-gcp-project-id"
- ```
-
-2. **Configure Gemini CLI Settings:**
- In your workspace (`.gemini/settings.json`) or user (`~/.gemini/settings.json`) settings file, enable telemetry. **Crucially, do not set `telemetryOtlpEndpoint`, or ensure it's an empty string.**
-
- ```json
- {
- "telemetry": true,
- "sandbox": false
- // "telemetryOtlpEndpoint": "" // Leave empty or omit this line
- }
- ```
-
- _Note: Telemetry is not compatible with sandbox mode at this time. Ensure `"sandbox": false`._
-
-3. **Run Gemini CLI:**
- That's it! The Gemini CLI will now automatically detect the `GOOGLE_CLOUD_PROJECT` variable and send telemetry data directly to your Google Cloud project.
-
-For more detailed configuration options, including using a local collector, other OTLP backends, or advanced Google Cloud collector setups, please refer to the sections below.
-
## Enabling Telemetry
You can enable telemetry in multiple ways. [Configuration](configuration.md) is primarily managed via the `.gemini/settings.json` file and environment variables, but CLI flags can override these settings for a specific session.
@@ -64,47 +25,13 @@ Add these lines to enable telemetry by in workspace (`.gemini/settings.json`) or
{
"telemetry": true,
"sandbox": false
- // Optional: Specify a custom OTLP/gRPC endpoint for your collector.
- // If commented out or empty, behavior depends on GOOGLE_CLOUD_PROJECT env var.
- // "telemetryOtlpEndpoint": "http://localhost:4317"
}
```
-The Gemini CLI determines where to send telemetry data based on the following priority:
-
-1. **`telemetryOtlpEndpoint` in Settings**: If `telemetryOtlpEndpoint` is configured in `.gemini/settings.json` (and is a valid OTLP/gRPC endpoint), telemetry data will be sent to this specified endpoint. This is typically used for sending data to a local collector or a specific third-party observability platform.
- _Example for a local collector:_
-
- ```json
- {
- "telemetry": true,
- "sandbox": false,
- "telemetryOtlpEndpoint": "http://localhost:4317"
- }
- ```
+## Running an OTEL Collector
-2. **`GOOGLE_CLOUD_PROJECT` Environment Variable**: If `telemetryOtlpEndpoint` is not set or is empty, the CLI checks for the `GOOGLE_CLOUD_PROJECT` environment variable. If this variable is set, telemetry data (traces, metrics, and logs) will be sent directly to the corresponding Google Cloud services (Cloud Trace, Cloud Monitoring, Cloud Logging) for that project.
- _To enable direct Google Cloud export, ensure `GOOGLE_CLOUD_PROJECT` is set in your environment and `telemetryOtlpEndpoint` is omitted or empty in your settings:_
-
- ```json
- {
- "telemetry": true,
- "sandbox": false
- // "telemetryOtlpEndpoint": "" // or omit the line
- }
- ```
-
-3. **Console Exporter (Default Fallback)**: If neither `telemetryOtlpEndpoint` is configured nor the `GOOGLE_CLOUD_PROJECT` environment variable is set, telemetry data will be exported to the console. This is useful for quick local debugging without setting up a collector or cloud services.
-
-## Running an OTEL Collector (Optional)
-
-While the Gemini CLI can send telemetry directly to Google Cloud (if `GOOGLE_CLOUD_PROJECT` is set) or to the console (as a fallback), you might choose to run an OpenTelemetry (OTEL) Collector in specific scenarios:
-
-- **Local Debugging/Inspection**: To view all telemetry data locally in your terminal.
-- **Custom Processing/Routing**: If you want to receive telemetry data, process it, and then forward it to one or more backends (including Google Cloud or other observability platforms).
-- **Using a Non-GCP Backend**: If you want to send data to a different OTLP-compatible backend that requires a collector.
-
-An OTEL Collector is a service that receives, processes, and exports telemetry data. When a collector is used, the CLI sends data to it using the OTLP/gRPC protocol.
+An OTEL Collector is a service that receives, processes, and exports telemetry data.
+The CLI sends data using the OTLP/gRPC protocol.
Learn more about OTEL exporter standard configuration in [documentation][otel-config-docs].
@@ -125,11 +52,10 @@ mkdir .gemini/otel
### Local
-To use a local collector, you must explicitly set the `telemetryOtlpEndpoint` in your `.gemini/settings.json` file to the collector's address (e.g., `"http://localhost:4317"`). See the "Enabling Telemetry" section for details on setting this value.
+This is the simplest way to inspect events, metrics, and traces without any external tools.
+This setup prints all telemetry from the Gemini CLI to your terminal using a local collector.
-This setup then prints all telemetry from the Gemini CLI to your terminal using that local collector. It's the simplest way to inspect events, metrics, and traces locally without any external tools when you've configured the CLI to send data to it.
-
-**1. Configure `telemetryOtlpEndpoint`**
+**1. Create a Configuration File**
Create the file `.gemini/otel/collector-local.yaml` with the following:
@@ -207,74 +133,35 @@ Use this method if you prefer not to use Docker.
### Google Cloud
-The Gemini CLI can send telemetry data directly to Google Cloud services (Cloud Trace, Cloud Monitoring, Cloud Logging) if the `GOOGLE_CLOUD_PROJECT` environment variable is set and no `telemetryOtlpEndpoint` is configured in your settings. This is the simplest way to integrate with Google Cloud.
-
-**Direct Export to Google Cloud (Recommended for most GCP users):**
-
-1. **Prerequisites**:
- - A Google Cloud Project ID.
- - **APIs Enabled**: Ensure Cloud Trace API, Cloud Monitoring API, and Cloud Logging API are enabled in your Google Cloud project.
- - **Authentication**: The environment where the Gemini CLI runs must be authenticated to Google Cloud with permissions to write traces, metrics, and logs. This is typically handled via Application Default Credentials (e.g., by running `gcloud auth application-default login`) or a service account with the necessary roles (`Cloud Trace Agent`, `Monitoring Metric Writer`, `Logs Writer`).
-2. **Configuration**:
-
- - Set the `GOOGLE_CLOUD_PROJECT` environment variable to your project ID.
- ```bash
- export GOOGLE_CLOUD_PROJECT="YOUR_PROJECT_ID"
- # Optionally, set GOOGLE_CLOUD_LOCATION if needed by your setup
- # export GOOGLE_CLOUD_LOCATION="YOUR_PROJECT_LOCATION" # e.g., us-central1
- ```
- - Ensure `telemetryOtlpEndpoint` is **not set** or is empty in your `.gemini/settings.json` file.
- ```json
- {
- "telemetry": true,
- "sandbox": false
- // "telemetryOtlpEndpoint": "" // Ensure this is empty or commented out
- }
- ```
-
-3. **Run Gemini CLI**: With these settings, the CLI will automatically detect `GOOGLE_CLOUD_PROJECT` and send telemetry directly to Google Cloud. No separate collector is needed for this direct export.
-
-**Using an OTEL Collector with Google Cloud (Advanced/Custom Scenarios):**
-
-You might choose to use an OTEL Collector if you want to:
+This setup sends all telemetry to Google Cloud for robust, long-term analysis.
-- Perform custom processing, batching, or filtering of telemetry data before sending it to Google Cloud.
-- Aggregate telemetry from multiple sources before exporting.
-- Send data to Google Cloud from an environment where the CLI cannot directly authenticate or reach Google Cloud endpoints, but the collector can.
+**1. Prerequisites**
-If you opt for this route, the setup involves running an OTEL Collector configured to export data to Google Cloud. The Gemini CLI would then be configured to send its telemetry to this collector's endpoint.
+- A Google Cloud Project ID.
+- **APIs Enabled**: Cloud Trace, Cloud Monitoring, Cloud Logging.
+- **Authentication**: A Service Account with the roles `Cloud Trace Agent`, `Monitoring Metric Writer`, and `Logs Writer`. Ensure your environment is authenticated (e.g., via `gcloud auth application-default login` or a service account key file).
-**1. Prerequisites (for Collector Setup)**
+**2. Set environment variables**
-- All prerequisites for direct export (Project ID, APIs enabled, Authentication for the _collector_).
-- An OTEL Collector setup (e.g., `otelcol-contrib` binary or Docker).
+Set the `GOOGLE_CLOUD_PROJECT`, `GOOGLE_CLOUD_LOCATION`, and `GOOGLE_GENAI_USE_VERTEXAI` environment variables:
-**2. Configure Gemini CLI to Send to Your Collector**
-Update your `.gemini/settings.json` to point `telemetryOtlpEndpoint` to your collector's listening address (e.g., `http://localhost:4317` if the collector is local).
-
-```json
-{
- "telemetry": true,
- "sandbox": false,
- "telemetryOtlpEndpoint": "http://localhost:4317" // Or your collector's address
-}
+```bash
+GOOGLE_CLOUD_PROJECT="YOUR_PROJECT_ID"
+GOOGLE_CLOUD_LOCATION="YOUR_PROJECT_LOCATION" # e.g., us-central1
+GOOGLE_GENAI_USE_VERTEXAI=true
```
-**3. Create a Collector Configuration File**
-Create `.gemini/otel/collector-gcp.yaml` for your collector. This file tells the collector to receive data (e.g., on `0.0.0.0:4317`) and export it to Google Cloud.
-_(The existing `collector-gcp.yaml` content provided in the document can be used here, it correctly defines an OTLP receiver and a Google Cloud exporter.)_
-Ensure the `project` field within the `googlecloud` exporter configuration in this YAML is correctly set, typically by referencing the `GOOGLE_CLOUD_PROJECT` environment variable available to the collector.
+**3. Create a Configuration File**
-```bash
-# Ensure GOOGLE_CLOUD_PROJECT is set in the environment where the collector runs
-# export GOOGLE_CLOUD_PROJECT="YOUR_PROJECT_ID"
+Create `.gemini/otel/collector-gcp.yaml`:
+```bash
cat <<EOF > .gemini/otel/collector-gcp.yaml
receivers:
otlp:
protocols:
grpc:
- endpoint: "0.0.0.0:4317" # Collector listens on this endpoint
+ endpoint: "0.0.0.0:4317"
processors:
batch:
@@ -282,12 +169,12 @@ processors:
exporters:
googlecloud:
- project: "${GOOGLE_CLOUD_PROJECT}" # Collector uses this to export to GCP
+ project: "${GOOGLE_CLOUD_PROJECT}"
metric:
prefix: "custom.googleapis.com/gemini_cli"
log:
default_log_name: "gemini_cli"
- debug: # Optional: for debugging the collector itself
+ debug:
verbosity: detailed
service:
@@ -300,49 +187,66 @@ service:
exporters: [googlecloud]
logs:
receivers: [otlp]
- exporters: [googlecloud, debug] # Sending to GCP and console for debug
+ exporters: [googlecloud]
EOF
```
**4. Run the Collector**
-You can run the collector for Google Cloud using either Docker or a locally installed `otelcol-contrib` binary, ensuring it has access to Google Cloud credentials and the `GOOGLE_CLOUD_PROJECT` environment variable.
-_(The existing Docker and `otelcol-contrib` run commands provided in the document can be used here. Ensure the collector's environment has `GOOGLE_APPLICATION_CREDENTIALS` set if using a service account key, or that it can pick up ADC.)_
-**_Option 1: Use Docker_**
-(Ensure `GOOGLE_CLOUD_PROJECT` is available to the Docker container if your `collector-gcp.yaml` relies on it for the `project` field, or hardcode it in the YAML.)
+You can run the collector for Google Cloud using either Docker or a locally installed `otelcol` binary.
+
+**_Option 1: Use Docker _**
+
+This method encapsulates the collector and its dependencies within a container.
+
+1. **Run the Collector**:
+ Choose the command that matches your authentication method.
+
+ - **If using Application Default Credentials (`gcloud auth application-default login`)**:
-- **If using Application Default Credentials (`gcloud auth application-default login`)**:
+ ```bash
+ docker run --rm --name otel-collector-gcp \
+ -p 4317:4317 \
+ --user "$(id -u):$(id -g)" \
+ -v "$HOME/.config/gcloud/application_default_credentials.json":/etc/gcp/credentials.json:ro \
+ -e "GOOGLE_APPLICATION_CREDENTIALS=/etc/gcp/credentials.json" \
+ -v "$(pwd)/.gemini/otel/collector-gcp.yaml":/etc/otelcol-contrib/config.yaml \
+ otel/opentelemetry-collector-contrib:latest --config /etc/otelcol-contrib/config.yaml
+ ```
- ```bash
- docker run --rm --name otel-collector-gcp \
- -p 4317:4317 \
- --user "$(id -u):$(id -g)" \
- -v "$HOME/.config/gcloud/application_default_credentials.json":/etc/gcp/credentials.json:ro \
- -e "GOOGLE_APPLICATION_CREDENTIALS=/etc/gcp/credentials.json" \
- -e "GOOGLE_CLOUD_PROJECT=${GOOGLE_CLOUD_PROJECT}" \ # Pass the env var
- -v "$(pwd)/.gemini/otel/collector-gcp.yaml":/etc/otelcol-contrib/config.yaml \
- otel/opentelemetry-collector-contrib:latest --config /etc/otelcol-contrib/config.yaml
- ```
+ - **If using a Service Account Key File**:
+ ```bash
+ docker run --rm --name otel-collector-gcp \
+ -p 4317:4317 \
+ -v "/path/to/your/sa-key.json":/etc/gcp/sa-key.json:ro \
+ -e "GOOGLE_APPLICATION_CREDENTIALS=/etc/gcp/sa-key.json" \
+ -v "$(pwd)/.gemini/otel/collector-gcp.yaml":/etc/otelcol-contrib/config.yaml \
+ otel/opentelemetry-collector-contrib:latest --config /etc/otelcol-contrib/config.yaml
+ ```
-- **If using a Service Account Key File**:
- ```bash
- docker run --rm --name otel-collector-gcp \
- -p 4317:4317 \
- -v "/path/to/your/sa-key.json":/etc/gcp/sa-key.json:ro \
- -e "GOOGLE_APPLICATION_CREDENTIALS=/etc/gcp/sa-key.json" \
- -e "GOOGLE_CLOUD_PROJECT=${GOOGLE_CLOUD_PROJECT}" \ # Pass the env var
- -v "$(pwd)/.gemini/otel/collector-gcp.yaml":/etc/otelcol-contrib/config.yaml \
- otel/opentelemetry-collector-contrib:latest --config /etc/otelcol-contrib/config.yaml
- ```
+2. **Check Status**:
+ Your telemetry data will now appear in Google Cloud Trace, Monitoring, and Logging.
+
+3. **Stop the Collector**:
+ ```bash
+ docker stop otel-collector-gcp
+ ```
**_Option 2: Use `otelcol-contrib`_**
-(Ensure `GOOGLE_CLOUD_PROJECT` is set in the shell where you run this command.)
-```bash
-./otelcol-contrib --config="file:$(pwd)/.gemini/otel/collector-gcp.yaml"
-```
+Use this method if you prefer not to use Docker.
+
+1. **Run the Collector**:
+
+ ```bash
+ ./otelcol-contrib --config="file:$(pwd)/.gemini/otel/collector-gcp.yaml"
+ ```
-With this collector setup, the Gemini CLI sends data to your collector, and the collector then forwards it to Google Cloud.
+2. **Check Status**:
+ Your telemetry data will now appear in Google Cloud Trace, Monitoring, and Logging.
+
+3. **Stop the Collector**:
+ Press `Ctrl+C` in the terminal where the collector is running.
---
diff --git a/packages/core/src/config/config.ts b/packages/core/src/config/config.ts
index 4962d2a7..48408201 100644
--- a/packages/core/src/config/config.ts
+++ b/packages/core/src/config/config.ts
@@ -143,7 +143,8 @@ export class Config {
this.telemetry = params.telemetry ?? false;
this.telemetryLogUserPromptsEnabled =
params.telemetryLogUserPromptsEnabled ?? true;
- this.telemetryOtlpEndpoint = params.telemetryOtlpEndpoint ?? '';
+ this.telemetryOtlpEndpoint =
+ params.telemetryOtlpEndpoint ?? 'http://localhost:4317';
this.fileFilteringRespectGitIgnore =
params.fileFilteringRespectGitIgnore ?? true;
this.checkpoint = params.checkpoint ?? false;
diff --git a/packages/core/src/telemetry/sdk.ts b/packages/core/src/telemetry/sdk.ts
index e1750a86..61f501a6 100644
--- a/packages/core/src/telemetry/sdk.ts
+++ b/packages/core/src/telemetry/sdk.ts
@@ -71,77 +71,34 @@ export function initializeTelemetry(config: Config): void {
'session.id': config.getSessionId(),
});
- const otlpEndpointSetting = config.getTelemetryOtlpEndpoint();
- const gcpProjectId = process.env.GOOGLE_CLOUD_PROJECT;
+ const otlpEndpoint = config.getTelemetryOtlpEndpoint();
+ const grpcParsedEndpoint = parseGrpcEndpoint(otlpEndpoint);
+ const useOtlp = !!grpcParsedEndpoint;
- let spanExporter;
- let logExporter;
- let metricReader;
-
- if (otlpEndpointSetting && otlpEndpointSetting.trim() !== '') {
- const grpcParsedEndpoint = parseGrpcEndpoint(otlpEndpointSetting);
- if (grpcParsedEndpoint) {
- diag.info(`Using user-configured OTLP endpoint: ${grpcParsedEndpoint}`);
- spanExporter = new OTLPTraceExporter({
+ const spanExporter = useOtlp
+ ? new OTLPTraceExporter({
url: grpcParsedEndpoint,
compression: CompressionAlgorithm.GZIP,
- });
- logExporter = new OTLPLogExporter({
+ })
+ : new ConsoleSpanExporter();
+ const logExporter = useOtlp
+ ? new OTLPLogExporter({
url: grpcParsedEndpoint,
compression: CompressionAlgorithm.GZIP,
- });
- metricReader = new PeriodicExportingMetricReader({
+ })
+ : new ConsoleLogRecordExporter();
+ const metricReader = useOtlp
+ ? new PeriodicExportingMetricReader({
exporter: new OTLPMetricExporter({
url: grpcParsedEndpoint,
compression: CompressionAlgorithm.GZIP,
}),
exportIntervalMillis: 10000,
- });
- } else {
- diag.warn(
- `Invalid user-configured OTLP endpoint: "${otlpEndpointSetting}". Falling back to console exporter.`,
- );
- spanExporter = new ConsoleSpanExporter();
- logExporter = new ConsoleLogRecordExporter();
- metricReader = new PeriodicExportingMetricReader({
+ })
+ : new PeriodicExportingMetricReader({
exporter: new ConsoleMetricExporter(),
exportIntervalMillis: 10000,
});
- }
- } else if (gcpProjectId) {
- diag.info(
- `No OTLP endpoint configured, GOOGLE_CLOUD_PROJECT detected (${gcpProjectId}). Exporting telemetry to Google Cloud.`,
- );
- const gcpTraceUrl = 'https://trace.googleapis.com:443';
- const gcpMetricUrl = 'https://monitoring.googleapis.com:443';
- const gcpLogUrl = 'https://logging.googleapis.com:443';
-
- spanExporter = new OTLPTraceExporter({
- url: gcpTraceUrl,
- compression: CompressionAlgorithm.GZIP,
- });
- logExporter = new OTLPLogExporter({
- url: gcpLogUrl,
- compression: CompressionAlgorithm.GZIP,
- });
- metricReader = new PeriodicExportingMetricReader({
- exporter: new OTLPMetricExporter({
- url: gcpMetricUrl,
- compression: CompressionAlgorithm.GZIP,
- }),
- exportIntervalMillis: 10000,
- });
- } else {
- diag.info(
- 'No OTLP endpoint or GOOGLE_CLOUD_PROJECT detected. Using console exporters.',
- );
- spanExporter = new ConsoleSpanExporter();
- logExporter = new ConsoleLogRecordExporter();
- metricReader = new PeriodicExportingMetricReader({
- exporter: new ConsoleMetricExporter(),
- exportIntervalMillis: 10000,
- });
- }
sdk = new NodeSDK({
resource,