OpenTelemetry Collector

In this step, we will redirect the traces we are collecting to an OpenTelemetry Collector. You have 2 different options to connect between the OpenTelemetry SDK and the collector: HTTP or GRPC.

For this workshop, we will use GRPC but the process is mostly the same for HTTP with the main difference being the OpenTelemetry library to import and the protocol and port for the exporter endpoint. The learn more about the HTTP method, go here.

Configure the Services

1. Add the following dependency to the package.json file for each service (/web and /users)

   "dependencies": {
       "@opentelemetry/exporter-trace-otlp-grpc": "^0.46.0",
       ...
   }
   

2. In the opentelemetry/src/instrumentation.js file, add the follow at the top to import the GRPC library:

   const {
     OTLPTraceExporter,
   } = require("@opentelemetry/exporter-trace-otlp-grpc");
   

3. In the same file, replace this line:

   traceExporter: new opentelemetry.tracing.ConsoleSpanExporter(),
   

   with this one:

   traceExporter: new OTLPTraceExporter({url: 'http://otel-collector:4317'}),
   

Now you have configured your services to export trace data to the collector over GRPC. Let’s setup the collector now.

Configure the Collector

1. Create a folder named conf in the /opentelemetry directory

2. Add a file for your collector configuration named config.yaml in the new /opentelemetry/conf directory

3. Add the following content to the config.yaml file:

   receivers:
     # Receiver to collect trace, metrics and logs from grpc and http protocols
     # Default is port 4317 for grpc and 4318 for http
     otlp:
       protocols:
         grpc:
         http:
   
   processors:
     # Accepts spans, metrics, or logs and places them into batches
     # to better compress the data and reduce the number of outgoing connections
     batch:
   
   exporters:
     # Exports data to the console
     debug:
       verbosity: detailed
   
   service:
     pipelines:
       traces:
         receivers: [otlp]
         processors: [batch]
         exporters: [debug]
   

4. Edit the docker-compose.yml file in the root directory. Add the following code to add an OpenTelemetry Collector container:

   otel-collector:
     image: otel/opentelemetry-collector-contrib:0.92.0
     container_name: otel-collector
     ports:
       # This is default port for listening to GRPC protocol
       - 4317:4317
       # This is default port for listening to HTTP protocol
       - 4318:4318
     volumes:
       # Path to the collector config file
       - ./opentelemetry/conf:/etc/otelcol-contrib/
   

Now we are ready to test. If your containers are already running, press Ctrl+C in the terminal and wait for them to gracefully stop. Rebuild and restart your containers by running the following command.

docker-compose up --build

Then make requests to http://localhost:4000 and http://localhost:4000/api/data in your browser. You should see output in the console similar to this:

otel-collector  | Span #0
otel-collector  |     Trace ID       : fd59153ee2f72d038ed1b522bd3fb1a2
otel-collector  |     Parent ID      : fc76be759e16a619
otel-collector  |     ID             : 3586e00ce56154db
otel-collector  |     Name           : middleware - query
otel-collector  |     Kind           : SPAN_KIND_INTERNAL
otel-collector  |     Start time     : 2022-10-20 22:47:50.046786816 +0000 UTC
otel-collector  |     End time       : 2022-10-20 22:47:50.047485184 +0000 UTC
otel-collector  |     Status code    : STATUS_CODE_UNSET
otel-collector  |     Status message :
otel-collector  | Attributes:
otel-collector  |      -> http.route: STRING(/)
otel-collector  |      -> express.name: STRING(query)
otel-collector  |      -> express.type: STRING(middleware)

Next we will update the collector to send data to Cloud Observability.