Grapl has several tools to aid in debugging issues.
Grapl Debugger Container
Grapl Debugger Container¶
This is a container that can attach to running containers and includes debug tools inside. See the debugger docs for details.
We currently have tracing enabled for local grapl. Our current backend is Jaeger
In your browser go to
http://localhost:16686. You should see the Jaeger front-end.
On the left side, are search options.
Select the service you’re interested in and click search. You can also use any of the additional filters on the left (such as http code, etc). If your service does not appear, it’s possible that a) it doesn’t have any traffic (ie the web ui needs a web request), b) there are no traces from within the Lookback window if
In the center a list of traces will appear.
Click on one. You will go to a page with detailed trace information, including performance data for each span.
Click on a span, and then click on tags to get more detailed information, such as http code, etc
On the top-right, there is a drop-down menu with Trace Timeline selected. Clicking on it will provide a few additional options
To have pulumi send traces to Jaeger run
WITH_PULUMI_TRACING=1 make $MAKECOMMND where $MAKECOMMAND can be
Tracing docker buildx bake¶
docker buildx supports sending traces to a backend. Since we build prior to
running Jaeger, you will need to explicitly set Jaeger up, either via running
make up first or by running it manually in either docker or as a standalone
This tracing is meant to help debug docker build issues including performance issues.
This generates a LOT of traces, enough to potentially crash Jaeger.
This slows down the build process significantly :(
To run tracing for the docker build:
Do a one-time setup
make setup-docker-tracingif you haven’t already.
WITH_TRACING=1 make build-local-infrastructureor any other build command that uses bake such as
build-test-e2e. Alternatively, you can run traces for individual services via
docker buildx bake --file $FILE --builder=grapl-tracing-builder