Services¶
Network Diagram¶
Grapl Services - main pipeline¶
Unless otherwise specified, the input to a service is the output of the one described above it.
Graph Generator¶
Input: Event logs (e.g. sysmon logs, osquery logs, Cloudtrail logs) from the customer’s S3 bucket.
Work: Turns these events into a standalone subgraph, independent of existing Dgraph state.
Output: Stores the subgraph to S3 for Node Identifier.
Node Identifier¶
Work: Identifies nodes in the incoming subgraph against the canonical identities of nodes in DynamoDB. The incoming nodes may be new, or they may represent something already known in the master graph.
For instance, an incoming subgraph may refer to a process
{PID:1234, name: "coolthing.exe", started at 1:00AM}; it’s possible that
Dgraph already has a node representing this exact same process. We’d like to
de-duplicate this process node.
Output: TODO
Graph Merger¶
Work: Write the new edges and nodes from Node Identifier to Dgraph.
Output: TODO (it looks like exactly the same as node identifier’s output, just re-encoded?)
Engagement Creator¶
Work: Simply appends the risks and lenses described in its input to Dgraph.
Output: Nothing is written to S3; the desired output is a mutation to Dgraph.
Other services¶
Metric Forwarder¶
Input: This services receives stdout from other lambdas via Cloudwatch
Logs. We subscribe only to logs that contain the reserved keyword MONITORING|.
Work: Metric Forwarder parses these log lines - which are statsd
serialized metrics - and forwards them to Cloudwatch as metrics.
Model Plugin Deployer¶
TODO
Engagement View (aka UX)¶
Provides the main customer interaction with Grapl. This is not actually a standalone service, but hosted as static assets inside Grapl Web UI.
Graphql Endpoint¶
Graphql interface into our Dgraph database.
Grapl Web UI¶
Provides authn/authz functions, and acts as a router to other services:
Graphql Endpoint (/graphqlEndpoint)
Model Plugin Deployer (current undergoing rewrite)
Also, hosts static assets like Engagement View.