AWS setup¶
Warnings¶
NOTE that setting up Grapl will incur AWS charges! This can amount to hundreds of dollars a month based on the configuration. This setup script is designed for testing, and may include breaking changes in future versions, increased charges in future versions, or may otherwise require manually working with CloudFormation. If you need a way to set up Grapl in a stable, forwards compatible manner, please get in contact with us directly.
Preparation¶
Local AWS credentials¶
See full instructions here.
You should have a local file ~/.aws/credentials
, with an entry resembling this
format:
[my_profile]
aws_access_key_id=...
aws_secret_access_key=...
aws_session_token=...
You will need the profile to configure your account, if you haven’t already:
aws configure --profile "my_profile"
If your profile’s name is not “default”, then note it down, as you will need to include it as a parameter in later steps.
Installing Dependencies¶
You’ll need to have the following dependencies installed:
Pulumi: https://www.pulumi.com/docs/get-started/install/
AWS CLI:
your choice of the following:
pip install awscli
https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2-docker.html
helpful alias:
alias aws='docker run --rm -it -v ~/.aws:/root/.aws -v $(pwd):/aws -e AWS_PROFILE amazon/aws-cli'
Clone Grapl Git repository¶
git clone https://github.com/grapl-security/grapl.git
cd grapl/
The remaining steps assume your working directory is the Grapl repository.
Build deployment artifacts¶
Previously we supported uploading deployment artifacts (Docker images) directly
from your dev machine, but the current state of Grapl requires that the Docker
images be downloaded from Dockerhub or Cloudsmith. If you truly wish to upload
an image to Cloudsmith, try bin/upload_image_to_cloudsmith.sh
Spin up infrastructure with Pulumi¶
(This section is actively under development, and as of Dec 2021 requires infrastructure defined in the private repository https://github.com/grapl-security/platform-infrastructure )
See pulumi/README.md for instructions to spin up infrastructure in AWS with Pulumi. Once you have successfully deployed Grapl with Pulumi, return here and follow the instructions in the following section to provision Grapl and run the tests.
graplctl
¶
We use the graplctl
utility to manage Grapl in AWS.
Installation¶
To install graplctl
run the following command in the Grapl checkout root:
make graplctl
This will build the graplctl
binary and install it in the ./bin/
directory.
You can familiarize yourself with graplctl
by running
./bin/graplctl --help
Usage notes for setup¶
If your AWS profile is not named ‘default’, you will need to explicitly provide it as a parameter:
as a command line invocation parameter
as an environmenal variable
Usage with Pulumi¶
Several commands will need references to things like S3 buckets or AWS log groups. While you can pass these values directly, you can also pull them from a Pulumi stack’s outputs automatically.
To do this, you will need to export GRAPLCTL_PULUMI_STACK
in your environment,
and then use the ./bin/graplctl-pulumi.sh
wrapper instead of invoking
graplctl
directly.
For further details, please read the documentation in that script.
Testing¶
Follow the instructions in this section to deploy analyzers, upload test data, and execute the end-to-end tests in AWS.
Deploy analyzers¶
TBD
Upload test data¶
TBD
Logging in to the Grapl UI with the test user¶
You may use the test user to log into Grapl and interact with the UI. The test
username is the deployment name followed by -grapl-test-user
. For example, if
your deployment was named test-deployment
, your username would be
test-deployment-grapl-test-user
.
To retrieve the password for your grapl deployment, navigate to “AWS Secrets Manager” and click on “Secrets”.
Click on the “Secret name” url that represents your deployment name followed by
-TestUserPassword
. The link will bring you to the “secret details” screen.
Scroll down to the section labeled “Secret Value” and click the “Retrieve Secret
Value” button. The password for your deployment will appear under “Plaintext”.
DGraph operations¶
You can manage the DGraph cluster with the docker swarm tooling by logging into
one of the swarm managers with SSM. If you forget which instances are the swarm
managers, you can find them by running graplctl swarm managers
. For your
convenience, graplctl
also provides an exec
command you can use to run a
bash command remotely on a swarm manager. For example, to list all the nodes in
the Dgraph swarm you can run something like the following:
bin/graplctl swarm exec --swarm-id my-swarm-id -- docker node ls
If you forget which swarm-id
is associated with your Dgraph cluster, you may
list all the swarm IDs in your deployment by running bin/graplctl swarm ls
.