At Velocidex we have been running open source endpoint monitoring tools for our clients in order to detect and respond to incidents. One of our favorite tools is GRR, developed by Google internally and then released as open source. GRR is a very powerful tool, with a polished UI and good documentation.
Unfortunately the open source version released by Google suffers from some shortcomings and so we have decided to develop a new project, built on the shoulders of giants called Velociraptor.
These are Velociraptor's design goals:
- Focus on data collection. Velociraptor's primary use case is to collect data and export it to other systems. Velociraptor does no analysis itself and therefore has no need for a complex data model.
- Flexibility - Velociraptor can adapt easily to new requirements without needing to redeploy either clients or servers. Using VQL (Velocidex Query Language) provides flexibility in the type and number of queries that are used to rapidly adapt to changing requirements. VQL allows us to collect just the information needed and no more in an adaptive way.
- Remove abstractions. Velociraptor aims to be as simple to understand as possible. The default data store simply stores files in the file system which may be easily inspected by the user. No special tooling is required to script or manage Velociraptor. Reduce demand on the data store. Rather than increase the data store requirements, we want to simplify the design to the point that requirements on the data store are so low, one can run a medium to large sized deployment with very few resources (down to perhaps a single server machine). In fact the default data store does not even use a database, but simply uses flat files.
- Simplify everything! Velociraptor aims to be very simple to run and administer. We remove a lot of the GRR functionality that we dont find we use often. Velociraptor ships as a single, statically linked executable which can perform all actions necessary for deployers.
In short we really wanted something like this:
In this section we go through a typical deployment scenario of Velociraptor.
Velociraptor ships as a single statically compiled binary. It has no external dependencies and does not require installing as a package. Simply download the binary from the release page for the OS you will be deploying on. The same binary is used for the client and the server.
Velociraptor needs a configuration file to start serving requests. You can generate a new configuration file using the velociraptor config generate command. Note that this prints a new configuration into stdout so you might want to redirect it into a new file (NOTE: The file contains key material so ensure it has appropriate permissions). Below we highlight the parameters you will probably want to change:
$ velociraptor config generate | tee /etc/velociraptor.config.yaml Client: server_urls: - http://localhost:8000/ writeback_linux: /etc/velociraptor.writeback.yaml writeback_windows: /Program Files/Velociraptor/velociraptor.writeback.yaml Datastore: filestore_directory: /tmp/velociraptor implementation: FileBaseDataStore location: /tmp/velociraptor Frontend: bind_address: 127.0.0.1 bind_port: 8000 GUI: bind_address: 127.0.0.1 bind_port: 8889
The configuration file contains default values for most settings and new keys for cryptographic material so it is expected that you edit the file to customize it for your local deployment. The config file is divided into sections. Here is a quick overview:
Clients receive a subset of the complete configuration which enables them to connect to the server. You can extract the client's configuration using the velociraptor config client command.
$ velociraptor --config /etc/velociraptor.config.yaml config client Client: ca_certificate: | -----BEGIN CERTIFICATE----- MIIDIDCCAgigAwIBAgIQEPaF6CPMLOlixEmpgHhvsTANBgkqhkiG9w0BAQsFADAa abIwLMojhIxVFXZOZ0p2ZhYkeKJwNGbiA9rBJR2iKxeJOa0B -----END CERTIFICATE----- nonce: UwtTRfezXIU= server_urls: - http://localhost:8000/ writeback_linux: /etc/velociraptor.writeback.yaml writeback_windows: /Program Files/Velociraptor/velociraptor.writeback.yaml
Start the server using the frontend command:
$ velociraptor --config velociraptor.config.yaml frontend INFO:2018/08/08 15:39:09 Launched gRPC API server on 127.0.0.1:8888 INFO:2018/08/08 15:39:09 GUI is ready to handle requests at 127.0.0.1:8889 INFO:2018/08/08 15:39:09 Frontend is ready to handle client requests at 127.0.0.1:8000
You can now verify the server is working by connecting to the GUI with a web browser:
The client is run using the s configuration
$ velociraptor --config /etc/velociraptor.client.config.yaml client velociraptor: error: Unable to load writeback file: open /etc/velociraptor.writeback.yaml: no such file or directory Genering new private key.... Wrote new config file /etc/velocirpator.writeback.yaml INFO:2018/08/08 16:02:22 Starting Crypto for client C.039f18494e6dae95 INFO:2018/08/08 16:02:22 Starting HTTPCommunicator: [http://localhost:8000/] INFO:2018/08/08 16:02:22 Sending unsolicited ping. INFO:2018/08/08 16:02:22 Updated server serial number in config file /etc/velociraptor.writeback.yaml to 1 INFO:2018/08/08 16:02:22 Received PEM for VelociraptorServer from http://localhost:8000/ INFO:2018/08/08 16:02:22 Received response with status: 406 Not Acceptable INFO:2018/08/08 16:02:22 Enrolling INFO:2018/08/08 16:02:23 Received response with status: 406 Not Acceptable INFO:2018/08/08 16:02:25 Sending unsolicited ping. INFO:2018/08/08 16:02:25 Received response with status: 200 OK INFO:2018/08/08 16:02:25 Checking foreman INFO:2018/08/08 16:02:26 Received response with status: 200 OK INFO:2018/08/08 16:02:27 Sending unsolicited ping. INFO:2018/08/08 16:02:27 Received response with status: 200 OK
We can see that when a new client starts for the first time it goes through a number of steps:
This post introduces Velociraptor - a new end point monitoring and IR tool built upon GRR's groundwork and experience. To be clear, we reused some of GRR's code and some design elements, but Velociraptor is a new project and is largely a rewrite of GRR's codebase. Like GRR, Velociraptor is released under an open source license and is a community project hosted on https://gitlab.com/velocidex/velociraptor.
It is still very early days and we would love to receive feedback and suggestions. This is the first technology preview release and we hope to make a more stable and comprehensive release in the coming months. As Velociraptor becomes more battle tested we hope the codebase will stabilize.
The near term roadmap is:
Please play with it and send feedback to firstname.lastname@example.org