NAV Navbar
shell
API V1

Datawaves API Reference

Datawaves API enables you to record your events, then query it in a meaningful way for you and your users.

Note: The API currently works through HTTP only, we don't provide any SDKs.

API Base URI

All API calls should start with https://datawaves.io/api/v1

Performance

We do not enforce you to set a timeframe, but note that operating on a large set of events that happened over a long period of time will slow your query.

Authorization

$ curl "API_endpoint_here" \
  -H "Authorization: {secret_key}"

Make sure to replace {secret_key} with your API key.

To authorize your API calls, include an HTTP header called “Authorization” with all of your write requests, which should contain your API secret key.

You can find API keys on your project page. Projects

Recording Events

Record a single event

$ curl {base_URI}/projects/{projectID}/events/{collection} \
  -H "Authorization: {secret_key}" \
  -H 'Content-Type: application/json' \
  -d '{
        "id": 123,
        "product": "iPhone XS charger"
      }'

Make sure to replace {projectID} with your project ID, {collection} with your collection name, and {secret_key} with your API secret key.

Use this endpoint to record a single event.

HTTP Request

POST {base_URI}/projects/{projectID}/events/{collection}

Timestamp

Every event gets assigned a timestamp field with the current date and time. You can override that by specifying a timestamp field in your event data. The timestamp's format should be YYYY-MM-DDTHH:mm:ss.sssZ

Record multiple events

$ curl {base_URI}/projects/{projectID}/events
  -H "Authorization: {secret_key}" \
  -H 'Content-Type: application/json' \
  -d '{
        "orders": [
          {"id": 123, "product": "iPhone XS charger"},
          {"id": 456, "product": "Pixel 4 charger"}
        ],
        "signups": [
          {"plan": 10},
          {"plan": 20}
        ]
      }'

Make sure to replace {projectID} with your project ID, and {secret_key} with your API secret key.

Use this endpoint to record multiple events.

HTTP Request

POST {base_URI}/projects/{projectID}/events

Timestamp

Every event gets assigned a timestamp field with the current date and time. You can override that by specifying a timestamp field in your event data. The timestamp's format should be YYYY-MM-DDTHH:mm:ss.sssZ

Query Operations

Count

$ curl {base_URI}/projects/{projectID}/queries/{collection}/count
  -H 'Content-Type: application/json'

Make sure to replace {projectID} with your project ID, {collection} with your collection name.

# Response
{
  "result": 24
}

Note that some query parameters could change the response structure, like
group_by. Check the Query Parameters section for more details.

Count the number of events in the collection matching given criteria.

This type of operation can help you answer questions such as:
- How many orders have been made in the previous two weeks?

HTTP Request

POST {base_URI}/projects/{projectID}/queries/{collection}/count

Optional Parameters

Parameter Description
filters Filter events based on one or more rules.
group_by Group events using a certain property.
interval Specifies the size of time interval by which to group results.
timeframe Limit operation to a specific period of time when the events occurred.
timezone Assigns a timezone offset to relative timeframes.

Limits

Queries are rate limited at 200/minute.

Count Unique

$ curl {base_URI}/projects/{projectID}/queries/{collection}/count_unique
  -H 'Content-Type: application/json'
  -d '{
        "target_property": "TARGET_PROPERTY"
      }'

Make sure to replace {projectID} with your project ID, {collection} with your collection name.

# Response
{
  "result": 12
}

Note that some query parameters could change the response structure, like
group_by. Check the Query Parameters section for more details.

Count the number of events with unique values for a target property in a collection matching given criteria.

This type of operation can help you answer questions such as:

HTTP Request

POST {base_URI}/projects/{projectID}/queries/{collection}/count_unique

Required Parameters

Parameter Description
target_property Specifies the name of the event property to operate on.......

Optional Parameters

Parameter Description
filters Filter events based on one or more rules.
group_by Group events using a certain property.
interval Specifies the size of time interval by which to group results.
timeframe Limit operation to a specific period of time when the events occurred.
timezone Assigns a timezone offset to relative timeframes.

Limits

Queries are rate limited at 200/minute.

Min

$ curl {base_URI}/projects/{projectID}/queries/{collection}/min
  -H 'Content-Type: application/json'
  -d '{
        "target_property": "TARGET_PROPERTY"
      }'

Make sure to replace {projectID} with your project ID, {collection} with your collection name.

# Response
{
  "result": 20
}

Note that some query parameters could change the response structure, like
group_by. Check the Query Parameters section for more details.

Calculate the minimum value for a target property.

HTTP Request

POST {base_URI}/projects/{projectID}/queries/{collection}/min

Required Parameters

Parameter Description
target_property Specifies the name of the event property to operate on.......

Optional Parameters

Parameter Description
filters Filter events based on one or more rules.
group_by Group events using a certain property.
interval Specifies the size of time interval by which to group results.
timeframe Limit operation to a specific period of time when the events occurred.
timezone Assigns a timezone offset to relative timeframes.

Limits

Queries are rate limited at 200/minute.

Max

$ curl {base_URI}/projects/{projectID}/queries/{collection}/max
  -H 'Content-Type: application/json'
  -d '{
        "target_property": "TARGET_PROPERTY"
      }'

Make sure to replace {projectID} with your project ID, {collection} with your collection name.

# Response
{
  "result": 20
}

Note that some query parameters could change the response structure, like
group_by. Check the Query Parameters section for more details.

Calculate the maximum value for a target property.

HTTP Request

POST {base_URI}/projects/{projectID}/queries/{collection}/max

Required Parameters

Parameter Description
target_property Specifies the name of the event property to operate on.......

Optional Parameters

Parameter Description
filters Filter events based on one or more rules.
group_by Group events using a certain property.
interval Specifies the size of time interval by which to group results.
timeframe Limit operation to a specific period of time when the events occurred.
timezone Assigns a timezone offset to relative timeframes.

Limits

Queries are rate limited at 200/minute.

Sum

$ curl {base_URI}/projects/{projectID}/queries/{collection}/sum
  -H 'Content-Type: application/json'
  -d '{
        "target_property": "TARGET_PROPERTY"
      }'

Make sure to replace {projectID} with your project ID, {collection} with your collection name.

# Response
{
  "result": 20
}

Note that some query parameters could change the response structure, like
group_by. Check the Query Parameters section for more details.

Calculate the sum of all values for a target property.

HTTP Request

POST {base_URI}/projects/{projectID}/queries/{collection}/sum

Required Parameters

Parameter Description
target_property Specifies the name of the event property to operate on.......

Optional Parameters

Parameter Description
filters Filter events based on one or more rules.
group_by Group events using a certain property.
interval Specifies the size of time interval by which to group results.
timeframe Limit operation to a specific period of time when the events occurred.
timezone Assigns a timezone offset to relative timeframes.

Limits

Queries are rate limited at 200/minute.

Avg

$ curl {base_URI}/projects/{projectID}/queries/{collection}/avg
  -H 'Content-Type: application/json'
  -d '{
        "target_property": "TARGET_PROPERTY"
      }'

Make sure to replace {projectID} with your project ID, {collection} with your collection name.

# Response
{
  "result": 20
}

Note that some query parameters could change the response structure, like
group_by. Check the Query Parameters section for more details.

Calculate the average value for a target property.

HTTP Request

POST {base_URI}/projects/{projectID}/queries/{collection}/avg

Required Parameters

Parameter Description
target_property Specifies the name of the event property to operate on.......

Optional Parameters

Parameter Description
filters Filter events based on one or more rules.
group_by Group events using a certain property.
interval Specifies the size of time interval by which to group results.
timeframe Limit operation to a specific period of time when the events occurred.
timezone Assigns a timezone offset to relative timeframes.

Limits

Queries are rate limited at 200/minute.

Median

$ curl {base_URI}/projects/{projectID}/queries/{collection}/median
  -H 'Content-Type: application/json'
  -d '{
        "target_property": "TARGET_PROPERTY"
      }'

Make sure to replace {projectID} with your project ID, {collection} with your collection name.

# Response
{
  "result": 20
}

Note that some query parameters could change the response structure, like
group_by. Check the Query Parameters section for more details.

Calculate the median value for a target property.

HTTP Request

POST {base_URI}/projects/{projectID}/queries/{collection}/median

Required Parameters

Parameter Description
target_property Specifies the name of the event property to operate on.......

Optional Parameters

Parameter Description
filters Filter events based on one or more rules.
group_by Group events using a certain property.
interval Specifies the size of time interval by which to group results.
timeframe Limit operation to a specific period of time when the events occurred.
timezone Assigns a timezone offset to relative timeframes.

Limits

Queries are rate limited at 200/minute.

Percentiles

$ curl {base_URI}/projects/{projectID}/queries/{collection}/percentiles
  -H 'Content-Type: application/json'
  -d '{
        "target_property": "TARGET_PROPERTY"
      }'

Make sure to replace {projectID} with your project ID, {collection} with your collection name.

# Response
{
  "result": {
    "1.0": 9.998,
    "25.0": 15,
    "5.0": 10,
    "50.0": 30,
    "75.0": 45,
    "95.0": 50,
    "99.0": 50
  }
}

Note that group_by and interval are not allowed with this query.

Calculate the percentiles values for a target property.

HTTP Request

POST {base_URI}/projects/{projectID}/queries/{collection}/percentiles

Required Parameters

Parameter Description
target_property Specifies the name of the event property to operate on.......

Optional Parameters

Parameter Description
filters Filter events based on one or more rules.
timeframe Limit operation to a specific period of time when the events occurred.
timezone Assigns a timezone offset to relative timeframes.

Limits

Queries are rate limited at 200/minute.

Select Unique

$ curl {base_URI}/projects/{projectID}/queries/{collection}/select_unique
  -H 'Content-Type: application/json'
  -d '{
        "target_property": "TARGET_PROPERTY"
      }'

Make sure to replace {projectID} with your project ID, {collection} with your collection name.

# Response
{
  "result": [
    10,
    20
  ]
}

Note that some query parameters could change the response structure, like
group_by. Check the Query Parameters section for more details.

Return a list of unique property values for a target property.

This operation can help you get values such as:

HTTP Request

POST {base_URI}/projects/{projectID}/queries/{collection}/select_unique

Required Parameters

Parameter Description
target_property Specifies the name of the event property to operate on.......

Optional Parameters

Parameter Description
filters Filter events based on one or more rules.
group_by Group events using a certain property.
interval Specifies the size of time interval by which to group results.
timeframe Limit operation to a specific period of time when the events occurred.
timezone Assigns a timezone offset to relative timeframes.

Limits

Queries are rate limited at 200/minute.

Standard Deviation

$ curl {base_URI}/projects/{projectID}/queries/{collection}/standard_deviation
  -H 'Content-Type: application/json'
  -d '{
        "target_property": "TARGET_PROPERTY"
      }'

Make sure to replace {projectID} with your project ID, {collection} with your collection name.

# Response
{
  "result": 5.25
}

Note that some query parameters could change the response structure, like
group_by. Check the Query Parameters section for more details.

Calculate the standard deviation value for a target property, among all events in a collection matching given criteria.

HTTP Request

POST {base_URI}/projects/{projectID}/queries/{collection}/standard_deviation

Required Parameters

Parameter Description
target_property Specifies the name of the event property to operate on.......

Optional Parameters

Parameter Description
filters Filter events based on one or more rules.
group_by Group events using a certain property.
interval Specifies the size of time interval by which to group results.
timeframe Limit operation to a specific period of time when the events occurred.
timezone Assigns a timezone offset to relative timeframes.

Limits

Queries are rate limited at 200/minute.

Query Parameters

Filters

$ curl {base_URI}/projects/{projectID}/queries/{collection}/count
  -H 'Content-Type: application/json' \
  -d '{
        "filters": [
          {
            "property_name" : "price",
            "operator" : "gte",
            "property_value" : 0.99
          },
          {
            "property_name" : "on_sale",
            "operator" : "eq",
            "property_value" : true
          }
        ]
      }'

Make sure to replace {projectID} with your project ID, {collection} with your collection name.

Filters enable you to refine the scope of events to be included in an operation.

Filters are sent as an array of JSON objects. Each JSON object (except Or Filters) has three properties, all of which are required:

Property Description
property_name Specifies the name of the property to filter.
operator Specifies the filter operator to use.
property_value The value to compare to the property specified by the property_name.

Available Operators

Operator Description
eq Equal to.
ne Not equal to.
lt Less than.
lte Less than or equal to.
gt Greater than.
gte Greater than or equal to.
exists Whether or not a specific property exists on an event record. The value passed in must be either true or false.
in Whether or not the property value is in a given set of values. The value passed in must be a JSON array of values. Example: [1,2,3,4,5].
contains Whether or not the string property value contains the given sequence of characters.
not_contains Whether or not the string property value does not contain the given sequence of characters.
regexp Matching property name with regular expression in property value.
or “Or” - This is a special filter that matches an event if any of its component filters match. See Or Filters.

Not all filter operators make sense for different property data types, so only certain operators are valid for each.

Data Type Valid Operators
string eq, ne, lt, gt, exists, in, contains, not_contains, regex ....................
number eq, ne, lt, lte, gt, gte, exists, in
boolean eq, exists, in

Or Filters

$ curl {base_URI}/projects/{projectID}/queries/{collection}/count
  -H 'Content-Type: application/json' \
  -d '{
        "filters": [
          {
            "operator" : "or",
            "operands" : [
              {
                "property_name" : "price",
                "operator" : "gte",
                "property_value" : 9.99
              },
              {
                "property_name" : "customer.tier",
                "operator" : "eq",
                "property_value" : "premium"
              }
            ]
          }
        ]
      }'

Make sure to replace {projectID} with your project ID, {collection} with your collection name.

This type of filters allow you to filter on events that match one or more of an array of filters, but not necessarily all of them.

Group By

$ curl {base_URI}/projects/{projectID}/queries/{collection}/count \
  -H 'Content-Type: application/json' \
  -d '{
        "group_by": "country"
      }'

Make sure to replace {projectID} with your project ID, {collection} with your collection name.

# Response
{
  "result": [
    {
      "count": 20,
      "country": "USA"
    },
    {
      "count": 50,
      "country": "UK"
    }
  ]
}

Notice how the result value changed to an array instead of a single value.

The group_by parameter groups results categorically, by co-occurrence of a specified property.

This query parameter can help you answer questions such as:

Structure

Adding the group_by parameter changes the structure of the response from a single value to an array of objects containing:

  1. The unique value for the specified group_by property
  2. The result of the analysis

Order

$ curl {base_URI}/projects/{projectID}/queries/{collection}/count \
  -H 'Content-Type: application/json' \
  -d '{
        "group_by": "country",
        "order": {"by": "key", "direction": "DESC"}
      }'

Make sure to replace {projectID} with your project ID, {collection} with your collection name.

The order parameter orders the results returned by a group_by query.

Ordering can be performed in two ways. Let's say you want to return the top 10 most purchased products. Pass the value key to the parameter by if you want to order alphabetically by the product name, or pass the value count if you want to order by the count of the products.

This query parameter can help you answer questions such as:

Required Parameters

Parameter Description
by Order using key or result. Allowed values: key, count.
direction Direction of ordering. Allowed values: ASC, DESC.

Interval

$ curl {base_URI}/projects/{projectID}/queries/{collection}/count \
  -H 'Content-Type: application/json' \
  -d '{
        "interval": "month"
      }'

Make sure to replace {projectID} with your project ID, {collection} with your collection name.

{
  "result": [
    {
      "count": 1400,
      "start": "2020-01-01T00:00:00.000Z"
    },
    {
      "count": 2000,
      "start": "2020-02-01T00:00:00.000Z"
    }
  ]
}

Note: If you specified an interval, group_by and order will be ignored.

The interval parameter helps you to build a histogram, it groups results into sub-timeframes.

This query parameter can help you answer questions such as:

Calendar and fixed intervals

When specifying an interval, the interval can be specified in two manners: calendar-aware time intervals, and fixed time intervals.

Calendar-aware intervals understand that daylight savings changes the length of specific days, months have different amounts of days, and leap seconds can be tacked onto a particular year.

Fixed intervals are, by contrast, always multiples of SI units and do not change based on calendaring context.

Avaliable calendar intervals

Value Description
minute Group events into sub-timeframes of minutes.
hour Group events into sub-timeframes of hours.
day Group events into sub-timeframes of days.
week Group events into sub-timeframes of weeks.
month Group events into sub-timeframes of months.
year Group events into sub-timeframes of years. .........................................

Available fixed intervals

Value Description
{number}ms Number of milliseconds. Ex: 3ms
{number}s Number of seconds.
{number}m Number of minutes.
{number}h Number of hours.
{number}d Number of days. .............................................................................

Timeframe

$ curl {base_URI}/projects/{projectID}/queries/{collection}/count \
  -H 'Content-Type: application/json' \
  -d '{    
        "timeframe": {
            "from": "2012-08-13T19:00:00.000Z",
            "to": "2013-09-20T19:00:00.000Z"
          }
      }'

Make sure to replace {projectID} with your project ID, {collection} with your collection name.

The timeframe parameter specifies a period of time over which to run an operation. This refines the scope of events that are included in the operation, based on when each event occurred.

Required parameters

Parameter Description
from A starting time value with the following format YYYY-MM-DDTHH:mm:ss.sssZ
to An end time value with the following format YYYY-MM-DDTHH:mm:ss.sssZ

Timezone

$ curl {base_URI}/projects/{projectID}/queries/{collection}/count \
  -H 'Content-Type: application/json' \
  -d '{    
        "timeframe": {
            "from": "2012-08-13T19:00:00.000Z",
            "to": "2013-09-20T19:00:00.000Z"
          },
        "timezone": "+01:00"
      }'

Make sure to replace {projectID} with your project ID, {collection} with your collection name.

Specify a certain timezone to be respected by interval or timeframe parameters.

Valid values

Valid values are ISO 8601 UTC offsets, such as +01:00 or -08:00, and IANA time zone IDs, such as America/Los_Angeles.

Rate limits

To protect you from misuse of the service by some users, we apply rate limits to our API.

The rate limits are enforced per client, so for example if the limit is 200/minute and a client display your data from the browser and from a mobile device, the limit for him will be 400/minute for all Datawaves projects.

When the threshold is exceeded, the client will receive a 429 error page until the Block time has expired.

Errors

The API uses the following error codes:

Code Meaning
400 Bad Request -- Your request is invalid (mostly an invalid jSON).
401 Unauthorized -- You are not authorized to do this action.
404 Not Found -- The specified project could not be found.
420 You have reached your usage limit for this billing period.
429 Rate limit has been reached.
500 For now this could mean a lot of things. Examples: the specified collection could not be found, there is something wrong with your query, or internal server error (try again later).