Skip to main content

Keep Axiom query compute low

Every APL query that runs in your Axiom organisation uses Axiom query compute, measured in GB-hours: how long the query runs times the memory it needs. Axiom plans include a monthly allowance. Huntbase is built to stay well inside it, and shows you what it used.

The more events a query scans, the more compute it uses. A query over one dataset and a short time range, with field filters, is cheap. A full-text search over every dataset for 30 days is not.

What uses compute​

What runsHow oftenHow it uses compute
Queries, hunts and ScoutWhen you or Scout run themOne APL query per run, over the time range or the hunt's time window.
Watcher deployments and correlation rulesAt most once an hour each, within the query budgetOne query per deployment per run, over the time since the last run.
MonitorsOn the monitor's schedule, in AxiomAxiom runs the monitor. Huntbase also reads its history every 5 minutes.
Threat intel look-backWhen it checks your threat intel against AxiomSearches your datasets, or only the ones you name, for your indicators.
Normalised queriesWhen a query over an hb_ surface runsOne APL query per mapped dataset.
Risk roll-upEvery 15 minutes, when onOne small query on the risk dataset.
Schema sync and format previewsWhen the schema syncs, or you click Preview rowsA few small queries.

Sending scheduled results or risk events to Axiom is ingest, not query compute.

See what Huntbase used​

Open the Axiom connection's page. Next to the query budget, Axiom query compute used by Huntbase shows the Last 24 hours and the Last 30 days:

  • how many queries ran, the compute they used in GB-hours, their query time, and how many rows Axiom examined;
  • the same split by what ran them: Watchers, Monitors, Hunts, Scout, Intel look-backs, Schema discovery, Queries and schedules and Other;
  • when the allowance is set, what share of the monthly GB-hours the last 30 days used.

Where Axiom reports a query's cost, Huntbase uses it. Otherwise it estimates the cost from the query time, and the figure says (estimated). A number Huntbase can't read says not known.

Monitors run inside Axiom, so their own runs don't appear here. Only the queries Huntbase runs for them do. For the whole picture, check your organisation's usage and billing in Axiom.

Keep it low​

Point Huntbase at the right datasets.

  • Give Huntbase a token that can read only the datasets you want hunted.
  • Exclude datasets Huntbase shouldn't search, such as application logs, on the connection's page. See Connect Axiom.
  • Set Datasets to check for threat intel on the connection, so intel look-backs search your security datasets, not every dataset.
  • Deploy watchers to the one dataset that holds their events.

Keep time ranges short.

  • Start hunts with the shortest window that covers the activity. Set the hunt's time window to the dates you care about rather than the last 90 days.
  • Watcher deployments already search only the time since their last run.

Write cheaper APL.

  • Filter on fields first: where EventID == 4625 near the top of the query.
  • Use search and other filters across every field sparingly. The Search a dataset for an indicator template is the quickest way to answer "have we seen this?", but it scans every field.
  • Use simple comparisons instead of regular expressions where they work.
  • Use take to cap rows when you only need a sample.

Control how often things run.

  • Leave watcher deployments on the hourly default unless a rule needs to be faster. The connection's query budget caps scheduled queries per hour.
  • A monitor that runs every 5 minutes runs 12 times as often as an hourly one. Pick the slowest schedule that still catches what you need.

Set a limit in Axiom. Axiom lets you set hourly and daily Query cost limits on an API token. A token over its limit gets errors until the window resets, so Huntbase stops querying instead of using more. Leave headroom, or watchers and hunts will fail until the limit resets.

When Axiom says the limit is reached​

If your Axiom organisation runs out of query compute, Axiom refuses queries until the limit resets. Huntbase shows the run as failed with Axiom's message. Queries work again once Axiom's limit resets. Huntbase's own query budget helps you stay clear of the limit in the first place.

Next steps​