Tips for a successful Firebolt evaluation

Getting Started

Choose a representative use case

For your initial testing, we typically recommend taking one of two approaches:

  • Identify a specific test case from your live production workloads and replicate that test case in Firebolt. This often consists of taking the queries behind one or two critical charts from a dashboard and running those queries in Firebolt and any other products being tested.
  • Define a set of representative queries (including filters, joins, aggregations) based on your projected workloads and run those queries in both Firebolt and any other products being tested. This is the most common approach.

Each of these approaches allow you to fully validate Firebolt’s ability to handle your expected analytical query workloads.

Define your expectations

Know what you’re looking to achieve before beginning your tests. This will help you determine whether Firebolt meets your needs!

If you need help, here are a few example success criteria from other Firebolt evaluations:

  • Reduce query execution time to < 1.5 seconds for all test queries
  • Enable complex aggregations to run over 100GB of data in less than 1 second
  • Support 50 concurrent queries with a response time of < 2 seconds

Understand the evaluation process

After you have selected your use case, you will want to simulate how your organization will interact with Firebolt in a real scenario. Typical Firebolt evaluations follow four common steps:

  1. Ingest initial data
  2. Run test queries
  3. Optimize performance using indexes
  4. Test integrations for ingestion and querying

Once you complete these steps, you should have a clear understanding of whether Firebolt will work for your organization. You can then proceed with building an MVP use case and putting Firebolt into production.

Understand Firebolt billing

Firebolt’s billing is based on engine uptime, measured in seconds. We also pass through AWS S3 storage costs, although this ends up being a relatively minimal cost (pennies on the dollar). Consequently, the amount that you spend is entirely dependent on which engines you use and how long those engines are running.

Your Firebolt account includes a $200 credit to allow you to try the product. You should be able to complete your evaluation without spending more than $200.

Recommended engine configurations

Evaluation Step

Recommended Engine Configuration

Cost Range

Ingest initial data

Single node M2

$3 to $5

Run test queries

Single node M2

$5 to $20

Optimize performance

Varies based on workloads

$20 to $100

Test integrations

Varies based on workloads

$20 to $50

Estimating ingestion costs:

  • You can assume that Firebolt ingests data at a rate of approximately 3GB/minute. Your actual data ingestion rate may be faster than this, but this metric can provide a conservative ballpark range for how much you will spend. For example, if you use the M2 engine ($1.30/hour) for one hour, you will be able to ingest approximately 180GB of data during that time (assuming each of the files being ingested fits in the RAM of one node of your engine).

Estimating query costs:

  • Since Firebolt bills based on engine uptime, your estimated query costs are entirely dependent on your final engine configuration. We recommend creating a full cost estimation only after your queries are optimized and efficiently pruning data with indexes.

Step 1: Ingest Initial Data

Recommended Engine Configuration

Cost Range

Single node M2

$3 to $5

Tips

  • Remember to connect your AWS account to Firebolt before you begin ingesting data.
  • The data that you want to ingest should be located on an S3 bucket that Firebolt can access. Refer to this guide to configure your AWS permissions properly.
  • Pick a reasonable amount of data to load for testing—enough to run your defined queries and tests, but not your full production workload. Most customers load around 150GB of data for testing.
  • Don’t worry too much about perfectly defining primary indexes when doing your initial data ingestion. It’s easy to recreate a table with new primary indexes after you have run your initial test queries–Firebolt will even suggest the best indexes to use!

Key Resources

Step 2: Run Test Queries

Recommended Engine Configuration

Cost Range

Single node M2

$5 to $20

Tips

  • Firebolt is fully PostgreSQL-complaint, so you can write your queries using standard PostgreSQL syntax. Our full SQL function glossary can be found at this link.
  • Try to test a mix of analytical queries that include filters, aggregations, and joins.
  • Do not use external tables in analytical SELECT queries. They are not actual tables and it is slow to access data associated with them - there are no indexes and data is being read line-by-line from the associated files.
  • Use your initial test queries as a way to understand Firebolt’s performance without significant optimizations. However, to fully experience Firebolt’s speed, you will want to spend time creating indexes in step 3.

Key resources

Step 3: Optimize Performance

Recommended Engine Configuration

Cost Range

Varies based on workload

$20 to $100

Tips

  • Creating primary and aggregating indexes is the main way to optimize your query performance in Firebolt. Improving your indexing will have a more significant impact on query run times than changing your engine sizes. Consequently, during this stage, focus on creating effective indexes first, and then experiment with different engine configurations.
  • Use the information_schema.query_history table to understand the performance of each of your queries. In particular, the “scanned_rows” column shows how many records were read by Firebolt. By comparing this value to the total number of rows in your table, you can assess the pruning power of your indexes. The fewer rows scanned, the better!
  • Once you have created effective primary and aggregating indexes, you can experiment with comparing query performance across different engine configurations. In general, size up (bigger nodes) before scaling out (more nodes).

Key resources

Step 4: Test Integrations

Recommended Engine Configuration

Cost Range

Varies based on workload

$20 to $100

Tips

  • Testing Firebolt’s integrations will allow you to validate that your downstream BI and reporting applications can connect to data in Firebolt.
  • Use the Firebolt SDKs to natively incorporate Firebolt into your application or custom-coded pipelines. The JDBC driver can allow you to connect to Firebolt from many BI tools.

Key resources