Tips for a successful Firebolt evaluation

1

Getting Started

Choose a representative use case

For your initial testing, we typically recommend taking one of two approaches:

  • Identify a specific test case from your live production workloads and replicate that test case in Firebolt. This often consists of taking the queries behind one or two critical charts from a dashboard and running those queries in Firebolt and any other products being tested.
  • Define a set of representative queries (including filters, joins, aggregations) based on your projected workloads and run those queries in both Firebolt and any other products being tested. This is the most common approach.
    Each of these approaches allow you to fully validate Firebolt’s ability to handle your expected analytical query workloads.

Define your expectations

Know what you’re looking to achieve before beginning your tests. This will help you determine whether Firebolt meets your needs!
If you need help, here are a few example success criteria from other Firebolt evaluations:

  • Reduce query execution time to < 1.5 seconds for all test queries
  • Enable complex aggregations to run over 100GB of data in less than 1 second
  • Support 50 concurrent queries with a response time of < 2 seconds

Understand the evaluation process

After you have selected your use case, you will want to simulate how your organization will interact with Firebolt in a real scenario. Typical Firebolt evaluations follow four common steps:

  • Ingest initial data
  • Run test queries
  • Optimize performance using indexes
  • Test integrations for ingestion and querying
    Once you complete these steps, you should have a clear understanding of whether Firebolt will work for your organization. You can then proceed with building an MVP use case and putting Firebolt into production.

Understand Firebolt billing

Firebolt’s billing is based on engine uptime, measured in seconds. We also pass through AWS S3 storage costs at the rate of $23/TB. Consequently, the amount that you spend is primarily dependent on which engines you use and how long those engines are running.
Your Firebolt account includes 600 Firebolt Units (a $200 value) to allow you to try the product. You should be able to complete your evaluation without consuming more than 600 FBU.

Recommended INITIAL engine configurations

Evaluation Step Recommended Initial Engine Configuration FBU Range
Ingest initial data S engine, 1 node 4 to 16 FBU
Run test queries S engine, 1 node 8 to 32 FBU
Optimize performance Varies based on workloads. See:
* Optimizing your ingestion workloads
* Optimizing your analytical workloads		 32 to 240 FBU
Test integrations Please refer to the previous row for helpful links. 32 to 240 FBU
  • During your initial ingestion step, we recommend gathering benchmarks about your data ingestion rate using Firebolt’s information schema tables. You can then use this data to extrapolate your expected ongoing ingestion costs.
    Estimating query costs:
  • Since Firebolt charges are based on engine uptime, your estimated query costs depend entirely on your final engine configuration. We recommend creating a full cost estimation only after optimizing your queries and efficiently pruning data with indexes.

Step 1: Ingest Initial Data

Recommended Engine Configuration FBU Range
S engine, 1 node 4 to 16 FBU

Tips

  • Remember to connect your AWS account to Firebolt before you begin ingesting data.
  • The data that you want to ingest should be located on an S3 bucket that Firebolt can access. Refer to this guide to configure your AWS permissions properly using a key and secret.
  • Pick a reasonable amount of data to load for testing—enough to run your defined queries and tests, but not your full production workload. Most customers load around 150GB of data for testing.
  • Don’t worry too much about perfectly defining primary indexes when doing your initial data ingestion. It’s easy to recreate a table with new primary indexes after you have run your initial test queries. You can simply ask Firebolt for PI recommendations.

Key Resources

Step 2: Run Test Queries

Recommended Engine Configuration FBU Range
S engine, 1 node 8 to 32 FBU

Tips

  • Firebolt is fully PostgreSQL-complaint, so you can write your queries using standard PostgreSQL syntax. Our full SQL function glossary can be found at this link.
  • Try to test a mix of analytical queries that include filters, aggregations, and joins.
  • Do not use external tables in analytical SELECT queries. They are not actual tables and it is slow to access data associated with them - there are no indexes and data is being read line-by-line from the associated files.
  • Use your initial test queries as a way to understand Firebolt’s performance without significant optimizations. However, to fully experience Firebolt’s speed, you will want to spend time creating indexes in step 3.

Key resources

Step 3: Optimize Performance

Recommended Engine Configuration FBU Range
Varies based on workloads. See:
* Optimizing your ingestion workloads
* Optimizing your analytical workloads		 32 to 240 FBU

Tips

  • Creating primary and aggregating indexes is the main way to optimize your query performance in Firebolt. Improving your indexing will have a more significant impact on query run times than changing your engine sizes. Consequently, during this stage, focus on creating effective indexes first, and then experiment with different engine configurations.
  • Use the information_schema.query_history table to understand the performance of each of your queries. In particular, the “scanned_rows” column shows how many records were read by Firebolt. By comparing this value to the total number of rows in your table, you can assess the pruning power of your indexes. The fewer rows scanned, the better!
  • Once you have created effective primary and aggregating indexes, you can experiment with comparing query performance across different engine configurations. In general, size up (bigger nodes) before scaling out (more nodes).

Key resources

Step 4: Test Integrations

Recommended Engine Configuration FBU Range
Varies based on workloads. See:
* Optimizing your ingestion workloads
* Optimizing your analytical workloads		 32 to 240 FBU

Tips

  • Testing Firebolt’s integrations will allow you to validate that your downstream BI and reporting applications can connect to data in Firebolt.

  • Use the Firebolt SDKs to natively incorporate Firebolt into your application or custom-coded pipelines. The JDBC driver can allow you to connect to Firebolt from many BI tools.

Key resources