BigQuery
- Architecture
- Queries
- Optimizations
- query-optimizations
- storage-optimizations
- Documentation
- modeling
- slots
- data-masking
- interview-questions
BigQuery is a serverless data analytics platform. You don't need to provision individual instances or virtual machines to use BigQuery. Instead, BigQuery automatically allocates computing resources as you need them. You can also reserve compute capacity ahead of time in the form of slots, which represent virtual CPUs.
Pricing
- Queries (on-demand) - $6.25 per TiB - The first 1 TiB per month is free.
- Pricing | BigQuery: Cloud Data Warehouse | Google Cloud
- Quotas and limits | BigQuery | Google Cloud
- Estimate and control costs | BigQuery | Google Cloud
- Pricing | BigQuery: Cloud Data Warehouse | Google Cloud
- Estimate and control costs | BigQuery | Google Cloud
Storage pricing
Storage pricing is the cost to store data that you load into BigQuery. You pay for active storage and long-term storage.
- Active storage includes any table or table partition that has been modified in the last 90 days.
- Long-term storage includes any table or table partition that has not been modified for 90 consecutive days. The price of storage for that table automatically drops by approximately 50%. There is no difference in performance, durability, or availability between active and long-term storage.
- Metadata storage includes storage for logical and physical metadata for datasets, tables, partitions, models and functions stored in the BigQuery metastore.
The first 10 GiB of storage per month is free.
Mumbai (asia-south1)
Operation | Pricing |
---|---|
Active logical storage | $0.023 per GiB per month |
Long-term logical storage | $0.016 per GiB per month |
Active physical storage | $0.052 per GiB per month |
Long-term physical storage | $0.026 per GiB per month |
Metadata storage | $0.052 per GiB per month |
S3 Standard - General purpose storage for any type of data, typically used for frequently accessed data - First 50 TB / Month - $0.023 per GB
Pricing is same as AWS S3 Standard storage
Others
BigQuery
should not be used if you expect OLTP behavior or performance.- Write data from Kafka to BigQuery with Dataflow | Google Cloud
References
- https://github.com/GoogleCloudPlatform/training-data-analyst/blob/master/courses/machine_learning/deepdive/02_generalization/repeatable_splitting.ipynb
- https://github.com/GoogleCloudPlatform/training-data-analyst
- php - Speed of inserting to BigQuery - should this be batched in background? - Stack Overflow
- BigQuery SQL - YouTube
- Advanced BigQuery Concepts - YouTube
- Strategies for optimizing your BigQuery queries - YouTube
- How does BigQuery store data? - YouTube
- BigQuery Spotlight - YouTube