Comprehensive Snowflake Documentation
Overview
Connecting to Snowflake
- Ecosystem
- Snowsight
- Databases
- Overview
- Database Details
- Schema
- Table Details
- View Details
- Pipe Details
- Function Details
- Procedure Details
- Classic Console
- SnowSQL
- Visual Studio Code SQL Extension
- Client Download
- Client Configuration
- Connecting to Third-Party Systems
- About the Snowflake Connector for ServiceNow
- Known Limitations During Preview
- Preparing Your ServiceNow Instance
- Installing and Configuring the Connector with Snowsight
- Installing and Configuring the Connector Using SQL Commands
- Setting Up Data Ingestion for Your ServiceNow Data
- Accessing the ServiceNow Data in Snowflake
- Monitoring the Connector
- Managing the Snowflake Connector for ServiceNow
- Cost Governance of the Snowflake Connector for ServiceNow
- Troubleshooting the Connector
Virtual Warehouses
- Overview
- Multicluster
- Considerations
- Working with Warehouses
- Query Acceleration Service
- Monitoring Load
- Snowpark-optimized Warehouses
Databases, Tables, & Views
- Table Structures
- Temporary and Transient Tables
- External Tables
- Amazon S3
- Google Cloud Storage
- Azure Blob Storage
- Iceberg Tables
- Configuration
- Configure an External Volume
- Configure a Catalog Integration
- Work with Iceberg Tables
- Create
- Manage
- Convert
- Snowflake Iceberg Catalog SDK
- Iceberg table data types
- Search Optimization Service
- Views
- Views
- Secure Views
- Materialized Views
- Considerations
- Views, Materialized Views and Dynamic Tables
- Table Design
- Cloning
- Data Storage
Data Types
Data Loading
- Overview
- Feature Summary
- Considerations
- Preparing to Load Data
- Staging Files Using Snowsight
- Loading Data Using the Web Interface
- Monitor data loading activity
- Loading Semi-Structured Data
- Introduction
- Supported Formats
- Considerations
- Loading Unstructured Data
- Introduction
- Directory Tables
- Amazon S3
- Google Cloud Storage
- Azure Blob Storage
- REST API
- Processing with UDF and Procedure Handlers
- Sharing
- Troubleshooting
- Bulk Loading
- Local File System
- Amazon S3
- Option 1: Configuring a Snowflake Storage Integration
- Option 2: Configuring an AWS IAM Role (Deprecated)
- Option 3: Configuring AWS IAM User Credentials
- Google Cloud Storage
- Microsoft Azure
- Troubleshooting
- Snowpipe
- Overview
- Auto Ingest
- REST Endpoints
- Error Notifications
- Troubleshooting
- Managing
- Snowpipe Costs
- Snowpipe Streaming
- Overview
- Configurations and Examples
- Recommendations
- Snowpipe Streaming Costs
- Kafka Connector with Snowpipe Streaming
- Schema Detection and Evolution for Kafka Connector with Snowpipe Streaming
- Accessing Data in Other Storage
- Amazon S3-compatible Storage
- Querying and Transforming Data
- Querying Data in Staged Files
- Querying Metadata for Staged Files
- Transforming Data During Load
- Evolving Table Schema Automatically
- Continuous Data Pipelines
- Overview
- Dynamic tables
- Create dynamic tables
- About managing dynamic tables
- Manage dynamic tables refresh
- Monitor dynamic tables
- Streams
- Tasks
- Enabling Error Notifications using AWS SNS
- Enabling Error Notifications using MS Azure Grid Events
- Enabling Error Notifications for Tasks using Google Pub/Sub
- Integrating Task Error Notifications with Tasks
- Task Error Payload Example
- Examples
Data Unloading
- Overview
- Features
- Considerations
- Preparing to Unload Data
- Unloading into a Snowflake Stage
- Unloading into Amazon S3
- Unloading into Google Cloud Storage
- Unloading into Microsoft Azure
Queries
- Joins
- Subqueries
- Querying Hierarchical Data
- Common Table Expressions (CTE)
- Querying Semi-structured Data
- Window Functions
- Match Recognize
- Sequences
- Persisted Query Results
- Distinct Counts
- Similarity Estimation
- Frequency Estimation
- Estimating Percentile Values
- Monitor query activity with Query History
- Query Profile
- Query Hash
- Cancel Statements
Snowflake Cortex ML-Based Functions
Data Sharing & Collaboration
- Introduction to Secure Data Sharing
- About listings
- About Snowflake Marketplace
- Legal requirements for providers and consumers of listings
- Using listings as a provider
- Becoming a provider of listings
- Accessing Provider Studio
- Managing your provider profile
- Preparing data for a listing
- Creating and publishing a listing
- Configuring listings
- Configuring Cross-Cloud Auto-fulfillment
- Paid listings pricing models
- Managing listing requests
- Modifying published listings
- Defining listing referral links
- Removing listings as a provider
- Monitoring usage of your listing
- Monetization usage views
- Using listings as a consumer
- Becoming a consumer of listings
- Exploring listings
- Accessing and installing listings as a consumer
- Paying for listings
- Data sharing for providers
- Getting started
- Working with shares
- Sharing data from multiple databases
- Sharing data across regions and cloud platforms
- Using secure objects to control data access
- Sharing data protected by a policy
- Data sharing for consumers
- Consuming shared data
- General data sharing tasks
- Managing reader accounts
- Configuring reader accounts
- Enabling non-Admins to perform sharing tasks
- Granting privileges to other roles
- Sharing from Business Critical to non-Business Critical
- Data Exchange
- About
- Admin and membership
- Accessing a Data Exchange
- Becoming a data provider
- Managing data listings
- Configuring and using Data Exchanges
- Requesting a new Data Exchange
Alerts & Notifications
- Snowflake Alerts
- Email Notifications
- [SYSTEMSEND_EMAIL")
- Snowpipe Error Notifications
- Tasks Error Notifications
Security
- Authentication
- API authentication and secrets
- Federated authentication and SSO
- Key-pair authentication and rotation
- Multi-factor authentication (MFA)
- OAuth
- Partner applications
- Custom client integration
- Managing user consent
- Azure integration
- Okta integration
- PingFederate integration
- Power BI integration
- Custom authorization server
- Partner applications
- Networking and private connectivity
- Network policies
- Network rules
- AWS VPC interface endpoints for internal stages
- Azure private endpoints for internal stages
- AWS PrivateLink
- Azure Private Link
- Google Cloud Private Service Connect
- Administration and authorization
- Sessions and session policies
- SCIM
- Access control
- End-to-End encryption
- Encryption key management
Data Governance
- Data Sensitivity & Access Visibility
- Object Tagging
- Data Classification
- ASSOCIATE_SEMANTIC_CATEGORY_TAGS
- Data Access Policies
- Masking Policies
- Using Dynamic Data Masking
- Using External Tokenization
- Row Access Policies
- Data Lineage & Dependencies
- Access History
- Object Dependencies
Organizations & Accounts
- Organizations
- Introduction
- Getting Started
- Managing Accounts
- Connecting to Your Accounts
- Accounts
- Account Identifiers
- Trial Accounts
- Parameter Management
- User Management
Business Continuity & Data Recovery
- Replication
- Introduction
- Considerations
- Configuration
- Security Integrations and Network Policy Replication
- Stage, Pipe, and Load History Replication
- Understanding Cost
- Failover
- Account Failover
- Client Redirect
- Overview
- Data Recovery
- Time Travel
- Fail-safe
- Storage Costs
Performance Optimization
Memory Spillage
Performance degrades drastically when a warehouse runs out of memory while executing a query because memory bytes must "spill" onto local disk storage. If the query requires even more memory, it spills onto remote cloud-provider storage, which results in even worse performance.
Options for Resolving Memory Spillage
When memory spillage is the issue, you can convert your existing warehouse to a Snowpark-optimized warehouse, which provides 16x more memory per node and 10x the local cache compared to a standard warehouse. Though a larger warehouse also has more memory available, a query might not require its expanded compute resources.
If you want to try resolving the spillage of a query without adjusting the warehouse that runs it, use the Query Profile to identify which operation nodes are causing the spillage.
Performance impact from local and remote disk spilling
Resolving memory spillage | Snowflake Documentation