Snowflake Architecture Cheat Sheet

Architecture

  1. No software, No Hardware, No maintenance. Snowflake is provided as Software-as-a-Service (SaaS) that runs completely on cloud infrastructure
  2. Snowflake uses a central data repository for persisted data that is accessible from all compute nodes in the data warehouse. (Shared Disk)
  3. Similar to shared-nothing architectures, Snowflake processes queries using MPP (massively parallel processing) compute clusters where each node in the cluster stores a portion of the entire data set locally (Shared Nothing)
  4. Shared disk architecture & Shared nothing architecture (SDA/SNA) (multi cluster architecture)
  5. This approach offers the data management simplicity of a shared-disk architecture, but with the performance and scale-out benefits of a shared-nothing architecture.
  6. Snowflake’s unique architecture consists of three key layers:
    • Cloud Services (The brain of the system)
    • Query Processing (The muscles of the system)
    • Database Storage
  7. Snowflake Manages everything for customer
    • Authentication
    • Configuration
    • Resource Management
    • Data Protection
    • Availability (it has built in redundancy)
    • Optimization

Database Storage

  1. When data is loaded into Snowflake, Snowflake reorganizes that data into its internal optimized, compressed, columnar format.
  2. All data is encrypted AES 256 strong encryption
  3. Snowflake stores this optimized data in cloud storage.
  4. Snowflake manages all aspects of how this data is stored — the organization, file size, structure, compression, metadata, statistics, and other aspects of data storage are handled by Snowflake.
  5. The data objects stored by Snowflake are not directly visible nor accessible by customers; they are only accessible through SQL query operations run using Snowflake

Query Processing

  1. Snowflake allows to create multiple, independent compute clusters for query processing and they are called virtual warehouses.
  2. They all access same data source without any contention (Unlimited scale)
  3. When a virtual warehouse is resized, all subsequent queries take advantage of new resources.

Cloud Services

  1. Services layer is fully maintained by snowflake and distributed across multiple availability to ensure high availability
  2. The cloud services layer is a collection of services that coordinate activities across Snowflake.
  3. These services tie together all of the different components of Snowflake in order to process user requests, from login to query dispatch
  4. The cloud services layer also runs on compute instances provisioned by Snowflake from the cloud provider.
  5. The key component of service layer is the metadata store which powers number of snowflake unique features
    1. Zero copy cloning
    2. Time travel
    3. Data sharing
    4. Among the services in this layer:
      1. Authentication & session management
      2. Infrastructure management
      3. Metadata management
      4. Query parsing and optimization
      5. Access control

Connecting Snowflake

Virtually all operations can be performed through these client

  1. A web based user interface
  2. Command Line Interface (SnowSQL)
  3. ODBC & JDBC
  4. Native Connector
  5. Third Party Connector

Query Lifecycle

  1. Authenticate and submit a query via a client ( SnowSQL or WebUI) after initiating a session
  2. Specify the virtual warehouse to the session (else a default will be taken)
  3. Services layer check if you are authorized to access the data and database (both) and operation specified in the operation (it may be alter or delete or drop or select etc)
  4. The service layer creates an optimized query plan
  5. Services layer send the instruction to virtual warehouse, allocate resources, get the data needed for processing and execute the query
  6. (caching might come but in simple one it is not mentioned)
  7. Results are then return to you

Snowflake SnowPro Practice & Reading Guide

SnowProc Certification Cheat Sheet

Refer topic wise important notes and cheat sheet

SnowProc Certification Practice Test