Cloud services
Authentication, metadata, query planning and coordination. Think managed control plane, rather than a service you deploy.
YOU KNOW AWS. START HERE.
Snowflake,
translated.
A field guide to the data platform, through an AWS lens. From “where are the files?” to understanding a dbt pipeline.
The one-sentence version
Snowflake is a managed data platform that combines capabilities you might otherwise assemble from Redshift, S3, Glue, orchestration and governance services. It is not just another name for Redshift.
AWS comparisons here are mental models, not exact product equivalences. This is a general guide, not a description of any particular client's system.
01 / ORIENTATION
One platform. Three layers.
Snowflake runs on AWS, Azure or Google Cloud. Your account belongs to a particular cloud and region. Snowflake manages the underlying infrastructure; you work with data objects, compute and permissions.
Compute
Independent clusters called virtual warehouses run SQL and transformations against shared data.
Storage
Native managed tables, or external storage accessed through external and Iceberg table arrangements.
The familiar AWS picture
Sources→S3→Glue / EMR / Redshift→BI
A possible Snowflake picture
Azure files→Ingestion→Raw tables→dbt SQL→Data marts
An example, not a requirement: sources can also stream rows directly, and analytical data can remain in externally stored Iceberg tables.
02 / COMPUTE
A warehouse is not where the tables live.
In Snowflake, a virtual warehouse is a compute cluster. Databases and schemas organise objects. Warehouses supply processing power. Suspending a warehouse does not delete the tables.
AWS lens
Imagine independent Redshift-like compute clusters working against one shared, managed data layer, without copying each dataset into each cluster.
LOADING_WHDBT_WHBI_WHDEV_WH
↓ independent compute, shared access ↓
DATABASE → SCHEMA → TABLES & VIEWSSeparate warehouses isolate compute capacity: a heavy dbt run need not compete with dashboard queries for the same cluster. They do not create security boundaries by themselves; roles and grants control access to data.
Scale up
Resize a warehouse to give work more resources. It may help a large query, but more compute is not automatically more efficient.
Scale out
Multi-cluster warehouses primarily address concurrency: more simultaneous work, not simply more horsepower for one query.
Auto-suspend / auto-resume lets compute stop when idle and start when needed. This is a major cost-control lever for intermittent workloads.
03 / STORAGE
“Storage is in Azure” needs unpacking.
Azure could be a landing zone, the home of the analytical tables, or simply the cloud underneath Snowflake. Those are different architectures.
Native Snowflake tables
Loaded data is reorganised into Snowflake's compressed, columnar format and micro-partitions. Snowflake manages the physical storage.
AWS lens: Redshift-managed table storage.
On Azure, the underlying infrastructure is Azure storage, but it is not normally a collection of Parquet files in your own storage account.
External tables
Files remain in client-managed Blob Storage or ADLS. Snowflake provides a table interface over the files.
AWS lens: Athena or Redshift Spectrum over S3.
Do not assume native-table performance or feature support. External tables are generally read-oriented.
Iceberg tables
Analytical data and metadata use Apache Iceberg, potentially in client-managed Azure storage.
AWS lens: Iceberg on S3, used by Athena, Glue or EMR.
Who manages the catalog and table matters. Read/write capabilities and supported features depend on that configuration.
Micro-partitions, briefly
Native table data is automatically divided into small storage units with metadata that helps Snowflake prune irrelevant data during queries. You don't normally hand-create partitions as you might in a Hive-style data lake. Clustering and query design still matter at scale.
04 / INGESTION
Files, rows, or connectors?
First separate extraction from loading. A file loader does not necessarily know how to extract records from a SaaS application or capture changes from a source database.
Stages + COPY INTO Batch files
A stage describes a file location. An external stage references Azure storage or S3; an internal stage uses Snowflake-managed file storage. A stage is not a table.
COPY INTO loads staged CSV, JSON, Parquet and other supported files into tables. Storage integrations provide governed cloud-storage access without embedding credentials in every command.
AWS lens: Redshift COPY from S3.
Snowpipe Arriving files
Automatically loads files as they appear, commonly using cloud storage notifications. Snowflake manages ingestion compute.
AWS lens: S3 events → managed ingestion → warehouse.
Not source extraction: something must still create those files. Also plan for error handling, duplicates and replay—not just the happy path.
Snowpipe Streaming Incoming rows
Applications and connectors send rows directly through supported APIs/SDKs, without first producing staged files.
AWS lens: the ingestion role of Kinesis / Firehose feeding analytics.
Not a general event bus: don't treat it as a drop-in replacement for Kafka or Kinesis with arbitrary downstream consumers.
Openflow & connectors Source integration
Openflow is built on Apache NiFi and provides connector-based data movement. Snowflake and third-party connectors cover additional sources.
AWS lens: parts of AppFlow, Glue ingestion and integration tooling.
Check the specific connector's deployment model, supported change capture, schema evolution, recovery and cost.
Map each source all the way through: extract → transport → land → load → validate → replay.
05 / TRANSFORMATION
dbt writes the instructions.
Snowflake does the processing.
For ordinary SQL models, dbt compiles templated SQL, resolves model dependencies and sends statements to Snowflake. Snowflake's compute creates views, builds tables or updates incremental models. dbt also executes the tests you declare.
AWS lens
Exactly the distinction you'd make with dbt targeting Redshift: the dbt runner coordinates; the database executes the SQL. Your data isn't normally downloaded into the dbt process for transformation.
Two execution questions, not one
- Where does the dbt runner run? dbt Cloud, CI, a container, or Snowflake itself?
- Which Snowflake warehouse runs its SQL? What size, permissions and concurrency does it have?
dbt Projects on Snowflake can deploy and execute dbt projects within Snowflake, with native scheduling and monitoring. That is an option—not something to infer just because the database is Snowflake.
Other native transformation tools
Dynamic tables
Declare a query and target freshness. Snowflake manages refreshes and dependencies.
AWS lens: managed materialized views + declarative ETL.
A target lag is a freshness objective, not simply a cron interval. Incremental support and refresh mode matter. Useful overlap with dbt models, not a blanket dbt replacement.
Streams
Expose changes to a source object relative to a tracked point. They don't operate as independent message queues.
AWS lens: a table change feed, loosely like DynamoDB Streams.
Consumption semantics, retention and staleness matter. A stream does not provide indefinite event retention.
Tasks
Execute SQL or stored procedures on schedules or triggers. Tasks can be linked in dependency graphs.
AWS lens: EventBridge scheduling + a focused Step Functions-style DAG.
Can use a selected warehouse or serverless compute. Useful for native SQL work and scheduling dbt project execution.
Snowpark
APIs for Python, Java and Scala transformations near the data; also used with stored procedures and UDFs.
AWS lens: some Glue / EMR programming use cases.
Not inherently a Spark cluster. Many DataFrame operations execute in Snowflake; collecting a large result into Python memory changes the performance picture.
Raw table→Stream tracks changes→Task runs MERGE→Curated table
06 / OPERATIONS
Managed infrastructure.
Still your data responsibilities.
| Snowflake capability | AWS mental model | What to remember |
|---|---|---|
| Roles & grants | IAM + database grants + Lake Formation | Snowflake has its own role hierarchy and object privileges. Separate service identities and apply least privilege. |
| Horizon Catalog | Glue Data Catalog + governance tooling | Discovery, metadata and governance integrated into the platform. |
| Masking & row access policies | Fine-grained Lake Formation / Redshift controls | Govern access in the data platform, not just in the dashboard. |
| Query, task & load history | CloudWatch + database system views | Use to diagnose failures, queueing, scans, latency and spend. |
| Time Travel | Historical query / point-in-time recovery concepts | Query or restore earlier states within retention. Not a complete disaster-recovery plan. |
| Zero-copy cloning | Copy-on-write snapshots | Fast dev/CI copies. Changed data and retained history can still consume storage. |
| Secure Data Sharing | Redshift data sharing | Governed access without a conventional export/import pipeline; cross-region/cloud arrangements can involve replication. |
07 / THE WIDER PLATFORM
Names you might hear in meetings.
- Snowpark Container Services
- Run containerised applications within Snowflake. Loosely overlaps with ECS/EKS use cases, not the whole AWS container ecosystem.
- Streamlit in Snowflake
- Host interactive Python data applications close to their data.
- Cortex AI & Snowflake ML
- Managed AI functions and model tooling. Overlaps with parts of Bedrock and SageMaker.
- Marketplace & Native Apps
- Acquire or distribute datasets and applications through Snowflake.
- Hybrid tables
- Indexed, transactional tables for lower-latency reads and writes. Different from standard analytical tables.
- Snowgrid
- Cross-region and cross-cloud collaboration, governance and business-continuity capabilities. Not a promise that all your data is automatically replicated everywhere.
Feature support, editions, regions and commercial terms vary. Snowflake is US-headquartered; running an account in an Azure EU region does not make the provider European-owned.
08 / COST & PERFORMANCE
No servers to patch.
Still plenty to optimise.
The main meters
- Warehouse compute: size, running time and active clusters.
- Serverless services: separately metered ingestion, maintenance and other features.
- Storage: tables, retained history and changed clone data.
- Transfer: especially across regions or clouds.
Common dbt cost traps
- Frequent full refreshes of large models.
- “Incremental” models scanning most of the source.
- Warehouses left running while idle.
- Oversizing—or slow, undersized compute.
- BI and batch jobs competing on one cluster.
- Duplicate transformations in dbt and native jobs.
Start with query history and query profiles, not a larger warehouse. Look for poor pruning, join explosions, spills, repeated scans and queueing. Measure runtime and credits together.
09 / TAKE THIS INTO THE MEETING
Seven questions to map the system.
- Which cloud and region hosts the Snowflake account?Establish residency, network boundaries and transfer implications.
- What exactly is in the Azure account?Landing files, external/Iceberg analytical tables, or both?
- What extracts and loads each source?Name the connector, file producer, pipe or streaming writer. How do replay and schema changes work?
- Where does dbt run, and what schedules it?Separate the runner, the SQL warehouse and the orchestrator.
- What transforms data besides dbt?Find tasks, streams, dynamic tables, procedures and Python code.
- Which warehouses serve which workloads?Check sizing, isolation, auto-suspend, concurrency and service identities.
- How do you know the data is correct and fresh?Failed loads, failed tests, stale models, alert ownership and recovery procedures.
Your goal is an architecture map, not product trivia.
Once you can trace one important dataset from source to dashboard—and explain its permissions, freshness, failure recovery and cost—you have a useful foundation for reviewing the system.
DESK REFERENCE
The quick AWS dictionary.
Approximate overlaps only. Read the caveats above before treating anything as interchangeable.
| If you know… | Start with… |
|---|---|
| Redshift compute | Virtual warehouses |
| S3 landing location | External stages referencing Blob/ADLS or S3 |
| Redshift COPY | COPY INTO |
| S3-event-driven file ingestion | Snowpipe |
| Streaming delivery into analytics | Snowpipe Streaming |
| Athena / Spectrum | External tables; Iceberg where appropriate |
| Glue / EMR transformations | SQL, dbt, Snowpark, dynamic tables |
| EventBridge + workflow DAGs | Tasks and task graphs |
| Table change feeds | Streams |
| Glue Catalog / Lake Formation | Horizon Catalog, roles and governance policies |
| CloudWatch + system views | Query, task, load and account usage history |
GO DEEPER
Official reading.
- Architecture & core capabilities ↗The best starting point for the overall platform.
- Data loading overview ↗Batch, continuous and streaming ingestion.
- dbt Projects on Snowflake ↗Native execution, orchestration and monitoring.
- Dynamic tables ↗Declarative pipelines and target freshness.
- Streams & tasks ↗Change tracking and native execution.
- Iceberg tables ↗External open-format storage and catalog options.