Loading Continuously Using Snowpipe¶
Snowpipe is Snowflake’s continuous data ingestion service. Snowpipe loads data within minutes after files are added to a stage and submitted for ingestion. The service provides REST endpoints and uses Snowflake-provided compute resources to load data and retrieve load history reports. The service can load data from any internal (i.e. Snowflake) or external (i.e. AWS S3 or Microsoft Azure) stage. With Snowpipe’s server-less compute model, Snowflake manages load capacity, ensuring optimal compute resources to meet demand. In short, Snowpipe provides a “pipeline” for loading fresh data in micro-batches as soon as it’s available.