GCP Data Platform Architecture | Google Cloud | Symhas
GCP · BigQuery · Dataflow · Pub/Sub · Cloud Storage · Data Catalog · dbt
BigQuery Is the Fastest Data Warehouse at Any Scale.
Most Deployments Use 10% of What It Can Do.
BigQuery as a destination for scheduled SQL exports is not a data platform — it is a slow reporting database with a different billing model. BigQuery with Dataflow streaming pipelines, partitioned and clustered tables, BI Engine for sub-second dashboard response, and Data Catalog for governed discovery is a different category of capability. Symhas builds the latter.Symhas designs and deploys the full GCP data platform — BigQuery data warehouse with optimal schema design, Dataflow batch and streaming pipelines, Pub/Sub event ingestion, Cloud Storage as the raw data lake, and dbt for transformation — production-ready, cost-optimised, and governed from day one.
BigQueryServerless petabyte-scale warehouse — the data platform anchor for every Symhas GCP engagement
<1sDashboard query response on BigQuery BI Engine — sub-second on tables of any size
↓40%BigQuery cost reduction from partitioning, clustering, and slot commitment optimisation
8wkGCP data platform — BigQuery, Dataflow, Pub/Sub, dbt — production-ready, fixed price
GCP certified architects available nowActive
BigQueryServerless petabyte-scale warehouse — partitioned, clustered, and BI Engine-accelerated
<1sDashboard query response on BigQuery BI Engine — sub-second regardless of table size
↓40%BigQuery cost reduction from partitioning, clustering, and slot commitment optimisation
8wkGCP data platform — BigQuery, Dataflow, Pub/Sub, dbt — production-ready, fixed price
What We Deliver
Core Capabilities.
Production-Grade on GCP.

Every capability designed, deployed, and documented by Symhas GCP-certified data and AI architects. Fixed price. Production-ready.

BigQuery Data Warehouse DesignPartitioning · Clustering · BI Engine · Authorised views · Column-level security
BigQuery schema design that performs and costs predictably — partition pruning reducing bytes scanned, clustering ordering data for filter efficiency, BI Engine reservations caching hot tables for sub-second dashboard response, and column-level security restricting sensitive fields without duplicating tables.
Schema design — denormalised star schema optimised for BigQuery columnar storage
Partitioning — date and integer-range partition keys reducing bytes scanned per query
Clustering — column clustering within partitions for secondary filter efficiency
BI Engine reservations — reserved capacity caching frequently queried tables for sub-second response
Column-level security — policy tags restricting PII and financial fields to authorised roles
BigQuery that performs at any scale and bills predictably — not a warehouse that surprises at month-end
Dataflow Batch & Streaming PipelinesApache Beam · Batch · Streaming · Exactly-once · Auto-scaling
Google Cloud Dataflow runs Apache Beam pipelines for both batch and streaming data processing — exactly-once processing semantics preventing duplicate records, auto-scaling workers to match throughput, and native integration with Pub/Sub, Cloud Storage, and BigQuery.
Apache Beam pipeline design — batch and streaming unified programming model
Exactly-once semantics — duplicate prevention for streaming pipelines writing to BigQuery
Auto-scaling — Dataflow workers scale up and down automatically based on backlog and throughput
Windowing and triggers — tumbling, sliding, and session windows for streaming aggregations
Dataflow Flex Templates — containerised pipelines deployable on demand or on schedule
Data flowing from source to BigQuery reliably — batch and streaming, no duplicates, auto-scaled to throughput
Pub/Sub Event Streaming & IngestionPub/Sub · Push · Pull · Ordering · Dead-letter · BigQuery subscription
Cloud Pub/Sub as the event backbone — decoupling data producers from consumers, guaranteed at-least-once delivery with ordered topics where sequence matters, dead-letter topics capturing failed messages, and Pub/Sub BigQuery subscriptions writing directly to BigQuery without a Dataflow pipeline for simple ingestion.
Topic and subscription design — fan-out, ordered, and partitioned topics per use case
Pub/Sub BigQuery subscription — direct write to BigQuery table without Dataflow for simple events
Dead-letter topics — failed message capture and reprocessing workflow
Message ordering — per-key ordering for event streams where sequence matters
Push subscriptions — webhook delivery to Cloud Run or Cloud Functions consumers
Event streams delivered reliably to BigQuery and downstream consumers — no lost messages, no processing gaps
Cloud Storage Data Lake & dbt TransformationCloud Storage · Data lake · dbt · Medallion · Data lineage · Data Catalog
Cloud Storage as the raw data lake — structured landing of source files before transformation, with dbt managing the transformation layer from raw to curated BigQuery datasets, Data Catalog providing governed data discovery, and a medallion architecture separating raw, cleansed, and aggregated data tiers.
Cloud Storage bucket design — raw, processed, and archive tiers with lifecycle policies
dbt project structure — source definitions, staging models, intermediate, and mart layers
Medallion architecture — bronze, silver, and gold BigQuery datasets with lineage documentation
Data Catalog — all BigQuery tables and columns tagged, classified, and searchable
dbt tests — schema tests and custom data quality tests run on every pipeline execution
Every table in BigQuery traced from source — dbt lineage showing where data came from and how it was transformed
Delivery Model
Assessment to Production.
Fixed Price. Fixed Timeline.

Four phases with go/no-go gates. Scope and price agreed before week one.

01
Data Architecture DesignWeeks 1–2

Source system inventory and data volume profiling. BigQuery schema design. Dataflow pipeline architecture. Pub/Sub topic design. dbt project structure and medallion layer design. Cloud Storage bucket and lifecycle design. Architecture approved at go/no-go gate.

02
Infrastructure & Pipeline BuildWeeks 3–5

BigQuery datasets created with partitioning and clustering. Cloud Storage buckets with lifecycle policies deployed. Dataflow pipelines built and unit tested. Pub/Sub topics and subscriptions configured. dbt project scaffolded with source definitions.

03
dbt Models & Data QualityWeeks 6–7

dbt staging, intermediate, and mart models built. dbt tests for schema validation and data quality implemented. BI Engine reservations configured. Data Catalog tags applied to all BigQuery tables. End-to-end pipeline tested with production-volume data.

04
Go-Live & Team CertificationWeek 8

Production pipelines live. Cost monitoring alerts configured. Data engineering team certified on dbt workflow and Dataflow operations. Runbooks for pipeline failure and backfill documented. Symhas moves to advisory.

Retail · GCP Data Platform500+ Location Retailer.
33% Forecast Accuracy Gain. 21% Holding Cost Reduction. 12 Weeks.

A national retailer with 500+ locations needed unified demand forecasting across all sites — previously running 12 separate reporting databases with no single source of truth for inventory and sales data. Forecast accuracy was measured at under 60% on slow-moving SKUs.

Symhas built a GCP data platform — BigQuery as the unified warehouse ingesting POS, inventory, and supplier data via Dataflow, dbt transformation producing a medallion architecture, and Vertex AI forecasting models trained on the BigQuery feature dataset. Forecast accuracy rose to 93% on slow-moving SKUs.

↑33%Forecast accuracy
↓21%Inventory holding costs
500+Locations connected
12wkTo production AI
Discuss Your Programme
What was delivered

GCP Data Platform — Retail Production Deployment

BigQuery — unified warehouse ingesting POS, inventory, and supplier data from 500+ locations
Dataflow — batch pipeline processing 4.2M daily POS transactions from 12 source formats into BigQuery
Pub/Sub — real-time inventory events from warehouse management system, 180K messages/hour
dbt — 47 models across bronze, silver, and gold layers, 340 dbt tests running on every pipeline execution
Cloud Storage — raw data lake storing all source files before Dataflow processing, 7-year retention
Data Catalog — all 847 BigQuery columns tagged with business glossary terms and data owner

“We had 12 reporting databases that disagreed with each other on every metric. BigQuery gave us one number. dbt gave us the lineage to prove where that number came from. That is a different conversation with the business.”

— Chief Data Officer, National Retailer

GCP Services Deployed
The Specific GCP Services
We Configure for This Capability.
GCP
Google BigQuery

Serverless petabyte-scale warehouse — schema design, partitioning, clustering, BI Engine, and column-level security.

Schema and partition design
Clustering and BI Engine
Column-level security
Slot commitment optimisation
GCP
Google Cloud Dataflow

Batch and streaming pipelines — Apache Beam, exactly-once semantics, auto-scaling, and Flex Templates.

Pipeline design and build
Exactly-once streaming
Auto-scaling configuration
Flex Template packaging
GCP
Google Cloud Pub/Sub

Event streaming — topic and subscription design, BigQuery subscriptions, dead-letter, and message ordering.

Topic and subscription design
BigQuery direct subscription
Dead-letter topic configuration
Message ordering per key
GCP
Google Cloud Storage

Data lake — bucket design, lifecycle policies, raw/processed/archive tiers, and transfer automation.

Bucket and lifecycle design
Raw, processed, archive tiers
Transfer Service configuration
Retention and lock policies
GCP
dbt (data build tool)

Transformation layer — medallion architecture, source definitions, mart models, and data quality tests.

dbt project structure
Bronze/silver/gold models
Schema and custom tests
Lineage documentation
GCP
Google Cloud Data Catalog

Data governance — table and column tagging, business glossary, PII classification, and data lineage.

Tag template design
Business glossary
PII auto-classification
Lineage from Dataflow and dbt
Why Symhas
GCP Expertise Built from Production Deployments.
Schema Designed for BigQuery, Not Ported From RDBMSNormalised schemas ported from SQL Server or Oracle to BigQuery perform poorly and cost unpredictably. Symhas designs denormalised BigQuery schemas from scratch — optimised for columnar storage, partition pruning, and BI Engine caching.
BI Engine Reservations Before Dashboard Go-LiveDashboards on BigQuery without BI Engine reservations degrade under concurrent user load. Symhas configures BI Engine reservations and validates sub-second query response before dashboard go-live — not after users complain.
dbt Medallion Before Analysts Get AccessAnalysts writing queries directly on raw BigQuery tables create inconsistent metrics that differ by team. Symhas builds dbt gold layer marts before the first analyst query runs in production — one definition of every metric.
Exactly-Once Semantics Configured, Not AssumedDataflow streaming pipelines without exactly-once configuration silently duplicate records during worker restarts. Symhas configures exactly-once delivery on every streaming pipeline — duplicates are prevented, not cleaned up after the fact.
Cost Optimisation Built Into Schema DesignBigQuery costs are almost entirely driven by bytes scanned. Symhas designs partition keys and clustering columns based on the actual query patterns — cost reduction from schema design, not from adding monitoring after the bills arrive.
Data Engineering Team Certified on dbt and DataflowBy handover your data engineering team builds new dbt models, deploys Dataflow pipelines, and manages BigQuery independently. Certified and operating solo before Symhas steps back.
Next Step
Tell Us What Data Sources You Are Trying to Unify.
We Will Design the BigQuery Platform That Connects Them.
A 30-minute GCP data assessment with a Symhas BigQuery architect. We will review your source systems, data volumes, and reporting requirements — and design the data platform architecture before the engagement price is agreed.No commitment. No pitch deck. An honest conversation about your data and AI ambitions.