2327 IT & Software Developer jobs in the UK
Requirements
Must:
- Significant commercial experience in Data Engineering with years of delivering production workloads on Azure and Databricks.
- Strong expertise in PySpark, Spark SQL, and distributed data processing.
- Proven experience in Medallion/Lakehouse delivery using Delta Lake.
- Solid proficiency in dimensional modeling (Kimball), including surrogate keys, SCD types 1/2, and merge strategies.
- Operational experience with SLAs, observability, idempotent pipelines, reprocessing, and backfills.
- A strong understanding of secure Azure Landing Zone patterns.
- Comfort with Git, CI/CD, automated deployments, and modern engineering standards.
- A clear communicator capable of translating technical decisions into business outcomes.
Responsibilities
- Engineer scalable ELT pipelines using Lakeflow Declarative Pipelines, PySpark, and Spark SQL across a complete Medallion Architecture (Bronze - Silver - Gold).
- Implement ingestion patterns for files, APIs, SaaS platforms (like subscription billing), SQL sources, SharePoint, and SFTP using Azure Data Factory and metadata-driven frameworks.
- Apply Lakeflow expectations for data quality, schema validation, and operational reliability.
- Build clean, conformed Silver/Gold models aligned to enterprise business domains, including customers, subscriptions, deliveries, finance, credit, logistics, and operations.
- Deliver star schemas, harmonization logic, Slowly Changing Dimensions (SCDs), and business marts to enhance high-performance Power BI datasets.
- Apply governance, lineage, and fine-grained permissions via Unity Catalog.
- Design and optimize orchestration using Lakeflow Workflows and Azure Data Factory.
- Implement monitoring, alerting, SLAs/SLIs, runbooks, and cost optimization across the platform.
- Build CI/CD pipelines in Azure DevOps for notebooks, Lakeflow pipelines, SQL models, and Azure Data Factory artifacts.
- Ensure secure, enterprise-grade platform operation across Development and Production using private endpoints, managed identities, and Key Vault.
- Contribute to platform standards, design patterns, code reviews, and the future roadmap.
- Collaborate closely with BI and Analytics teams to deliver curated datasets that power dashboards across the organization.
- Influence architectural decisions and uplift engineering maturity within a growing data function.
Description
We are embarking on a major data modernization program, transitioning away from legacy systems, manual Excel reporting, and fragmented data sources into a fully automated Azure Enterprise Landing Zone coupled with Databricks Lakehouse. You will have a unique opportunity to join this transformation effort at an early stage, helping to shape the architecture and define engineering standards, pipelines, curated layers, and best practices that will support Operations, Finance, Sales, Logistics, and Customer Care. If youre eager to help build a best-in-class Lakehouse from scratch, then this role is designed for you. Tech Stack: - Databricks: Lakeflow Declarative Pipelines, Workflows, Unity Catalog, SQL Warehouses - Azure: ADLS Gen2, Data Factory, Key Vault, Virtual Networks & Private Endpoints - Languages: PySpark, Spark SQL, Python, Git - DevOps: Azure DevOps Repos, Pipelines, CI/CD - Analytics: Power BI, Fabric Nice to Have: - Databricks Certified Data Engineer Associate - Streaming ingestion experience (Auto Loader, structured streaming, watermarking) - Subscription/entitlement modeling experience - Advanced Unity Catalog security (RLS, ABAC, PII governance) - Terraform/Bicep for Infrastructure as Code (IaC) - Fabric Semantic Model / Direct Lake optimization
Something wrong or incorrect with this job? Tell us in the chat 💬 on the right ➡️
You can find Data Engineer salaries in the UK here.
How many Data Engineer jobs are in the UK?
Currently, there are 2327 Data openings. Check also: Spark jobs, Snowflake jobs, Kafka jobs, Hadoop jobs - all with salary brackets.
Is the UK a good place for Data Engineers?
The UK is one of the best countries to work as a Data Engineer. It has a vibrant startup community, growing tech hubs and, most important: lots of interesting jobs for people who work in tech.
Which companies are hiring for Data Engineer jobs in the UK?
Mas & Pas Limited, ProCheckUp Limited, Central Technology Ltd, EG Group, LendingCrowd, DCBL, Computershare among others, are currently hiring for Data roles in the UK.
The company with most openings is Experis as they are hiring for 90 different Data Engineer jobs in the UK. They are probably quite committed to find good Data Engineers.
The company with most openings is Experis as they are hiring for 90 different Data Engineer jobs in the UK. They are probably quite committed to find good Data Engineers.