4512 IT & Software Developer jobs in the UK
Requirements
Must:
- Strong capability in at least three areas of engineering practice
- Experience with Databricks (Spark, PySpark, Delta Lake, Unity Catalog advantageous)
- Proficiency in Azure data services (Azure Data Factory, Data Lake, Azure Functions advantageous)
- Experience contributing to CI/CD pipelines (Azure DevOps, GitHub Actions, Terraform)
- Scripting and programming skills (Python advantageous)
- Good understanding of DevOps and automation concepts (e.g. YAML pipelines, Infrastructure as Code)
- Solid understanding of cloud security, compliance and governance principles
- Degree level IT or technical/scientific subject (or equivalent experience)
- Microsoft Azure Data Engineer or Solutions Architect certification (desirable)
- Databricks Certified Data Engineer or Machine Learning Associate (desirable)
- Experience with streaming solutions (Kafka, Event Hubs, Spark Streaming) (desirable)
- Knowledge of machine learning and AI on Databricks (desirable)
Responsibilities
- Interpret outcomes and user stories, translating them into technical solutions
- Create innovative solution designs for domain and enterprise data products
- Oversee Data Analysts to support detailed data discovery
- Oversee data modelling for finance and enterprise data products
- Lead Product Increment planning to break down solutions into features and epics for incremental delivery
- Design and implement scalable data solutions on Azure and Databricks within the assigned domain
- Ensure appropriate engineering standards are applied to maintain data quality, performance, and reliability
- Work with Product Owners and Business Analysts to understand outcomes, refining user stories
- Direct and collaborate with Data Analysts on detailed data discovery and requirements refinement
- Oversee logical and physical data modelling for finance and enterprise data products, working closely with architecture as needed
- Implement and maintain data pipelines and ETL workflows in Databricks (PySpark, Delta Lake)
- Contribute to CI/CD pipelines for data applications using Azure DevOps and infrastructure-as-code (Terraform)
- Apply security, access control, and compliance standards for Azure and Databricks in collaboration with platform and security teams
- Support monitoring, logging, and basic cost optimisation for the teams data products
- Support the development of DevOps practices within the team, including reducing technical debt and improving automation over time
Description
We are seeking a Lead Data Engineer to join our team in London, Watford, or Bristol, with hybrid working options available. This role involves hands-on technical leadership in Azure cloud and Databricks-based solutions within our Enterprise Data Platform. Youll lead a cross-functional team to build, deploy, and support high-performance data-driven solutions. We offer a dynamic work environment focusing on collaboration and innovation, with opportunities for professional development and growth.
Something wrong or incorrect with this job? Tell us in the chat 💬 on the right ➡️
You can find Data Engineer salaries in the UK here.
How many Data Engineer jobs are in the UK?
Currently, there are 4512 Data openings. Check also: Spark jobs, Snowflake jobs, Kafka jobs, Hadoop jobs - all with salary brackets.
Is the UK a good place for Data Engineers?
The UK is one of the best countries to work as a Data Engineer. It has a vibrant startup community, growing tech hubs and, most important: lots of interesting jobs for people who work in tech.
Which companies are hiring for Data Engineer jobs in the UK?
Collect Solutions Ltd, Arctic Shores, SwapTix, HM Land Registry, Wilson James, Companies House, Hamilton Jones Consulting among others, are currently hiring for Data roles in the UK.
The company with most openings is Experis as they are hiring for 111 different Data Engineer jobs in the UK. They are probably quite committed to find good Data Engineers.
The company with most openings is Experis as they are hiring for 111 different Data Engineer jobs in the UK. They are probably quite committed to find good Data Engineers.