1200 IT & Software Developer jobs in the UK
PySpark Developer - South East London
£78,000 - 91,000
Randstad Digital
Hopton Street, South East London
£78,000 - 91,000
Requirements
Must:
- 5 years of hands-on experience writing scalable, production-grade PySpark/Spark SQL
- Strong proficiency in AWS Data Stack, including EMR, Glue, S3, Athena, and Glue Workflows
- Solid foundation in SAS for understanding and debugging legacy logic
- Expertise in ETL/ELT, dimensions, facts, SCDs, and data mart architecture
- Experience with parameterisation, exception handling, and modular Python design
Responsibilities
- Lead the end-to-end migration of SAS code (Base SAS, Macros, DI Studio) to PySpark using automated tools (SAS2PY) and manual refactoring
- Design, build, and troubleshoot complex ETL/ELT workflows and data marts on AWS
- Optimise Spark workloads for execution efficiency, partitioning, and cost-effectiveness
- Implement clean coding principles, modular design, and robust unit/comparative testing to ensure data accuracy
- Maintain Git-based workflows, CI/CD integration, and comprehensive technical documentation
Description
We are seeking a Lead PySpark Engineer to drive a large-scale data modernisation project, transitioning legacy data workflows into a high-performance AWS cloud environment. This is a hands-on technical role focused on converting legacy SAS code into production-ready PySpark pipelines within a complex financial services landscape. The role is fully remote, offering 33 days of holiday entitlement (pro-rata), and is a great opportunity to collaborate internally with our team.
Something wrong or incorrect with this job? Tell us in the chat 💬 on the right ➡️
You can find DevOps salaries in the UK here.
How many DevOps jobs are in the UK?
Currently, there are 1200 DevOps openings. Check also: Cloud jobs, AWS jobs, Azure jobs, GCP jobs, Kubernetes jobs, Docker jobs, Terraform jobs - all with salary brackets.
Is the UK a good place for DevOps?
The UK is one of the best countries to work as a DevOps. It has a vibrant startup community, growing tech hubs and, most important: lots of interesting jobs for people who work in tech.
Which companies are hiring for DevOps jobs in the UK?
SwapTix, Arctic Shores, TBG Solutions Ltd, Companies House, FCDO Services, MOD - Defence Digital, Counter Terrorism Policing among others, are currently hiring for DevOps roles in the UK.
The company with most openings is Morson Edge as they are hiring for 38 different DevOps jobs in the UK. They are probably quite committed to find good DevOps.
The company with most openings is Morson Edge as they are hiring for 38 different DevOps jobs in the UK. They are probably quite committed to find good DevOps.