3045 IT & Software Developer jobs in the UK
Data Engineer Lead
£117,000 - 124,800
Infoplus Technologies UK Ltd
Manor Way Unit 12, Sheffield
£117,000 - 124,800
Requirements
Must:
- Hands-on experience building streaming data pipelines with Kafka (producers/consumers, schema registry, Kafka Connect/KSQL/KStream)
- Proficiency with OpenShift/Kubernetes telemetry (OpenTelemetry, Prometheus) and CLI tooling
- Experience integrating telemetry into Splunk (HEC, UF, sourcetypes, CIM), building dashboards and alerting
- Strong data engineering skills in Python (or similar) for ETL/ELT, enrichment, and validation
- Knowledge of event schemas (Avro/Protobuf/JSON), contracts, and backward/forward compatibility
- Familiarity with observability standards and practices; ability to drive toward Level 4 maturity (proactive monitoring, automated insights)
- Understanding of hybrid cloud and multi-cluster telemetry patterns
- Knowledge of security and compliance for data pipelines: secret management, RBAC, encryption in transit/at rest
- Good problem-solving skills and ability to work in a collaborative team environment
- Strong communication and documentation skills
Responsibilities
- Design, implement, and maintain data pipelines to ingest and process OpenShift telemetry (metrics, logs, traces) at scale
- Stream OpenShift telemetry via Kafka (producers, topics, schemas) and build resilient consumer services for transformation and enrichment
- Engineer data models and routing for multi-tenant observability; ensure lineage, quality, and SLAs across the stream layer
- Integrate processed telemetry into Splunk for visualization, dashboards, alerting, and analytics to achieve Observability Level 4 (proactive insights)
- Implement schema management (Avro/Protobuf), governance, and versioning for telemetry events
- Build automated validation, replay, and backfill mechanisms for data reliability and recovery
- Instrument services with OpenTelemetry; standardize tracing, metrics, and structured logging across platforms
- Use LLMs to enhance observability capabilities (e.g., query assistance, anomaly summarization, runbook generation)
- Collaborate with platform, SRE, and application teams to integrate telemetry, alerts, and SLOs
- Ensure security, compliance, and best practices for data pipelines and observability platforms
- Document data flows, schemas, dashboards, and operational runbooks
Description
We are a technology-forward company committed to enhancing our observability capabilities and enabling our teams to deliver high-quality products. Our focus is on building and maintaining scalable data pipelines that ensure reliability and compliance in processing telemetry data. As part of our dynamic team, you will have the opportunity to work alongside skilled platform, SRE, and application teams in a collaborative environment, with a strong emphasis on security and best practices. We offer competitive benefits and a supportive work culture, allowing for professional development and career growth.
Something wrong or incorrect with this job? Tell us in the chat 💬 on the right ➡️
You can find Data Engineer salaries in the UK here.
How many Data Engineer jobs are in the UK?
Currently, there are 3045 Data openings. Check also: Spark jobs, Snowflake jobs, Kafka jobs, Hadoop jobs - all with salary brackets.
Is the UK a good place for Data Engineers?
The UK is one of the best countries to work as a Data Engineer. It has a vibrant startup community, growing tech hubs and, most important: lots of interesting jobs for people who work in tech.
Which companies are hiring for Data Engineer jobs in the UK?
Bluefruit Software Limited, Roots Plants, OKTO Integrated Services, EG Group, RevOps Automated, Leger Shearings Group, Bank of America among others, are currently hiring for Data roles in the UK.
The company with most openings is Experis as they are hiring for 92 different Data Engineer jobs in the UK. They are probably quite committed to find good Data Engineers.
The company with most openings is Experis as they are hiring for 92 different Data Engineer jobs in the UK. They are probably quite committed to find good Data Engineers.