Lead Data Engineer

- Extensive enterprise experience with Hadoop, Spark, and Splunk. - Proficiency in object-oriented and functional scripting, particularly in Python. - Skilled in handling raw, structured, semi-structured, and unstructured data (SQL and NoSQL). - Experience integrating large, disparate datasets using modern tools and frameworks. - Strong background in building and optimizing ETL/ELT data pipelines. - Familiarity with source control and implementing Continuous Integration, Delivery, and Deployment via CI/CD pipelines. - Experience supporting and collaborating with BI and Analytics teams in fast-paced environments. - Ability to pair program and work effectively with other engineers. - Excellent analytical and problem-solving abilities. - Knowledge of agile methodologies such as Scrum or Kanban is a plus. - Comfortable representing the team in standups and problem-solving sessions. - Capable of driving the creation of technical test plans and maintaining records, including unit and integration tests, within automated test environments to ensure high code quality. - Promote SRE (Site Reliability Engineering) culture by addressing challenges through data engineering. - Ensure service resilience, sustainability, and adherence to recovery time objectives for all delivered software solutions. - Demonstrated ability and enthusiasm for enhancing team performance. - Strong active listening and effective communication skills. - Self-mastery, with a focus on positive mindsets and professional behaviours. - Maintains up-to-date expertise in current tools, technologies, and key areas such as cybersecurity, data privacy, consent, and data residency regulations. - Engages with industry groups and external vendors to represent and advance our interests and influence. - Takes accountability for ensuring control and compliance throughout the engineering process. - Champions innovation and the adoption of advanced technologies and best practices within the domain.