Senior Data Engineer

- Monitor and support daily ETL processes, data pipelines, and batch jobs to ensure timely and accurate data delivery. - Troubleshoot and resolve production issues, job failures, and performance bottlenecks across ETL and data warehouse systems. - Work closely with the Data platform team to resolve data load issues. - Perform root cause analysis of recurring issues and implement permanent fixes. - Collaborate with development teams to transition projects smoothly into production and ensure operational readiness. - Implement and maintain monitoring, alerting, and logging solutions for proactive issue detection. - Ensure data quality, consistency, and availability through ongoing validation and health checks. - Apply best practices for production support, including incident management, change management, and problem management. - Work closely with business users, data analysts, and other stakeholders to resolve data-related queries. - Document runbooks, support procedures, and knowledge base articles to streamline production operations. - Continuously optimize processes for reliability, performance, and scalability in production environments. - Ensure compliance with data security, access controls, and audit requirements in production systems. - Check system dashboards, logs, and alerts for failures or anomalies. - Verify data quality and integrity checks (row counts, duplicates, missing data, schema changes). - Review ETL/ELT job runs, data pipeline executions, and batch processes. - Validate data loads into staging, warehouse, and downstream systems for critical tables. - Monitor real-time and scheduled jobs to ensure SLAs are met. - Investigate and resolve production issues (job failures, data inconsistencies, performance delays). - Collaborate with business users to resolve data access or reporting issues. - Coordinate with development/engineering teams for fixes, hot patches, or re-runs of failed jobs. - Track and document incidents, resolutions, and preventive measures in ticketing systems. - Participate in daily/weekly operations meetings to report status and highlight issues. - Handover critical ongoing issues to on-call/offshore support (if applicable). - Enhance existing models with the addition of fields as per the requirements. - Help with deployments and initial loads during Go-live. - Perform root cause analysis for recurring or high-severity incidents. - Fine-tune ETL workflows and SQL queries to improve performance. - Implement monitoring scripts and automation to reduce manual intervention. - Restructure the load plans to improve efficiency. - Review security and access controls to ensure compliance. - Update documentation (runbooks, troubleshooting guides, SOPs) for operational continuity.