AWS Integration Engineer

- Experience in deploying comprehensive cloud infrastructure for various products, including Astronomer Airflow and AccelData environments. - Strong ability to facilitate cross-functional integration between vendor products and other systems, such as data lakes, storage, and compute services. - Knowledge of best practices for cloud security, scalability, and performance. - Proficiency in managing and configuring vendor product deployments, ensuring the setup and maintenance of environments. - Proven skills in ensuring high availability, scalability, and fault tolerance of Airflow clusters. - Experience implementing monitoring, alerting, and logging for Airflow and related components. - Ability to perform upgrades and patches for platform-related components. - Skills in capacity planning, resource allocation, and optimization of Airflow workers. - Experience maintaining and configuring integrations with source control systems (e.g., GitHub, GitLab) for version control. - Familiarity with collaborating with cloud providers (e.g., AWS) for pipeline integration and scaling requirements. - Ability to identify, design, and implement internal process improvements, including redesigning infrastructure for greater scalability, optimizing data delivery, and automating manual processes. - Experience in developing infrastructure for optimal extraction, transformation, and loading of data from various sources using AWS and SQL technologies. - Strong collaboration skills with stakeholders, including design, product, and executive teams, to address platform-related technical issues. - Proven ability to build analytical tools to leverage the data pipeline, providing actionable insights into key business performance metrics.