Data Engineer

DKatalis

We are seeking a hands-on senior data engineer to help us build out and manage our data infrastructure, which will need to operate reliably at scale using a high degree of automation in setup and maintenance. The role will involve both setting up and managing the data infrastructure, as well as building and optimizing key ETL pipelines on both batch and streaming data. The ability to work with the teams from product, engineering, BI/analytics and data science is essential. Ownership needs to be taken of data model design and data quality. Automation and the use of data science to manage and improve data quality would be valued. The individual will also play active roles in ensuring data governance policies and tooling are implemented and adhered to


The individual will also need to be able to manage multiple stakeholders at an executive level and make well informed architectural choices when required. A high degree of empathy is required for the needs of the downstream consumers of the data artefacts produced by the data engineering team, i.e. the software engineers, data scientists, business intelligence analysts, etc and the individual needs to be able to produce transparent and easily navigable data pipelines. Value should be assigned to consistently producing high quality metadata to support discoverability and consistency of calculation and interpretation.


Candidates should have a wide set of experience across the following systems and languages:



  • Apache Kafka

  • Apache Flink

  • Apache Airflow

  • Cloud data warehouses such as Redshift or BigQuery

  • Python and Java

Apply

👉 Please mention in your application that you found the job on pyremote, this helps us get more companies to post here!

This job is sourced from Stack Overflow Jobs. When clicking on the button to apply above, you will leave pyremote and go to the job application page. pyremote accepts no liability or responsibility as a consequence of any reliance upon information on there (external sites) or here.