Senior Big Data Engineer

Kubevisor

Description

A successful candidate will be a person who enjoys diving deep into data, doing analysis, discovering root causes, and designing long-term solutions. It will be a person that wants to innovate in the world of cloud big data engineering and machine learning. Major responsibilities include:



  • Understand the client's business need and guide them to a solution using Apache Spark, Hadoop, Kubernetes, AWS etc.

  • Lead the customer projects by being able to deliver a Spark project on AWS from beginning to end, including understanding the business need, aggregating data, exploring data and deploying to AWS (EMR, S3, Step Functions etc.) to deliver business impact to the organization.


 
Basic Qualifications



  • Degree in computer science or a similar field

  • 5+ years work experience in big data engineering

  • Experience in managing and processing data in a data lake

  • Able to use a major programming, preferably Scala/Python (preference in that order) on Spark to process data at a massive scale

  • Experience working with a wide range of big data tools, especially Spark on AWS



Preferred Qualifications



  • Experience with processing large datasets with Spark on AWS

  • Experience in data modelling, ETL development, and data warehousing.

  • Experience building/operating highly available, distributed systems of data extraction, ingestion, and processing of large data sets

  • Experience developing software projects

  • Experience using Linux to process large data sets

  • Combination of deep technical skills and business savvy enough to interface with all levels and disciplines within our client's organization

  • Demonstrable track record of dealing well with ambiguity, prioritizing needs, and delivering results in a dynamic environment


Ideally you are in the GMT to GMT+4 timezone.

Apply

👉 Please mention in your application that you found the job on pyremote, this helps us get more companies to post here!

This job is sourced from Stack Overflow Jobs. When clicking on the button to apply above, you will leave pyremote and go to the job application page. pyremote accepts no liability or responsibility as a consequence of any reliance upon information on there (external sites) or here.