Senior Big Data Engineer

Senior Big Data Engineer Job Description Template

Our company is looking for a Senior Big Data Engineer to join our team.


  • Work in scrum methodology, effectively prioritise and delegate backlogged commitments;
  • Engage with liaise with customers over requirements;
  • Using customer requirements to create robust technical solutions;
  • Taking a lead and utilising the team to solve complex problems;
  • Development of the new and existing data pipelines utilizing Amazon AWS/EMR;
  • Cooperating effectively with data scientists and developer, mentor more junior data engineers;
  • Parse and handle huge data sets;
  • Manage requests and stakeholder requirements in a prioritised manner;
  • Manage data integration with third party applications;
  • Design and implement internal process improvements;
  • Deliver large scale algorithms in production;
  • Building suitable environments for deployment;
  • Build and manage data pipeline architecture.


  • Interest in machine learning and AI;
  • Knowledge of data management, handling and processing data quality, preparation, cleansing and modelling;
  • Hands-on experience with Spark, Airflow, Hadoop, or Databricks;
  • 5-7 years of experience as a data engineer or software engineer working with data;
  • Excellent knowledge of Python;
  • Significant Amazon AWS experience including EMRInterest in machine learning and AI;
  • Experience working with privacy and GDP;
  • Experience working with AWS (EMR, S3, Athena, Redshift);
  • Experience with Scala/Java and Python;
  • Previous experience working in an Agile Environment;
  • TDD and Automation;
  • Bachelor’s degree in computer science, engineering, or equivalent;
  • Experience using Spark, PySpark.