Data Engineer


Leading consultancy is urgently recruiting for some experienced Data Engineers with Java OR Scala and Azure ideally or any other Cloud solutions along

  • Java\ J2EE or Scala
  • GCP or Azure (Azure is preferred)
  • Spark Batch, Spark Streaming, Hive, Oozie or Airflow, Kafka, NoSQL DBs, Mongo,
Technical Skills:
  • Working experience on the following is required:
    • Overall 8-10 years of experience and at-least 6+ years of hands-on-experience in Java\ J2EE or Scala is a must
    • Hands on experience in working with Low latency and high throughput data platforms
    • Strong working knowledge on following areas would be required:
      • Data Engineering technologies like Spark Batch, Spark Streaming, Hive, Oozie or Airflow, Kafka, NoSQL DBs, Mongo, HBase
      • Good hands on experience on Data Platforms like HDP, CDH, Databricks
      • Well versed with different data storage formats like ORC, Parquet, Avro
    • Hands on experience in developing data platform on any cloud – GCP or Azure (Azure is preferred)
    • Appreciation of DevOps, working experience on Containerized platform using Docker and Kubernetes
    • Jenkins or any other CI/ CD pipelines
    • Understanding of TDD/BDD methodologies
    • Git
    • Good knowledge about the security of data platforms – SSL, RBAC, Kerberos
  • Develop and peer review of LLD (Initiate/ participate in peer reviews) of Data pipeline designs.
  • Ability to debug data pipeline failures, suggest enhancements and apply continuous improvements to the overall Data ingestion and curation process.
  • Experience of working in a fast-paced environment.
  • Prior experience of having worked in Retail Banking domain will be a big plus.
Behavioural Skills:
  • Ensure test documentation / reporting is accurate, meaningful and aligns with buContinuesiness objectives.
  • Communicate effectively the daily status – risks / issues / dependencies, track progress and manage issue workflow to resolution.
  • Collaborate with development and business teams to ensure seamless communication and coordination with team members.
  • Participates as a team member and fosters teamwork by inter-group coordination within the modules of the project.
  • Train and coach members of project groups to ensure effective knowledge management activity.
Good communication, analytical and problem-solving abilities.