Senior Data Engineer

Comment: 0   

Sand Technologies

Nairobi, Cairo, Lagos, Johannesburg

Egypt, Kenya, South Africa, Nigeria

Min 7 yrs experience

70k-95k USD per annum

https://www.sandtech.com/

What we are looking for

Sand Technologies is looking for a Senior Data Engineer. the Senior Engineer will focus on building the data infrastructure for a best-in-class intelligence platform and be responsible for ensuring data availability, quality, and security.

What does the job involve

  • Data Pipeline Development: Lead the design, implementation, and maintenance of scalable data pipelines using tools such as Databricks, Python, and PySpark.
  • Data Architecture: Architect scalable and efficient data solutions using modern architectures.
  • Data Modeling: Design and optimize data models and schemas for efficient storage, retrieval, and analysis of structured and unstructured data.
  • ETL Processes: Develop, optimize, and automate ETL workflows.
  • Big Data Technologies: Utilize technologies like Spark, Kafka, and Flink for distributed data processing and analytics.
  • Cloud Platforms: Deploy and manage data solutions on AWS, Azure, or Google Cloud Platform (GCP).
  • Data Quality and Governance: Implement and oversee data governance, quality, and security measures.
  • Monitoring, Optimization, and Troubleshooting: Monitor and optimize pipeline performance, troubleshoot issues.
  • DevOps: Build and maintain CI/CD pipelines, commit code to version control, and deploy data solutions.
  • Best Practices: Stay current with emerging technologies and best practices in data engineering, cloud architecture, and DevOps.
  • Technology Selection: Evaluate and recommend technologies, frameworks, and tools.
  • Performance Optimization: Optimize software performance through architectural design decisions and performance tuning.

Qualifications

  • Programming Languages: Proficiency in Python, Java, Scala, or SQL.
  • Data Modeling: Strong understanding of data modeling concepts and techniques.
  • Big Data Technologies: Experience with Databricks, Spark, Kafka, and Flink.
  • Modern Data Architectures: Experience with lakehouse architectures.
  • CI/CD and Version Control: Experience with CI/CD pipelines, Git, and containerization (e.g., Docker).
  • ETL Tools: Knowledge of tools like Apache Airflow, Informatica, or Talend.
  • Data Governance: Strong understanding of data governance and best practices.
  • Cloud Platforms: Experience with AWS, Azure, or GCP services.
  • SQL and Real-Time Data Streaming: Proficiency in SQL and real-time data streaming technologies like Apache Spark Streaming and Kafka.
  • Cloud Data Tools: Experience using data tools in at least one cloud service - AWS, Azure, or GCP (e.g., S3, EMR, Redshift, Glue, Azure Data Factory, Databricks, BigQuery, Dataflow, Dataproc).

Benefits

  • TBD
     
    Contact: admin@globaljobservices.vn

0 comments

Write a comment

Note: HTML is not translated!