Data Engineer

  • Shah Alam, Selangor
  • Permanent
  • Full-time
  • 1 month ago
We are seeking an experienced Data Engineer with 2 years of professional experience in building and maintaining data pipelines and working with cloud services. The ideal candidate will have strong technical skills in SQL, experience with cloud platforms (preferably AWS), and the ability to develop and optimize scalable data solutions. Experience with workflow orchestration tools like Apache Airflow, knowledge of CI/CD practices, and expertise in data ingestion into data marts for analytics and reporting are highly desirable. ROLES & RESPONSIBILITIES: Design, develop, and maintain robust ETL/ELT pipelines for large-scale data processing. Manage and optimize data pipelines using SQL and cloud technologies. Develop and manage DAGs (Directed Acyclic Graphs) in Apache Airflow for workflow orchestration and scheduling. Work with AWS services such as S3, Lambda, Redshift, and RDS to store, process, and transform data. Design and implement efficient data ingestion strategies from diverse sources into data marts to support business intelligence and reporting needs. Collaborate with data scientists and analysts to understand data requirements and ensure data is accurate, accessible, and secure. Write complex SQL queries to extract, transform, and analyze data for reporting and analytics. Implement CI/CD pipelines to automate testing, deployment, and monitoring of data workflows. Monitor data pipeline performance and troubleshoot issues to ensure data accuracy and timeliness. Optimize data storage and retrieval strategies for performance and scalability. Maintain data integrity and compliance with data governance policies. TO BE A SUCCESSFUL CANDIDATE: 2 years of hands-on experience in data engineering or related fields. Strong proficiency in SQL and experience working with relational databases (e.g., PostgreSQL, MSSQL). Experience with AWS cloud services (e.g., S3, Redshift, Lambda, RDS). Experience developing and managing workflows using Apache Airflow . Solid experience in data ingestion, transformation, and loading into data marts or analytical databases. Familiarity with CI/CD tools and practices (e.g., Git, GitHub Actions, Jenkins, or similar). Familiarity with ETL/ELT processes, data pipeline frameworks and Medallion Architecture (tiered architecture). Experience with data warehousing concepts and tools. Strong analytical skills and the ability to work with large datasets. Proficiency in Python or another scripting language is a plus. Good understanding of data security and governance best practices. Bachelor's degree in Computer Science, Engineering, or a related field. Show more Show less

foundit

Similar Jobs

  • Hiring Data Engineer

    • Shah Alam, Selangor
    • RM 5,000-6,500 per month
    Description We are seeking a talented and experienced Hiring Data Engineer to join our team in Southeast Asia. The ideal candidate will have a strong background in data engineering…
    • 8 days ago
  • Data Engineer Analyst I

    • Shah Alam, Selangor
    As a Data Engineer Analyst I you will responsible for building, maintaining, optimizing, and overseeing data retrieval, storage, and distribution to support the business requiremen…
    • 1 month ago
  • Data Engineer Analyst I

    • Shah Alam, Selangor
    As a Data Engineer Analyst I you will responsible for building, maintaining, optimizing, and overseeing data retrieval, storage, and distribution to support the business requiremen…
    • 1 month ago