ADF Data Engineer

@ Neshent Technologies
Neshent Technologiesneshenttechnologies.com

ADF Data Engineer

Magnolia Park, Texas
Posted today

About the job

Our company focuses on data engineering solutions. This role involves designing and optimizing data pipelines using Azure Data Factory and Databricks for large-scale data processing.

Requirements

  • Azure Data Factory experience
  • Azure Databricks expertise
  • Advanced Python or Scala
  • Expert SQL skills
  • Knowledge of data storage technologies

Qualifications

  • 8-10 years professional experience
  • 3+ years Azure experience
  • Strong coding standards
  • Experience with CI/CD

Full job description

Experience : 8–10 years overall, with 3+ years of Azure experience.

Required Skills
  • Strong hands-on experience with Azure Data Factory (ADF) — pipelines, data flows, triggers, integration runtimes, parameterization, error handling, and retries.
  • Strong experience with Azure Databricks — Notebooks, Jobs/Workflows, PySpark/Scala, Delta Lake, cluster configuration, and optimization.
  • Advanced Python or Scala with strong coding standards, modularization, and unit testing (pytest).
  • Expert SQL — CTEs, window functions, query tuning, and large datasets.
  • Experience with ADLS Gen2, Delta Lake, Parquet, partitioning, Z-Ordering, OPTIMIZE, and VACUUM.
  • Experience with Git, branching strategies, and CI/CD for ADF/Databricks.
  • Exposure to Infrastructure as Code (IaC) is a plus.
Roles and Responsibilities
  • Coding & Development: Design, develop, and optimize ADF/Databricks data pipelines and solutions.
  • UI Testing: Perform functional/UI testing, data validation, and troubleshooting.
  • Stakeholder Management: Collaborate with stakeholders, understand requirements, and communicate project status/issues effectively.
Show full description

Similar jobs in Magnolia Park, Texas

See all ADF Data Engineer jobs