Databricks Data Engineer

We are seeking a Senior Databricks Data Engineer with strong experience designing, developing, and optimizing data engineering solutions using Databricks, PySpark, Delta Lake, and modern data engineering practices. The ideal candidate will have expertise in data pipelines, cloud data platforms, governance, automation, and CI/CD processes.

Roles & Responsibilities

• Design, develop, and maintain scalable data pipelines using Databricks and PySpark.
• Build and optimize Delta Lake solutions using Delta Tables and Declarative Pipelines.
• Implement Databricks Asset Bundles for deployment and lifecycle management.
• Manage data governance, security, and permissions using Unity Catalog.
• Configure and enforce cluster policies and platform governance standards.
• Develop clean, maintainable, and reusable Python code.
• Implement CI/CD workflows using Git/GitHub best practices.
• Collaborate with cross-functional teams in an Agile environment.
• Optimize data processing performance and troubleshoot data engineering issues.
• Support automation initiatives using tools such as Terraform and Bash scripting.

Required Technical Skills

• Strong hands-on experience with Databricks Data Engineering.
• Expertise in PySpark, Delta Tables, and Declarative Pipelines.
• Experience with Databricks Asset Bundles (DAB).
• Strong knowledge of Unity Catalog, access controls, and permission management.
• Experience with cluster policies, governance, and Databricks administration best practices.
• Strong Python development skills with understanding of Object-Oriented Programming (OOP).
• Knowledge of clean code principles and software development best practices.
• Experience with CI/CD processes and automation.
• Strong understanding of Git/GitHub, including pull requests and branching strategies.
• Experience working in Agile development environments.

Preferred Skills

• Strong SQL skills.
• Experience with Terraform for infrastructure automation.
• Familiarity with Bash scripting and command-line tools.