Job Summary
We are seeking an Azure Databricks Solution Architect to lead the architecture, design, and implementation of scalable, secure, and high-performance data platforms on Microsoft Azure. This is a hands-on role requiring both architectural leadership and direct technical contribution to solution design, development, optimization, and delivery. The ideal candidate will drive platform strategy while building proof-of-concepts, reference architectures, and production-grade data solutions.
Roles and Responsibilities
-
Own the end-to-end architecture of Azure Databricks-based data platforms, including data ingestion, transformation, storage, governance, and consumption layers.
-
Design reusable data frameworks, ELT patterns, and scalable data models.
-
Develop and optimize Spark jobs and data pipelines with a focus on performance, scalability, reliability, and cost optimization.
-
Architect solutions using Azure Databricks, Delta Lake, Azure Data Lake Storage (ADLS), Azure Data Factory (ADF), and other Azure services.
-
Define platform governance, security, coding standards, and best practices.
-
Collaborate with business analysts, functional teams, and engineering teams to translate business requirements into technical solutions.
-
Conduct architecture reviews, code reviews, troubleshooting, and production support for complex data workloads.
-
Drive cloud modernization initiatives and adoption of Azure-native data and analytics services.
Required Qualifications
-
12+ years of experience in Data Engineering, Data Architecture, or Platform Engineering.
-
Strong hands-on experience with Azure Databricks, Apache Spark, Python or Scala, SQL, and Delta Lake.
-
Proven experience designing and implementing enterprise data lakes, cloud data platforms, and analytical data stores.
-
Experience with Azure DevOps, CI/CD pipelines, and automated deployment practices.
-
Strong hands-on implementation experience with enterprise-scale Azure data solutions.
-
Excellent analytical, problem-solving, communication, and stakeholder management skills.
Preferred Qualifications
-
Experience in the Financial Services or Insurance domain.
-
Knowledge of Azure Event Hubs, Azure Cosmos DB, Azure Synapse Analytics, or other Azure analytics services.
-
Microsoft Azure and/or Databricks certifications.
-
Experience developing reusable frameworks, shell scripting, and supporting large-scale production environments.