Databricks Data & AI Analytics Architect

Role Description:

We are seeking an experienced Data & AI Analytics Architect / Lead Engineer to modernize enterprise analytics platforms and drive adoption of AI-powered analytical solutions on Databricks. The role focuses on building scalable data ingestion frameworks, metadata-driven architectures, conversational analytic capabilities, and governed data products that improve business decision-making and reduce time to insight.

The organization operates multiple reporting and analytics solutions across Trading, Risk, Finance, and Operations using a mix of legacy data warehouse technologies and custom data pipelines.

Current challenges include:

·       Fragmented ingestion and transformation patterns resulting in higher maintenance effort.

·       Ongoing migration of analytical workloads from legacy platforms to Databricks.

·       Increasing demand for self-service analytics, conversational data access, and AI-assisted decision support.

·       Need for improved data quality, lineage, governance, and auditability.

·       Requirement to accelerate insight generation and improve analytical scalability, reliability, and performance.

Skills:

·       Strong expertise in Databricks, Spark, Delta Lake, Unity Catalog, SQL Warehouses, Databricks Workflows, Databricks Genie, and Genie One.

·       Experience designing enterprise-scale data ingestion, ETL/ELT, metadata-driven, and watermark-based processing frameworks.

·       Experience modernizing and migrating workloads from legacy data warehouses to Databricks.

·       Strong SQL skills including query optimization, semantic modeling, and metadata-driven analytics.

·       Experience building AI/LLM-powered analytics solutions, conversational interfaces, and agent-based architectures.

·       Experience developing analytical assistants/copilots leveraging business metadata, governance rules, and semantic context.

·       Ability to design conversational data exploration with drill-down analysis, context retention, and root-cause investigation.

·       Experience with automated insight generation including KPI monitoring, trend analysis, anomaly detection, correlation analysis, and executive summaries.

·       Good understanding of cloud data and analytics architectures covering ingestion, transformation, modeling, and serving layers.

·       Knowledge of visualization and dashboarding concepts enabling business-friendly analytical experiences.

·       Ability to work effectively across business and technology teams to improve data availability, reliability, and performance.

·       Familiarity with Trading, Risk, Finance, Operations, or Supply Chain domains is highly desirable.