Databricks Solution Architect

Clearstead Advisors is seeking an experienced Databricks Solution Architect to lead the design, development, and ongoing management of our enterprise data and AI platform. As a founding member of our AI & Data team, this individual will establish architecture standards, develop scalable data solutions, and ensure the security, reliability, and performance of our Databricks environment.

This is a highly technical, hands-on position combining strategic architecture leadership with active data engineering responsibilities. The successful candidate will collaborate with internal stakeholders, technology partners, and fellow team members to advance Clearstead's data capabilities and AI initiatives.

KEY RESPONSILBILITIES

Data Architecture & Platform Management

• Lead the architecture and development of our Azure Databricks platform, including Delta Lake, Unity Catalog, and medallion architecture.

• Establish technical standards, security controls, and scalable data integration frameworks.

• Oversee platform performance, monitoring, operational readiness, and cost optimization.

• Manage the transition of platform ownership from external implementation partners.

Data Engineering & Integration

• Design, build, and maintain production data pipelines using PySpark, SQL, and Databricks Workflows.

• Integrate information from Salesforce, Orion, Addepar, PARis, and other enterprise systems.

• Develop data transformation, reconciliation, and quality assurance processes.

• Implement automated testing, deployment, and infrastructure management practices.

AI Enablement & Data Innovation

• Develop data infrastructure supporting AI applications, intelligent agents, and advanced analytics.

• Support document ingestion, vector search, model serving, and retrieval-augmented generation.

• Collaborate with the AI Engineer and Data Management Lead to deliver secure, scalable solutions.

Technical Leadership & Collaboration

• Serve as the technical design authority for the enterprise data platform.

• Evaluate vendor solutions and oversee implementation partner deliverables.

• Partner with business and technology leaders to translate requirements into effective solutions.

• Mentor developing engineers and promote engineering best practices.

Role Scope and Working Relationships

• Direct Reports: None

• Decision-making responsibility: Recommendations with approval

• Key internal partner(s): IT Partners

• Key external relationship(s): Vendors, or other external partners

• Primary systems/tools: Databricks, Azure, Databases, Other Dev tools

• Travel Requirements: Limited to None