Data Engineering Lead

Location: Warren, NJ or White Plains, NY

Time Type: Full-Time; Hybrid In-Office 3 days/Remote 2 days

About the Role

We are seeking a highly skilled and experienced hands-on Senior Data Engineer to lead the design, development, and optimization of scalable data pipelines using Databricks on Azure. This role will be instrumental in driving data architecture initiatives, enhancing data quality, and enabling advanced analytics across the enterprise. 

What You'll Do

• Databricks Development & Optimization: Build and optimize distributed data processing jobs using Apache Spark on Databricks. Implement Delta Lake, DLT pipelines, dbt transformations and Medallion architecture for scalable and reliable data workflows. 

• ETL & Data Integration: Design and automate ETL pipelines using Azure Data Factory, Databricks, and Synapse Analytics. Integrate data from diverse sources including Duck Creek, Intacct, Workday and external APIs. 

• Data Modeling & Warehousing: Develop dimensional models (Star/Snowflake schemas), stored procedures, and views for data warehouses. Ensure efficient querying and transformation using SQL, T-SQL, and PySpark. 

• Cloud & DevOps Integration: Leverage Azure DevOps, CI/CD pipelines, and GitHub for version control and deployment. Utilize Azure Logic Apps and ML Flow for workflow automation and model training. 

• Security & Governance: Implement role-based access control (RBAC), data encryption, and auditing mechanisms. Ensure compliance with enterprise data governance policies. 

• Collaboration & Leadership: Work closely with data scientists, analysts, and business stakeholders to deliver high-quality data solutions. Mentor junior engineers and contribute to code reviews and architectural decisions. 

What You'll Need

• Bachelor’s or Master’s degree in Computer Science, Engineering, or related field. 

• 5+ years of experience in data engineering with at least 2 years on Databricks. 

• Proficiency in Python, Scala, SQL, and Spark. 

• Hands-on experience with Azure Data Services (ADF, ADLS, Synapse). 

• Strong understanding of ETL, data warehousing, and data modeling concepts. 

• Experience with Microstrategy, Power BI, including DAX and advanced visualizations. 

• Familiarity with MLflow, LangChain, and LLM integration is a plus. 
• Knowledge of Duck Creek a plus.
• Insurance Domain knowledge preferred.

Preferred Certifications 

• Databricks Data Engineering Professional 

• Azure/AWS Data Engineering Certifications 

‌

WHAT WE DO

We're a member-owned property and casualty insurer designed exclusively for financially successful families and driven by a purpose of doing what is right for our members.  Our reciprocal model focuses on service and doing what is right for the membership: we provide exceptional service, hospitality and care, we partner with our members to help prevent losses and we create smart insurance solutions at fair prices.

We aim for our members to love their insurance. It is our mission is to create a membership experience so compelling that ​our members never want to leave.

WHO WE ARE

We want to be transparent about what we expect from each other.  From PURE, you can expect:

Opportunities to stretch and grow: your professional and personal development matters to us. We’re committed to providing experiences through on-the-job learning and professional development that increase your impact and rewards.

Clarity and kindness: you can rely on us to be open, honest and supportive, offering clarity on what success looks like.

Support in good times and bad: we believe in showing up for each other consistently, not only when it’s easy. You can expect a thoughtful partner, even when we disagree.

A community that cares: we are committed to sustaining a community in which each person feels cared for as an individual. We lift each other up, celebrate wins together and support one another through challenges in work and life.

WHO YOU ARE

All of the strongest relationships are a partnership- a two way street. So here’s what we ask of you:

• Aim to bring your best every day: you’re here because you want to be part of a team that makes a real impact and aims high.

• Be a student and a teacher: share your knowledge and talents and be willing to listen and learn from those around you.

• Get comfortable being uncomfortable: we face tough moments and obstacles with a “courage over comfort” approach and a positive, solutions-oriented mindset.

• Be a culture builder:  building a positive culture is everyone's responsibility, based on care, respect and openness to diverse perspectives.

‌

‌

To ensure a successful onboarding experience, all new hires must work onsite at one of our offices during their first week of employment. Candidates should apply only if they are able to meet this requirement.

‌

Want to Learn More?

• [Our Values]

• [Our Benefits]  

• [Our Community Impact]

• [Our Leadership]