Senior Data Engineer

Role Overview

We are seeking a highly skilled, autonomous Lead Data Engineer to architect and implement an automatedoperational integration pipeline. The effort is focused on building an automated operational integration thatextracts required data from the Eclipse data share, snapshots into an internal Ops database, generatesInvestorTools-compatible files, and securely delivers those files through an SFTP process on a scheduledbasis with an initial full file followed by ongoing delta files.The core objective of this initiative is to replace manual data movement and reconciliation activities with areliable, scalable, and fully supportable integration solution. As the Lead Engineer on this initiative, you must bea self-starter capable of driving end-to-end technical execution with minimal oversight while engaging directlywith stakeholders and client teams.

PRIMARY RESPONSIBILITIES

1. Data Integration & ETL EngineeringDesign and deploy robust ETL/ELT pipelines to extract datasets from the Eclipse data share, create historicalsnapshots within the Ops database, and implement delta/change-data-capture (CDC) extraction logic.2. Workflow OrchestrationWrite, schedule, monitor, and troubleshoot automated DAGs using orchestrators such as Apache Airflow to guaranteereliable, scheduled pipeline execution and file delivery.3. Database & SQL ManagementApply advanced SQL skills to perform complex data transformations, query data-share/warehouse environments, andmanage relational schemas within the Ops database.4. Automation & SFTP File GenerationUtilize Python to construct custom file parsing, formatting, and schema validation routines for InvestorToolscompatible file outputs and secure automated SFTP transmission.5. Client Communication & LeadershipAct as the primary technical point of contact for client stakeholders. Communicate system architecture, progress, and technical requirements clearly while working independently as a self-starter

Qualification Matrix

Minimum Qualifications5+ Years Experience: Deep hands-on experiencedesigning ETL/ELT pipelines, data snapshots, andCDC extractions.Orchestration Expertise: Practical proficiencywriting, scheduling, and troubleshooting DAGs inApache Airflow.Advanced SQL: Strong experience in relationalschema management, warehouse queries, anddata transformation.Python Scripting: Proficient in Python for customfile parsing, formatting, schema validation, andSFTP automation.Financial Domain Exposure: Direct experiencewith financial datasets (portfolio accounting,trading, or fixed income).Client Communication: Clear verbal and writtencommunication skills for direct clientengagement.Independent Worker: Strong self-starter ability tooperate with minimal dependencies orsupervision.Nice to HavePlatform Experience: Direct prior experience withEclipse data share environments orInvestorTools software integrations.Fixed-Income Analytics: Understanding of fixed income security processing, yield calculations,and bond accounting rules.Data Security: Experience with automatedencryption (PGP/SSH), secure key management,and enterprise SFTP protocols.CI/CD & DevOps: Exposure to automateddeployment pipelines (Docker, GitHub Actions,Jenkins) for data workflows.Data Quality Frameworks: Knowledge ofautomated data testing frameworks (Great expectation dbt tests)

Key Operational Objectives

Manual Effort Elimination: Completely replace existing manual spreadsheet uploads and ad-hoc reconciliationactivities with automated, supportable data pipelines.Robust Delta Processing: Ensure fault-tolerant execution of initial full data loads and continuous daily incrementaldelta generations without data loss or duplication.Stakeholder Alignment: Serve as an effective technical lead who communicates milestone status, technical risks,and interface requirements directly to client managers and engineering peers.