Senior Data Engineer - Databricks

Rutherford, NJ/ Jersey City, NJ

• Build scalable, production-grade ETL/ELT pipelines using Databricks (PySpark, Spark SQL, Delta Live Tables, Workflows).
  
• Ingest structured, semi-structured, and streaming data into Bronze, Silver, and Gold layers.
  
• Develop optimized transformations, data quality rules, and reusable framework components.
  
• Implement best practices for job orchestration, monitoring, alerting, and automation.
  
• Hands-on experience: Spark, Delta Lake, Workflows, Unity Catalog.
  
• Strong SQL programming and performance tuning skills.
  
• Experience with cloud environments (AWS/Azure/GCP).
  
• Experience with modern data lakehouse concepts and distributed systems.
  
• Strong understanding of Lakeflow Connect, LSDP/Lakehouse, Medallion Architecture, Data Validations, Genie, and Agent Bricks/RAG use cases.
  
• Should be able to explain these concepts using real project examples and architecture decisions.
  
• Knowledge of medallion architecture, DLT and unity catalog within Databricks.
  

•  Strong Python (PySpark) and SQL programming
  
•  Databricks — Spark, Delta Lake, Workflows, Unity Catalog
  
•  ETL/ELT pipeline development — Medallion Architecture (Bronze/Silver/Gold)
  
•  Delta Live Tables, Auto-Loader, Structured Streaming
  
•  Data modeling — dimensional (star/snowflake), normalization/denormalization
  
•  CI/CD, Git, job orchestration
  
•  Cloud experience — AWS, Azure, or GCP
  
•  7–10+ years in data engineering
  
• Knowledge of medallion architecture, DLT and unity catalog within Databricks.
  

Nice-to-Have Skills

•  Lakeflow Connect, LSDP/Lakehouse, Genie, Agent Bricks/RAG use cases
  
•  Data governance, metadata management, Unity Catalog advanced features
  
•  Airflow, dbt, or similar orchestration tools
  
•  Data security, compliance, and access models
  
•  Cost optimization and performance tuning in cloud environments
  
•  Corporate/enterprise data warehousing background