Description
We are looking for a Staff Data Engineer to lead the data engineering and data architecture behind it. You will own the data model and the pipeline contracts other teams build against, move pipelines from prototype into production the business depends on, and make ownership attribution and risk enrichment a service instead of repeated one-off work. Success here takes deep distributed data engineering experience, real data judgment, and the ability to hold a technical direction across a large, matrixed engineering organization.
Minimum Qualifications
15+ years of experience working with Spark and other distributed data technologies (e.g. Hadoop, Presto, Flink, Druid) for building efficient & large scale data pipelines Highly proficient in at least one of Java, Python or Scala Deep expertise in Data Principles, Data Architecture & Data Modeling, Strong SQL skills Strong problem solver with meticulous attention to detail, capable of taking on loosely defined problems Experience working in a complex, matrixed organization involving cross-functional, and/or cross-business projects Strong communication and collaboration skills & ability to lead high-level discussions on technology strategy and approach Conceptually familiar with AWS cloud resources (S3, EC2, RDS etc) Conceptually familiar with OSI model and understanding of how network works (Load Balancer, Routers, network tagging, NetFlow) Conceptually familiar of the full technology stack (from BMCs, Firmware, to OS layer, to containers and applications) primitives
Preferred Qualifications
Experience with Cloud Computing platforms like Amazon AWS, Google Cloud Experience with building stream-processing applications using Apache Flink, Spark-Streaming, Apache Storm, Kafka Streams or others Experience with Search systems (such as ElasticSearch, Solr), NoSQL datastores (such as HBase, Cassandra, MongoDB) Experience building distributed, high-volume data services is a plus
Learn more about this Employer on their Career Site
