Data Engineer

Job Description

Base-2 Solutions is seeking a Data Engineer who will work with team members to build, deploy, and operate data pipelines and data engineering components to enable data brokering and exchange capabilities across a diverse range of producers and consumers. In this role, you will be responsible for building robust, scalable data pipelines while working collaboratively with cross-functional teams to ensure seamless  integration, optimized performance, and efficient delivery of solutions.

Capabilities

• Work with team members to operationalize data pipelines and supporting cloud infrastructure.
• Collaborate with external data producers and consumers to obtain and provide data through interfaces such as REST APIs and S3.
• Provide day-to-day support of deploying Python-native data pipelines and performing data engineering tasks to enable data brokering and exchange capabilities.
• Provide Tier 2/3 troubleshooting and incident resolution support for data pipelines in Production.

Qualifications

• 4+ years of proven experience in data engineering, with expertise in designing, developing, and maintaining data ingestion, transformation, and loading pipelines and components.

Required Skills/Experience

• Demonstrated experience in designing and deploying data pipelines leveraging AWS cloud infrastructure across multiple classification domains (e.g., IL5 to IL6+).
• Experience with infrastructure-as-Code (IaC) tools, including Terraform, CloudFormation, or Ansible, to automate deployment of data pipeline cloud infrastructure.
• Understanding of RMF security principles and hands-on experience implementing security controls for data pipelines in cloud environments.
• Strong scripting and programming skills in languages such as NiFi, Hadoop, HDFS, and Kafka. Experience implementing data pipelines in the Cloudera Data Platform environment in highly preferred.
• Strong communication skills, with the ability to clearly convey complex technical concepts.