Data Pipeline Engineer

Data Pipeline Engineer

About the Data Pipeline Engineer Position

  • We iterate quickly in a multi-account cloud architecture, with numerous data sources and models – that’s where you come in.
  • You’ll work closely with our engineers, data scientists and security team to manage and maintain ETL processes including data ingestion, modeling, implementation and deployment.
  • You will help move our software development practices forward and can help mentor others on design and best practices.
  • You will be an active participant in product and platform architecture, keeping security and cost in mind.
  • You will be responsible for ensuring quality data and tooling for the Data Science team to leverage for machine learning breakthroughs.

About You

  • Demonstrated hands on software development projects in Python are required. You have direct experience in software build, deployment, version control, configuration and testing of applications using modern best practices.
  • You have experience working with AWS cloud architecture such as EC2, RDS, S3, ECS, Lambda, DynamoDB, and machine learning operations in general.
  • You’re opinionated about tooling and curious about new trends and technologies in the software development world.
  • You like to work collaboratively with developers, data scientists and UX teams to improve the quality and resiliency of the products you’re releasing.
  • You have a background in data collection and management, with a focus on automation and stability.
  • You have working knowledge of data processing, parallelization and performance improvements including Pandas, SciPy and NumPy, or other toolkits.
  • You have 4+ years of Python development experience, and are proficient with SQL
  • You have excellent verbal and written communication skills with a focus on good technical writing.

Skills You Might Have

  • Experience working in highly regulated industries like finance, health care or defense.
  • Understanding of networking and network design principles
  • Experience working with Linux, RedHat and/or CentOS operating systems
  • Knowledge of virtualization and/or containerization strategies and technologies (Docker, VMWare, Kubernetes, etc.)
  • A working knowledge of JavaScript. Additional languages are a bonus.
  • Experience working in an Agile team