About the Data Pipeline Engineer Position
- We iterate quickly in a multi-account cloud architecture, with numerous data sources and models – that’s where you come in.
- You’ll work closely with our engineers, data scientists and security team to manage and maintain ETL processes including data ingestion, modeling, implementation and deployment.
- You will help move our software development practices forward and can help mentor others on design and best practices.
- You will be an active participant in product and platform architecture, keeping security and cost in mind.
- You will be responsible for ensuring quality data and tooling for the Data Science team to leverage for machine learning breakthroughs.
- Demonstrated hands on software development projects in Python are required. You have direct experience in software build, deployment, version control, configuration and testing of applications using modern best practices.
- You have experience working with AWS cloud architecture such as EC2, RDS, S3, ECS, Lambda, DynamoDB, and machine learning operations in general.
- You’re opinionated about tooling and curious about new trends and technologies in the software development world.
- You like to work collaboratively with developers, data scientists and UX teams to improve the quality and resiliency of the products you’re releasing.
- You have a background in data collection and management, with a focus on automation and stability.
- You have working knowledge of data processing, parallelization and performance improvements including Pandas, SciPy and NumPy, or other toolkits.
- You have 4+ years of Python development experience, and are proficient with SQL
- You have excellent verbal and written communication skills with a focus on good technical writing.
Skills You Might Have
- Experience working in highly regulated industries like finance, health care or defense.
- Understanding of networking and network design principles
- Experience working with Linux, RedHat and/or CentOS operating systems
- Knowledge of virtualization and/or containerization strategies and technologies (Docker, VMWare, Kubernetes, etc.)
- Experience working in an Agile team