Senior Data Engineer

As a Data Engineer in Research & Insights, you will be instrumental in developing our clients Decision Maker Insights product, whose objective is to showcase to customers the efficacy of their platform in achieving their desired outcomes. You will join a distributed team of skilled software engineers dedicated to the continuous delivery of business value. Daily responsibilities include building automated data ingestion and ETLs pipelines, aggregations, designing database schemas and performance optimization.

Who You Are

You are a software developer, who specializes in data modeling, ETL pipelines, distributed computing and databases. You work with data warehouses, data lakes and all flavors of databases and know how to orchestrate/automate jobs processing large batch data.

Qualifications:

• You have 6+ years of professional software development experience.
• You have strong computer science fundamentals.
• You have strong software development skills in a server-side language (Python, Java, Scala, etc…).
• You have designed relational schemas and worked with no-sql databases. You are intimate with data storage formats (JSON, Parquet, Avro, etc…).
• You have experience tuning/indexing databases including column-based systems.
• You are a SQL expert.
• You have worked with distributed computing frameworks such as Spark, Hadoop, etc
• You provision your own cloud resources.
• You have architected data infrastructure and ETL pipelines.
• You are an excellent verbal and written communicator.

Preferred Qualifications:

• Your language of choice is Python.
• You have experience leveraging PySpark.
• You cloud computing environment of choice is AWS.