Data Engineer

Anthill’s trusted partner, CCC Intelligent Solutions Inc., has picked Bulgaria as its first European location for building an R&D center. CCC has been continuously pioneering solutions that power better decisions for insurers, automotive manufacturers, and collision repairers.

As a member of the engineering team, you will work closely with data scientists and play a key role in designing and building state-of-the-art data platforms supporting the company’s predictive models of training and inferencing tasks.

Responsibilities:

• Build end-to-end data pipelines from sources to fully curate and enhance data sets
• Locate and analyze source data; create data flows to extract, profile, and store ingested data;
• Define and build data cleansing and imputation; map to a common data model; transform to satisfy business rules and statistical computations; and validate data content
• Produce software data building blocks, data models, and data flows for varying client demands, such as dimensional data, data feeds, dashboard reporting, and data science research and exploration
• Assist with producing automated software tests of data flow components and automate business-specific tests for data content quality
• Help automate orchestration and error handling for use by production operation teams

Requirements:

• 4+ years of experience in Python, Spark
• Open-source big data tools (Hive, Spark, Kafka)
• AWS eco-system (Amazon EMR, S3, Presto)
• Airflow for scheduling and monitoring of big data ETL pipelines
• SQL for data profiling and data validation
• Unix commands and scripting
• Hadoop fundamentals and architecture (HDFS, map-reduce, job performance)

What we offer:

• Remote Work Options
• Comprehensive Health Insurance
• Flexible Benefits Program
• Sport and Entertainment Perks
• Support for Seminars, Conferences, Trainings, and Certifications
• Childbirth Allowance
• Employee Assistance Program (EAP)
• Vacation and Travel Allowance

Ready to level up? Only shortlisted candidates will be contacted for further steps in the recruitment process.