Senior Data Engineer (Data Ingestion)

WE ARE

SoftServe is a global digital solutions company headquartered in Austin, Texas, founded in 1993. Our associates work on 2,000+ projects with clients across North America, EMEA, APAC, and LATAM. We are about people who create bold things, make a difference, have fun, and love their work.

Big Data & Analytics is the data consulting and engineering arm of our Center of Excellence. Hundreds of data engineers and architects build end-to-end data & analytics solutions: from strategy and architecture through technical design, proof of concept, and full-scale implementation. We work with clients across healthcare, finance, manufacturing, retail, and energy industries.

We hold top-level partnerships with major cloud providers and collaborate with technology leaders such as AWS, GCP, Microsoft, Databricks, Snowflake, Confluent, and others.

IF YOU ARE

• An experienced Data Engineer with 5+ years of hands-on practice in building and scaling data solutions
• Confident working with Databricks, including developing and maintaining ingestion pipelines
• Skilled in Python and SQL for data manipulation, transformation, and integration
• Comfortable using PySpark to handle distributed data processing at scale
• Experienced in web scraping and integrating semi-structured or unstructured data from external sources
• Familiar with working with large datasets and building data pipelines optimized for performance and reliability
• Communicative in English (intermediate level or higher), able to collaborate effectively with stakeholders across teams

AND YOU WANT TO

• Work on data ingestion, where data from various sources will be ingested into Databricks and then into Postgres
• Automate and scale the existing name normalization proof of concept
• Build and support infrastructure around the solution
• Create a data store for reference and integrate it with external systems for validated names
• Work on enriching metadata and ensuring sufficient process performance
• Handle large datasets (up to 1,000,000 records) and integrate the solution into the onboarding flow for future normalization
• Collaborate with stakeholders to iterate on the proof of concept, expand features, and ensure scalability and performance
• Influence the direction of the process and contribute with your technical ideas and solutions

TOGETHER WE WILL

• Address different business and technology challenges, engage in impactful projects, use top-notch technologies, and drive multiple initiatives as a part of the Center of Excellence
• Support your technical and personal growth, as we have a dedicated career plan for all roles in our company
• Investigate new technologies, build internal prototypes, and share knowledge with the SoftServe Data Community
• Upskill with full access to Udemy learning courses
• Pass professional certifications, encouraged and covered by the company
• Adopt best practices from experts while working in a team of top-notch Engineers and Architects
• Collaborate with world-leading companies and attend professional events