Senior Data Engineer with DataBricks

*This position is fully remote only for employment in Bulgaria. However, people can also work in one of our offices in Sofia or Varna if they prefer to.

About DataArt

DataArt is a global software engineering firm and a trusted technology partner for market leaders and visionaries. Our world-class team designs and engineers data-driven, cloud-native solutions to deliver immediate and enduring business value.

We promote a culture of radical respect, prioritizing your personal well-being as much as your expertise. We stand firmly against prejudice and inequality, valuing each of our employees equally.

We respect the autonomy of others before all else, offering remote, onsite, and hybrid work options. Our Learning and development centers, R&D labs, and mentorship programs encourage professional growth.

Our long-term approach to collaboration with clients and colleagues alike focuses on building partnerships that extend beyond one-off projects. We provide the ability to switch between projects and technology stacks, creating opportunities for exploration through our learning and networking systems to advance your career.

Position Overview

Our client is a premier home improvement company boasting a network of over 2,000 stores across eight European countries.

As a Data Engineer, you will offer engineering support and deliver data insights through the collection, analysis, and summarization of development and service issues. Your expertise will help steer the direction of our data strategies and ensure the highest quality in our technology offerings. You’ll be integrated into a dynamic data engineering team led by an experienced Data Lead. Your new team is a blend of talented Data Engineers, Data Designers, meticulous QA experts, and an agile Scrum Master, all dedicated to building exceptional data solutions.

Responsibilities

• Design, build, and maintain ETL processes, transforming raw data into actionable insights
• Manage and optimize large-scale data pipelines, ensuring data quality and integrity
• Develop SQL queries for efficient data extraction, transformation, and analysis
• Collaborate with cross-functional teams to understand data requirements and implement effective solutions
• Utilize Python and PySpark for various data engineering tasks, including scripting and automation
• Employ Azure Databricks for advanced data processing and analytics
• Monitor and troubleshoot data workflows to maintain excellent performance standards

Requirements

• Proven experience as a Data Engineer within a professional setting
• Strong programming skills in Python and a solid understanding of ETL processes
• Proficiency in SQL and experience with large datasets and distributed computing environments
• Hands-on experience with Azure Databricks and Azure Data Factory
• Familiarity with data modeling, data warehousing, and performance tuning
• Proficiency with Azure and ADF (Azure Data Factory)
• Expertise in Databricks, including Delta tables and Delta Live tables
• Strong programming skills in Python and Pyspark
• Advanced SQL capabilities
• Experience with Unity Catalogs
• Proficiency in DBT (Data Build Tool)
• Essential knowledge of the Medallion architecture

What We Offer

• Unique corporate culture – no micromanagement, friendly atmosphere, freedom, and mutual respect
• Flexible schedule – ability to change projects, to work from home, and to try yourself in different roles
• Professional Development Map – a comprehensive map of your professional development within DataArt
• We hire people not for a project but for the company. If the project (or your work in it) is over, you go to another project or to a paid “Idle”.
• Social benefits – additional health insurance, life insurance, sports card, etc.
• Opportunity to work from another DataArt office in a different city or country (temporary or permanently)
• Free English courses
• Cozy office with a great atmosphere
• Snacks, drinks, and fruits are always available

^{reference number: DE00135}