Databricks Engineering Consultant

About Baringa:

We set out to build the world’s most trusted consulting firm – creating lasting impact for clients and pioneering a positive, people-first way of working. We work with everyone from FTSE 100 names to bright new start-ups, in every sector.

You’ll find us collaborating shoulder-to-shoulder with our clients, from the big picture right down to the detail: helping them define their strategy, deliver complex change, spot the right commercial opportunities, manage risk, or bring their purpose and sustainability goals to life. Our clients love how we get to know what makes their organisations tick – slotting seamlessly into their teams and being proudly geeky about solving their challenges.

We have hubs in Europe, the US, Asia and Australia, and we work all around the world – from a wind farm in Wyoming to a boardroom in Berlin. Find us wherever there’s a challenge to be tackled and an impact to be made.

Our Data, AI, Solutions & Engineering (DAISE) practice is looking for an experienced Databricks Engineering Consultant to join the team.

In DAISE, we are focused on delivering value-adding, sustainable data capabilities, aligned to our client’s specific needs. This expertise is applied across clients in all of our industry market sectors (Financial Services, Products & Services, Energy & Resources, Pharmaceutical & Lifesciences and Government).

What you will be doing

We are seeking passionate and dynamic Databricks engineers who enjoy solving complex big data challenges and brings a balance of technical expertise and consulting skills to client engagements. We’re looking for people who can both advise our clients and, when needed, get hands on in technical delivery to bring a solution to life. You will be using your experience to help our clients solve their most important data challenges. You would be also responsible to support the growth of our team, helping them to build the skills they need to solve our client’s challenges. Depending on the level, you can be a part of our leadership team, shaping the direction of the practice, growing the business and leading our people.

• Designing and implementing scalable cloud-based data platforms using Databricks on Azure, AWS, or GCP, enabling organizations to efficiently manage and analyse large datasets.
• Building and optimising data pipelines with Databricks, ensuring high-performance ETL / ELT workflows that integrate data from various sources.
• Performing maturity assessments across clients’ data engineering and Databricks capabilities and recommending improvements.
• Building Databricks technology blueprints and advising clients on the different technology options, translating business requirements (both functional and non-functional) into solutions using the whole Databricks stack, ensuring compliance with the organisations strategy, policies and standards and in some cases, help customers to define new policies, principles and standards.
• Helping organisations migrate from traditional data platforms to Databricks-based cloud solutions, leveraging relevant solutions.
• Working with clients in key areas of data governance, such as in defining principles including, but not limited to Master Data Management, Data Lineage and Data Security.

Your skills and experience

Below we list some of the skills we are looking for. Of course, we do not expect you to be an expert in all areas and we understand that experiences vary based on the background and years of experience:

• Design & develop scalable data pipelines using Apache Spark and Databricks. Three years of hands-on experience in Data Engineering with Databricks, implementing ETL workflows, data transformations, and data integrations
• Competent in Python, PySpark and SQL, with the ability to write efficient, distributed data processing code.
• Good understanding of Delta Lake, Lakehouse architecture. Working with structured and unstructured datasets storing data securely on cloud databases and other storage services
• Experienced in fine-tuning and optimising Spark jobs for performance & cost efficiencies
• Passionate individual who is excited by problems with data and can bring a good mix of technical delivery and core consulting skills in client engagements.
• Ability to own and run client engagements, interact with leaders across industry, work with senior stakeholders to help them understand and frame their problems, assess their current state, and make impactful recommendations which help shape their thinking.
• Good understanding and expertise in delivering data architectures, data pipelines and solutions that are robust and scalable within Databricks ecosystem
• Experience in setup and use of Databricks technologies, ideally on at least one of the major cloud-providers (Azure, AWS, GCP).
• Knowledge of modern data engineering practices, including CI/CD for Databricks, Infrastructure as Code (i.e. Terraform), and version-controlled development with Git and Databricks Repos.
• A good knowledge of key core concepts like distributed computing, batch & stream processing, data lake architecture, functional and object-orientated programming, how pipelines are built and deployed on cloud, pipeline schedules and SLAs.
• Well-versed with documentation and artefacts that need to go along with the solution design and delivery work.
• Be a ‘lifelong learner’ and can demonstrate a drive to always be learning and developing your skillsets and you are keen to help develop those around you.
• Hold a bachelor’s degree across any discipline. A master’s degree in a technical discipline (computer science, engineering or another STEM discipline) would be helpful but is not essential for success

What a career at Baringa will give you

Putting People First.

• Baringa is a People First company and wellbeing is at the forefront of our culture. We recognise the importance of work-life balance and flexible working and provide our staff amazing benefits. Some of these benefits include:
• Generous Annual Leave Policy: We recognise everyone needs a well-deserved break. We provide our employees with 5 weeks of annual leave, fully available at the start of each year. In addition to this, we have introduced our 5-Year Recharge benefit which allows all employees an additional 2 weeks of paid leave after 5 years continuous service.
• Flexible Working: We know that the ‘ideal’ work-life balance will vary from person to person and change at different stages of our working lives. To accommodate this, we have implemented a hybrid working policy and introduced more flexibility around taking unpaid leave.
• Corporate Responsibility Days: Our world is important to us, so all our employees get 3 every year to help social and environmental causes and increase our impact on the communities that mean the most to us.
• Wellbeing Fund: We want to encourage all employees to take charge and prioritise their own wellbeing. We’ve introduced our annual People Fund to support this by offering every individual a fund to support and manage their wellbeing through an activity of their choice.
• Career Progression: No one develops at the same pace. That’s why we have quarterly rather than annual promotion reviews. We don’t have any quotas: if you’re ready and delivering at the right level, you’ll get that promotion.
• Profit Share Scheme: All employees participate in the Baringa Group Profit Share Scheme so everyone has a stake in the company’s success.

Diversity and Inclusion.

We are proud to be an Equal Opportunity Employer. We believe that creating an environment where everyone feels a sense of belonging is central to our culture and that diversity is paramount to driving creativity, innovation, and value for our clients and for our people.

An award-winning workplace.

You can be a part of our ‘Great Place to Work’ – with our commitment to women and well-being in the workplace for all. Click here to see some of our recent awards and how we’ve achieved this.

Using business as a force for good.

We’re proud of our recent exploration of kindness and its role in business in our first-ever brand campaign: Economics of Kindness. You can see some additional recent impact work by clicking here.

Join us

All applications will receive consideration for employment without regard to race, ethnicity, religion, gender, gender identity or expression, sexual orientation, nationality, disability, age, faith or social background. We do not filter applications by university background and encourage those who have taken alternative educational and career paths to apply. We would like to actively encourage applications from those who identify with less represented and minority groups. We operate an inclusive recruitment process, ensuring reasonable adjustments where needed. Please contact a member of our Recruitment Team to discuss further .