Data Platform Engineer – Azure Databricks & DevOps

About Abylon

Established in 2016 Abylon Consulting is a rapidly growing and innovative IT solutions provider, with a strong focus on providing innovative BI solutions and consulting services. We have high expertise in Microsoft’s data analytics and BI and AI products, hence many of our solutions are based on or tightly integrated with the relevant Microsoft products.

Abylon is a Certified Microsoft Solution partner in Data & AI and Digital & App innovation categories. In 2024 Abylon became an official Databricks Consulting partner.

Our dynamic growth was recognized when Abylon Consulting made it to the Financial Times’ annual list of Europe’s 1000 fastest growing companies for three years in a row (2022, 2023, and 2024), as well as to the Deloitte Technology Fast 50 list in 2022.

Building on our proven track record of success in Hungary, we expanded our expertise and solutions to the UK market, establishing a subsidiary and opening our London office.

If you would like to find out more about us, visit our About us page or our  LinkedIn page.

We are currently seeking a Data Platform Engineer with strong experience in Azure-based data platforms, Databricks, CI/CD, DevOps practices, and cloud infrastructure operations.

(This role is not a traditional Data Engineer position. It combines platform engineering, DevOps, Azure infrastructure, Databricks administration, deployment automation, and client-facing technical coordination. The ideal candidate understands how enterprise data platforms are built, deployed, operated, and maintained in a cloud environment.)

 

Your responsibilities:

  • Developing, maintaining, and improving Azure DevOps CI/CD pipelines
  • Managing deployment processes for Databricks jobs, database changes, and platform components
  • Supporting CI/CD processes from development to production environments
  • Support and evolve the Databricks Lakehouse environment (Unity Catalog, Delta Lake, compute clusters, access control)
  • Collaborate with data engineers and analytics teams to ensure platform stability and scalability
  • Working with Git methodologies, branching policies, and software development lifecycle practices
  • Enable development squads through self-service tooling and developer experience improvements
  • Setting up and maintaining Azure-based platform environments, including production environments
  • Supporting Azure infrastructure components such as app registrations, service principals, Key Vault, managed identities, virtual machines, and access rights
  • Managing Databricks platform components with admin-level responsibilities
  • Enforce security, cost governance, and compliance at the platform level
  • Maintaining Databricks catalogs, external locations, credentials, connections, cluster policies, and permissions
  • Supporting Databricks Asset Bundle deployments
  • Working with SQL and DDL deployment processes, including metadata database deployments
  • Supporting Python-based Databricks jobs and data platform workloads
  • Working with Python, PySpark, and Pandas when needed
  • Coordinating infrastructure, DevOps, framework, and general architecture-related tasks
  • Communicating directly with the client and supporting platform-related discussions
  • Troubleshooting deployment, access, configuration, and platform operation issues
  • Collaborating with data engineering, DevOps, infrastructure, and client-side teams

You are the perfect candidate if you have…

  • Proven experience in platform engineering, DevOps, data platform operations, or a similar role
  • Hands-on experience with Azure DevOps pipelines and CI/CD processes
  • Experience deploying solutions to Databricks (Unity Catalog)
  • Practical knowledge of Databricks administration and platform configuration
  • Understanding of Databricks jobs, cluster policies, catalogs, external locations, credentials, connections, and permissions
  • Experience with Azure cloud services and platform components
  • Familiarity with app registrations, service principals, Key Vault, managed identities, and Azure access management
  • Strong understanding of Git, branching strategies, and development lifecycle processes
  • Solid SQL knowledge and experience with database deployment processes
  • Python knowledge, including PySpark and Pandas
  • Ability to work with infrastructure, deployment, and platform configuration topics
  • Good problem-solving skills and a structured, ownership-driven mindset
  • Ability to communicate clearly with clients and technical stakeholders
  • Strong command of written and spoken English
  • Bash scripting
  • Experience with Azure Data Factory, DDL deployments, setting up production environments from scratch
  • Previous Data Engineering background
  • Experience supporting enterprise-scale data platforms

Why Join Us?

How to Apply?

If you feel like you would like to join us based on the description, all you have to do is send your Hungarian or English CV to our email address: job@abylon.io

For more information on Abylon and other open positions follow our social media channels!

Please provide your name and email to access the full package details.

Please provide your name and email address to download the whitepaper

Please provide your basic info to view the Demo

Download Whitepaper on Rapid Smart Excel Add-In