Key Skills and Experience
Essential
Previous experience in a Data Engineering role:
1. We're looking to meet experienced Data Engineers who live and breathe Data, who seek to take this best practice and use it in a dynamic environment while making waves in the industry.
2. In-depth knowledge in most of the following technologies: o Proficient in Spark/PySpark with hands-on experience. o Azure data related technologies such as Azure Databricks, Azure Data Factory, Azure Event Hubs and Azure Service Bus. o Solid programming skills in Python or Scala, and strong command of SQL. o Experience with testing frameworks like pytest, ScalaTest, or similar. o Open table formats such as Delta, Iceberg or Apache Hudi. o Experience in leveraging CI/CD workflows to automate the building, testing, publishing, releasing, and deployment of code using tools like Azure DevOps, GitHub Actions, and distributed version control systems such as GIT. o Understanding of cloud infrastructure and Infrastructure as code (Terraform or Bicep)
3. Excellent communication skills.
4. Good analytical and problem-solving skills.
5. Broad expertise in the delivery of data product solutions.
6. Commitment to provide a high level of customer service to internal and external users.
7. Proven experience in the successful delivery and ongoing maintenance of complex, high volume, high performing, and high-quality software systems.
8. Experience of implementing/supporting cloud-based solutions.
9. Familiarity with agile methodologies, such as Scrum or Kanban.
Nice to Have
1. Experience in retail and/or e-commerce.
2. Understanding of Big Data and Distributed Computing.
3. Understanding of streaming technologies (Spark Structured streaming, Apache Flink, etc)
4. Other programming languages: PowerShell, Bash.
5. Understanding of Databricks Ecosystem (Unity Catalog, Workflows, Delta Live Tables).
6. Understanding of any Data Observability or Data Quality Framework.