JOB DESCRIPTION
Job Rank: Director
Department: Front Office IT
Section / Function: Insert Section/Function
Travel Required: No
SMCR Status: N/A
Effective from:
Front Office Supervisor (FOS)
Dual Hat Department: Non-employing entity Dual Hat Department
Dual Hat Reports to: Non-employing entity Manager’s Job Rank/Corporate title and functional title
Dual Hat SMCR: Client Dealing Function, Material Risk Takers, Manager of Certified Employees, Significant Management Function, CASS Oversight function, Algorithmic trading or N/A
Profile Summary
Qualifications, Skills and Experience
We are seeking a highly skilled and experienced Lead Data Engineer to join our talented team. The ideal candidate will have a strong background in data engineering, with experience in AI and a proven track record of designing and implementing robust data pipelines.
Responsibilities:
- Lead the design, development, and maintenance of scalable and efficient data pipelines to support our data-driven applications
- Collaborate with cross-functional teams, including data scientists, software engineers, and product managers, to understand data requirements and deliver high-quality solutions
- Architect and implement data ingestion, transformation, and storage processes using tools such as Apache Spark, Azure Data Factory, and other similar technologies
- Optimize data pipeline performance, ensuring data accuracy, reliability, and timely delivery
- Mentor and guide junior data engineers, fostering a culture of continuous learning and improvement
- Stay up-to-date with the latest trends and best practices in data engineering and AI, and apply them to our projects
- Participate in the evaluation and selection of new technologies and tools to enhance our data engineering capabilities
- Contribute to the development of AI-powered features and products, leveraging your knowledge of machine learning and data science concepts
Required Qualifications:
- Bachelor's or Master's degree in Computer Science, Engineering, or a related field
- 7+ years of experience in data engineering, with at least 3 years in a leadership role
- Strong proficiency in Apache Spark, Azure Data Factory, and other data pipeline tools
- Experience with cloud platforms such as Azure, AWS, or GCP
- Solid understanding of data modeling, data warehousing, and ETL/ELT processes
- Familiarity with AI and machine learning concepts, with hands-on experience in implementing AI-driven solutions
- Excellent problem-solving skills and the ability to design and optimize data architectures
- Strong programming skills in languages such as Python, Scala, or Java
- Experience with data quality, data governance, and data security best practices
- Excellent communication and collaboration skills, with the ability to work effectively in a team environment
- Ability to thrive in a fast-paced, dynamic environment and manage multiple priorities
Preferred Qualifications:
- Advanced degree in Computer Science, Data Science, or a related field
- Certifications in relevant technologies, such as Azure Data Engineer or Databricks Certified Developer
- Experience with real-time data processing and streaming technologies like Apache Kafka or Azure Event Hubs
- Knowledge of data visualization tools, such as Power BI or Tableau
- Contributions to open-source projects or active participation in the data engineering community
