- Develop and test Data pipelines for availing data for analysis and reporting.
- Fixing any data quality issues raised by the BI team or business.
- B.Sc. Computer Science or equivalent
- 2-4 years of working experience in Data Engineering
- Have a deep understanding of various data modeling methodologies.
- Strong SQL skill for enterprise class RDBMS.
- Good knowledge about Big Data technologies (Hadoop, Spark, Nifi,….).
- Awareness of Office and desktop applications (word, excel…,etc).
- Work experience in telecommunication environment is a plus.
- Database design/modeling experience (relational/dimensional modeling experience).
- Good knowledge in Hadoop system.
- Strong Programming and scripting skills and shell scripting in Python, Java or Scala. Also Shell scripting is needed.
- Good knowledge and experience in Big Data technologies specially Spark, Nifi. AirFlow is a plus.
- Experience in any of the known ETL tools (DataSatge, Informatica, …….) is a plus.
- Fluent English and Arabic is a must.
- Flexibility in dealing with people with diverse working styles.
- Excellent analytical skills.
- Good communication skills.
- Good interpersonal and teamwork skills.
- High attitude for research and development.
- Ability to work under pressure & tight deadlines
- Ability to work in a team
Transformation & Operational Efficiency
- Develop complete data pipelines to load and integrate any source into data architecture.
- Perform unit testing for data pipelines.
- Develop model on DWH or Data lake which will be used for reporting and further analysis.
- Analyze the needed changes for any source which will affect the data architecture.
- Incorporate these changes to be reflected on the developed pipelines.
- Update the data model if needed to accommodate these changes.
- Handover the needed scripts or jobs to the support team with any rollback scripts if needed.
- Document all changes and modifications and all development activities.
- Monitor jobs performance and outputs quality and sustain quality throughout all activities.
- Investigating in any data quality issues raised by the BI or business team.
- Develop any fixes needed to any quality issues raised by the BI or business team.