Senior Data Engineer at Discovery Limited

Key Purpose

• Provide continuous monitoring and support of Linux environments.  Design and implement containerized solutions for machine learning models. Technical leadership and ownership of Linux environments.

Areas of responsibility may include but not limited to:

• Administration & continuous monitoring of Linux environments
• Optimize machine learning models & solutions produced by Data Scientists & Actuaries
• Participate in the architecture of ML environments and frameworks
• Provide ongoing support and enhancement to ML models
• Estimate time and resource requirements for business requirement
• Container management and automation
• Implement Redhat/Linux best practice and standards
• Conduct root cause analysis on production issues
• Technical leadership of entire information management process of both structured and unstructured data
• Implementing machine learning algorithms in production through integration
• Configuration of the Big Data infrastructure and environment for optimal performance
• Work closely with Data Scientists & Actuaries in productionisation of ML model
• Producing relevant technical documentation and specifications
• Develop data processing functions (DPF’s) using Java or Python, etc.

Education and Experience

• Matric    (Essential)
• Minimum 3 – 5 years’ experience
• Redhat certification (Advantageous)
• National Diploma in IT (BTech) or appropriate certification (Advantageous)
• Bachelor of Science (Information Systems, Computer Science, Mathematics) Advantageous
• Linux administration (Intermediate) 
• Linux Shell Scripting (Intermediate)
• Distributed programming skills on a cluster environment (Intermediate)
• Container setup (e.g., Docker) (Intermediate)
• Python/R/Scala Programming (Intermediate)
• Git versioning
• Spark Configuration (Intermediate)
• CI/CD
• Systems Development Life Cycle (SDLC) (Intermediate)
• Data Security and Protection Policies (Intermediate)
• Big Data using Hadoop (Intermediate)
• SQL (Basic)
• Java/.net Programming (Advantageous)
• Big Data Ingestion using Sqoop/Kafka (Advantageous)
• MS Excel (Basic)