Big Data Engineer
The required skillset to be successful at this job consists of:
- Experience in Requirements Engineering, Solution Architecture, Design, Development and Deployment
- Experience in integrating data into analytical platforms
- Experience with ingestion technologies (Spark, Kafka, Storm, Apache NiFi & Minifi) and storage (HDFS, HBase, Hive)
- Experience in data profiling, source-target mappings, ELT development, SQL optimization, testing and implementation
- Expertise in SQL & other data manipulation languages essential (Pig, Python, Java)
- Experience with structured/unstructured data
- Track record of implementing databases and data access middleware and high volume processing involving large data stores