- Drive the integration of the data pipeline to feed AI/ML models
- Design and implement big data solutions to meet specific use cases
- Works hands-on with code for big data processing
- Analyze structural requirements for new software
- Improve system performance by conducting tests, troubleshooting and integrating new elements
- Coordinate with Data Scientists to identify future needs and requirements
- May lead and direct the work of others
- 3+ years of data engineering industry experience
- Demonstrate ability to work with structured and unstructured data
- Excellent skills in data management, data processing and cleaning
- Deep understanding of data structures and stores
- Knowledge of Natural Language Processing (NLP) and ML
- Fluent in at least two programming languages (Python, R, Scala, Java, C#)
- Advanced knowledge of Cloud infrastructures (Azure, AWS, or GCP)
- Proficiency in common machine learning tools (sklearn, tensorflow…) with big data tools (Spark/Hadoop, Hive, Pig, Hbase, ..) and NLP tools (NLTK)
- Excellent understanding and experience with relational and non-relational databases
- Experience with index search tools (Elasticsearch, solr)
- Understanding of Enterprise search tools
- Coordinate with Data Scientists to identify future needs and requirements
-
Great communication skills (English written and verbal)