The Senior Data Engineer builds data processing frameworks that handle the business’s growing database. He works with senior data science leadership as well as other Data and Analytics teams in leveraging data with reporting and scientific tools, for example, Tableau, and Spark. The Senior Data Engineer strives to continuously develop new and improved data engineering capabilities.

Essential Functions

• Integrating data solutions with the current platform to ingest data into appropriate data structures (RDBMS vs Redis vs kafka vs dynamo vs others)


• Leverage big data technologies to aggregate and analyze user behavior data (perhaps Redshift, Tableau, and all the modelling tools in between)


• Take a data-driven approach when designing and testing search and recommendation relevance models


• Monitor data platforms and web services for data integrity and system reliability


• Work with other teams to create microservice applications and API interfaces (accent on proper DB design, injection and retrieval of data)


• Designing and building robust and efficient data pipelines which support recommender systems, search engine indexing, and a variety of web services

• Strive to gain a deeper understanding of customer behaviors and sales trends in order to deliver actionable insights


• Provide technical expertise on large scale distributed systems, data collection and storage, and information retrieval (reliability, localization, distribution, scalability, etc)


• Implement and maintain data streams

• Strong verbal and written communication skills


• Experience designing and implementing MapReduce or Spark jobs


• Experience working with a variety of SQL and NoSQL data storage solutions

• Relevant Bachelor’s Degree and 7 years of relevant industry or academic experience, or Relevant Master's Degree plus 4 years of relevant industry or academic experience, or 11 years of practical experience.


• Programing language expertise in C#


• Mastery of at least one NoSQL repository (DynamoDB, MongoDB, redis)


• Expert with at least one Queueing technology such as (SQS, S3, Kafka, Rabbit MQ)


• Fluent with Visual Studio


• Expert with SQL, and RDBMS Technologies such as (MS SQL Server, Oracle, PostGres)


• Expert working with AWS Technologies like EC2, Kinesis, S3, Lambda, etc


• Expert understanding of Elastic Search technologies

• Experience with (Clojure, Go, java)


• Expert understanding of .NET architectures (ASP, Core, MVC, Entity Framework)


• Fluent with a monitoring technology (New Relic, Data Dog, Scalyr)


• Fluent with AWS API Gateway