Senior/Lead Data Engineer

InViz is Bangalore Based Startup helping Enterprises simplifying the Search and Discovery experiences for both their end customers as well as their internal users. We use state-of-the-art technologies in Computer Vision, Natural Language Processing, Text Mining, and other ML techniques to extract information/concepts from data of different formats- text, images, videos and make them easily discoverable through simple human-friendly touchpoints. Senior/ Lead Software Development Engineer - Data Experience: 4-11 years Responsibility:

• The person will be responsible for leading the development and implementing advanced analytical approaches across a variety of projects, domain and solutions.

• Should have a mix of analytical, technical skills, someone who can work with business requirements and develop them into a useable and scalable solution.
• One should fully understand the value proposition of data mining and analytical methods.
• Should be able to oversee the maintenance and enhancements of existing models, algorithms and processes as well as oversee the development and maintenance of code and process documentation.

• Good hands-on experience on Kafka, Hive, Airflow, Shell scripting, No-SQL database

• Good exposure to RDBMS and SQL.
• Should have skills in data ingestion, transformation, staging and storing of data, analysis of data from Parquet, Avro, JSON, and other formats.
• Experience in building and optimizing “big data” data pipelines, architectures and data sets.
• Good hands-on experience on Python/Pyspark/Scala-spark. Having exposure to data science libraries is a plus.
• Good experience in BigQuery,DataFlow,Pub/Sub,Composer,Cloud Functions. 

• Create custom software components (e.g., specialized UDFs) and analytics applications.
• Hands on experience in Statistical Methods such as Regression, Logistic regression, decision trees, random forest, other segmentation & clustering methods.
• Build high-performance algorithms, prototypes, predictive models and proof of concepts. 
• Experience in e-commerce domain is a plus .