Senior/Lead Data Engineer
InViz is Bangalore Based Startup helping Enterprises simplifying the Search and Discovery experiences for both their end customers as well as their internal users. We use state-of-the-art technologies in Computer Vision, Natural Language Processing, Text Mining, and other ML techniques to extract information/concepts from data of different formats- text, images, videos and make them easily discoverable through simple human-friendly touchpoints. Senior/ Lead Software Development Engineer - Data Experience: 4-11 years Responsibility:
- The person will be responsible for leading the development and implementing advanced analytical approaches across a variety of projects, domain and solutions.
- Should have a mix of analytical, technical skills, someone who can work with business requirements and develop them into a useable and scalable solution.
- One should fully understand the value proposition of data mining and analytical methods.
- Should be able to oversee the maintenance and enhancements of existing models, algorithms and processes as well as oversee the development and maintenance of code and process documentation.
- Good hands-on experience on Kafka, Hive, Airflow, Shell scripting, No-SQL database
- Good exposure to RDBMS and SQL.
- Should have skills in data ingestion, transformation, staging and storing of data, analysis of data from Parquet, Avro, JSON, and other formats.
- Experience in building and optimizing “big data” data pipelines, architectures and data sets.
- Good hands-on experience on Python/Pyspark/Scala-spark. Having exposure to data science libraries is a plus.
- Good experience in BigQuery,DataFlow,Pub/Sub,Composer,Cloud Functions.
- Create custom software components (e.g., specialized UDFs) and analytics applications.
- Hands on experience in Statistical Methods such as Regression, Logistic regression, decision trees, random forest, other segmentation & clustering methods.
- Build high-performance algorithms, prototypes, predictive models and proof of concepts.
- Experience in e-commerce domain is a plus .