Senior/Lead Data Engineer | Online Jobs | Optimhire

Senior/Lead Data Engineer

InViz is Bangalore Based Startup helping Enterprises simplifying the Search and Discovery experiences for both their end customers as well as their internal users. We use state-of-the-art technologies in Computer Vision, Natural Language Processing, Text Mining, and other ML techniques to extract information/concepts from data of different formats- text, images, videos and make them easily discoverable through simple human-friendly touchpoints. Senior/ Lead Software Development Engineer - Data Experience: 4-11 years Responsibility:

  • The person will be responsible for leading the development and implementing advanced analytical approaches across a variety of projects, domain and solutions.
  • Should have a mix of analytical, technical skills, someone who can work with business requirements and develop them into a useable and scalable solution.
  • One should fully understand the value proposition of data mining and analytical methods.
  • Should be able to oversee the maintenance and enhancements of existing models, algorithms and processes as well as oversee the development and maintenance of code and process documentation.
Required Skillset:
  • Good hands-on experience on Kafka, Hive, Airflow, Shell scripting, No-SQL database
  • Good exposure to RDBMS and SQL.
  • Should have skills in data ingestion, transformation, staging and storing of data, analysis of data from Parquet, Avro, JSON, and other formats.
  • Experience in building and optimizing “big data” data pipelines, architectures and data sets.
  • Good hands-on experience on Python/Pyspark/Scala-spark. Having exposure to data science libraries is a plus.
  • Good experience in BigQuery,DataFlow,Pub/Sub,Composer,Cloud Functions. 
  • Create custom software components (e.g., specialized UDFs) and analytics applications.
  • Hands on experience in Statistical Methods such as Regression, Logistic regression, decision trees, random forest, other segmentation & clustering methods.
  • Build high-performance algorithms, prototypes, predictive models and proof of concepts. 
  • Experience in e-commerce domain is a plus . 

Job Type


Must have Skills

  • Python
  • Shell Scripting
  • Big Data
  • Apache-Kafka
  • Data mining
  • BigQuery
  • Data Engineer - 3 Years
  • Google Cloud Platform - IAAS

21 - 47 K/Year USD (Annual salary)

Longterm (Duration)

Fully Remote


Rohit S