DINESH P.

DINESH P.

Senior Spark developer

Pune , India

Experience: 6 Years

DINESH

Pune , India

Senior Spark developer

49955.4 USD / Year

  • Notice Period: Days

6 Years

Now you can Instantly Chat with DINESH!

About Me

Overall 6+ Years of IT experience in Software development Big Data, Hadoop and Spark Technologies. Hands on Experience in Hadoop Technologies like HDFS, Hive, Sqoop, Spark, Spark SQL, Spark Streaming, Kafka, Apache Nifi and Oozie. Involved in capabil...

Show More

Portfolio Projects

Description

Michaels is the largest such arts and crafts chain in the United States, while competitors include Hobby Lobby, Ben Franklin, Jo-Ann Stores, A.C. Moore and several smaller chains Michaels sells a variety of arts and crafts products, including scrapbooking, beading, knitting, rubber stamping, home decor items etc.

Show More Show Less

Description

The purpose of this project is research a medicine and development of Drugs. It is analysis of web-enabled product suite that improves communication; optimize promotional activations for new product launches of pharmaceutical companies. It helps to medical representative to do their daily activities and it provides sales forces with a single, centralized location from which access product information and evaluate competitors.

Show More Show Less

Description

RD is a Data migration project which gets the data from RDBMS to staging area which is relocates in HDFS. The purpose of this project converting all traditional data (RDBMS) into Hadoop technology and validating Meta files which is coming from RDBMS. Those data mart is used in presentation layer to view the data on reporting tool.

Show More Show Less

Description

Description: the project which will deliver a pilot model for a centralised data storage and

data management platform. This will be used to store and process the data from the log files

that come from various IT and Network applications into ArcSight.

Role &Responsibilities:

  • Build the nifi job for Kafka topic to hdfs (raw zone)
  • Build the stream flow for the serving layer(stream data).
  • Develop DDL for store the data into hive table

Implemented performance tuning

Show More Show Less

Description

Description: Adtech ingestion is a ingestion the data from multiple source location to data lake and the date lake is on S3 premises for storage. The data manipulation and cleansing perform using spark and python. Validate the data in data warehouse using Hive and redshift.

Role &Responsibilities:

  • Involved in manage data coming from different DB to S3 using spark python.
  • Writing CLI commands using HDFS and S3.
  • Involved in creating Hive tables, loading with data which will run internally in Map Reduce way.
  • Implemented complex Hive and red shift queries for validate data.

Show More Show Less

Description

  • Develop automation for Sqoop job for multiple table with provided mapper.
  • Develop spark code for moving data from staging layer to schema, raw and archive layer with add additional audit column
  • Develop spark code for collecting data from Kafka topic to staging layer(HDFS).
  • Execute all script using batch scripting.

Show More Show Less

Description

This Project is responsible for Developing Data lake to analyse customer data of a leading bank which allows client to have clear picture of customers and their activities. Project has two phases as Ingestion and Data Manipulation. First phase of this project is to develop a Ingest mechanism to pull data from source systems like DB2 and CDC to HDFS platform.

Show More Show Less