Shashank T.

Shashank T.

Data Engineer

Noida , India

Experience: 2 Years

Shashank

Noida , India

Data Engineer

134400 USD / Year

  • Start Date / Notice Period end date: 2020-09-30

2 Years

Now you can Instantly Chat with Shashank!

About Me

Experienced Data Engineer with interest in building solutions in Big Data distributed computing frameworks, back-end applications and cloud computing....

Show More

Portfolio Projects

Description

● A complete end to end solution for organizations needing to
utilize the power of cloud and data analytics.
● Extracting valuable information from the data in a scheduled and
automated manner.
● The data is processed using spark and scala and stored in various
DB’s across pipelines after finishing and changing data into
required format.
● AWS S3 was used for Cloud Storage. Designed Monitoring and
Alerting Platform using AWS Lambda, Apache NiFi, ELK, Python
and SQS.

Show More Show Less

Description

A complete end to end solution using a combination of DataEngineering and Data Science for Call Allocation of serviceengineers.Migration of data from on-prem to Cloud using Aws DMS andusing spark scala for data processing and view the data in Athenausing AWS Glue, Process data using S3 to another bucket usingLambda function by adding jobs as steps in EMR, Visualizing datausing Tableau.

Show More Show Less

Description

  • The platform migrates data from different sources to Cloud andprocesses data using Apache Spark.
  • Reading blob data using spark and ingesting it and storing it in various DB’s across pipelines after finishing and changing datainto therequired format. AWS was used for Cloud
  • ELK for Monitoring and Logging Platform.

Show More Show Less

Description

  • The platform integrates all the data sources, internal and externalto the company, to the Data Lake.
  • It stores all kinds of data and makes it available immediately to different business and development teams for their work.
  • The data is processed using spark scala and stored in s3. The Platform was built on a serverless infrastructure (Lambda), thusoptimizing cost and enabling auto-scaling capabilities.
  • AWS services like S3, EMR, KMS/IAM were used andimplemented in the architecture to provide reliable storage,
    compute and security.

Show More Show Less

Description

  • A complete end to end solution using a combination of DataEngineering and Data Science forCall Allocation of service
    engineers.
  • Migration of data from on-prem to Cloud using Aws DMS and using spark scala for data processing and view the data in Athena using AWS Glue, Process data using S3 to another bucket using Lambda function by adding jobs as steps in EMR, Visualizing data using Tableau.

Show More Show Less

Description

Show More Show Less

Description

Show More Show Less

Description

Show More Show Less

Description

Show More Show Less

Description

Show More Show Less

Description

Show More Show Less