● A complete end to end solution for organizations needing to
utilize the power of cloud and data analytics.
● Extracting valuable information from the data in a scheduled and
automated manner.
● The data is processed using spark and scala and stored in various
DB’s across pipelines after finishing and changing data into
required format.
● AWS S3 was used for Cloud Storage. Designed Monitoring and
Alerting Platform using AWS Lambda, Apache NiFi, ELK, Python
and SQS.

• The platform integrates all the data sources, internal and external to the company, to the Data Lake.
• It stores all kinds of data and makes it available immediately to different business and development teams for their work.
• The data is processed using spark scala and stored in s3. The Platform was built on a serverless infrastructure (Lambda), thus optimizing cost and enabling auto-scaling capabilities.
• AWS services like S3, EMR, KMS/IAM were used and implemented in the architecture to provide reliable storage,
  compute and security.

• The platform migrates data from different sources to Cloud and processes data using Apache Spark.
• Reading blob data using spark and ingesting it and storing it in various DB’s across pipelines after finishing and changing data into the required format. AWS was used for Cloud
• ELK for Monitoring and Logging Platform.

• A complete end to end solution using a combination of Data Engineering and Data Science for Call Allocation of service
  engineers.
• Migration of data from on-prem to Cloud using Aws DMS and using spark scala for data processing and view the data in Athena using AWS Glue, Process data using S3 to another bucket using Lambda function by adding jobs as steps in EMR, Visualizing data using Tableau.