Our project involved capturing bonus points for all our users from various discrete sources and third party channels

This data comes in flat files, json files, web pages etc.

We have to capture , cleanse, process, transformand load the data on cloud and then generate analytics for the business users.

• Working on Dataflow, BigQuery, Cloud spanner,Cloud-SQL, BigTable, Datastore.

• Have implemented one complete migration project with ETL pipeline from on-premise hadoopset up to GCP cloud.

• Working on setting up Apache Beam pipeline on google cloud.

Our project involved capturing service now data.

This data helped reflect the type of incidents, requests and changes within the organisation.

We have to capture, load and process the data for analytics.

Our tasks included :

• Setting up data pipeline on Azure environment.
• Worked on Azure data factory team to help set up the pipeline on Azure
• Also Workedon spark , spark streaming , kafkaand python to process the data on Hadoop cluster on Azure.

• Working on tuning hadoop and spark jobs and hive queries
• Applying best practices to tune the jobs
• Working closely with developers to help fix the performance of their spark jobs on hadoop environment
• Working on impala queries and with end users to tune their queries so that their dashboards can work / load.

Working on managing hadoop cluster.

Working on helping users and applications to deploy and run the jobs

Ensuring that the cluster is available and stable.

• Working as an ETL developer ( abinitio )
• Capture data from various sources and load on database.
• working with end users on their analytical queries

• Worked on Shell scripting in developing various applications
• Worked on supporting these jobs
• Production Support 24 * 7 * 365 environment
• worked on developing ETL scripts on abinitio also
• Supported the project from Richmond, VA