Worked on Hadoop and spark projects for converting data from XML (which is received from MQ ) to AVRO with certain business logics and exporting it to SQL database using sqoop jobs

Worked on project consisting of Hive , SFTP , Hadoop for integration of two applications. I got so much exposure to all these technologies as I was the only person working on this project

Worked as a sole data engineer in recommendation engine for a leading CPG client which suggests products to buy for outlets on the whole India country level. Worked on migration of Azure to GCP platform with tools like dataproc, airflow, BigQuery. Worked on scaling up recommendation from 100K stores to 1M stores. Worked on changes in recommendation as per new business logic Currently working on migrating this to databricks.

Worked on Hadoop and spark project for converting data from XML (which is received from MQ) to AVRO with different business logics and exporting it to SQL database using Sqoop jobs.

Worked on a project consisting of Hive, SFTP, Hadoop for integration of two applications. got csv files from SFTP server, integrated the data from csv file with already existing data in Hive using Hive queries. Pushed the final generated file back to SFTP. We performed all SFTP operations like push and pull from Hadoop, as there were storage space issues in the local file system.

Developed a Spark Application which takes the XML file from SFG server, transforms it into hbase and finally pushes it to DDC.

Developed Active and passive Anti-ransomware for protecting PC through ransomware attacks.

Developed and hosted an android app for ZENITH 2K18 (a national level tech fiesta of our college).

Application for steganography using image processing.

Automate the process of report generation with Excel macros using VBA.

Doubt solving system for students and teachers using spring and Angular.

Developed android application for solving problems of students and mess vendors using Flutter