Mithu W.

Mithu W.

Big Data Engineer

Pune , India

Experience: 3 Years

Mithu

Pune , India

Big Data Engineer

5142.85 USD / Year

  • Immediate: Available

3 Years

Now you can Instantly Chat with Mithu!

About Me

Software professional with 2.9 years of experience in IT programming, specializing in Java technologies and Hadoop. Skilled in developing and supporting applications, with expertise in Hadoop ecosystem and Apache components....

Show More

Portfolio Projects

Description

Description: Moven is platform independent mobile application for Banking system.

➢ Design a connector to Transfer data securely across Amar Bank’s systems to Moven Mobile Application. ➢ Amar Bank’s back-end system will extract the required data and transfer data flat-files(CSV file with pipe delimiter) on Alibaba cloud. ➢ NIFI file watcher job pickup file from that location, process it and dump data into MongoDB. ➢ Mysql is used to maintains all configuration table. ➢ Python script used to post MongoDB record into Moven API. ➢ User got the email notification statistics report after completion of batch. ➢ All deployment done by using Docker on Alibaba cloud.

Show More Show Less

Description

Description: Regulatory BigData Reporting Project is Data Warehouse Project of Future Generali Insurance Company Mumbai for generating IRDA Report.

➢ This Project consists of two primary components:

A) A building a data warehouse

B) providing an end-user query/reporting front-end to the data warehouse.

➢ The data warehouse is built on a Hadoop cluster of Hortonwork platform . ➢ Hive is used as the Data warehouse for storing the OLAP data. ➢ Also Sqoop is used for one time-full loading the data from RDBMS system to Hive at month end . ➢ Hive uses Apache Tez as processing engine . ➢ Once loading the data from RDBMS to Hive , we are transforming and processing the data using spark and make base table in hive which is called as BAP and then loading the result set data of spark sql in to BAP table of Hive. ➢ After preparing the BAP, we hitting the BAP using presto query for generating the number of report This query result represented to the end user using the Jaspersoft BI tool. ➢ End user requesting the report for particular period by entering the input , it getting the result and download it in the form of excel sheet.  Responsibilities: ➢ Import data from RDBMS system to Hive using Sqoop. ➢ Development of ETL in Spark Sql for Extracting the required data on Hive. ➢ Implement the logic for developing the IRDA Report from the Data which is stored in Hive in presto SQL. ➢ Developing the Pestro queries for IRDA report on Hive. ➢ Development of Jaspersoft Report using that Data. ➢ Run the ETL at the Monthly, Quarterly and Annualy .

Show More Show Less

Description

Data extraction project for Abacus AI platform, involving data gathering, Docker deployment, Unix scripting, Presto clusters, and Python-Presto interaction.

Show More Show Less

Description

Interactive analytically querying platform for data access, involving connector creation, Presto installation, and testing with different databases.

Show More Show Less