Responsibilities:  Worked closely with business customers for Requirement gatherings  Developing Sqoop jobs with incremental load from heterogeneous RDBMS (IBM DB2) using native dB connectors  Designed Hive repository with external tables, internal tables, partitions, ACID property and UDF for incremental data load of parsed data for analytical & operational dashboards  Experienced in developing Hive Queries on different data formats like Text file, CSV file, Log files and leveraging time based partitioning yields improvement in performance using HiveQL  Created Hive external tables for the data in HDFS and moved data from archive layer to business layer with hive transformations.  Developed application using PySpark to determine the CSE Agents  Worked on Revision management system to install the changes in production

Responsibilities:  Analysis, Requirement gathering, Planning & designing  Preparing the unload scripts for the existing 36 Atomic tables.  Preparing scp scripts for the required 36 unloaded files to Hadoop  Creating Clojure application to clean and extract data from HDFS