About Me
- Overall 15 years of software industry experience
- 8 years experience with Hadoop ecosystem.
- Hands on experience in Azure Analytics, Azure data factory, storage, Azure HDInsight.
- Capable of proce...
- Capable of processing large sets of structured, semi-structured and unstructured data and supporting systems application architecture.
- Able to assess business rules, collaborate with stakeholders and perform source-to-target data mapping, design and review.
- Familiar with data architecture including data ingestion pipeline design, Hadoop information architecture, data modelling and data mining, machine learning and advanced data processing. Experience optimizing ETL workflows.
Skills
Programming Language
Software Engineering
Data & Analytics
Web Development
Database
Mobile Apps
Others
Operating System
Development Tools
Portfolio Projects
Company
Azure Devops for Oil refinery
Description
- Created data pipeline using Azure data factory
- Gathering the data stored in Azure data store, optimizing it and joining with internal datasets to gather meaningful information.
- Data cleaning using U-Sql code on the Azured data lake.
- Data aggregation on terabyte datasets done using U-Sql which saves IO/CPU time
- Adopted DAX for business logic calculations in providing analytics of system up time on periodical basis
Company
Analytical Solutions for OEM
Description
Facilitated insightful daily analyses of 100GB to 1TB of server data collected by server logs. Spawning recommendations and tips that increased page performance by 38%.
- Developed spark scala programs to parse the raw data, populate staging tables and store the refined data in partitioned tables in the EDW.
- Created Hive queries that helped market analysts spot emerging trends by comparing fresh data with EDW reference tables and historical metrics.
- Enabled speedy reviews and first mover advantages by using Oozie to automate data loading into the Hadoop Distributed File System and PIG to pre-process the data.
- Provided design recommendations and thought leadership to sponsors/stakeholders that improved review processes and resolved technical problems.
- Managed and reviewed Hadoop log files.
- Tested raw data and executed performance scripts.
- Shared responsibility for administration of Hadoop, Hive and Pig
Tools
Oracle cloud azure devopsCompany
Burt Metrics DW
Description
-
- Created HBase tables to load large sets of structured, semi-structured and unstructured data coming from UNIX, NoSQL and a variety of portfolios.
- Supported code/design analysis, strategy development and project planning.
- Created reports for the BI team using Sqoop to export data into HDFS and Hive.
- Developed multiple MapReduce jobs in Java (spark) for data cleaning and pre-processing.
- Assisted with data capacity planning and node forecasting.
- Collaborated with the infrastructure, network, database, application and BI teams to ensure data quality and availability.
- Administrator for Pig, Hive and Hbase installing updates, patches and upgrades.
Tools
PentahoCompany
DB-exoc
Description
· Performed analysis of current optimization model and provided recommendations to move to Oracle’s cost-based optimization. Lead the stress test initiative to evaluate the success of this approach
· Established and wrote the Performance Tuning guidelines for the current module to enhance the quality of our product for all current and future code development
· Steered data mart and data warehouse development using 10.2, taking full advantage of new RDBMS features as they become available and stable
· Followed OMS standards (as per CMMI 1.2) in developing and building new applications
Show More Show LessSkills
Oracle Shell ScriptingTools
shell script