Now you can Instantly Chat with Madhukar!
- Overall 15 years of software industry experience
- 8 years experience with Hadoop ecosystem.
- Hands on experience in Azure Analytics, Azure data factory, storage, Azure HDInsight.
- Capable of proce...
- Capable of processing large sets of structured, semi-structured and unstructured data and supporting systems application architecture.
- Able to assess business rules, collaborate with stakeholders and perform source-to-target data mapping, design and review.
- Familiar with data architecture including data ingestion pipeline design, Hadoop information architecture, data modelling and data mining, machine learning and advanced data processing. Experience optimizing ETL workflows.
Data & Analytics
Azure Devops for Oil refinery
- Created data pipeline using Azure data factory
- Gathering the data stored in Azure data store, optimizing it and joining with internal datasets to gather meaningful information.
- Data cleaning using U-Sql code on the Azured data lake.
- Data aggregation on terabyte datasets done using U-Sql which saves IO/CPU time
- Adopted DAX for business logic calculations in providing analytics of system up time on periodical basis
SkillsAzure DataFactory Azure DataLake Azure ML Azure DataBricks SQL Data Modeling Power BI Data Architect Oracle Cosmo DB MongoDB
Analytical Solutions for OEM
Facilitated insightful daily analyses of 100GB to 1TB of server data collected by server logs. Spawning recommendations and tips that increased page performance by 38%.
- Developed spark scala programs to parse the raw data, populate staging tables and store the refined data in partitioned tables in the EDW.
- Created Hive queries that helped market analysts spot emerging trends by comparing fresh data with EDW reference tables and historical metrics.
- Enabled speedy reviews and first mover advantages by using Oozie to automate data loading into the Hadoop Distributed File System and PIG to pre-process the data.
- Provided design recommendations and thought leadership to sponsors/stakeholders that improved review processes and resolved technical problems.
- Managed and reviewed Hadoop log files.
- Tested raw data and executed performance scripts.
- Shared responsibility for administration of Hadoop, Hive and Pig
ToolsOracle cloud azure devops
Burt Metrics DW
- Created HBase tables to load large sets of structured, semi-structured and unstructured data coming from UNIX, NoSQL and a variety of portfolios.
- Supported code/design analysis, strategy development and project planning.
- Created reports for the BI team using Sqoop to export data into HDFS and Hive.
- Developed multiple MapReduce jobs in Java (spark) for data cleaning and pre-processing.
- Assisted with data capacity planning and node forecasting.
- Collaborated with the infrastructure, network, database, application and BI teams to ensure data quality and availability.
- Administrator for Pig, Hive and Hbase installing updates, patches and upgrades.
· Performed analysis of current optimization model and provided recommendations to move to Oracle’s cost-based optimization. Lead the stress test initiative to evaluate the success of this approach
· Established and wrote the Performance Tuning guidelines for the current module to enhance the quality of our product for all current and future code development
· Steered data mart and data warehouse development using 10.2, taking full advantage of new RDBMS features as they become available and stable
· Followed OMS standards (as per CMMI 1.2) in developing and building new applicationsShow More Show Less
SkillsOracle Shell Scripting