Development Dimensions International (DDI) is an international human resources and leadership development consultancy. DDI works with organizations to make changes related to leadership development, leadership selection, succession management, and execution and performance. This project was started with the existing OnPrem Sqlserver,File based data feeds into the Azure Data Lake first using in ADF v1. Post ADF V2 GA , the existing jobs with the enhancement or change requests as required by business move to V2 version. Client has its exist mixed datasets of relational or File based like Asset,Course offered ,Topic,Usages,Skillset details which all for learning prospect in existing on prem system which need to be moved to Cloud with required transformation. Role covered all aspects of BI, including, integration, data modelling, Design discussion and Architectural brainstorming session with client in Sprint call or Grooming session as per Agile method, development of the ETL, providing analytical solution for business users and helping them with the reporting by Power BI. Am the One developer working with client development and QA team. Involved in requirements gathering, analysis, design, development, change management. Created dynamic pipelines in ADF v2 for delta Load and transformed data using USQL language. Authored Standard Template for making generic pipeline to handle multiple on prem data base instead of creating separate pipeline to Optimize the resource and make more reusable code. Implemented logic to optimize the performance and save cost while invoking pipeline like preventing zero byte file creation in landing/Staging, Copy & USQL activities Invocation dynamically depend on the Data availability. Authored the ARM template in for deployment in VSTS. Tested the different Schedule and rerun Trigger window scenarios considering Production system reference. Developed code more reusable and customizable to handle and Test the different scenarios using Configuration Object both in Database level and Json file which used in ADF pipeline building. Got appreciate from QA team and client for the Logic and pipeline authored. Developed pipelines using Azure Data Factory to ingest data from One Prem Sql server database ,File system into the ADLS ,Blob Storage Account.

This project delivered the initial set of Client OnPrem and Sales Force data feeds into the Azure Data Lake. Working with fellow Microsoft partners, with the Business Intelligence/Analytics team, responsible for BI development and the surrounding infrastructure for the UK, USA, Brazil ,APAC ,Australia and South Africa. Involved in end to end implementation like requirement gathering ,Functional discussion, Data Ingestion ,Data set & frequency setup documentation ,building ETL /ELT Logic for business users and the creation of front end reporting with mobile compatible dashboards using Power BI. Actively involved in developed pipelines using Azure Data Factory to ingest data from One Prem(File system),Sales Force Data, amongst others into the ADLS and Different Storage Account. These pipelines were delta enabled and had various activities (copy and Transformation) as the data moved from source to target. Worked with data frames to ingest, transform, analyze and help Front end team to visualize data. Staging – ELT stage layer where the data is loaded directly from Source files. In this step data are loaded from source files. Hierarchy File Build on Top of Dim Data– Pre-defined Files build on staging layer with user-friendly column names. In Pre-Core or Collated Layer ,The business logic and Transformation using USQL Jobs & PowerShell script. PowerShell script creation for Storage Access policy check and Access provisioning, ARM template deployment. Code repository for the different pipelines, Scripts being maintained using VSTS. Involved in Performance optimization in USQL job using Index and Partitioning, applying required filter and conditional Logic. Used Assembly in USQL for Diagnostic Log analysis. Pipeline being developed to Track ADLS user access, Storage account user metrics and Power bi report being generated as per user requirement.

Actively involved in Script and mapping development, improving existing mapping, performance. In the Overall load process to fact entities, the data is loaded in intermediate layers: Staging – ETL stage layer where the data is loaded directly from Source files. In this step data are loaded from source files. Views – Pre-defined views build on staging layer with user-friendly column names. Pre-Core – The history tables are the exact replica of staging tables with some additional date fields to keep track of history records. This layer allows to: simplify the load of delta extract provided by CERPS make data reprocessing simpler track and analyze data received by CERPS over time Core – ETL core layer where the data is loaded from Pre-Core entities. Load into this layer performs most of the transformations needed to match target data-model in Core. From Core tables, Different views are created and used for reporting purpose.

The scope of the Project is to extract the Dental Claims data from AEDW and WDW-ADV and merge it with Dental Claims files sent by WADW team from WADW data source. From AEDW, extract will be created for Dental Claims data and they will be joined with XREF table on the basis of Unique Member key and thus only eligible members for the required period will be selected. For provider details, DDW will be sourced and provider details will be added to the Dental Claims data from AEDW. From WDW-ADV members which are still not migrated to AEDW will be sourced and extract will be created for Dental Claims and they will be joined with MBR_XREF on the basis of Unque_id and mbr_num and thus only eligible members for the required period will be selected. WADW team will send the Claim file for WADW data. Dental Claim data from AEDW with appended provider data from DDW, WDW-ADV and data from WADW file will be merged. Dental Claims driver table will be created CLM_XREF and CLM_XREF for extracting source data. Dental Claims data will be extracted on monthly basis and rules will be applied on them. History files will be generated for required period.

On a Weekly basis, EES sends Synygy product, customers, employees and sale transactions. Synygy will merge this data together with data added directly into the Synygy application by the business users. Synygy perform some processing to associate sale transactions to the proper sales representative in order to calculate commission based compensation. On a weekly basis, Synygy will send EES files containing data created in Synygy as well as sale transactions with their associated sales representatives. This data received from Synygy will be used to support various internal EES applications which need access to the Synygy created data.

Informatica Migration Project and EES applications run jobs in Maestro now changed to Tidal Schedular tool. All jobs getting transferred , enabled in Tidal and Disable in Maestro.

This project delivered the initial set of Client OnPrem and Sales Force data feeds into the Azure Data Lake.

Working with fellow Microsoft partners, with the Business Intelligence/Analytics team, responsible for BI development and the surrounding infrastructure for the UK, USA, Brazil ,APAC ,Australia and South Africa.
Involved in end to end implementation like requirement gathering ,Functional discussion, Data Ingestion ,Data set & frequency setup documentation ,building ETL /ELT Logic for business users and the creation of front end reporting with mobile compatible dashboards using Power BI.

· Actively involved in developed pipelines using Azure Data Factory to ingest data from One Prem(File system),Sales Force Data, amongst others into the ADLS and Different Storage Account.

· These pipelines were delta enabled and had various activities (copy and Transformation) as the data moved from source to target. Worked with data frames to ingest, transform, analyze and help Front end team to visualize data.

· Staging – ELT stage layer where the data is loaded directly from Source files. In this step data are loaded from source files.

· Hierarchy File Build on Top of Dim Data– Pre-defined Files build on staging layer with user-friendly column names.

· In Pre-Core or Collated Layer ,The business logic and Transformation using Databricks,Pyspark and some case USQL Jobs & PowerShell script.

· PowerShell script creation for Storage Access policy check and Access provisioning, ARM template deployment.

· Code repository for the different pipelines, Scripts being maintained using VSTS.

· Involved in Performance optimization in USQL job using Index and Partitioning, applying required filter and conditional Logic.

· Used Assembly in USQL for Diagnostic Log analysis. Pipeline being developed to Track ADLS user access, Storage account user metrics and Power bi report being generated as per user requirement.

This project was started with the existing OnPrem Sqlserver,File based data feeds into the Azure Data Lake first using in ADF v1.

Post ADF V2 GA , the existing jobs with the enhancement or change requests as required by business move to V2 version with Data Lake Store Gen2 & Databricks From Gen1 phase wise make sure client production system not impacted.

Client has it’s exist mixed datasets of relational or File based like Asset,Course offered ,Topic,Usages,Skillset details which all for learning prospect in existing on prem system which need to be moved to Cloud with required transformation.

Health care project related to differnt claim and settlement

Medicine and health care project related to sales and profit from the RX and CX medicine.