Gupta P.

Gupta P.

Informatica developers, AWS Redshift,ORACLE,SHELL SCRIPTING,AWS SNS,AWS CLOUD,AWS GLUE

Wadgaon , India

Experience: 10 Years

Gupta

Wadgaon , India

Informatica developers, AWS Redshift,ORACLE,SHELL SCRIPTING,AWS SNS,AWS CLOUD,AWS GLUE

10 Years

Now you can Instantly Chat with Gupta!

About Me

I have expereince more than 10 years and i worked on these technologies

1) Informatica power center latest version

2) Informatica Intelligent Cloud Services (IICS

3) Oracle,MYSql, DB2 sql

4) AWS Redsh...

3) Oracle,MYSql, DB2 sql

4) AWS Redshift

5) AWS S3, SES, SNS,

6) AWS GLUE,

7) AWS LAMBDA

8) AWS Dynamodb

9) Unix shell scripting

10) Intermediate Python scripting

11) AWS DMS

Show More

Portfolio Projects

AWS DATA LAKE

Company

AWS DATA LAKE

Description

Project                   : DATALAKE

Software                 : Informatica 10, Informatica Cloud, AWS, Shell script, Python Script, AWS Lamdba

Database                : Oracle, Redshift, DB2,SQL SERVER,SAP,Hive

Role                       : Team Lead/Senior Software

 

Project Description: -

Schneider is maintaining data in several systems like SAP, Oracle, DB2, FlatFile ect. They want to bring all the data into AWS could system. From the cloud system the user’s analysis the data according to the source system. The existing Data ware house system is slowly moving into the AWS system (Redshift Database). From the Redshift database the reporting team will build reports according to the user requirements.

 

Contribution/Role:

Role: Team Lead and Developer

 

  1. Working on agile project and distributing the work based on the priority
  2. Acting as a scrum master and discussing with the product manager
  3. Allocating the Jira task to the developers based on the priority
  4. Design & implement work flows using ETL and Unix scripting to perform data ingestion on AWS platform
  5. Extracting the data from SAP source system in the form of flat file and load the data into Redshift database system.
  6. Some of the source system data is migrating using DMS
  7. Used Informatica cloud to migrate the data from the source system into directly S3 file system of AWS.
  8. Data loaded into dimensions and facts.
  9. Interacting with the source system to get the information about the tasks what they required.
  10.  Created Unix scripts to push the data to S3 bucket.
  11.  Interacting with source system to bring the data into AWS environment
  12. Created unix script to fetch the files from other SFTP server to our unix environment.

Aws Activities:

  1. Created tables in Redshift Database and copied the data into the tables using Copy command
  2. Created SNS to get the notification of the failures while loading the data using Lambda python
  3. Created Redshift partition for the data based on the user requirement.
  4. Unload the data from Redshift table or from Redshift spectrum based on the user requirement.
  5. Created python script to load the data into Redshift tables.
  6. Using AWS DMS (Data migration services) moving the data from source systems to Redshift database.
  7. Created triggers or events using AWS lambda python to copy the files from S3 bucket to Redshift tables.
  8. Performance tuning done to load the data into fact table for 8 years data in redshift.
  9. Done POC on AWS Glue.
  10.  Having knowledge on PYSpark.
  11.  Configure the DMS jobs to load the data from source system to Redshift tables.
  12.  Created partitions in the redshift spectrum on the table based on the requirement.

Python:

  1. Created python scripts to load the data into Redshift tables.
  2. Created script to unload the data from Redshift spectrum to get the changed data in the files between current run and previous run
  3. Created python scripts to vacuum the tables based on the business requirement.
  4. Created script to add additional field to the file using pandas and load into redshift spectrum.
  5.  

 

 

IICS: (Informatica Cloud)

  1. Reading the files from BOX application and move it to AWS S3 bucket using Informatica Cloud mappings, tasks and Unix script.
  2. Extract the data from the SQL server database to Redshift database using Informatica cloud.
  3. Created simple jobs in Informatica Cloud (IICS) read the data from files and load into s3 bucket.
  4. Created linear and parallel tasks as per the requirement
  5. Create depended jobs between informatica cloud and informatica power center.
  6. Used parameter in the mappings as per the requirement.
  7. Created Replication task to move the data from one source system to another system and schedule the job.

Show More Show Less

Tools

informatica,

GD Metrics

Company

GD Metrics

Description

Project                   : GD Metrics AND MSI Metrics

Software                 : Informatica 10

Database                : Oracle

Role                       : Team Lead/Senior Software

 

Project Description: -

Schneider is maintaining data in several systems like Remedy, Planview, Trust IPO and Flat files as source databases. They want to bring all the data into Oracle database system. The reports/Dashboards implemented on ITSM Domain, especially in the areas of Incident, Service Request, Problem and Change Management, Asset Management etc. These reports/dashboards will be used to know the performance of the support teams at global level, which will be used to implement charge back mechanism to the vendors based on SLA calculations.

 

Contribution/Role:

Role: Team Lead and Developer

 

  1. Involved in preparing the mapping specification based on the business rules
  2. Update the daily status report
  3. Created Dynamic parameter file generation
  4. Involved in preparing the Unix script
  5. Created mapping and workflow to generation 882 files for various table which are required
  6. Worked with multiple databases, flat files and Created Mapplets to reuse transformation Logic
  7. Created stored procedure based on the requirement
  8. Create reusable components where the same business logic applied.
  9. Data loaded into dimensions and facts.

Show More Show Less

Tools

Informatica

DWP

Company

DWP

Description

Client                      : DWP

Project                   : UCMI

Software                 : Informatica 9.1, Informatica Data Replication, Informatica Data Explore

Database                : Oracle

Role                       : Team Lead/Senior Software

 

Project Description: -

The Universal Credit (UC) Programme has been established to reform the system of benefits and tax credits for people of working age. It aims to improve the incentives to enter work, reduce benefit dependency and simplify administration while continuing to provide appropriate levels of support, especially for people with additional needs.


Management Information (MI) is longer term tactical and strategic decision making based on a more holis-tic view of the business, and for this it is necessary to summarise, combine and integrate data, often of quite disparate nature and drawn from different parts of the business, into a coherent whole. This allows the interaction of the different business areas to quantified, and for decisions to take account of wider impact.
 

 

Contribution/Role:

Role: Team Lead and Developer

  1. Involved in preparing and review of  ETL Specification
  2. Involved in preparing and review of Mapping sheets
  3. Involved in estimation (WBE)
  4. Interacting with client for the clarifications
  5. Creating the reference data hub
  6. Involved to prove the POC for the requirement of IDR and IDE

 

IDR Activities:

  1. Setup the Network mode environment
  2. Setup the connectivity for Source and Target database
  3. Map the tables and columns to get the data form source to target  database
  4. Assign the path where Archive log files exist of the Source Oracle database
  5. Create, import, edit, or export configuration files.
  6. Perform the Initial Sync, Extract and Apply by using SCN (System Change Number) concept
  7. Using the SQL Mode for the Data Replication
  8. Schedule Extractor, Applier, InitialSync, in IDR Schedule
  9. Creating the Recovery tables.
  10. Assigning the required permission at source database level to read the data like Enabling Archive log mode, Enabling Minimal Global Supplemental Logging, Assign the permission to the oracle user like select, alter, view, resource, alter session.

 

IDE Activities:

  1. Created the folders for the project
  2. Imported the Physical Data Objects (PDO) like Table and  Flat files
  3. Create the profile for the PDO objects
  4. Create column based profile
  5. Prepare scorecards for the profile data
  6. Assign the thresholds for the scorecards
  7. Drill down the data for the profiled data
  8. Create rule based profile (Business rule), filters
  9. Create the reference data from the profiled data

 

PowerCenter:

  1. Created Dynamic parameter file generation
  2. Involved in preparing the Unix script
  3. Created mapping and workflow to generation 882 files for various table which are required
  4. Worked with multiple databases, flat files and Created Mapplets to reuse transformation Logic
  5. Used Transformations such as Source Qualifier, Filter, Aggregator, Expression, Connected and Unconnected Lookup, Sequence Generator, Router, Update Strategy, etc to develop mappings

Show More Show Less

Tools

Informatica

General electric

Company

General electric

Description

Client                      : GE

Project                   : GECC

Software                 : Informatica 9.1

Graphic Tool            : Graphviz 2.28 ver (To design the dependency of the workflows)

Database                : Teradata 13.10 ver, Teradata SQL Assistant 13

Role                       : Senior Software Engineer

 

Project Description: -

   GE Fleet Services is one of the largest fleet management companies in the world. GE Fleet provides fleet leasing and management services that meets the needs of

 

companies with all sizes of car and truck fleets. To understand the EDW and implement a data model that integrates data from multiple existing data sources, including: the Fleet mainframe, the Fleet customer data base (“CDB”), Telematics (PUnit), Oracle, Siebel, and Collision Experts International (3rd party accident data “CEI”) to the existing GECA WH. CUSTOMER data model standards will be followed.
Created data source to data target (i.e., “source to target”) mappings and related business logic to support the population of the target data instantiated physical model (i.e., the updated version of the GECA WH).
Designed, developed, test and deploy extract, transfer and load (“ETL“) routines and components that schedule, extract, cleanse, and transform data as per the source to target specifications.
To define and document the business, lineage and technical metadata in all pertinent aspect of the Project, including, but not limited to, data modelling, ETL, data quality, access layer, history, and audit trail/controls
 

Created Mapping sheet and related business rules/logic to support the population of the target data instantiated physical model (i.e., the updated version of the GECA WH).

 

Contribution/Role:

Role: Team Lead and Developer

  1. Handled the Mapper team
  2. Involved in preparing the mapping specification based on the business rules
  3. Review the mapping specification and update to the PL/PM
  4. Interacting with client
  5. Update the daily status report
  6.  Worked on development and review activities
  7. Involved in preparing ETL design using Power Center Mapping Architect for Visio.
  8. Worked on different SORs systems like Cobal copybook, Oracle, Flat files ect.
  9. Involved in Data profiling for the mainframe file using Teradata Profiler.
  10. Involved in preparing the control Frame work for scheduling the jobs.

Show More Show Less