Kumar raja D.

Kumar raja D.

Expert in Spark/BigData/Teradata Development

New Delhi , India

Experience: 10 Years

Kumar raja

New Delhi , India

Expert in Spark/BigData/Teradata Development

28800 USD / Year

  • Immediate: Available

10 Years

Now you can Instantly Chat with Kumar raja!

About Me

  • Overall 9 years and 10 months of experience with  3.4 years of experience in Bigdata Technologies like Spark, Scala, Hadoop and Kafka and 6+ years... like Spark, Scala, Hadoop and Kafka and 6+ years of experience as Teradata developer.
  • Working on Bigdata technologies like Spark, Scala, Kafka, Hadoop, HDFS File system, Hive and NiFi.
  • Experienced in creating RDD, DF and DS for the required input data and performed the data transformations using Spark-core.
  • Worked on Spark-SQL, Data Frame, Pair RDD's and Spark YARN
  • Developed the producer and consumer jobs using Kafka
  • Preparation of loading job scripts using Teradata Fastload, Multiload utilities.
  • Developed BTEQ and Fastexport scripts to add the logic to get the related data from the source DB.
  • Worked with different teradata objects like PI, SI and Derived Tables.
  • Apply Collect statistics for each table, load collect stats and refresh collect stat scripts using UNIX scripting.
  • Extensively worked on Teradata SQL.
  • Knowledge in version control tool: GitHub
  • Knowledge in Cassandra, Ambari and AWS glue

Show More

Portfolio Projects

Walt Disney World (DVC-Member & Sales)

Company

Walt Disney World (DVC-Member & Sales)

Description

This project will remove the legacy property management capabilities from LMS to pave the way for its ultimate retirement once sales capabilities have been decommissioned. The project will include the replacement of the legacy PMS supports for front desk operations (guest facing activities) and back office (e.g., Room Assignment, Housekeeping, Revenue Accounting, etc.).  The new PMS will enable enhanced interactions between Cast and Guests by providing mobile technology to Front Office and Housekeeping Cast Members.

Responsibilities:                     

  • Extracting the data from SFTP servers and publishing to Amazon S3 bucket using NiFi workflow.
  • Converted the CSV data into Nested Json format using the Spark and Scala.
  • Developed RDD’s, DF’s and Data sets for the required input data and performed the data transformations using Spark-Core.
  • Loaded the data from servers directly into HDFS using Kafka.
  • Handled Hive queries using Spark SQL that integrates with Spark environment.
  • Scheduled the spark jobs using the cron scheduler

Show More Show Less

Walt Disney World (PMS2.0 & PPT2M)

Company

Walt Disney World (PMS2.0 & PPT2M)

Description

Description:

Disney Vacation Club (DVC) is migrating member data to a MariaDB system. IA will receive eventing messages and store them in the HDFS data lake. Data will then be exposed in familiar table structures via Hive for analytics to allow users to interact via HiveQL, a SQL-like language.

Responsibilities:                     

  • Developed the Kafka consumer job to read the JSON data from topic.
  • Developed RDD’s, DF’s and Data sets for the required input data and performed the data transformations using Spark-Core.
  • Loaded the data from servers directly into HDFS using Kafka.
  • Build the data pipeline to store raw, data-asset and consumable areas.
  • Developed the consumable are using Hive.
  • Scheduled the spark jobs using the cron scheduler.
  • Unit testing.

Show More Show Less

Tools

IDE putty WinSCP

Well Point /Anthem

Company

Well Point /Anthem

Description

Description:

Well Point, Inc. is one of the largest health benefits companies in the United States. Through its networks nationwide, the company delivers a number of leading health benefit solutions through a broad portfolio of integrated health care plans and related services, along with a wide range of specialty products such as life and disability insurance benefits, dental, vision, behavioral health benefit services, as well as long term care insurance and flexible spending accounts.

Edward as data warehouse extracts and stores data from across 13 different subject areas or domains across the enterprise. The different subject areas or domains are Members, Claims, Clinical, Rxclaims, RHI, Broker,  Revenue, Capitation, Groupers, Provider. Apart from subject area the data comes from various source systems depending upon location, Business divisions, partners, vendors etc. Edward is the wellpoints single source of truth to support all business needs, all reports and downstream applications feeds data from here.

Responsibilities:                     

  • Analysis of the specifications provided by the clients.
  • Involved in writing/modifying scripts for loading data to target data warehouse for BTEQ, FastLoad and MultiLoad.
  • SQL Assistant usage for data analysis and SQL queries.
  • Worked with different Teradata objects like PI, SI and Derived Tables.
  • Modifying the queries to use the Teradata features for performance improvement.
  • Modified or created UNIX Scripts and Informatica workflows.
  • Involved in developing and update existing scripts as per project requirements
  • Prepared Unit Test cases to check that data entered has been uploaded correctly into the database.
  • Modified or created WLM scheduled jobs, UNIX Scripts and Informatica workflows.Creating the Design Specification document as per the Requirements gathered

Show More Show Less

Tools

putty

Glaxo Smith Kline (GSK)

Company

Glaxo Smith Kline (GSK)

Description

GSK project is a enhancement project for the existing EDW. Source data from Oracle, SQL Server, Db2 or Flat files flow into Teradata Database transition tables; Extracting data from different source systems, data transformation and loading to Teradata staging tables (ETL process) is implemented through Informatica workflows or Data from Flat files are loaded to staging tables through Teradata Utilities like Fast load and Multi-load.

Responsibilities:                     

  • Write/Modify scripts using Teradata BTEQ, FastLoad, MultiLoad utilities to make load jobs.
  • Involved in building tables, views and Indexes
  • SQL Assistant usage for data analysis and SQL queries.
  • Worked with different Teradata objects like PI, SI, Volatile Tables, GTT and Derived Tables.
  • Modifying the queries to use the Teradata features for performance improvement.
  • Modified or created IPM scheduled jobs, UNIX Scripts and Informatica workflows.
  • Created Semantic Layer views.
  • Prepared Unit Test cases to check that data entered has been uploaded correctly into the database.

Show More Show Less

Tools

FileZilla
Share:

Verifications

  • Profile Verified

  • Phone Verified

Preferred Language

  • English - Fluent

Available Timezones

  • Eastern Daylight [UTC -4]

  • Eastern EST [UTC +3]

  • Dubai [UTC +4]

  • New Delhi [UTC +5]

  • China (West) [UTC +6]

  • Singapore [UTC +7]

  • Hong Kong (East China) [UTC +8]