The main purpose of this project is to transformed customer and account data for two banks and store into Hive after transformation in Spark and Scala and main the client account relationship is separate hive tables.

Processed data coming from multiple sources and apply transformation and store the data in Hive. We are developing data pipelines and schedule them on Kylo.

This project was a launch on work with Tech-M for the ATT client. The main purpose of SSOT graph DB was, to have a centralized data store that would help us to find other interconnected data sources. Final outcome needs to be seen in a graphical format. [Graph with edges and nodes]

Working on processing of real data json data stream using Spark and Kafka Integration. We store intermediate data in redis and final output we were stored in Mongo DB.

And We updated data in Mongo DB using real data stream.

Open Item batch is an item information maintenance system. It holds product information. An item identifies the Division, Category, Store location and Selling price (Retail) of the merchandise. It assigns the Key Stock Number (KSN) which signifies the product purchased by the customer. KSN identifies the style, size, color and flavor. Open Item batch is not an ordering system.

Data for tables is extracted from Teradata. This data is used by other systems such as Local Publishing or Dynamic Pricing. Item and Pricing information must be extracted from Enterprise Data Hub (HADOOP)

The main purpose of this project is to process data coming from multiple sources and apply transformation and store the data in Hive. We are developing data pipelines and schedule them on Kylo.

Technology Used -Spark, Scala, Nifi, Kylo, Azure