The project includes preparing and preprocessing of blockchain json block data. The json data is ingested in Hadoop Distributed File System with Object Relationship Mapping through Plain Old Java Objects with Jackson library and Encoders. The HDFS block data is fetched by the algorithm to find the block elements that contain specific buyers that are into the fradulent money trail. Later the gathered data by Spark or SparkSQL can be dumped into MongoDB through which it can be easily used by Tableau to do some data visualization.

Implemented Type2 Slowly Changing Dimensions for the Data Transformation Module in ETL Pipeline.

Language - Scala

Framework - Spark

Metadata - Oracle

The Engine was config driven, earlier it was designed to be called by spark submit but than we added an API call to the module from UI.