We are getting Data From Microsoft SQl server to Spark Perform a Data Transformation and AI/ML On that Data and after that stored That Data into ElasticSearch and that JSON formated Data used into Kibana's BI Report

Data is getting from the senser using Iot Device and with using MQTT Broker we get that Data Into Spark with Using Spark Streaming or else we get that Data from Kafka Spark Streaming it is continuesly getting Data From Kafka and We perform Data Transformation on Spark and After that we have to stored that Data into ElasticSearch and kinesis and that Data is used From Front End side as a JSON Token as per there requirements

Data is available in Hive and I need to used that Data With using Different File formats With using Sqoop.
In Sqoop it is a used For Data Ingestion Tool which processed and transfer Data From Hive to HDFS as per Client's requirements.

If they Required Something with using Spark then used Spark SQL directly