For an enterprise to truly leverage value from data, the enterprise must integrate and then operationalize process, analytics, and technology at scale, under an advanced organizational model

Pyspark scripts as backend service for azure data pipe line, these spark script can be executed through azure DF, azure DB services and Airflow

• Processing unstructured data like (social media data, Rss feeds, Industry Reports, Operational Systems, Finance Systems, Call/e-mail Log) to searchable text format.
• Building adapter for new feeds, social media, Message Providers (Rabbit MQ.), web crawler, ocr engine, Heterogeneous formats :( XML, Websites, Log Files), sensor data, using python Library and Hadoop technologies.
• Working on Message queue like Kafka, Oozie, MQ websphere, Rabbit MQ.
• Integration with twitter data broker like GNIP, Snipp3r and finding the sentiment for provided user queries.
• Working on internet thing using sklearn, sentiment analysis using Natural Language processing.

• Finding exceptions in T&E data through pyspark logarithms, Hive based on SME defined business rules.

For an enterprise to truly leverage value from data, the enterprise must integrate and then operationalize process, analytics, and technology at scale, under an advanced organizational model

• Processing unstructured data like (social media data, Rss feeds, Industry Reports, Operational Systems, Finance Systems, Call/e-mail Log) to searchable text format.
• Building adapter for new feeds, social media, Message Providers (Rabbit MQ.), web crawler, ocr engine, Heterogeneous formats :( XML, Websites, Log Files), sensor data, using python Library and Hadoop technologies.
• Working on Message queue like Kafka, Oozie, MQ websphere, Rabbit MQ.
• Integration with twitter data broker like GNIP, Snipp3r and finding the sentiment for provided user queries.

Working on internet thing using sklearn, sentiment analysis using Natural Language processing.

Cisco Frap is an ML based application, which will predict failure rate of cisco components.

My Role and Responsibility:

• Migrated data validation with pyspark, previously data was in spark and hive, it has been migrated to snowflakes, I have validated migrated data and shared reports with client to get data if anything missed.
• Developed python application to get the data from Snowflakes tables, apply transformation on the data and write back to snowflake tables.
• Validate data manually and make changes in scripts based on the data validation.
• Data extraction from different source like smartsheet, sharepoint and docexchange.
• Data preparation which involves data extraction from multiple sources, apply transformation and prepare data for ML model input, which will give the prediction, forecast, distribution data, which will used in data visualization.

Cisco KBS is an Elastic search-based application, able to search required keyword to get the required aggregation and search results.

My Role and Responsibility:

• Ingestion the source data(mssql table) into Elastic search with orchestration layer.
• Building backend api’s with python flask, configuring yaml file for api params.
• Writing the complex Elastic search backend queries to provide the results for user search.
• Automated scheduling of AWS Sagemaker service through cisco specific internal scheduler.

Description:

The digital ecosystem around us is fast changing with social media torchbearers Facebook and Twitter, Social media users are going beyond ‘networking ’ to discuss brands, products, services, and issues, sharing information with the world, and influencing perceptions of many. It has become challenging for Marketing and Communication professionals to understand these conversations, react judiciously and expeditiously to all the chatter around. MI combine end-to-end solutions in servicing, technology and technology consulting with a cloud-based offering and an evolved pricing model to help organizations intelligently manage costs while improving the effectiveness of their day-to-day operations.

Responsibilities in this project:

• Integration with twitter data broker like GNIP, Snipp3r and pushing Twitter data into kafka queue.
• From Kafka queue spark stream reads data and generates sentiment, theme, word cloud for twitter data, index data into Solr.
• UI reads data from solr and populates dash board.
• Building User authentication for user login using elastic search data.
• User will create project on UI, once after submit, tornado listener get Json request from UI, extracting required information Json with Json parser.
• From json extracted information, will get user required keywords, using Google api, we search those keyword and extract top 100 links and stored in hdfs location.
• Executing mapreduce to extract data from web pages and using python nltk to tag fraud related keyword to content extract from web.
• Indexing tagged to Elasticsearch and updating response back to UI, now user interface can read data from Elasticsearch.

Finding exceptions in T&E data through pyspark logarithms, Hive based on SME defined business rules.

Engine to get tweeter data provide with python post request handler, data from provider flows into Kafkaqueue From Kafka queue, we have spark consumer to process the tweets

Automated web search to fetch the links related to companies, stakeholders in context offraud keywords.From user provides keywords, using Google api, we search those keyword and extract top 100 links andstored in hdfs location, Executing mapreduce to extract data from web pages and using python nltk to tagfraud related keyword to content extract from web.