About Me

- Sai Krishna is with over 10 years in software development experience including around 4+ yrs Cloud/Bigdata/Spark Engineer Big Data/Hadoop developer, Azure, AWS & ETL related technologies.- Hands on experience on major components in Hadoop Ecosystem... Azure, AWS & ETL related technologies.- Hands on experience on major components in Hadoop Ecosystem including HDFS, YARN, Hive, HBase, PIG, Sqoop, Flume, Apache Spark and Apache Kafka, Cloudera Data Science Workbench, Stream Sets, Apache Nifi, Amazon Glue, Amazon Redshift, Amazon Kinesis.- Used Scala sbt to develop Scala coded spark projects and executed using spark-submit.- Implemented pre-defined operators in spark such as map, flat Map, filter, reduceByKey, groupByKey, aggregateByKey and combineByKey etc.- Experienced in working with spark eco system using Spark SQL and Scala queries on different formats like Text file, CSV file.- Experience in implementing Spark RDDs in Python (Pyspark) & Scala.- Exposure on ML (Machine Learning) algorithms like Linear regression, Logistic Regression, Naive Bayes, SVM, Decision Trees, Random Forest, Boosting, Kmeans and work closely with DS (data science) team.- Extensive experience in creating, configuring and optimizing for performance of EMR CLUSTER ON AWS.- Developed STAR SCHEMA DATA MODELS (Cubes) using ETL tool AZURE TIMEXTENDER and publish to PBI reports.- Set up standards and processing for Hadoop based application design and development.- AWS CERTIFIED ASSOCIATE ARCHITECT.- Used ADF (Azure Data Factory) service to develop ETL/ELT pipelines; Working with Azure delta lake to bring ACID properties (with Python & Spark SQL) to Spark.- Experience in performing data enrichment, cleansing, analytics, aggregations using HIVE and SparkSQL.- Setup CDSW in Hadoop environment & associated in developing models in R, Python, Scala.- Experience in Designing, developing and implementing connectivity products that allow efficient exchange of data between our core database engine and Hadoop ecosystem.- Experience in Disaster recovery and Backup activities, Multi-node setup, Performance tuning and benchmarking, Security integration, Monitoring, maintenance and troubleshooting of Hadoop Cluster.- Knowledge and able to associate towards implementation of DATA SECURITY using Kerberos authentication, User Privileges, Process ID etc.- Experience in DATA INGESTION (BATCH PROCESSING), by importing and exporting the data from HDFS to Relational Database systems and vice-versa & from Flat files (like .CSV, .txt) using Sqoop, Streamsets, and Nifi pipelines.- Developed DATA STREAMING (NEAR REAL-TIME) pipeline using Kafka & Flume, Spark Streaming.- Experience in DATA ANALYTICS, using HIVEQL, Apache Spark with R, Python, Scala and Spark SQL.- Knowledge on KUBERNETES to deploy applications and manage workloads and services with declarative configuration and automation.- Been part of production support team and resolve the tickets.- Worked on NoSQL database HBase for storing huge amounts of web logs.- Knowledge towards Data Warehouse (DW) & SQL DB.- Deployed Instances, provisioned EC2, S3 bucket, Configured Security groups and Hadoop eco system for Cloudera in AWS.- Knowledge on abinitio ETL tool & transformed the ETL logic to Spark Scala.- Created functions and assigned roles in AWS Lambda to run python scripts & perform event driven processing.- Experience in Data mining and BUSINESS INTELLIGENCE tools such as Tableau.- Knowledge on CI/CD pipelines using tools like Git, and Jenkins.- Knowledge towards MEMORY MANAGEMENT in Spark & Hive PERFORMANCE TUNING.- Exposure towards Splunk tool for LOG ANALYSIS.- Experience in application development using Core Java, RDBMS, and Linux shell scripting.- Major strengths are familiarity with multiple software systems, ability to learn quickly new technologies, adapt to new environments, self-motivated, team player, focused adaptive and quick learner with excellent interpersonal, technical and communication skills.- Good communication skills, work ethics and the ability to work in a team efficiently with good leadership skills.- Knowledge on AGILE methodology and Platinum & Silver agile certified (at State Street Corporation) with cross-team functionality.