Working with 2 projects as a big data developer.

Working with Spark, Python along Azure databricks.

Nia Data was implemented as the data lake solution and data ingestion processes were established. Ingested data was validated, transformed and harmonized through IIP models. This harmonized data was made available for analytics. Data exploration and reporting was performed on the harmonized data to provide business insights.

Client: Internal & External Clients

Team Size: 10+

Profile: Hadoop Developer

Duration: 8+ Months

Tools: Hadoop 2.7.1, HDFS 2.7.1, MaReduce Yarn 2.7.1, Hive 2.2.0, Solr 5.2.1, Spark 2.2.0, Sqoop Client 1.4.6.

Databases: Cassandra, HBase, PostgreSQL

Role and Responsibility:

• Hand on experience on Hadoop, Hive, Spark Yarn and Sqoop.
• Handing product issue and maintains using patch updates.
• Working with custom module and script for automation using Linux and Core Java.
• Manage and monitor Hadoop log files
• Develop Hive Query as per client requirement that works seamlessly on Nia Data Clusters
• Daily batch run and bug fixing in real time
• Adapter development from Oracle in Hive using SQOOP
• Working on installing, configuring and supporting Hadoop.
• Acquire complete knowledge of the Hadoop ecosystem and Hadoop Architecture
• Have worked on critical scenarios and given feasible and efficient solutions
• Product development and release management
• Jobs Execution monitoring

Client: RB, P&G

Team Size: 5

Profile: Hadoop Developer

Duration: 1 Year

Tools: HDFS, MapReduce, Pig, Hive, Hue, SQOOP, Flume, Cloudera Cluster

Databases: MySQL, SQL Server 2008, Oracle

Role and Responsibility:

• Worked in Development and Enhancement Environment
• Migration from MySQL, SQL Server and Oracle in Hive
• Transformation script writing in Pig
• Daily batch run and bug fixing in real time

• Have worked on critical scenarios and given feasible and efficient solutions

• Have stretched many times to deliver deliverables in time
• Data Analysis and pattern comparison
• Transformation script writing in Pig
• Worked on several semi-structured formats to process datasets in Pig like JSON, XML , CSV, Fixed length file
• Worked on bad files to reprocess in Pig using UDF
• Implemented custom UDF in Pig
• Worked on Hadoop and Spark integration from scratch

ETL using SQL server and Java:

• Requirement Analysis Distributors and Retailers into TradeEdge platform

• Design & Development of Requirement Specification, UAT, SOP, and Automation Script in Trade edge using SQL Server 2008R2 and Java
• Design the process Execution Plans for Reports and Operations automation using Java.

Importing different sources to database and processing on the database with high performance query and giving required output for report generation tools.

Database Activity in SIMS:

Working on live deployed application changes and working with login if any fails due to technical issue or flaws in coding. Work to repair the record and maintain the database integrity with details analysis in working of the application till the application will be stable.

Setting up modules on server using Linux scripts. Maintain and Support Linux server and database servers.

Attendance and Transport Management:

Mobile application for SIMS (Student Information Management System) for taking attendance from the android application. ERP for college attendance management and transport management

SIMS is used in different type of school, college and training institutes for student & management.

• Using the Roll-Call
• Using NFC technology
• Current Location using Alert
• Current Location using GPS

Client: 30+ Professional Institute and Colleges clients

Team Size: 5

Profile: Mobile Java Application Developer

Duration: 3 Years

Databases: SQLite, MySQL

Tools: Eclipse, SVN

Language: J2ME and Android 3.0 to 5.1