Work Summary:- Developed and deployed data pipelines using Apache Airflow to migrate data from MySQL SQL Server and Oracle Fusion to Snowflake ensuring seamless data transition. Implemented ngrok to manage Pipedrive events facilitating real-time database updates and improving data accuracy. Utilized Airbyte for ETL processes and pipeline scheduling optimizing data extraction transformation and loading operations. Created and managed views in Snowflake enhancing data accessibility and reporting capabilities. Flask to handle events from Pipedrive redirecting through ngrok to manage real-time updates.

Work Summary:- Developed and implemented a log storage system to streamline debugging of API failures during data extraction ensuring data integrity and proper formatting. Engineered functions for CRUD operations utilizing DynamoDB and S3 enhancing data management efficiency. Designed and created APIs to extract data from social platforms such as Facebook Instagram YouTube and TikTok expanding data collection capabilities. Secured services by integrating Azure Key Vault ensuring sensitive information such as credentials and API keys are stored and accessed securely. Collaborated with cross-functional teams to deliver high-quality solutions driving continuous improvement in data processing workflows.