I've participated in a project where we built a Data Infrastructure from scratch at the client. The client is a mining company in Brazil, our job was to bring all the data from the source systems in a unified Data Warehouse, in order to increase data maturity of the company, eliminate time wasted into preparing data manually by the analists and power data driven in the company. This project took 3 years and still in progress. We now have more than 300 data pipelines running in production, more than 200 users thoughout the company. For the technologies, we use resources in Azure:

• (Mostly) Data Factory and Databricks to pull data from the source systems and also perform ETL
• Azure Data Lake Gen2 to storage data
• Azure Data Warehouse (now it's called Azure Dedicated SQL Pool)
• Power BI to visualize data

I've helped the team since the beginning to not only creating data pipelines but also constantly evolving the infrastructure. Some of the work I did:

• Reusable functions and libraries to simplify development and avoid rework
• Unifying repositories from multiple tools in one to simplify development
• Implementation of CI/CD pipeline when deploying to production
• Checklist of good practices and quality standards that devs have to fill before deploying anything to production, to assure quality
• Data Quality dashboard, fed by the result of auto-generated SQL queries that scan all tables in production for quality issues (null values, duplicates, out of date tables, etc)
• Welcome-Kit for new people and videos showing how our infra work, automating the onboarding process
• Mandatory Kick-Off document to assure alignment of the scope and deadlines with the client
• Backlog sheet where everyone has to write "what I expect to do" in the beginning of the sprint and"what I actually did" at the end, to make management easier
• Prioritization score based on questions about the task that quantify effort and benefits, preventing prioritizing the wrong thing
• etc

I've worked in a project in a Bank in Brazil. Our purpose was creating ~30 software bots to automate already mapped repetitive processes. At that time we didn't had any robust RPA tool (like UiPath) and the client did not intend to buy one. So in order to complete the mission, we had to create one. I've created a system in C# to orchestrate software bots in several computers. It basically consisted of:

• Orchestrator: The "brain" of the robots. Responsible for storing schedule and parameter information, triggering robots and managing concurrency and back-end related data.
• Executor: A program that runs 24/7 in the desired computers. Waits an order from the Orchestrator to start a robot in the machine.
• Robot: The robot itself. We used a home-made software that could execute commands like "click button", "execute javascript", "type into text" (and dozens of other commands) to automate tasks.
• User Interface: The front-end part of the solution, where the user could see the robots executed or in execution, treat errors, trigger a manual execution or schedule new robots.

I developed the orchestrator, executor, the user interface as well as some robots, and we had another consultants that also helped me with the development and build the other robots.