Hello, I'm Rafael Nadalin, 22 years old, graduated in Systems Analysis and Development and this is my
DATA ENGINEERING PORTFOLIO
Projects .
ETL - Redesim Web Scraping
Introduction:
This project automates data extraction from the Redesim website, monitoring license protocol statuses. The application uses Firebase to store results and send email notifications when changes occur.
Project Goal:
The main objective is to perform automated data extraction and processing through web scraping, along with sending automatic update alerts.
Development Process:
The main challenges were configuring the scraping using `BeautifulSoup` and `requests` to access Redesim data and implementing email notifications with `smtplib`. Firebase integration enabled efficient data storage. Technologies used: Python, Firebase, Google Cloud Platform, Web Scraping (BeautifulSoup and Requests), SMTP.
Conclusion:
The project enhanced my skills in automation with web scraping and API integration. Future plans include implementing Airflow for cloud scheduling and adding Docker containers.
ETL - YouTube Data Pipeline
Introduction:
This project performs extraction, transformation, and loading of data from YouTube channels using YouTube Analytics API and Google Cloud Console.
Project Goal:
The focus was practicing API requests, authentication methods, and building an ETL pipeline.
Development Process:
Challenges included creating an authentication flow with YouTube Analytics API and building the data extraction and standardization pipeline. Technologies used: Python, GCP, YouTube Analytics API, Object-Oriented Programming, and RestAPI.
Conclusion:
I gained deeper understanding of OAuth2 authentication and API report extraction. Future plans include adding Airflow for cloud automation.
ETL - Checklist Fácil
Introduction:
This ETL extracts information from Checklist Fácil API, focusing on performance and non-compliance recurrence.
Project Goal:
The objective is to collect, process, and store audit data to facilitate non-compliance monitoring.
Development Process:
I used Python and GCP to perform API-based extraction, filtering completed audits and identifying non-conformities. The process was automated with Cloud Scheduler and stored in Cloud Storage.
Conclusion:
The project improved my automation and API integration skills. Future plans include optimizing data flow and expanding functionalities.
Web Application - RPG System
Introduction:
The idea came from a hobby and the difficulty of finding an online RPG system that matched my vision.
Project Goal:
The goal was to build a web application focused on hosting online RPG sessions, character creation, and skill systems.
Development Process:
Development required Flask application routes, MongoDB Atlas for storing class, enemy, and skill information, GCP for VM hosting and Cloud Storage for player files, websockets for real-time event updates, and JavaScript for backend communication.
Conclusion:
This project was a significant challenge that involved constant learning about FrontEnd, sockets, sessions, data manipulation, SSL, and NGINX. I'm extremely happy to have made it functional.
Test Access:
If you'd like to test the site, create your character and join a session, here are test credentials.
Login: test
Password: test
About .
Currently working at WPP Media Services as a Data Engineer
Graduated in Systems Analysis, currently working as a Data Engineer focused on evolving and specializing in Cloud Data Engineering.
💼 Tools: Python, SQL, GCP, PySpark, Docker, MySQL, MongoDB, Flask, Pandas, GitHub Actions.
🧳 Skills: Data processing, Rest API, Code versioning (Git), CI/CD, Cloud computing, Linux.
Experience .
Mirum Agency - Data Engineering Intern
2023
During my internship at Mirum Agency, I worked on:
- Maintenance and interpretation of codes used in ETL/ELT processes
- Configuration and maintenance of Google Cloud Platform (GCP) services
- SQL query development and BI team support
- Python endpoint creation for media API integration (TikTok, Twitter, Pinterest, Google Ads)
- Client technical support including account configuration and service monitoring
Safe7 - Junior Data Engineer
2024
As a Junior Data Engineer at Safe7, my main responsibilities included:
- Business context analysis and solution development to maximize team productivity
- Database restructuring and modeling for greater efficiency
- Development of internal web tool "Safe7 Toolkit" centralizing processes
- ETL pipeline creation using GCP services with Python, web scraping and REST APIs
WPP Media Services - Data Engineer
2024 - Present
As a Data Engineer at WPP Media Services, my responsibilities include:
- Developing and optimizing data pipelines in cloud environments (GCP)
- Automating data ingestion, transformation and storage processes (ETL/ELT)
- Monitoring and performing preventive maintenance on pipelines
- Collaborating with cross-functional teams to implement data-driven solutions
Contact .
Reach me via email: rafael.nadalin@outlook.com
or Phone: +55 (41) 98413-5989