Experienced Data Engineer skilled in designing, and optimizing data pipelines, Improved efficiency by 30% using Python, SQL, and AWS. Proficient in Spark, Hadoop, real-time analytics, data validation.
In this groundbreaking AWS Data Engineering series, we delve into the intricacies of building a robust real-time data pipeline using DynamoDB, Snowflake, and AWS Lambda. The content unfolds as a captivating narrative, showcasing hands-on demonstratio...
In today’s digital landscape, customer engagement plays a crucial role in business success. Automating SMS and Email notifications can significantly enhance customer interactions, ensuring timely communication. This blog will walk through the impleme...
Architecture
Overview The Airline Data Ingestion project aims to automate the ingestion of daily flights transactional data into a Redshift data warehouse using AWS services. The process begins with data landing in an S3 bucket, which triggers a series of ETL ope...
Tokyo Olympic Azure Data Engineering Project This project demonstrates a data engineering pipeline on Azure, designed to handle, process, and analyze Tokyo Olympic data. It highlights the use of various Azure services for end-to-end data ingestion, p...
Step 1 : Create RDS Database a. SQL Server Standard Edition b. In the security group allow TCP traffic from all sources on port 1433 Step 2 : Create Database, table , enable CDC on table and insert the data into table a. execute th...