Loading repository data…
ameetsinghmanyal / repository
spotify-etl-aws-data-pipeline-project
Welcome to the repository for an ETL pipeline that extracts data from the Spotify API, transforms it, and loads it into AWS. This pipeline provides a comprehensive end-to-end solution for retrieving and analyzing music data from Spotify.
#athena#aws-lambda#aws-s3#boto3#cloudwatch#crwaler
Live repository statistics
★ 3Stars
⑂ 0Forks
◯ 0Open issues
◉ 3Watchers
33/100
OPENREPOHUB HEALTH SIGNAL
Limited signals
A transparent discovery signal based on current public GitHub metadata.
Recent activity35% weight
10
Community adoption25% weight
7
Maintenance state20% weight
100
License clarity10% weight
0
Project information10% weight
75
This score does not audit code, security, maintainers, documentation quality, or suitability. Verify the repository and its current documentation before adoption.
README preview
Spotify ETL End-To-End Pipeline Project
Overview
Welcome to the repository for an ETL pipeline that extracts data from the Spotify API, transforms it, and loads it into AWS. This pipeline provides a comprehensive end-to-end solution for retrieving and analyzing music data from Spotify.
Architecture
About Spotify API
The Spotify API is a web-based platform that provides developers with access to Spotify's music data and services. It allows developers to build applications that can interact with Spotify's music streaming platform, enabling users to browse and play music, manage playlists, search for songs, and access metadata about artists, albums, and tracks.
In this project we have used the Top 50 - India Playlist as our data
Services Used
- Amazon S3: Amazon Simple Storage Service (S3) is a cloud-based object storage service provided by Amazon Web Services (AWS). It is designed to store and retrieve large amounts of data, such as photos, videos, and other types of files, from anywhere on the web
- CloudWatch: Amazon CloudWatch is a monitoring and observability service provided by Amazon Web Services (AWS). It is designed to collect and track metrics, collect and monitor log files, and set alarms
- AWS Glue Crawler: AWS Glue Crawler is a service that automatically crawls data stores, such as Amazon S3, RDS, and Redshift, to infer schemas and partition structures. It can also identify changes in schemas and update the Data Catalog accordingly.
- AWS Lambda: AWS Lambda is a serverless computing service provided by Amazon Web Services (AWS). It enables developers to run code without provisioning or managing servers. Lambda runs code in response to events and automatically scales up or down to match the incoming request traffic.
- : Amazon Athena is an interactive query service provided by Amazon Web Services (AWS) that makes it easy to analyze data stored in Amazon S3 using standard SQL. It allows users to quickly and easily query data without having to set up and manage complex infrastructure or data warehousing tools.