kaggle_house_price_predict_regression GitHub Details, Stars and Alternatives | OpenRepoFinder
omrfrkaytnc / repository
kaggle_house_price_predict_regression
This project involves predicting house prices using the LightGBM algorithm, showcased through a Kaggle competition notebook. The Jupyter Notebook demonstrates thorough data preprocessing, feature engineering, and model evaluation to achieve accurate price predictions. This portfolio piece highlights my proficiency in data science and machine learni
A transparent discovery signal based on current public GitHub metadata.
Recent activity35% weight
10
Community adoption25% weight
5
Maintenance state20% weight
100
License clarity10% weight
0
Project information10% weight
35
This score does not audit code, security, maintainers, documentation quality, or suitability. Verify the repository and its current documentation before adoption.
README preview
House Prices Prediction - Regression Project with LightGBM
This repository contains a Jupyter Notebook for the House Prices Prediction competition on Kaggle. The notebook demonstrates data preprocessing and the use of the LightGBM algorithm to predict house prices. This project is part of my portfolio to showcase my skills in data science and machine learning.
https://www.kaggle.com/competitions/house-prices-advanced-regression-techniques
Table of Contents
Introduction
Data Description
Methodology
Algorithm Used
Results
Conclusion
Acknowledgements
Author
License
Introduction
Predicting house prices is a critical task in real estate and finance. This project aims to build a regression model to predict house prices using a dataset provided by Kaggle. The notebook includes data preprocessing, feature engineering, model training, and evaluation steps.
Data Description
Ask a home buyer to describe their dream house, and they probably won't begin with the height of the basement ceiling or the proximity to an east-west railroad. But this dataset proves that much more influences price negotiations than the number of bedrooms or a white-picket fence. With 79 explanatory variables describing (almost) every aspect of residential homes in Ames, Iowa, this competition challenges you to predict the final price of each home.
Methodology
Data Preprocessing
Handling missing values: Missing values were filled with the mean, median, or mode.
Encoding categorical variables: Categorical variables were transformed into numerical values using label encoding and one-hot encoding.
Skewness correction: The target variable (SalePrice) was log-transformed to approximate a normal distribution.
Exploratory Data Analysis (EDA)
Data visualization: Various plots were used to understand the relationships between features and the target variable.
Correlation analysis: Correlations between features were analyzed, and highly correlated features were identified.
Feature Engineering
Creating new features: A total of 299 new features were derived from existing ones.
Feature selection: The most effective features were selected, and unnecessary features were removed to improve model performance.
Model Training
Training the LightGBM model: The LightGBM regressor was trained on the training data.
ALGORITHMICALLY RELATED
Similar Open-Source Projects
Selected from shared topics, language and repository description—not editorial ratings.
Beginner's tutorial for working with the Google Earth Engine Python API in Jupyter Notebooks. The tutorial walks through a GEE workflow of measuring urban development surrounding Denver International Airport. Future work will involve development of a linear regression model to predict urban development surrounding DIA. This repository serves as a capstone project submission for Earth Data Analytics Professional Certificate Final Project.
Project Overview Welcome to the Convolutional Neural Networks (CNN) project in the AI Nanodegree! In this project, you will learn how to build a pipeline that can be used within a web or mobile app to process real-world, user-supplied images. Given an image of a dog, your algorithm will identify an estimate of the canine’s breed. If supplied an image of a human, the code will identify the resembling dog breed. Sample Output Along with exploring state-of-the-art CNN models for classification, you will make important design decisions about the user experience for your app. Our goal is that by completing this lab, you understand the challenges involved in piecing together a series of models designed to perform various tasks in a data processing pipeline. Each model has its strengths and weaknesses, and engineering a real-world application often involves solving many problems without a perfect answer. Your imperfect solution will nonetheless create a fun user experience! Project Instructions Instructions Clone the repository and navigate to the downloaded folder. git clone https://github.com/udacity/dog-project.git cd dog-project Download the dog dataset. Unzip the folder and place it in the repo, at location path/to/dog-project/dogImages. Download the human dataset. Unzip the folder and place it in the repo, at location path/to/dog-project/lfw. If you are using a Windows machine, you are encouraged to use 7zip to extract the folder. Download the VGG-16 bottleneck features for the dog dataset. Place it in the repo, at location path/to/dog-project/bottleneck_features. (Optional) If you plan to install TensorFlow with GPU support on your local machine, follow the guide to install the necessary NVIDIA software on your system. If you are using an EC2 GPU instance, you can skip this step. (Optional) If you are running the project on your local machine (and not using AWS), create (and activate) a new environment. Linux (to install with GPU support, change requirements/dog-linux.yml to requirements/dog-linux-gpu.yml): conda env create -f requirements/dog-linux.yml source activate dog-project Mac (to install with GPU support, change requirements/dog-mac.yml to requirements/dog-mac-gpu.yml): conda env create -f requirements/dog-mac.yml source activate dog-project NOTE: Some Mac users may need to install a different version of OpenCV conda install --channel https://conda.anaconda.org/menpo opencv3 Windows (to install with GPU support, change requirements/dog-windows.yml to requirements/dog-windows-gpu.yml): conda env create -f requirements/dog-windows.yml activate dog-project (Optional) If you are running the project on your local machine (and not using AWS) and Step 6 throws errors, try this alternative step to create your environment. Linux or Mac (to install with GPU support, change requirements/requirements.txt to requirements/requirements-gpu.txt): conda create --name dog-project python=3.5 source activate dog-project pip install -r requirements/requirements.txt NOTE: Some Mac users may need to install a different version of OpenCV conda install --channel https://conda.anaconda.org/menpo opencv3 Windows (to install with GPU support, change requirements/requirements.txt to requirements/requirements-gpu.txt): conda create --name dog-project python=3.5 activate dog-project pip install -r requirements/requirements.txt (Optional) If you are using AWS, install Tensorflow. sudo python3 -m pip install -r requirements/requirements-gpu.txt Switch Keras backend to TensorFlow. Linux or Mac: KERAS_BACKEND=tensorflow python -c "from keras import backend" Windows: set KERAS_BACKEND=tensorflow python -c "from keras import backend" (Optional) If you are running the project on your local machine (and not using AWS), create an IPython kernel for the dog-project environment. python -m ipykernel install --user --name dog-project --display-name "dog-project" Open the notebook. jupyter notebook dog_app.ipynb (Optional) If you are running the project on your local machine (and not using AWS), before running code, change the kernel to match the dog-project environment by using the drop-down menu (Kernel > Change kernel > dog-project). Then, follow the instructions in the notebook. NOTE: While some code has already been implemented to get you started, you will need to implement additional functionality to successfully answer all of the questions included in the notebook. Unless requested, do not modify code that has already been included. Evaluation Your project will be reviewed by a Udacity reviewer against the CNN project rubric. Review this rubric thoroughly, and self-evaluate your project before submission. All criteria found in the rubric must meet specifications for you to pass. Project Submission When you are ready to submit your project, collect the following files and compress them into a single archive for upload: The dog_app.ipynb file with fully functional code, all code cells executed and displaying output, and all questions answered. An HTML or PDF export of the project notebook with the name report.html or report.pdf. Any additional images used for the project that were not supplied to you for the project. Please do not include the project data sets in the dogImages/ or lfw/ folders. Likewise, please do not include the bottleneck_features/ folder.
Hyperparameter optimization: Grid search and cross-validation were used to find the best hyperparameters.
Model Evaluation
Performance metric: The model's performance was evaluated using RMSE (Root Mean Squared Error).
Cross-validation: Cross-validation was used to assess the model's generalization ability.
Algorithm Used
The regression algorithm used in this project:
LightGBM Regressor
Results
Number of Features: 81 (Before feature engineering)
Target Variable: SalePrice (Log-transformed)
New Features Created: 371 (After feature engineering)
Model Performance Metric: Evaluated using RMSE
Best Model RMSE Value: 0.1281 (After Log-transformed)
Conclusion
This project demonstrates how the LightGBM algorithm can be applied to predict house prices. Detailed information about the model's usage and performance is provided. The results show that the model can accurately predict house prices.
Acknowledgments
Kaggle for providing the dataset and competition platform.
The data science community for sharing knowledge and resources.
Author
Ömer Faruk Aytunç
For more details and the complete notebook, visit the Kaggle Notebook.
License
This project is licensed under the MIT License - see the LICENSE file for details.
Konut Fiyatları Tahmini - LightGBM ile Regresyon Projesi
Bu depo, Kaggle'daki Konut Fiyatları Tahmini yarışması için bir Jupyter Notebook içermektedir. Notebook, veri ön işleme ve LightGBM algoritmasını kullanarak konut fiyatlarını tahmin etmeyi göstermektedir. Bu proje, veri bilimi ve makine öğrenimi konusundaki becerilerimi sergilemek amacıyla portföyümün bir parçasıdır.
Konut Fiyatları - İleri Düzey Regresyon Teknikleri
İçindekiler
Giriş
Veri Açıklaması
Yöntem
Kullanılan Algoritma
Sonuçlar
Sonuç
Teşekkür
Yazar
Lisans
Giriş
Konut fiyatlarını tahmin etmek, gayrimenkul ve finans alanında kritik bir görevdir. Bu proje, Kaggle tarafından sağlanan bir veri kümesini kullanarak konut fiyatlarını tahmin etmek için bir regresyon modeli oluşturmayı amaçlamaktadır. Notebook, veri ön işleme, özellik mühendisliği, model eğitimi ve değerlendirme adımlarını içermektedir.
Veri Açıklaması
Bir ev alıcısına hayalindeki evi tarif etmesini sorsanız, muhtemelen bodrum tavanının yüksekliği veya doğu-batı demiryoluna yakınlığı ile başlamaz. Ancak bu veri kümesi, yatak odası sayısı veya beyaz çitli bir bahçeden çok daha fazlasının fiyat müzakerelerini etkilediğini kanıtlıyor. Iowa, Ames'teki konutların (neredeyse) her yönünü tanımlayan 79 açıklayıcı değişkenle bu yarışma, her evin nihai fiyatını tahmin etmenizi zorlar.
Yöntem
Veri Ön İşleme
Eksik değerlerin ele alınması: Eksik değerler ortalama, medyan veya mod ile dolduruldu.
Kategorik değişkenlerin kodlanması: Kategorik değişkenler, etiket kodlama ve tek sıcaklık kodlama kullanılarak sayısal değerlere dönüştürüldü.
Çarpıklık düzeltmesi: Hedef değişken (Satış Fiyatı) normal dağılıma yaklaştırmak için log dönüşümüne tabi tutuldu.
2 Keşifsel Veri Analizi (EDA)
Veri görselleştirme: Özellikler ile hedef değişken arasındaki ilişkileri anlamak için çeşitli grafikler kullanıldı.
Korelasyon analizi: Özellikler arasındaki korelasyonlar analiz edildi ve yüksek korelasyonlu özellikler belirlendi.
Özellik Mühendisliği
Yeni özelliklerin oluşturulması: Mevcut özelliklerden toplam 299 yeni özellik türetildi.
Özellik seçimi: En etkili özellikler seçildi ve gereksiz özellikler model performansını artırmak için çıkarıldı.
Model Eğitimi
LightGBM modelinin eğitimi: LightGBM regressor eğitim verileri üzerinde eğitildi.
Hiperparametre optimizasyonu: En iyi hiperparametreleri bulmak için Grid Search ve çapraz doğrulama kullanıldı.
Model Değerlendirmesi
Performans metriği: Modelin performansı RMSE (Kök Ortalama Kare Hatası) kullanılarak değerlendirildi.
Çapraz doğrulama: Modelin genelleme yeteneğini değerlendirmek için çapraz doğrulama kullanıldı.
Kullanılan Algoritma
Bu projede kullanılan regresyon algoritması:
LightGBM Regressor
Sonuçlar
Özellik Sayısı: 81 (Özellik mühendisliğinden önce)
Hedef Değişken: Satış Fiyatı (Log dönüşümlü)
Oluşturulan Yeni Özellikler: 299 (Özellik mühendisliğinden sonra)
Model Performans Metriği: RMSE kullanılarak değerlendirildi
En İyi Model RMSE Değeri: 0.1281 (Log dönüşümünden sonra)
Sonuç
Bu proje, LightGBM algoritmasının konut fiyatlarını tahmin etmek için nasıl uygulanabileceğini göstermektedir. Modelin kullanımı ve performansı hakkında ayrıntılı bilgiler sağlanmaktadır. Sonuçlar, modelin konut fiyatlarını doğru bir şekilde tahmin edebildiğini göstermektedir.
Teşekkür
Kaggle, veri seti ve yarışma platformunu sağladığı için.
Bilgi ve kaynak paylaşan veri bilimi topluluğuna.
Yazar
Ömer Faruk Aytunç
Daha fazla ayrıntı ve tam notebook için Kaggle Notebook sayfasını ziyaret edin.
Lisans
Bu proje MIT Lisansı altında lisanslanmıştır - ayrıntılar için LICENSE dosyasına bakın.
This project involves segmenting customers using k-means clustering in Jupyter Notebook. Customer segmentation is a powerful technique used in marketing and business analytics to divide customers into distinct groups based on their behaviors, preferences, or demographics.
This project involves the analysis of student performance using Seaborn plots in Jupyter Notebook. The dataset contains information about students' demographics, study habits, and performance in various subjects. Through this analysis, we aim to gain insights into the factors that influence student performance.
This project involves predicting height based on age using polynomial regression in Jupyter Notebook. Polynomial regression is a variation of linear regression that models the relationship between the independent variable (age) and the dependent variable (height) as an nth-degree polynomial.
This project involves detecting fake news using a decision tree classifier in Jupyter Notebook. Fake news detection is an important task in the field of natural language processing and machine learning, as it helps identify and filter out misleading or false information.