{ "cells": [ { "cell_type": "markdown", "id": "226d3493", "metadata": {}, "source": [ "# 02 · Baseline: a first submission, end to end\n", "\n", "**Goal:** go from raw CSV to a scored submission as fast as possible. It doesn't need to be good, it needs to *work*.\n", "\n", "Two baselines:\n", "1. **Dumb baseline**: predict the same tier for every row. This is the *floor*: any real model must beat it.\n", "2. **First real model**: minimal cleaning + a Random Forest with default settings." ] }, { "cell_type": "code", "execution_count": 1, "id": "bd30a0d7", "metadata": { "execution": { "iopub.execute_input": "2026-10-23T06:11:30.026063Z", "iopub.status.busy": "2026-10-23T06:11:30.026063Z", "iopub.status.idle": "2026-10-23T06:11:33.611994Z", "shell.execute_reply": "2026-10-23T06:11:33.611994Z" } }, "outputs": [], "source": [ "import pandas as pd\n", "from sklearn.ensemble import RandomForestClassifier\n", "from sklearn.metrics import accuracy_score\n", "from utils import load_data, basic_prep, save_submission, TARGET, TIER_TO_NUM, SEED\n", "from score import score\n", "\n", "train, test = load_data()" ] }, { "cell_type": "markdown", "id": "46309e88", "metadata": {}, "source": [ "## Baseline 1: always predict the most common tier" ] }, { "cell_type": "code", "execution_count": 2, "id": "94bb5d5b", "metadata": { "execution": { "iopub.execute_input": "2026-10-23T06:11:33.611994Z", "iopub.status.busy": "2026-10-23T06:11:33.611994Z", "iopub.status.idle": "2026-10-23T06:11:33.636869Z", "shell.execute_reply": "2026-10-23T06:11:33.636341Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "most common tier: budget\n", "leaderboard: {'public': 0.4032}\n" ] } ], "source": [ "most_common = train[TARGET].mode()[0]\n", "print(\"most common tier:\", most_common)\n", "path = save_submission(test, [TIER_TO_NUM[most_common]] * len(test), \"02_majority\")\n", "print(\"leaderboard:\", score(path))" ] }, { "cell_type": "markdown", "id": "3fe7a2fd", "metadata": {}, "source": [ "Around ⅓, as expected with 3 balanced tiers. **Every idea from now on must beat this.**" ] }, { "cell_type": "markdown", "id": "6ec32a51", "metadata": {}, "source": [ "## Baseline 2: Random Forest\n", "\n", "Models only understand **numbers with no blanks**. `basic_prep` in `utils.py` makes three minimal fixes:\n", "- **Missing `area`** → fill with the *train* median. **Missing `furnishingstatus`** → fill with the most common value in train.\n", " We use train's values for test too, because at prediction time we can only use what we learned from training.\n", "- **yes/no columns** → 1/0.\n", "- **`furnishingstatus`** → unfurnished=0, semi-furnished=1, furnished=2. It has a natural order, so numbers in that order make sense (**ordinal encoding**).\n", "\n", "The target also becomes a number: budget=0, mid=1, premium=2." ] }, { "cell_type": "code", "execution_count": 3, "id": "adaf4280", "metadata": { "execution": { "iopub.execute_input": "2026-10-23T06:11:33.639949Z", "iopub.status.busy": "2026-10-23T06:11:33.638396Z", "iopub.status.idle": "2026-10-23T06:11:33.688684Z", "shell.execute_reply": "2026-10-23T06:11:33.688684Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "['area', 'bedrooms', 'bathrooms', 'stories', 'mainroad', 'guestroom', 'basement', 'hotwaterheating', 'airconditioning', 'parking', 'prefarea', 'furnishingstatus']\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
areabedroomsbathroomsstoriesmainroadguestroombasementhotwaterheatingairconditioningparkingprefareafurnishingstatus
07420.042310001212
18960.044410001302
29960.032210100211
37500.033110101211
45750.032411001110
\n", "
" ], "text/plain": [ " area bedrooms bathrooms stories mainroad guestroom basement \\\n", "0 7420.0 4 2 3 1 0 0 \n", "1 8960.0 4 4 4 1 0 0 \n", "2 9960.0 3 2 2 1 0 1 \n", "3 7500.0 3 3 1 1 0 1 \n", "4 5750.0 3 2 4 1 1 0 \n", "\n", " hotwaterheating airconditioning parking prefarea furnishingstatus \n", "0 0 1 2 1 2 \n", "1 0 1 3 0 2 \n", "2 0 0 2 1 1 \n", "3 0 1 2 1 1 \n", "4 0 1 1 1 0 " ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "tr, te = basic_prep(train, test)\n", "FEATURES = [c for c in tr.columns if c not in (\"id\", TARGET)]\n", "X, y = tr[FEATURES], tr[TARGET].map(TIER_TO_NUM)\n", "X_test = te[FEATURES]\n", "print(FEATURES)\n", "X.head()" ] }, { "cell_type": "code", "execution_count": 4, "id": "bfe8e1c4", "metadata": { "execution": { "iopub.execute_input": "2026-10-23T06:11:33.688684Z", "iopub.status.busy": "2026-10-23T06:11:33.688684Z", "iopub.status.idle": "2026-10-23T06:11:34.353785Z", "shell.execute_reply": "2026-10-23T06:11:34.353785Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "accuracy on the TRAINING data: 1.0000\n" ] } ], "source": [ "model = RandomForestClassifier(n_estimators=300, random_state=SEED, n_jobs=-1)\n", "model.fit(X, y)\n", "\n", "train_acc = accuracy_score(y, model.predict(X))\n", "print(f\"accuracy on the TRAINING data: {train_acc:.4f}\")" ] }, { "cell_type": "markdown", "id": "15b89bb5", "metadata": {}, "source": [ "Nearly 100%! Are we done? Let's submit and see." ] }, { "cell_type": "code", "execution_count": 5, "id": "1f9035b7", "metadata": { "execution": { "iopub.execute_input": "2026-10-23T06:11:34.356827Z", "iopub.status.busy": "2026-10-23T06:11:34.356827Z", "iopub.status.idle": "2026-10-23T06:11:34.455073Z", "shell.execute_reply": "2026-10-23T06:11:34.454540Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "leaderboard: {'public': 0.629}\n" ] } ], "source": [ "path = save_submission(test, model.predict(X_test), \"02_random_forest\")\n", "print(\"leaderboard:\", score(path))" ] }, { "cell_type": "markdown", "id": "0e168d25", "metadata": {}, "source": [ "## What just happened?\n", "\n", "The model is clearly useful: it's far above the dumb baseline.\n", "But **the training score lied**. The forest *memorised* the training rows, so asking it about rows it has already seen is like grading a student on the exact questions they studied.\n", "\n", "We can't submit to Kaggle after every idea. There's a daily limit, and the public leaderboard here is only ~60 houses, so it's very noisy.\n", "We need an **honest score we can compute ourselves** → **03_validation**." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.10" } }, "nbformat": 4, "nbformat_minor": 5 }