Lesson 2 of 6

First submission

A dumb baseline and a first Random Forest. Why the training score lies.

Left: what & whyRight: the code and its real output

02 · Baseline: a first submission, end to end

Goal: go from raw CSV to a scored submission as fast as possible. It doesn't need to be good, it needs to work.

Two baselines:

Dumb baseline: predict the same tier for every row. This is the floor: any real model must beat it.
First real model: minimal cleaning + a Random Forest with default settings.

import pandas as pd
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score
from utils import load_data, basic_prep, save_submission, TARGET, TIER_TO_NUM, SEED
from score import score

train, test = load_data()

Baseline 1: always predict the most common tier

most_common = train[TARGET].mode()[0]
print("most common tier:", most_common)
path = save_submission(test, [TIER_TO_NUM[most_common]] * len(test), "02_majority")
print("leaderboard:", score(path))

most common tier: budget
leaderboard: {'public': 0.4032}

Around ⅓, as expected with 3 balanced tiers. Every idea from now on must beat this.

Baseline 2: Random Forest

Models only understand numbers with no blanks. basic_prep in utils.py makes three minimal fixes:

Missing area → fill with the train median. Missing furnishingstatus → fill with the most common value in train. We use train's values for test too, because at prediction time we can only use what we learned from training.
yes/no columns → 1/0.
furnishingstatus → unfurnished=0, semi-furnished=1, furnished=2. It has a natural order, so numbers in that order make sense (ordinal encoding).

The target also becomes a number: budget=0, mid=1, premium=2.

tr, te = basic_prep(train, test)
FEATURES = [c for c in tr.columns if c not in ("id", TARGET)]
X, y = tr[FEATURES], tr[TARGET].map(TIER_TO_NUM)
X_test = te[FEATURES]
print(FEATURES)
X.head()

['area', 'bedrooms', 'bathrooms', 'stories', 'mainroad', 'guestroom', 'basement', 'hotwaterheating', 'airconditioning', 'parking', 'prefarea', 'furnishingstatus']

	area	bedrooms	bathrooms	stories	mainroad	guestroom	basement	airconditioning	parking	prefarea	furnishingstatus
0	7420.0	4	2	3	1	0	0	1	2	1	2
1	8960.0	4	4	4	1	0	0	1	3	0	2
2	9960.0	3	2	2	1	0	1	0	2	1	1
3	7500.0	3	3	1	1	0	1	1	2	1	1
4	5750.0	3	2	4	1	1	0	1	1	1	0

model = RandomForestClassifier(n_estimators=300, random_state=SEED, n_jobs=-1)
model.fit(X, y)

train_acc = accuracy_score(y, model.predict(X))
print(f"accuracy on the TRAINING data: {train_acc:.4f}")

accuracy on the TRAINING data: 1.0000

Nearly 100%! Are we done? Let's submit and see.

path = save_submission(test, model.predict(X_test), "02_random_forest")
print("leaderboard:", score(path))

leaderboard: {'public': 0.629}

What just happened?

The model is clearly useful: it's far above the dumb baseline. But the training score lied. The forest memorised the training rows, so asking it about rows it has already seen is like grading a student on the exact questions they studied.

We can't submit to Kaggle after every idea. There's a daily limit, and the public leaderboard here is only ~60 houses, so it's very noisy. We need an honest score we can compute ourselves → 03_validation.