{ "cells": [ { "cell_type": "markdown", "id": "4726ee4a", "metadata": {}, "source": [ "# 05 · Models and tuning\n", "\n", "Features are fixed now (from 04: the 12 raw columns). Here we change **the model**:\n", "1. try several model families with default settings, same CV,\n", "2. understand why **small data** changes which models win,\n", "3. learn **early stopping** (how many trees to build),\n", "4. tune each model a little, then compare fairly,\n", "5. try the **ordinal trick**: the tiers have an order, so treat the problem as regression." ] }, { "cell_type": "code", "execution_count": 1, "id": "e8e52c06", "metadata": { "execution": { "iopub.execute_input": "2026-10-23T06:13:03.589342Z", "iopub.status.busy": "2026-10-23T06:13:03.589342Z", "iopub.status.idle": "2026-10-23T06:13:09.521892Z", "shell.execute_reply": "2026-10-23T06:13:09.521892Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(408, 12) (137, 12)\n" ] } ], "source": [ "%matplotlib inline\n", "import warnings\n", "import numpy as np\n", "import pandas as pd\n", "import matplotlib.pyplot as plt\n", "from sklearn.linear_model import LogisticRegression\n", "from sklearn.preprocessing import StandardScaler\n", "from sklearn.pipeline import make_pipeline\n", "from sklearn.ensemble import RandomForestClassifier\n", "from sklearn.model_selection import train_test_split, RepeatedStratifiedKFold\n", "from sklearn.metrics import accuracy_score\n", "from lightgbm import LGBMClassifier, LGBMRegressor, early_stopping, log_evaluation\n", "from xgboost import XGBClassifier\n", "from catboost import CatBoostClassifier\n", "from utils import make_xy, cv_score, save_submission, load_data, basic_prep, add_features, SEED\n", "from score import score\n", "\n", "warnings.filterwarnings(\"ignore\")\n", "X, y, X_test, test = make_xy()\n", "print(X.shape, X_test.shape)" ] }, { "cell_type": "markdown", "id": "61291f20", "metadata": {}, "source": [ "## 1. Model shoot-out (default settings)\n", "| model | idea in one line |\n", "|---|---|\n", "| Logistic Regression | one weighted sum of the columns per tier; simple, needs scaled inputs |\n", "| Random Forest | many deep trees on random samples, then a **vote** |\n", "| LightGBM / XGBoost / CatBoost | **gradient boosting**: trees built one after another, each fixing the previous ones' mistakes |\n", "\n", "`StandardScaler` rescales every column to mean 0 / std 1. Logistic Regression needs it, because otherwise `area` (thousands) would drown out `bathrooms` (1–4). Trees don't care." ] }, { "cell_type": "code", "execution_count": 2, "id": "e08bda63", "metadata": { "execution": { "iopub.execute_input": "2026-10-23T06:13:09.527114Z", "iopub.status.busy": "2026-10-23T06:13:09.527114Z", "iopub.status.idle": "2026-10-23T06:13:52.113653Z", "shell.execute_reply": "2026-10-23T06:13:52.112145Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "LogisticRegression 0.7190 ± 0.0481\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "RandomForest 0.6961 ± 0.0303\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "LightGBM 0.6953 ± 0.0371\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "XGBoost 0.6879 ± 0.0401\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "CatBoost 0.7132 ± 0.0340\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
modelcv_accstd
0LogisticRegression0.71900.0481
4CatBoost0.71320.0340
1RandomForest0.69610.0303
2LightGBM0.69530.0371
3XGBoost0.68790.0401
\n", "
" ], "text/plain": [ " model cv_acc std\n", "0 LogisticRegression 0.7190 0.0481\n", "4 CatBoost 0.7132 0.0340\n", "1 RandomForest 0.6961 0.0303\n", "2 LightGBM 0.6953 0.0371\n", "3 XGBoost 0.6879 0.0401" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "defaults = {\n", " \"LogisticRegression\": lambda: make_pipeline(StandardScaler(), LogisticRegression(max_iter=3000)),\n", " \"RandomForest\": lambda: RandomForestClassifier(n_estimators=300, random_state=SEED, n_jobs=-1),\n", " \"LightGBM\": lambda: LGBMClassifier(random_state=SEED, verbose=-1),\n", " \"XGBoost\": lambda: XGBClassifier(random_state=SEED),\n", " \"CatBoost\": lambda: CatBoostClassifier(random_seed=SEED, verbose=0),\n", "}\n", "results = []\n", "for name, make in defaults.items():\n", " mean, std, _ = cv_score(make, X, y, verbose=False)\n", " results.append({\"model\": name, \"cv_acc\": round(mean, 4), \"std\": round(std, 4)})\n", " print(f\"{name:20s} {mean:.4f} ± {std:.4f}\")\n", "pd.DataFrame(results).sort_values(\"cv_acc\", ascending=False)" ] }, { "cell_type": "markdown", "id": "4438e09e", "metadata": {}, "source": [ "**The simplest model wins.** Logistic Regression is at the top, and default LightGBM/XGBoost are at the bottom.\n", "\n", "### Why? Small data.\n", "With 400 rows, flexible models (deep trees, hundreds of boosting rounds) have plenty of room to **memorise noise**.\n", "A simple model *can't* memorise much, and here the true pattern is fairly simple anyway: bigger + more bathrooms + AC + good area = pricier.\n", "The defaults of LightGBM/XGBoost were designed for datasets with thousands to millions of rows.\n", "\n", "Fix: **restrict** (regularise) the flexible models: smaller trees, fewer rounds, bigger leaves." ] }, { "cell_type": "markdown", "id": "1b8371a9", "metadata": {}, "source": [ "## 2. Early stopping: how many trees?\n", "A boosting model adds trees one by one. **Early stopping** watches a validation set and stops once the validation error hasn't improved for N rounds." ] }, { "cell_type": "code", "execution_count": 3, "id": "aea8c1fa", "metadata": { "execution": { "iopub.execute_input": "2026-10-23T06:13:52.114159Z", "iopub.status.busy": "2026-10-23T06:13:52.114159Z", "iopub.status.idle": "2026-10-23T06:13:52.660331Z", "shell.execute_reply": "2026-10-23T06:13:52.660331Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "best number of trees: 167\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAArMAAAFzCAYAAAAt54EyAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjExLjIsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvgI3uAAAAAAlwSFlzAAAPYQAAD2EBqD+naQAAbYxJREFUeJzt3Qd4FGXXBuAnvfdCCARC7713EBQUUFFEEEHAgn6KKIqK5bd8FlT8xC52sQAKSlVAmnTpvYSahCSkkN7b/td5h12TkECyJNn23Nc17OxmdneyQ3bPnjnvee10Op0OREREREQWyN7UO0BEREREZCwGs0RERERksRjMEhEREZHFYjBLRERERBaLwSwRERERWSwGs0RERERksRjMEhEREZHFcoQNKi4uRmxsLLy8vGBnZ2fq3SEiIiKiMmQqhIyMDISGhsLevuL8q00GsxLIhoWFmXo3iIiIiOgaoqOjUb9+/Qp/bpPBrGRk9S+Ot7e3qXeHiIiIiMpIT09XyUd93FYRmwxm9aUFEsgymCUism4FBQX48ssv1fqDDz4IJycnU+8SEVXBtUpCbTKYJSIi26q7S0xMNKwTkXVhNwMiIiIislgMZomIiIjIYrHMgIiIiCyirWZ+fr6pd4OqkdSvOzg4XPfjMJglIiIisyZB7Llz51RAS9bF19cXISEh19X3n8EsERERmS0ZtBcXF6cyeNKm6WrN88myjmt2djYSEhLU9bp16xr9WAxmiYjIqknGx8fHx7BOlqWwsFAFPTILlLu7u6l3h6qRm5ubupSANjg42OiSAwazRERk9XV5TzzxhKl3g4xUVFSkLp2dnfkaWiH3y19QpB+0scEsc/VERERk9phVt0521XC2hMEsEREREVksBrM1LSUS2PAGsP2jGn8qIiKqeDpbWWSdyBI99thjuP322029G2aJwWxNiz8CbH4H2PEJUKzV/RARUe2RUdOxsbFq4XS2VFsefvhhjB49mi94LeAAsBq236ULmtt7wiMjDojcDjTqV9NPSURERFbm448/NvUumC1mZmtYkZ0zVuZ3VevFhxfX9NMRERGRib344ouYN28elixZogY4ybJu3TpMmjQJo0aNwtSpU+Hv749mzZqp7V9//XXDdtJGbtCgQdi3b99Vywz0j/XYY4+p/rvyeBMnTkRWVhZsDYPZGtYxzBfrHPuq9eKjS4FCTsVHRER0Xc328wtNslS2TEWCUwlY77zzTnUfWYYMGaJ+tnTpUtSrVw9nz57FqVOnDMGvfjuZ6axfv3645ZZbkJGRcdXnkcdq0KABjhw5gq1bt2LTpk147733bO4/F8sMavoFdrCHS9OBSDz1EYLyUoGzG4HmQ2v6aYmIiKxSTkERWv/fGpM897HXhsLd+fpCpzZt2uD//u//Kvy5ZFhfe+01fPHFF9iyZYsKaivSrVs3PPPMM2pdMrrjxo1T97E1zMzWgv4tQ7CyqKd2haUGRERENqt169ZX3BYVFaUCUZnSVSYOkHKD+Ph4dfvV6MsU9Pz8/JCSkgJbw8xsLRjQPAgPF/XGZMc10J1YCbvcdMDVuzaemoiISswyRJbPzclBZUhN9dzXq7yZzMaOHaum6928ebMqG3BxcUF4eLiayvdqOJGEhsFsLajj7Yqc4E44nRKKpgWxwOFfgW7318ZTExHZPAkeZs6cafOvg7WQAO56T/XX1jTKxcXFlZqud9euXWqAmD7TGhcXh+jo6FrYS+vAMoNaMrBlHSwoukG7svdbqWCvracmIiKiWtawYUMcO3YMycnJV91OygqaNm2K+fPnIz09HWfOnMH48eMrFQiThsFsLZYaLCnqhzw4ARcPA7GlW24QERGR9Zg8ebIKaKVcQN+aqyI//vgjDh8+jODgYPTv3x+9e/e+oh6WKmans8HpUOSbj4z6S0tLg7d37dSu5hcWo9Nra/G67kOMctgGdJ4I3MopbomIappMYfvTTz+pdcl4yelfshy5ubmqXVWjRo3g6upq6t2hWjy+lY3XmJmtJc6O9ujTNBA/Fw7Wbji8BJCBYEREVKMkZxMZGakWG8zfEFk9BrO1aECLIOzWtUC0QxhQkKUNBCMiIiIiozGYreW6WcAO3+UN1G7gQDAiIiKi68JgthbV93NH02BPLC7shyJ7Z20gWAwHghEREREZi8GsCbKzafDEAa8S2VkiIiIiMgqD2Vo2sIWUGgBfZPXTbjgiA8HSans3iIiIiKwCg9la1i3cX02HtyazMXL9mgMF2cDe72t7N4iIbIq042JLLiLrxGC2lrk6OVzOztpho/9d2o07PwMK82t7V4iIbGY62+eff14tsk5E1oXBrAkMaxuiLj+I7wR4hgAZscCRxabYFSIiIiKLxmDWBAa1DIaTgx1OJOUjqe1k7cZtH0pnb1PsDhEREZmhTz/9FK+99lqF18vz7bff4sUXX6z25zZnDGZNwNvVSc0GJn53GAo4ewKJx4FTf5lid4iIrFphYSF+/vlntcg6kaU4duwY9u3bV+H18pw8eRJ79uyp0vN89NFHeOONN6763ObMpMFsfn4+Fi5ciIEDByIkJATbt2+v1P3kDalHjx4IDw/HyJEjceTIEViaYW20UoPlJ7KALpO0G7d/aNqdIiKyQsXFxTh16pRaZJ3IUj366KN4+eWXq/1xjx49iv3799fKc9UER1M++QsvvICoqCg89NBDGD9+vApur2Xx4sWYNGmSSn/36tULc+bMUcGwfIMIDg6GpRjSug7sfz+MwzFpiLttEur+8zlwfgsQsxeo18XUu0dERERG2rx5M/73v/+pmMXR8d9Qa8GCBVi3bh2+/vprtf7991o3Ix8fH3Tu3BnTpk2Du7t7hY+7ceNGJCUloVOnTobb1q1bp0oLHBwcMGjQoCvuc63nmT9/Pv744w8Vgw0bNkzdNnv2bJVgLPtc//zzj9r3+Ph4NGvWDI8//jgaNGhg+Pl7772nvjC2bNkSy5cvR2Zmpko63nPPPdYbzL799tuwt7fHhQsXKn2f119/Hffddx8eeOABdf3LL79EaGgoPvvsM4v5BiECPV3QNdwfu84l488oR0xpOxo4tFCrnR3DVl1ERETlkvEl0tbSFJzcATu7a27Wvn17rF69GmvXrsUtt9xiuF0CXEnACTnD7Ofnp9aTk5Mxb948/P7779ixYwfsKngOSdyVjJnWrFmDESNG4Omnn0aHDh3wzTffYOfOnejdu7dhm2s9jyQG27Vrh/T0dDzxxBNqu7CwsCuea8OGDSrYfeyxxzB48GAVJEtgfPjwYdStW1dtI+vy2PKYkniMjY1V8ZpOp1NJS6sMZiWQrQp5oQ8ePKjaq+jJNx55UeVbkCWWGkgwu/roRUy5/XEtmD2+HEg+C/g3NvXuERERmR8JZN8MNc1zPx8LOHtcczNfX18MHz4cP/30kyGYjYiIULWsX331lbreuHFjtejdfvvtqFOnDrZu3Yp+/fpVbneef16VA7z11lvq+h133IFGjRqV2uZazyMZVgleJQurz8yW55lnnsGUKVNUQC7GjBmjAmiptf34448N2/n7+2PFihWGvs4nTpzAL7/8UqPBrEUNAIuJiVGXchBKkusS/VckLy9PBcIlF3Mw9HKLrt3nk5Ho3hRoOgTQFQNb3jP1rhEREdF1kOBt6dKlyMrKUtclsG3btq0KAEVBQYE6/T958mQV+EogKhnM06dPV+rxJbY5cOBAqcyv9FG+6aabSm13vc8jpARBamqlZEBPsrq33nqrKj0oqVu3bqUmKJHgOi4uDjXJpJnZqpIXX5SsP9FfLyoqqvB+8o3l1Vdfhbmp5+uG9vV9cOhCGtYdj8e4Ac8Bp9cBBxYAfZ4EApuaeheJiIjMi5zqlwypqZ67kiRwlOBSAloJbGXwur5EUkydOlVlR6V+VepOXVxccOjQIUPwey1paWmqPtXb27vU7XK9ZPB4vc9T8rmk5rYkuS6lCyWVnZhEgt6aHnhpUcFsUJDMnAWVCi8pMTHxqoO/Zs2ahRkzZhiuS2ZWUurmYGibEBXMrjl6EeO6dweaDwMiVgOb3gRGf2Pq3SMiIjIvUk9aiVP9piZB4+jRo1VGtmnTpjhz5oxhIJQEd9LNSepLhw4dasi0pqSkVPrxJe7x8PBQj9uzZ0/D7SUzrpV9nopqdEvGX56enqojSN++fQ23S+lE2bIGU7CoMgN5MaUdl3zDKGnLli3oLoHgVf5DyTeVkou5kGBWbDudhPTcAmDQC9oPjiwBLlpeyzEiInMjmSIZICwLp7Ol2nTvvffir7/+wvvvv48BAwYYEmkyZsjV1VUFonry/zM3N7dKjy/B8fvvv6+6BggZ/CWDwvQq+zwSX128ePGav8vcuXORmpqqrh8/flwFyhMnToSpmX0w+9RTTxlG/gkZRSfF03v37lWlBVKILKPtpL2XJWoa7KmWgiId/joaD9RtD7S+XfvhOsvpzkBERESl9e/fX430X7RokQoGy3Z0krPG0nlABmFJtwB9V4CqdHgqKChAkyZNVAcBGZRVdvBYZZ7nrrvuUp0IJDEog8CkFrcsGeglZQWSZZbH6tq1K8aOHYsJEyaY/LDb6fSFqCYgB3f69OkqDS6lAtI6Qr41S4sJWYTUl8g3Df3ECLK7zz77LD755BN1P/k2IaPopAi5sqTMQA6I1ICYQ5b2g3Wn8P66CPRrFogf7u8BXDoDfNIDKC4Axi8Gmt1o6l0kIiIyCckinjt3Tp3OliyjpZEuTFLDKkGmlAWUJLfLjF0BAQFqcNi2bdtUXau+d6tkP+X31/d6LXtdFBUVqS4JMuiqdevWqn9/RkYGunTpUunnERITyeQJEiPJIK6EhIQrnktIyy7J4uq7IJQkAbHEadKaTE+OnWwvwXZVj29l4zWTBrM5OTlqB8uSugxZ9L+IfOuQA1CSTEkoB0vaX1yr1sPcg9nIS1kY8O4m2NsBO58fjGAvV2DNC8COj4HA5sAj2wGHf0cGEhFR5cnnhdQMilGjRl0xiJjMm6UHs3R11RHMmrTMwM3NTU1jW3bRB7JCdr5sICvkzUgyuVUNZM1RwwAPdG7gi2IdsOLg5RGIA54B3AOBpAhg99em3kUiIoslZ/EkmyQLp7Mlsj5mXzNrK0Z1qqcul+7XeunC1Qe44fJgsE1vAdmlW18QEREREYNZszG8fSgc7e1wOCYNpxO0UYnoNBEIbgPkpgKbZpt6F4mIiIjMDjOzZsLfwxkDmmt9dJcduJyddXAEhmlT1GH3V0DiSRPuIREREZH5YTBrRm6/XGrw+/4Yw2xnaDwAaDEc0BUBa5437Q4SERGZiAnHq5OZH1cGs2ZkSKs68HRxxIWUHOyNLDE7x03/BeydtKluT/1lyl0kIiKqVQ4ODuoyPz+fr7wVys7OVpfSWsxY7E9iRtycHTCsbQgW772gsrNdw/21HwQ0AXpM1Vp1SXa28UC26iIiIpsg3Yvc3d1VP3oJeGRWK7KOjGx2drbqZyttVvVfWoxh0j6zpmJufWZL2noqCfd+/Q983Z2w6/khcHa8/EebmwZ82BnITgJufkcLbomI6JrkY076lQsJhqyhpaOtkays9CJlazXr4+vrq9qylvd3Wdl4jZlZM9OrSQCCvVyQkJGHTScTcFObkNKtulY+CWx8E2h3F+B+OXNLREQVkg9JmV2SLJccP5lxiqUG1sXJyem6MrJ6DGbNjIO9HW7rGIovt5zDkn0X/g1m9a26dn0FJBzVWnXd8o4pd5WIiKjWSHkBZwCj8rDwxAyN7qLNdbz+eAKSMvP+/YFq1fWmts5WXURElZ7OdunSpWqRdSKyLgxmzVCLEC90CPNFYbEOv++73HNWTwZ/lWzVZXslz0REVSJ1lgcPHlQLay6JrA+DWTN1d1ctO7toT/SVPdhKtuo6+adpdpCIiIjIDDCYNVMjOtSFq5O9mtp2f3Rq6R9Kq67ej2nrfz4L5Gs92oiIiIhsDYNZM+Xt6oRb2tZV64t2RV+5Qf+ZgHd9IC0K2Pp+7e8gERERkRlgMGvGxnZvoC6XHYxBWrbWI9HA2QMY9pa2vm0ucOmMCfaQiIiIyLQYzJqxbuF+aFHHC7kFxVi878KVG7QaCTQZDBTla+UGHAxGRERENobBrJk3+p7Qq6Fa/3FnJIqLywwEk9kybnkXcHAGTv8FnFhlmh0lIiIiMhEGs2ZuVKd68HRxxLmkLGw7k3TlBmow2OPa+urngPysWt9HIiJzn2Xo6aefVousE5F1YTBr5jxcHHFn53pqff6OyPI36vcU4NMASIsG1rxQuztIRGQBZ7k8PDzUUt7870Rk2RjMWgB9qcH64/GISc25cgNnd+C2j+UtG9j7LcsNiIiIyGYwmLUATYO90KtxAKRk9ud/KsjONh4A9J6mrS97DMi4WKv7SERkrmQK21WrVqmF09kSWR8GsxZi4uXs7MJd0cgrLCp/oxteBELaAznJwNJHZA7H2t1JIiIzJFPY7tmzRy2czpbI+jhW9Q6pqalYvnw5Nm/ejAsXtHZRYWFh6N+/P2699Vb4+PjUxH7avCGt66COtwvi0/Ow+shF3NZRq6MtxdEFuPMrYN4A4MwG4J/PgV7/sfnXjoiIiKxXpTOzSUlJeOyxx1CvXj3MnDkTcXFxCA0NVUtsbCyeeuop9bNp06bh0qVLNbvXNsjJwR73dG949YFgIqgFMPQNbX3dy8DFw7W0h0RERERmHMy2adMGeXl52LhxI+Lj41Xt0TfffKMWWU9ISMD69euRm5urtqXqN657GBzt7bA3MgVHY9Mq3rDrFKD5zdpkCovvZ7suIiIislqVDmZ37NiBL7/8Et27d69wmx49eqhttm/fXl37RyUEe7tiaNsQwyQKFZLWM9LdwDMESDoJrJ7F15GIiIhsO5ht3LixYX3Tpk2V3paq14SeWqnB0v2xSMspqHhDj0Dgji+0dl37vgeOLOGhICIiIqtjVDeDIUOGQKcrM7Uq1YoejfzRvI4ncgqKsGSvNgCvQtKuq98MbX3po0DcwVrZRyIiIiKzDmbDw8MRERFR/XtD1ySz10zoFW4oNbjml4qBzwNNhwCFOcCCcUBGPF9lIrIpMoXt9OnT1cLpbImsj1HB7IsvvohJkyZh586dSE9PV4O+Si5Us0Z1qgdPF0ecTcrCttPX6Bzh4AiM/gYIbA6kxwA/jwHyMniIiMimkgC+vr5q4XS2RNbHqGB28uTJKpDt1auX6ivr5uZWaqGaJYHsHZ21PrM/7Dx/7Tu4+gDjFgLuAUDcAWDhPUABv3QQERGRDU6aILZs2VL9e0JVcm/Phqrf7F/H4hGbmoNQ32t8iQhoAoxfDHw/Eji3GVhyP3DX91rmlojIihUVFanWkWLw4MFwcHAw9S4RUTUyKpLp27dvde4DGaF5HS/0bOyPnWeTsWBXFJ66qcW171SvMzBuAfDjncCJlcDK6cCtH2utvIiIrDiYlfaSYuDAgQxmiayMUWUGoqCgANu2bcMPP/xguI0zf9WuCT21gWALdkUjr7Cocndq1F+robWzB/b/CPz1fzW7k0RERETmFsxGRUWhc+fO6nTNxIkTDbc/+OCDWLFiRXXuH13FTW3qoI63C5Iy87D8QGzlX6tWI4GRH2rr2z8Ets7l60xERES2E8w++eSTavCXdDIo6emnn8bs2bOra9/oGpwc7DG5TyO1/sXmsygurkLv384TgBv/q62vexnY+z1fbyIiIrKNYPbvv//Gm2++CWdn51K3t2vXDnv37q2ufaNKuKdHA3i5OOJUQiY2nkyo2mvW53GgzxPa+songGPL+ZoTERGR9Qez0kvW3l67a8mefXFxcXB3d6++vaNr8nZ1wj09G6j1eX+frforNuQVoPNEQFesdTg4e/WpiomIiIgsPpiV0aCff/55qWA2KysLM2fOVFPdUu2a0qcRnBzssOt8MvZGplTtznL8RszV6miL8oEF9wDn2HqNiIiILIOd7przoV7pxIkT6N+/P5o3b47t27fjzjvvNPSelQ4HTZo0gTmTWl+Z7CEtLQ3e3t6wBs8sPohf9lzA0DZ1MG9C16o/QGEe8PPdwNmNgIMLMOZ7oMXNNbGrRES1Sj7mEhMT1XpQUBBnASOyEJWN14zKzLZs2RJHjhzB0KFDMXLkSFV2MHXqVBw8eNDsA1lr9VD/xupy7bF4nEnMrPoDOLpos4S1GA4U5QELxwMHF1X/jhIR1TI5gxgcHKwWTmdLZH2MysyOHTsWCxcurPLPzIU1ZmbFA9/vwbrj8RjbLQyz72xv3IMUFQLLHgUOXT6Gt8wBuj9YrftJREREZNLM7KJF5WfsJC7+5ZdfjHlIqgYPD9Cys7/ti0FCeq5xDyLT297+GdB9qnb9j6eBda8AxcU8RkRksTOAbdq0SS2yTkTWpUrBbGpqqlpKruuX5ORkrFy5EnXr1q2pfaVr6Brujy4N/ZBfVKz6zhpNOlXc/DYw4Dnt+tb3gUX3AnlGlC8QEZmYBLDSUlIWBrNENh7M+vn5qaXkun4JCAjA7bffriZUINN5fHAzdfnDzkjEG5ud1Xc5GDQLGPWFNiDs5Crgm6FAalT17SwRERHRdXKsysa7d+9Wl926dTOs6zk5OSEsLAz+/v7Xu090Hfo3C1TZWWnR9dmmM3jl1jbX93p2uBvwbwwsvAeIPwJ8MQgY+xPQoCePExEREVlWZrZr165qGTFihGFdv3To0EEFsjIAjExHRurOuLG5Wv/5nyjEpeVc/4OGdQMe3ACEtAOyk4DvRwIHfr7+xyUiIiK6TkYNAJPa2PJwAJh56N0kAN0b+ava2Y83nK6eB/UNA6asAVqO0CZXWPoIsPYloJiDKYiIiMh0OADMyrOzv+yJRnRydvU8sLMHMOYHoP8z2vXtH2rlB3kZ1fP4RERERFXEAWBWqmfjAPRpGoCCIh0+2VhN2Vl9p4MbXgDu/BpwdAUiVgNf3wSknK++5yAiIiKqiUkT9uzZYxUDwKx10oSy9pxPxujPd8DB3g4bnhqAhgEe1fsEF/ZqmdnMi4B7AHD3j0DD3tX7HERE16m4uBhxcXFqXdpH2suXciKymnjNqBnAzp8/j/DwcFgqWwlmxcRvdmFzRCJu7xiKuWM7Vf8TpMcCC8YBcQcAe0fgxv8CPR/RWnsRERERmeMMYBLIFhQUYNu2bfjhhx8Mt1+6dMm4vaUaM/OmFupy6YFYHIzWJryoVt6hwOQ/gbZ3AsWFwJpZwK+TgILr6HFLREREVElGBbNRUVHo3LkzBg8ejIkTJxpuf/DBB7FixQpjHpJqSLv6PrijUz21/saq46rjRLVzdtdqaG9+F7B3Ao4tBRaNZ0BLRGZBZv2S5IssnAGMyPoYFczKLF+9evVS6d+Snn76acyePbtKj5WZmYkFCxZg7ty5WL9+faXuExMTg/nz5+ODDz7A77//jry8vCo9p615emgLuDrZY9f5ZKw5erFmnkTKCno8BEz4DXByB06vu9zpgFPgEpFpSQC7bt06tTCYJbI+RgWzMr/1m2++CWdn51K3t2vXDnv37q3040hQ2r59e8yZMwdHjx7F+PHj1aQLV8se/vLLL2jSpAmWLFmCs2fP4pVXXkHz5s0RHR1tzK9iE0J93fBgv8Zq/a0/TyC/sLjmnqxRf2D8r1pAe2Y98OUgIOF4zT0fERER2TSjgtnc3FzDaFDpaaono0Xd3d0r/TjPPfcc/Pz8sGPHDnz55ZfYsGEDFi9erLKtFXnttddUacOyZctUZnbXrl3qm/a8efOM+VVsxsMDmiDIywWRl7Ixf0cNt9EK7wtMXAZ4hQJJEdoUuMdZfkJERERmEswOHDgQn3/+ealgNisrCzNnzsSQIUMq9RgSgErQet999xkyvK1bt0bfvn3x66+/Vng/Nzc3uLi4GK47OjqqxcOjmttOWRkPF0c8fZM2kcKH608hJSu/Zp8wrDvw8Bag8SCgMAf4ZSKw+6uafU4iIiKyOUYFs1IWIDWuEnhKScBdd92lTv3/888/eOuttyo9iEwC4BYttNH2enL9+PGKT0t/8cUXKpM7efJkVWJw4403qvrdadOmVXgfqamV+t6Siy0a3SUMLUO8kJ5biA/Wn6r5J/QIBMYvBjrfB+iKgVVPAetfk3mPa/65iYiIyCYYFcy2bNkSR44cwdChQzFy5EhVdjB16lQcPHhQBbWVHfglpH9YSb6+voaflUeeKycnBwkJCUhOTlbtwOQ2aRVWEQmw5Xn0i0zuYItk8oQXh7dW6z/sjMTJi7UwDa2DIzDyA2DQC9r1Le8Byx4Diio+XkRERESV5QgjBQcH46WXXjL27oaygLJZUmmMW1HJQGFhIe644w5MmjTJkAGW2zp16oRnn31WZW3LM2vWLMyYMcNwXZ7TVgPavs0CMaxNCFYfvYiXlh3Bood6lqp7rhHy+AOeATzrACufBA78CKRFA3d9B7ib/4xxREREZIXBrJQI/Pzzz4aSAKl3veeeeyo9AKxBgwZwdXXF6dOncdNNNxlul+vSnaA8MsDs4sWLGDRo0L+/gKOjKneQgWAVkRrbknW2tu6lka2xKSIBu84lY9mBWNx+uQ9tjetynxbQLp4CnPsb+GIAcPdPQN32tfP8RGST5HNCxmfo14nIuhhVZrBnzx40btxYZTz379+vFulMICUG+/btq9RjyBvKiBEj8OOPPxr6/kmrLWn7JdlXvT///BNff/21Wg8NDVXBstTMlpxzWwLZZs2aGfOr2KR6vm6YdoP2er3xx3Gk59biKf8Ww4AH1gF+4UBqFPDVYGDHJ3Iga28fiMimSPcdmblSFn0nHiKyHnY6I6aE6tq1q5oB7MMPP1TZVSF1q48//rgKbHfv3l2pxzl37hx69+6tBn11795d9ZCVDO/KlSsNbzgPPPAAdu7cqWp0hXRRmD59uhp0JgG1NMGOiIjA1q1bVS1vdc71a83yCotw89wtOJuUhSl9GuH/Rmq1tLUmOxlY+h8g4k/tev3uQL+ngGY3ySdP7e4LERERmZ3KxmtGBbPSHksmKQgMDCx1e1JSkiofyM7OrvRjyQCuRYsWIT4+Xk26IFnZkt+cf/vtN9X54IknnjDcduLECRXEyn0bNmyo7lOVoJTBrGZzRCImfrNLDQxb9XhftAyp5cBe/uvt+QZY84LWvkvU7aANGAvtVLv7QkRWS87+6Sf06dKlCxwcHEy9S0Rk6mC2bdu2WLhwobosSbKnMoOXPotqrhjM/us/P+3FH4cvonu4PxY+1BP29nYmOCBxwM5PgT3fAvkZgJ090O1BoOcjgH+j2t8fIrIq+fn5hkHDUh5XdvZKIrLseK3S53OljEC/SGeAu+++G3/88QcSExNVmyxZHzNmDJ566qnq+h2oFkirLjcnB+w6n4wFu6NM85p71wVu+i/w+D6g7Z1aT9pd84APOwI/jgYu7DHNfhEREZHZq3Rmtirtm4xI9tYqZmZL+2brOby28hg8XRyx5sn+aoCYSZ1erw0KO7P+39taDAdueAGo08aUe0ZEFoiZWSLLVNl4rdI9SrZs2VJd+0ZmZlLvcPxxOA57IlPw3JJDmD+le833nr2apoO1JfkssPk94ODPwMlVwMk/gHajgYGzgIDKTc5BRERE1q3Swaz0ciXrJHWy74xuj5s/2IItp5Lw654LGNPNDCaV8G8M3P4J0Gc6sPEN4NhS4PCvwJHfgM4TtFnFPINNvZdERERkQuyBRErjIE/MuFGbrOK/q47hYlqu+bwyQc2BMd8DUzdrrbt0RcDe74BPegBHlph674iIiMiEGMySwQP9GqNDmC8ycgvxwu+Hza/2Wdp2jf8VmLwaCGkH5CRrs4ktGAcknzP13hEREZEJMJglA+k3O2d0ezg72GP9iQQsPRBjnq9Ow17AAxuAAc8B9o5aLa1kadf/F8jPMvXeEZGZkRknx40bpxZOZ0tkfRjMUinN6nhh+hBtqttXlh9DQoYZlRuU5OgMDJoFPLwNaDwQKMoDtswBPuoKHF6sTchARHR5OtvmzZurhdPZElkfBrN0hYf6N0abUG+k5RTgpaVHzK/coKTglsCEpcDdPwG+DYCMWGDJ/cC3twBxh0y9d0RERGSuwezAgQPLXSfL5+Rgj3dHd4CjvR3WHI3H8oOxMGvSRqzVCODRXcCgFwFHNyBqO/DFAGDlDCA72dR7SEQmns72wIEDapF1IrIuRgezf//9d7nrZB1ah3pj2g1aucGLvx9B1KVsmD0nN2DATGDaHqDNHdpMYnu+Bj7oAKyepfWtJSKbIwHssmXL1MJglsj6sMyAKvTooCbo2tAPGXmFeHzhfhQUFVvGq+VTH7jrW2DSKqBOOyAvHdj5KfBhZ+Dnu7UZxoot5HchIiKiq2IwSxVydLDH3LEd4e3qiAPRqZi7LsKyXq3wvlpv2vGLgaY3ykTLQMRq4Mc7gE+6aVPm5qabei+JiIjoOjCYpauq7+eO2Xe2V+ufbjqD7aeTLOsVs7cHmt0I3LsYeGwv0H0q4OwFXDoNrHkemNsW2PgmcOkMOyAQERFZIAazdE23tKuLcd3DVLerJxYdQHJWvmW+aoFNgVveAZ46AYx4HwhsDuSmAX+/DXzUGXivBbDsMeD8NpYhEBERWQgGs1Qp/zeiDZoGeyIhIw/PLD5o3u26rsXFE+g6BfjPTuCu74CGfQEHZyAzHtj/A/DdLcA7jYD5twPrXwOOrwQyE02910RERFQOx/JuJCrLzdkBH47thNs/3YZ1xxMwf0ck7usdbtkvlL0D0GaUthTkAtE7gcO/AkeXAbmpwNmN2iLsHIBmNwFt7wA8gwHPOkBQS60tGBEREZmMnc7IFJudnZ0hO1dy3RKkp6fDx8cHaWlp8Pb2NvXuWJTvt5/Hy8uPwtnRHkv/00e18LI6hflAwlEgZh8Qu0+7TDh25XZ+jYB2o4G2o7XJG4jILBUXF+P48eNqvVWrVpwFjMjK4jWjg9nCwkLDHNcl1y0Bg1njyX+XB+fvUdnZxoEeWPpYH3i7OsHqJUYA++cD0bu1OtvUSKCgRO/dOm2BsB5AcCugyQ1AQBNT7i0REZHFq/Fg1pIxmL0+MgBsxIdbEJuWiyGt6uCLCV1gb29jp9vzs4CTfwKHFwOn1wHFBaV/Xr8b0P5ubfIGjwBT7SUREZHFYjBbDS8OVezQhVSM/nwH8guLMePG5nh8sDZbmE2S6XJlIob4I1pZwvmt2uxjwt5RC2wDmwEBTYGAy5f+jQAHG8hoE5kBlhkQXYXkNGWRMSD6cSA5KUDiSW1gtAyQLi4C0i4AadFAi1uA8D4wp3jNcmoDyKy0r++L129vi2cWH8L76yLQtp43bmhZBzbJ3R9ofxcAWQBkxANHFgOHFgFxB4GoHdpSkr0T4N8YCGqutQiTRQJdaR/m6mOSX4PIWkkp3OLFi9X6rFmz4OzsbOpdIqoeWUlagJl9CSgqBJxcAUc3wNEFKMjRkizpsf9+zkh5XFYCkHQaSDqpBazSd70w9/IDSkBrD+iKKn5ON99aC2Yri8EsGW1M1zCVof1xZxSmLzyA5Y/1RaNAD76iXnWAXo9qS9IpIPaA9mZx6ZR2KW8iBVnaG4ksZUmnBDd/wNlD65wg0/NKTa5M/uAdyteXiMhWFOZpAasEoJIoyYgDMi5qWdILu4Ck6p6ZU/dvIOtdH/CpBxQVaBlb73qAbwPtbKOZMSqY3bRpEwYOHFj9e0MW2X/2eFwG9kam4KH5e7DkP71tY0BYZUl5gSwlFRcDGbHaN2J5I1LLKW3JvKid1pGlPF6hgKu3FtQ26A007A3U66J9GyciIvOXlwlc2A1EbtcyqlKOJu/hrr5ahjTukJb8yErUBhxflR3gFQK4B2qla3J/ycjKpZwBlEHJ8nkhnzPJZwFnT8A94PKZwRbaIhlbNz+tlEBK5GSRZIp81lgIo4LZIUOGoKCgQLXkItsmLbo+G98ZIz7ailMJmXj0p334ZlI3ODlwPo6rTrEr2VZZmg4u/TN545I3nNx0IC9DC25To4HIbcCFPVoQrALhE8CZDdp9pJ5JMrfy5iNvZvKmJtnh5jdrwS7/TomIro9kJ1MigeQz2ql66T0uJWG+YUBmAnB8hfa+7ChBqY+WxPAL1ybcSYsCUmWJ1koCJEitCgl21ft6COBV9/JliPa+L+/x7v42f3SNCmbDw8MRERGBFi1a2PwLSECwtyu+vq8bxszbgS2nkvDS0iN46452/LJjDHkTDO1U8UAzaQkmga58y5YAV77ZSxZXBp6Vtf0jILgNUK+zFuhKgX9RHuDkrpUy6N8UPQIBJzftW3xR/r89dCXoJiKyNvnZQMo5LXGQfPlSAkx5H5RFak7Vuru2vX7cg6GutBrIKfzwvoBfQy0jKgFyTqr2vlunHVCnNeB5+f1ZMrZ8P67+YPbFF1/EpEmT8P7776N169ZXFNO7uvKUp61pV98HH43rhId+2IOFu6PRIsQLk/s0MvVuWRf59q3/Bt54AND9QS1AlTfi+KNaezCZ8CE7CUg4ARxZok3+IEtVeQRrzyEDASRDLG/schpKAl79aShZZJCB2q+A0otsKz9THwp8PyCiGibvhfKFP+4AkHBc++IuJV1S/1lcCKTHae+VEsRK3akxJLj1b6KdfpcANEcSDFGAgwvQYpiWJZXbpXRAsrSSiZVxD1JnKotP2OX1MO09kqqNUX1mr1VeYO6ta9maq+Z8teUsXl91HA72dvjh/u7o3SSwBp+Nrkpaq8ipL8ncSiZCAlMpScjP1G5TAwnitTdeyThIVlZOkclldWYg5M1fZYc9tcEF8mZet6OWHRYSKMu6i/ycqPrl5+fjrbfeUuvsZlDDJJhLj9G+1MoZoYrIl2QJBOX9RzKPEihKnaicdZLHkOnGpbZUphaXulB5n5D3JjmlL+9Pcl0GR0lwmhajlWhdbQR+eWfB5L1J2iRK/ai8B8njS4a0QF93Kks+ENIWaDxIqy8tG//o4x2Wc1lea64tW7Zcz76RFbu/byMcjU3H7/tjVP3sskf7okHA5VM1VLvkm3/niVW/n3xARO0Eondp2VUXL+2NXbIQ8iEjQbF+kTd7yYbIIj+XUbeSGZbTZRK4Cqkxk+VaVDsYnfacKpvREGgyCGh2k1Zf7CLZkEJt/2SfrOXDQ35n+eIhwb5jibNc8iEqH+xSWiJfPuSLRtMh2mtDVeLg4IDbbrvNsE5VIIGd1O7L37h8AZaOLBKEyv9ZCTYl6JRgMy9d+/uX7KdkRaWmtG577f+1tIaSv115T5KAUzKWKkitARKghrTTvhzLPsj+yXuLBKsStErwKmVU1VVnai3vQxaOM4Bx0oRql1tQhNGfb8eRmHSEB7hj8SO9EejpUv1PROYdoMmHlwS1ctpP+u3KdbldAjS5LhliuS6XJacGrgz5UKzbQevuIGUMkuWRD0zJ8sjsbFICIe3N3P0uX0r5g/+/63KaUGrk5ENVyjPkw07un3BMO2XYqB8Q3Fp7HPmZ+uCzA2L2avsvgbYM8JDHSDmvZbxlnySzIx/ykvWRLhNyX6lrlseVAED2TUYXS09h+TCX9jpSIiJfAHC59Y38PjLYRAUAZWaWk23qtNECfslkNeoPNBoA5MtgwQStBjqopVYPzQ9Zuhb5+5TgU/9lVA1SitSCUvl/rJrkX/j3i2lVBizJ33tl/o49grSsqvwNSyAa3FL7/y1fnOX/uPQ0FXJ2Sb7sSQmU/I1IkC2BqgyykrM98rct20rGlqxGjc8AJt0Mdu3ahbNnz2LChAnqtkuXLiEgwPyn7mSZQc1LSM/FqE+3IyY1Bx3q++DnB3vCw4VtjakC8kEmWR4JwOQUopQ/SLPviNVaMCgBry1y8QECmmhBqmTFYvdX8n7eWrsd+bCXgFsCAznlq1/ktK5kqCR4FhK0q7Y8RSXa88ilTmvrI4MSJVhQxylDC54lYJGAnyOpzY8cQ3WcLv/dyLGTbKl8GbuwF4jZo/W/lt6llSFnBeQx5KyAzGAoAaQcd33pkDyffIGT4FT+X8mXPfkyJn1Q5Wfyf0jq7fXZWFU7GqadYSEyVTAbFRWF4cOH49SpU8jLyzPUyN5xxx2YPHkyRo4cadYHh8Fs7TiTmInRn21HSnYBBjQPwlf3dWXLLjKOnHKXD2cHRy2IUtndQ1r2SNXPuWvNvaV9jaq9tdMyTdkpWrZWrSf/uy4ZKRklLB+qksmRLJJkiCRrKt0izm3WMlLyfPoBHVJqEdpR6xAh2atLZwDPIO3UpdomWcscyQe6BOMSOEhAGNZdy9LKB7w8l2RiJZsrwYBkUCWTKosEHtL6R2WKHbT2anKfkhlWfSZXgk0pPTj1l9avUrLNEmhI7aBk2qpSO3i95HeQbLG8ltJ2SPZBXlu5XV5XybrLutQdymeFvHYSBElWWU73yu8kk4hIgCSPoU5bV/90tqdPn1brTZs2hX1tjwxXv3eUdszk/5L8/nJ2QF4rmThFMvYle3xKpl/+f0iGUjKS8oVE34daSm7k/4UEmBJIys9kcJM6w5GjZVXli09l/w/I6Xf5vyP/b2WEvYyuVxO3+GlfoiRbKseTmX6ytmD2zjvvVBnYjz/+GC4uLoZgdvv27Zg5cya2bdsGc8Zgtvbsi0rBPV/uRG5BMe7sXB9z7mrPll1ENR34S42yTMohZQ4SiEhWTALG/MuLBEDSkkiCHtlG1RU6lLiUmujLQaUE7VIWIveXTJpkfSUYk+sSoFUnCeIkqNXXNMpzyXNK1k+COMkKSqb6agOLqjoATF4vCSD1XTfkDIHMsCT9nOVLj/53lsBcspuyvZSTGNblMk8L2vX9oWWR2yXglFrSyB2XS0lqmRx3oS9XkS9e9boC9btqX7BkQBOzo2SrwWxgYCBOnDihLqWzgf4hMjIyEBQUhNzcahwJXQMYzNau9cfj8dAPe1FUrMPDA5rguZtb1vIeENF1qWjEtgRt0gZJssWSYZVTxxJ0SkZcAmWZxEOy0ZJxlm0kWJasomQoz/2tZdZV3bOblrHV9zm+Fv1MeJKVl2Bcgl39Iu3gJNCWfW7QU7VLyk+9iLcW71F3ndUqGs45MsteopbplH0QTh7aY0kJRU2QQF0y1BKkyxcK9btLZr6Fdgrf8KVCSm3ytEy/BMQqiyvlJk217aVTgHyJ0Neg67Pg8npI8CxZejmtL5eS2ZVt5DipOvLL9adEFqJGuxlIsKo/TVOyTVdcXBzc3TlynUob3KoO3hrVDs8sOYTP/z4Dfw8nPNS/CV8mIktR0SlmyepJGYUsRg0SLNJKOYSsS0AsWWUJftWgoHQt2ym1lnKbjKSXMhH9THjXcmLl5RVHwO5xbfXY75KOvXJbCYD1nL20U+xyql1O30swKOUtEhxKgC6DBNW687+XUuoiAaXK5Hpp2+hH9cvrE9q5dLeK2jx2FjQtKZExjApmBw4ciM8//xzPP/+8IZjNyspSJQYy1S1RWWO6hSE5Ox+z/zyBN/84AV93Z4zpGsYXishWyWeHPpAVamR6Q225GslYSumDtIWTTKRkKNXpfQl607S6UdWDNBc4u0mrXfYMA/Rn+Qe/DHhLnWiQNsOSBK2SFdW3npPrPPVOZP3B7Jw5c9C/f3/88ccfqsTgrrvuMvSeNfd6WTIdKTFIycrHvM1n8dySQ/Bxc8LQNiE8JERk3Ex419J7mnaZnw9crplFj4eBMrNWKjwFT2SxjBrS2bJlSxw5cgRDhw5VnQuk7GDq1Kk4ePAgmjTh6WOqmNTLjulaH8U6YNqC/dh+xgSDIoiIiMi2M7OSfe3evTteeuml6t8jsmpSlvLmqHZIzS7A2mPxeOD7PZg/pTu6hlfTbCxERERkU4zqZiDtuBwdHdG7d29VPyuLBLdOTpfbgJg5djMwj1nCHpy/B1tOJcHTxRE/PtADHcM40paIql9RURH27t2r1rt06cIpbYksRGXjNaPKDFJSUrB06VIVwErd7KBBg+Dr64sbb7wRb7zxxvXsN9kIVycHfDGhK3o29kdmXiEmfP0PjsSkmXq3iMgKOTg4qM8rWWSdiKyL0dPZliQzgUkQ++OPP6pvwNXwkDWKmVnzkZVXiEnf7sLu8ynwdXfCzw/0ROtQtpEhIiKydek1mZm9ePEiFi5cqAZ9tWjRAu3bt8e5c+fwwgsvYMOGDdez32RjPFwc8e3k7ujUwFfV0Y7/aicztERU7dPZnj9/Xi2yTkTWxajMrAzikdm/HnroIdVXtlevXnB1vTwVoAVgZtb8pOcWYMLXu3AwOhVeLo74bko3dGnIQWFEdP2uOp0tEdlmZnb8+PFqENhHH32Ed999V13u3r1blRgQGcPb1Qk/3t8d3cP9kaFqaHdh+2m27SIiIqKrMyqYldrYCxcuqNGho0aNwoEDB9Slv7+/6jtLZAwvVyd8P6U7+jULRHZ+ESZ9txsbTsTzxSQiIqLqDWb1wsPD0bZtW7Rp0watWrVSU9quXr36eh6SbJybswO+uq8rbmxdB/mFxXho/l6sOhRn6t0iIiIiawpmZ8+ejWHDhsHPz09Na7ts2TJ07twZK1asUG27iK6Hi6MDPh3fGbd2CEVhsQ7TFuzDr3ui+aISERFR9cwAJj1mBwwYgOnTp6Nfv37w9PQ05mGIKuTkYI/37+4Id2cHLNwdjZmLDyE9txD3923EV42IiIiuL5jduXOnMXcjqhIHezu8dUc7uDs74ptt5/DflceQkJ6LZ4e1hL29HV9NIiIiMi6Y1bt06RIiIiLUJAnSbzYgIIAvKVUraQP30ohWCPJywdurT2De5rNIyMjD23e2h7PjdZV8E5GNkFm/pI2kfp2IrItR0UBOTo6aMKFOnTro3bs3+vTpo9blNvkZUXUHtI8MbII5d3VQ2drf98fg/u93q2lwiYiuRQJY+ZyShcEskfUxKpidOXOmmulr8eLFiIuLUzOCyfr69evVz4hqwugu9VWnAzcnB2w5lYRxX+xEUmYeX2wiIiIbZtQMYEFBQVi7di06depU6vb9+/dj6NChSEhIgDnjDGCW7UB0KqZ8txvJWfloGOCO+VO6o2GAh6l3i4jMlExhK4kXUbduXdjbs0SJCLY+A1hGRgYaNmx4xe1ymzyxMYydL9uIWJwsXMcwXyx+uBfq+7kh8lI27vxsB47FGvf/joisX2FhIb766iu1yDoRWRejglnpKfvOO++UCiRlXfrPdunSpUqPJfNlS72tk5MT2rVrp8oXruXo0aO45ZZb4ObmhuDgYDz33HPIy+PpZlvSOMgTvz3SG63qeqtSg7u/2IFd55JNvVtERERkCcHsnDlz8Omnn6J58+a455571CLrn3/+Od59991KP45s/+abb+Knn35SKeQ77rgDI0aMwLlz5yq8z+nTp1URf6NGjRATE4OoqChV9iAlDmRbgr1dsfChnugW7oeM3EJM+PofrD/O6W+JiIhsiVE1syI+Ph6fffaZypLKaPPWrVvjkUceUVnWypIAWDKsc+fOVddlV6RUYdy4cXj77bfLvc+YMWNw6tQp7Nu3Tz2vMVgza11y8ovw2M/7sP5Egup28M6d7XFnl/qm3i0iMhP5+fnqLKCYNWsWnJ2dTb1LRFSN8ZrRfWYlaH3llVeuq0etBKUyk5ieBKdyfceOHeXep6ioCH/88QdefPFFta3U2bKQn9ycHfD5hC54dskh/LYvBk/9ehCRl7LwxJDmnFyBiIjIylU6mE1KSqr0gwYGBlYqsyukRKAkqYHdtWtXufdJTExEVlaWyuB27doVBw4cUEH1fffdpwLrir5tSz1tyZpaYwepkXlPfztndAcEe7ni87/P4MMNp3EmMUv1ppVgl4iIiGw8mC0bdF5NVSoXynYxkOsVlQ/oH1cGny1btkxN2LB7925VZysZ2tdff73c+8nppVdffbXS+0SWSaa4fe7mlmgS5IHnfz+MVYfjcCElG19O7Krqa4mIiMiGg9nqHmAlvf5E2Z60cj0kJKTCjK90PRg/fjz69++vbuvVqxcmT56M3377rcJgVmqkZsyYUSozGxYWVo2/DZmTu7qGoYG/Ox7+cS8OXkjDbZ9sUwFt23o+pt41IjIBmfVLX9LGGcCIbDiY7dixY7U+sZ+fnxo0tnHjRowePdqQlZXrEpzqSU9AuV1KCCSQ7dmz5xV9AgsKCtTPKuLi4qIWsh09Ggdg6aN91OQKUm5w1+c7MHdsRwxtU/4XJSKyXhLADhw40NS7QUSmbs01bdq0Ss3sJVPbyraV8eyzz+Kbb77BkiVLEBsbq7KnmZmZqiuC3sMPP6z62uo9//zzqpXXihUrkJycjD///BPffvstJkyYUNlfhWyEzAr223/6oF+zQOQUFGHqD3vx8rIjyMpj03QiIiKby8y6urqiadOmGD58OEaOHKkmR5DBV1LHKgGs1K4uX74ca9aswdSpUyv1mBMnTlTBq5QByIAwmTThr7/+Qv36/7ZVkoxryazqsGHD8PXXX6uOBpGRkWjQoIEqL6hsAE22xcfNCd9O6obXVx3Hd9vP4/sdkaqF1+w72qNvs2sPVCQiyyefUzKAWD/+w9i2jkRkBX1mz5w5gw8++ACLFi26Iksrge3YsWNVUNmkSROYM/aZtU1bTiXiuSWHEZOao67f3TUML45oBS/XiktUiMjysc8skWWqbLxm9KQJ0iM2OjpafcOVTGqzZs1gKRjM2i4pMXh3zUmVpRWNAz3wxcQuaBrsZepdI6IawmCWyDLV+KQJErxaUgBLJDxcHPHKrW1wS7u6eGLhfpxNysJtH2/D66Pa4vaO9Xj6kYiIyFoHgBFZk+6N/LF8Wl/0bOyPrPwiPLnoIO77djeik7NNvWtERERUBQxmyWYFerrgx/t74OmbmsPZ0R6bIxIxbO5mLNgVVaWJP4iIiMh0GMySTXN0sMdjNzTDn9P7oVu4n8rSzvrtMCZ9uxsX03JNvXtERERUW8FsRkYGs1lksZoEeWLhQ73w4vBWKkv7d0Qibnr/byzee4H/r4mIiKwtmD1+/Lia8EBv+vTpapRZaGgoDhw4UJ37R1RrHOzt8EC/xvjj8b7oUN8H6bmFePrXg2r2sCMxaTwSRBY8A5hMfS4Lp7Mlsj5Gtea6+eab8dRTT2HIkCGIiIhQM3RJ79n169fjyJEjWLt2LcwZW3PRtRQWFePLLefw4fpTavYwezvgPwOb4vHBzVTmloiIiCy4z6w8sEw/6+HhgY8++gg7d+5UU8ympqaiUaNGSElJgTljMEuVFZeWgzdWHcfKQ3HqessQL7x2W1vVDYGIiIhMH68ZlWKSqW1lClvx559/4oYbbjA0pnZ2djZ2n4nMTl0fN3x8T2d8Or4z/D2cceJiBsbM24FHf9rHAWJEFkJyNpJskYWdSoisj1HB7PDhw3HHHXdgypQp2LJlC0aOHKlulzKDG2+8sbr3kcjkZJKFv57sj3t6NFAlB6sOx2HI//7GDzvOo7iYbbyIzFlBQYGail0WWSci62JUMPvxxx/j1ltvRWFhIVasWIHg4GB1uwS2L7/8cnXvI5FZCPB0wZuj2mHltH7o1MAXmXmFeGnZUYz+fDtOXsww9e4RERHZJKNqZitqzeXp6WkR04GyZpauV1GxDj/9E4l3Vp9UQa2Tgx0m9Q7Ho4OawtedpTZE5kRK4N566y21PmvWLJbDEVmIGq2ZZWsusnXSxmtir3D8NaM/bmxdBwVFOtX9oP87G/HF5jPILSgy9S4SERHZBKOC2RkzZhhqY6U119dff42VK1di3LhxeOaZZ6p7H4nMeoDYlxO74rvJ3VSnA+lN++YfJzD4vb+xdH8M62mJiIhqmKMxd9q+fTsWL16s1tesWYPbbrtNDQrr06ePas1FZGsGtghGv2ZB+G3fBby3NgIxqTl4YtEBfLnlLJ6/pRX6NA009S4SERFZJbbmIqrG0oO7uoZh49MDMXNoC3i6OOJobDrGf/UP7vtmF47FpvO1JiIiMofMrL41V5cuXVQHg++++07dztZcRICbs4MaCDa2Wxg+2nAaP+6MxN8RiWoZ0qoO/jOoCTo38ONLRVRL7O3t0bVrV8M6EVkXo7oZZGdnq5GhkZGRqtfswIED1e3/+c9/8OSTT6JZs2YwZ+xmQLXpfFIW3l17En8cjoP+r61nY381PW6/ZoEW0QGEiIjIqqaztXQMZskUTidkYt7fZ/D7/hgUXp5ooW09bxXUDm0TosoUiIiIqBaD2UuXLqluBvIQLVq0QEBAACwBg1kypdjUHDUwbOGuaORcbuHVONADDw9ogts6hcLF0YEHiKgayWeUnFEU7u7uPBtCZCFqNJjNycnBE088oVpyFRVpH8YODg64//77MXfuXLi5ucGcMZglc5CclY/vtp/H99vPIy1Hm2IzyMsFE3o2VNPmBnq6mHoXiawCJ00gskw1OmnCzJkzsWHDBtWeKy4uDhcvXlTrMgBMfkZE1+bv4YwZNzbHtuduwAu3tEKItysSM/Lwv78i0Hv2Bjyz+CBOXGQHBCIioqsxKjMbFBSEtWvXolOnTqVu379/P4YOHYqEhASYM2ZmyRwVFBWrQWLfbD2HgxfSDLf3aRqAyb0bYVDLYNbVEhmBmVkiy1TZeM2o1lwZGRlo2LDhFbfLbfLERFR1Tg72uK1jPdzaIRT7olLwzdbz+PNIHLadvqSWMH83TOwZjjFdw+Dj7sSXmIiIyNgyg86dO+Odd95RRfV6sj579mzVe5aIjCeturo09Mcn4ztj8zOD8FD/xvBxc0J0cg7e+OM4ery1DrN+O4TjcfziSEREZFRmds6cORg2bBiWLFmCbt26qdt2796N+Ph4rF69mq8qUTWp7+eupsN9ckhzLDsQowaMnbiYgQW7otXSPdwf43qE4ea2deHqxC4IRERke4zKzPbu3RunTp3C+PHjUVBQgMLCQrUut8nPiKj6ZxUb270B/pzeD79M7YXh7eqq+tld55Px5KKD6PbGOry09AiOxPxba0tERGQLOGnCVQqKiczZxbRc/LInWi0XUnIMt7cJ9VZT6d7asZ4qTyCydZJwWblypVofMWIEHB2NOilJRJbeZzYpKanSTx4YGAhzxm4GZE2Ki3XYfuYSFu6Owtqj8cgvKla3uzja45Z2dXF3tzD0aOTPRvFERGTbwWxV5o839xlyGcyStUrJylfT5S7aHY2T8RmG28MD3DGmWxhGd66PYG9Xk+4jERGRSYLZAwcOoLI6duwIc8Zglqyd/FlLr9pFu6Ow/EAssvIvz9Rnb4dBLYJVGcLAFkFwdDCqbJ7I4v4eZHyHcHJy4lkKIgtRo9PZWjoGs2RLsvIKsepwnMrW7o1MMdwe7OWC0V3qq7614YEeJt1HoprESROILFONTppQVnh4OM6fP18dD0VE1czDxVEFrLKcTshQQe1v+2KQkJGHTzedUUvPxv6qtpYtvoiIyNJUSzAbGRlZHQ9DRDWsabAXXhjeGjOHtsT64/FYuDsam08lYufZZLX837KjuL1jPRXYtq3nw+NBRERmj/1JiGyQs6M9bm5XVy0xqTlYvOeCavEl6z/sjFRL23reuLsrW3wREZENBLMdOnSojochIhOo5+uG6UOaYdoNTbHtTJLK1v51NB5HYtJxJOYoXl91HDe1CcGtHUIxoHmQCoSJiIisKpitSqcDIjJP9vZ26NcsSC3JhhZfUYiIz8SKg7FqkUkYbm4bgpEdQlXvWnZDICIiiwxmL1y4UOHPXFxcEBAQAHt7Zm+ILJW/hzPu79sIU/qE49CFNCw7EIuVh2LVoDHJ3Mri5+6Em1qHYFi7EPRpEsiMLRERmYRRrbmuNYGCtE+47777MGfOHDg7O8PcsDUXUdUVFevwz9lLWH4wFmuOXkRKtta3U3i7OqpShBHt66JP00A4sX8tmdl0tr///rtaHzVqFKezJbIQNdpn9pNPPsHbb7+Nl156CZ07d1bB7Z49e/Daa69hxowZqFOnDl544QVMmDAB//3vf2FuGMwSXZ/ComLsOpeMP49cxOqjF5GYkWf4mWRsh7UNwfB2oarlF0sRiIjI7IJZCWA/++wz9OjRo9TtO3fuxKOPPoq9e/di8+bNuP/++3Hq1CmYGwazRNWnuFiH3eeTsfJQHP48EoekzHzDzwI9nVVgO6J9KLqF+6sZyIiIiEwezLq7uyM2Nha+vr6lbk9NTUW9evWQlZWFzMxM1K1bFxkZ/84Pby4YzBLVXMb2n3MS2MZi9ZHSpQgy49gt7eqqUoTODfzUgDMiIiKTzADWoEEDfPTRR6rMoKS5c+eqn4mjR4+iY8eOxjw8EVkoKSmQmllZXrutLbafuYSVl2tsZfDYd9vPq6Wuj6shsO0Y5nvNOnyi68HpbImsm1HBrAStUkS/aNEidOnSBZLcldKCs2fPGorsFy9ejNdff72695eILIQMApO+tLK8Maodtp5OxMqDcVh7LB5xabn4eus5tdT3c8Pw9nVxS9u6aFfPhxlbIiKqEqPKDPTtuebNm4fjx4+rrErLli0xdepU1K9fH+aOZQZEppNbUIS/IxKx6lAc1h2PR3Z+Uaka2wHNgzGwRRD6NwuCj7sTDxVdN2ZmiSxTjZYZCAlazbFTARGZN1cnBwxtE6KWnPwibDyZoGps/z6ZqAaPLdl3QS2O9nbo3zwIt3UMxcDmwQxsiYio+mcAu3TpEiIiIlSZQYsWLdRkCUREleXm7KBqZ2XJLyzGnvPJ2BSRiI0nEnAqIRMbTiSoRUpqpQRBanH7Ng1UnRE4rS4RERkdzObk5OCJJ57A119/jaIi7RShg4ODasUl9bRubm58dYmoSiQ47d00UC3P39IKpxMysexAjOplK+syE5ksn206oyZpkAB4aNsQ9GwUoIJiIiKyTUYFszNnzsSGDRvUIK+ePXuqmtkdO3bg6aefVj/7+OOPq39PicimNA32xFM3tVDLxbRcbDudpJbNp5KQlPnvtLoSBPdo5K8GmklZQrNgT3ZHICKyIUYNAAsKCsLatWvRqVOnUrfv378fQ4cORUJCAswZB4ARWfi0uucuqUkaNp1IQGxabqmfS9svGTwmga2UJHAQGcl0tr/88ot6IcaMGcPpbIksRI1OmuDq6qomTfD39y91e3JyMkJDQ5GbW/rDxdwwmCWyDvL2dSYxE5tOJqqM7T9nLyGvsNjwc5mXoVMDP5W1lQ4JbUPZ+ouIyFLUaDDbu3dv9O/fH2+99ZbhdJ48zLPPPott27apxZwxmCWy3rZfMgPZ5ohE1f5Lam1LCvBwVjW5fZoEqMFkYf7uJttXIiIyYTC7fft2DBs2DHXq1EG3bt3Ubbt370Z8fDxWr16tgl1zxmCWyDZcSMnG5ogk/B2RgG2nLyEzr7DUzxv4u6NP0wD0bhKIno0DEOTlYrJ9JSKiWgxmhQSun332mZq2VrKzrVu3xiOPPKICXHPHYJbI9kjrr/1RKdh25hK2n07CgehUFBaXfvsL83dT3RGkB27fZoGqJy5Zx6QJc+bMUesyUNnZ2dnUu0RE5hDMVpeYmBjMnz9fBcft2rXDhAkTKv1GI+UMP/zwAwYPHoy77rqr0s/JYJaIJEu769wllbGVLgkn4zNQ8t3QxdEenRr4onu4P7o3CkDnhr5wd76u1txkIpwBjMgyVfsMYElJSZV+8sDAwEptd+LECVWS0LdvX/To0QPvvvsuvv/+e9X2y9HR8Zr7c++996pfVAakVSWYJSLydHHEDS3rqEWk5xbgQFSqmqRhzdGLiEvLxc6zyWoBTqsZydrW81FtwAa1DFYTNzjICDMiIjIpx6q046qsyiZ7ZcBYhw4dsGzZMlWqMHnyZDRq1Ag//vgjJk2adNX7ys8ffvhhLFiwoNL7RURUEW9XJ9XOS5aXR7bGmcQs7DqXjN3nk1WXBGkBJqUJsszbfBb+Hs4qaysZ284N/FSgy7IEIiIzDmalh2x1KigoUIPFPvroI0NHBGnrNWjQICxfvvyqwez777+PzMxMNUEDg1kiqm7yniSTNshyT48GhsFkEtxuPZ2ksrfJWflYffSiWoSTgx1ah/qgcwNf9GgUoGpuJftLREQ1q9LvtB07dqzWJ46MjFR1TJKJLUmuX6211969ezF79mzVPcHe3r5Sz5WXl6cWPSlNICKqivp+7mq5o3N9FBYVY390KvZGpmCfLFGpalayg9Gpavl223kV3HZp6KcWydxKv1vJ5hIRUfUyWdogJydHXXp5eZW6XQp8s7Ozy72PZGPHjh2LDz/8EA0aaNmSypB+uK+++up17jERkcbRwV7VzMqiL626kJKDfVEpKsCVHreRl7JL1NxqwgPctcBWBbi+aFHHSz0WERFZYDCrH5WWmppa6vaUlJQKR6xJLa0M/Nq4caNaxIULF9SAMamf/fTTT8vN1s6aNQszZswolZkNCwur5t+IiGy5LEEmYJDlto71VHB7NkmrudUytymqBvf8pWy1/LY/Rt3P3dkBHer7qrrbTmF+6NyQ2duaOj4NGzY0rBORdTFZMCvBpGRljx07piZg0JPrbdq0Kfc+/fr1U1nWklatWqUGp0kZREVvUi4uLmohIqoN8l7UJMhTLeO6a2eRUrPzVWnC/sgUdSmdEzLyCrHj7CW16DF7W/2cnJyuOaiYiCyXSfvMPvDAA/jnn3/U4u7uruphZUaxFStWYPjw4YZs7Llz5/DSSy+V+xgSxA4cOBBz586t9POyzywRmVpRsU5NtytZ25LZ27Ike9u+vgws09fe+iLAk1/Oicj6pVvCpAmJiYmqe4F0NpAWXX/99RfGjBmDefPmlQp4d+7ciSNHjpT7GAxmichapGUXYH+0NqBMZivTZ2/L0mdvZQreXk0CVHkDEZG1sYhgVkhHg3Xr1qkZwNq3b48uXbqU+vnff/+tfiZBbnkWL16sWnrJ5AuVxcwsEVlS9lYCW5XBjUpV18uq5+uG3k20wFaWuj5uJtlfcyWfMx988IFanz59OqezJbIQFhPMmgKDWSKy9OztnvMpqtZWWoEVFuuuyNz2ahKoBbeNAxDkZdtlCZzOlsgyVft0tkREZHo+7k4Y2CJYLSIrr1DNUiaB7c4zl3A4Ju1y14QoLNgVpbaRyR8kcyuTOXQN90Mdb1cT/xZERNWHwSwRkQXzcHEsFdym5xZg11ktuN1+5hKOx6Wr0gRZ5u+IVNvU93NTkzl0lX63Df3QMsQbDvZsWUVElonBLBGRFfF2dcKQ1nXUIlKy8vHPOS2w3X0+BScvpqsJHmRZdiBWbePh7KBmKNPPWNaxga96HCIiS8BglojIivl5OGNY27pqERm5BThweSpeWfZHpSIzrxBbTyepRUjL7kaBHmhV1xudwnxV7W2rEG/YM3tLRGaIwSwRkQ3xcnVCv2ZBatF3TIiIzzAEt7JEJWfjbGKWWlYdilPb+bg5oUcjf1V7O6BFsAp2iYjMAYNZIiIbJrWykoGV5d6e2pSviRl5OBaXjqOxadh9LlmVJ6TlFGDtsXi1YMUxVXerTeTgi37Ng9A40MNsp4qV/ZIWjvp1IrIubM11lVYPREQEFBYVqy4JMqhs66kk1T2hoEh3Ra/bAS2C0L9ZEHo29oevuzNfOiK6LuwzWw0vDhERXUlqbGV2MpnMYee5S9h9LgX5RcWltmlexxNdw/3RXZZG/gj15UQORFQ1DGar4cUhIqJry84vxD9nk/F3RCK2nErEmcSsK7aRsgQJaqXutnujADWxA0/5E9HVMJithheHiIiq7lJmnqqz3XM+GbvOJ+NobLoaaFaSzEomWdtODXxVr9s2od5wcXSokZe7oKAAn3zyiVp/9NFH4eTEtmNEloAzgBERkUkEeLpgWNsQtejLEvZFpmDXuWS1SGswGWS26nCcWoSzoz3ahnprg8pkMocGfgjxqZ6ZymTWdkle6NeJyLqwmwEREdUoTxdH9G8epBaRW1CEg9Gp2KP63KZgX1QqkrPy1aUs2HpObRfq44pOlwPbDvV9VMcFmfGMiKgkvisQEVGtcnVyQI/GAWrRZ0sjL2VjnwpsU7AvMhUnLqYjNi0XsYfiDL1u9ZM5tAn1UVPxDm4VjPp+7jx6RDaOwSwREZmUDAQLD/RQyx2d66vbsvIKcfCCdExIVSUKR2LTEJ+eZ5jMYcXBWLy8/CgaB3mgfT0fFeC2qeetLmWCByKyHQxmiYjI7Eg5Qe8mgWrRkzpbmcjhSEwaNp9KUgPM9MHt0gOxhu0a+LurAWVt6/mgdag3WgSxLRiRNeOkCexmQERkkVKz87E/OhVHYyTATVfZ2wspOVds54giTHDbr9Y9uo9Gm/r+qv5Wgl57e84IRmSu2M2AiIismswyNqhFsFpKBrjSCkyyt+oyNg1RSelIKdY6I8z/+wyKcF6tuzs7oEWIl2E639Z1tXV3Z560JLIkzMwyM0tEZNWk/vZ4XLohyD1xMQMn4zOQX1h61jL9ILPGgR6qRKGt1OGGXq7DdWcdLlFt46QJ1fDiEBGRdSosKsa5pCwcv5ihAl39IoPMyhPm74Y2dX3QVgaZqQFn3gj2qp4+uERUPgazV8FgloiIypOQkasyuEdLlClEJ19ZhyuCvVxUBlcC25Yh3mga7InwQPcam8mMyNakVzL5yDIDZmaJiKyaTGf75ZdfqvUHH3ywytPZpmUX4GhcGo5eHmQmpQpnk7JQ3mRiDvZ2amBZkyBPFdzK0ryOdslaXKKq4QAwIiKiy5MyJCYmGj2drdTLlm0TJnW4MrGD6qIQk4aIhEycSchUU/dK+YIs647Hl6rFre/nhubBXiqwbSLL5YCXfXGJrg+HbBIRERnRB7dLQ3+16EmgnJCRh9MJmWo5k5iJU/GZOJWQgaTMfFWuIMv6EwmlHivQ0wVNgz0Mwa3+sq6Pq5pQgoiujsEsERFRNZDAs463q1r6NP03iysuZeYh4nJgKxnc04mSyc3CxfRcJGXmqWXn2eRS95HWYfrAtrW0DgvVWoj5ezjzeBGVwGCWiIiohgV4uqCXLE0CSt2ekVugZjDTZ3L1l5GXspGdX4TDMWlq+X1/jOE+Id6uaHa5DleWZpdLFxjkkq1iMEtERGQiXq5O6BDmq5aSCoqKVUArwW1E/L/tw85fylbZXFm2nEoqdR8JZv8NcP8NdiX4ZbkCWTMGs0RERGbGycHeEIwOaxtiuF0GmJ28mK6CXKnHlXIFWZdpfJOz8rHrXLJaSvJ0cVQDzpoGeWoZ3culCzIgzdHB3gS/HVH1YjBLRERWTbKS0qtSv27JPMsZeCay8wtVuYLU5JYMdCW7KwHwwehUtZTkaG+HMH93hAe4o2GABxoFeqBhgLu6rOfLQJcsB/vMss8sERFZKZmy9/ylrCsyuWcTM5FXznS+ZQNdCW7DGeiSiXDShGp4cYiIiKxRcbFO1d2el564l7JUBld640ZeylJ1uRIEVzbQlcxueKCW2WVGl6oTg9lqeHGIiIhsOdCVwFYyu/pAV4LeqmR0GejS9eAMYERERJens/3uu+/UazFp0qQqT2dra+zt7RDq66aW3k2vHehq6/8GuvoZ0ABt1rWSga4MOmsQ4IEG/m5q2t8G/h4ID3RHQ38PuDk71O4vSlaDA8CIiMiqycxcsbGxhnUyXaCr3Z5d7mPLjGeS0ZXAtkGAu8rwagGvO/zcnSx+8B7VHAazREREVCuBrgS10cnZiEzOQlRyDqIulzCk5xYiLi1XLWVnQhNeLo6o5+eGEB9XFdy2DfVRM6I1DvKAuzNDGVvH/wFERERUa4Fu2VnQJFuekl1gqMuNSs5WiwS90ck5KgjOyCvEiYsZailLJoWQwWcS2P57qfXRlX69ZP0YzBIREZHJSPmAzF4mS5eGflf8PLegCBdSshGbKpnbHJxJzMLhC2k4cTFdBcH6GdF2nL10RY2uZHFLBrj69WAvF5YtWBEGs0RERGS2XJ0c0DTYSy1lpWbnq4yuTBihH3h2Vl1mIregWK3Lsv7ElZNPyCxozevI1L9eama05nW8VN0ua3MtD4NZIiIiski+7s7o1EAWvytqdOMzclWQq4JbFexmqmA3OiVHzYp2IDpVLRUFuZLJVQPSLs+QJj8j88QjQ0REVs/d3d3Uu0C1XKNb18dNLX2aBpY7K5rMiBYRn6GmAI6Iz1SdFyoKcoVMCNG+vg/a1vNBu3o+aBnihSCWK5gFTmfLSROIiIhsnj7IlQBXpvyVzgv6tmLJWfnlvj5uTg5a+zDVUsxdDTqr7+eO+v7aJbO514eTJhARERFVkrOjvaqblaWstOwCHI1Lw5GYNByOSVeX0nkhp6AIJ+Mz1FIe6Y+rglsV5F4OdEtcerB0oVowM8vMLBERERmRyY1JzTG0E5MMbkxKDi6kZuNCSg5Sswuu+RjSweHKQNcN9XzdUdfXFd6utj1bXXp6Onx8fJCWloarxWusmSUiIqufzvann35S6+PHj+d0tlRtmVxp9SVLeTJyC1SweyE5R7UWkwBXlujL62k5Bap8QZZDF9LKfQwpU5CJIupeXkJ83BCqLl1Vz1659HJxtPkODAxmiYjIqklT/sjISMM6UW3wcnVCyxBZys8opkuweznA/TfY/TfolWBXBqRJ/a4sFfFwdkBdXzdVuyuD0lqEeKFVXW8VZNvKpBEMZomIiIhqmZQQeNd1UoFnebLzC3Hx8hS/aknNQZxMEJGWi9hUbWY0KWXIyi8yBLwbTiQY7u9gb4c6Xi6GmddkqeerZXT1i7erdWR1GcwSERERmRl3Z0c0DvJUS0VKBrxnEzNx/GIGTl5eJKsbK4FvWi4QmVJhVreiYFdakdXxdlXlFOaOwSwRERGRhQe8fUr005Vymvj0PMSm5agsblxqrqrflXXttlxVqytZ3VMJmWopjyRtgzxdDMFtqK8rBrUIRu8yvXtNjcEsERERkRWxs7NTg8Nk6VxmdjS9nPwiQ7CrLbmlgl0JfqVjQ0JGnlr0E0n4uDkxmCUiIiIi03JzdkCTIE+1lEeyu5ey8ksHuqk56BruD3PDzCwREVk9Jyfb7tdJZEx2N9DTRS3t68OscdIETppAREREZLGTJpj/EDUiIiIiogowmCUiIiIii8WaWSIismqFhYX45Zdf1PqYMWPg6MiPPiJrYhZ/0RkZGbh06RLq1atX6SL9xMRE9Ybk51d+ywkiIiJRXFyMU6dOGdaJyLrYm/rb8tSpUxEYGIju3bujTp06+Omnn656n3nz5qFZs2Zo06YNGjVqpC43b95ca/tMRERERObDpMHs7Nmz8fvvv+Pw4cNISEjAnDlzcN999+HQoUPlbl9UVIT9+/dj9erVavukpCTceOONuPXWW9U6EREREdkWkwazkmV94IEH0Lx5c3V9ypQpaNq0Kb788styt3dwcMDnn3+OJk2aqOtSZvDss8+qlg179uyp1X0nIiIiIhsOZi9evIgLFy6gZ8+epW7v1asX9u7dW+nHOXbsmLqsX9/MO/oSERERkfUEszLgS0i9bElyvbIlAzJwbNq0abj55pvRtm3bCrfLy8tTjXdLLkRERERk+UzWzUBKBkR+fv4VgWdl2qbk5OTgtttug729PX744YerbvvWW2/h1VdfveJ2BrVERNZPPmdyc3MN7/vOzs6m3iUiqgR9nKbT6cxzOlvJqsoUZdK9YNy4cYbbpQdgamoq1q5dW+F95U1p5MiRiI2NxcaNGxEcHHzV55IAWRa9mJgYtG7dupp+EyIiIiKqKdHR0VctJzVZZtbLywtdunRRnQn0wax8e163bh2efvrpUrW1koWVNlz6QFa6F0hAWplAVri4uKhFz9PTU70wsg92dnaojW8WYWFh6jmvNrcwmRaPk2XgcbIMPE6WgcfJMtjqcdLpdCr5GRoaar6TJsipfwlM27dvrwZ+/e9//4OrqysefvhhwzYvvvgidu7ciSNHjqjWXKNGjcLBgwfVbC4pKSlqEXXr1lWZ3sqQ0gRTDBiT/4C29J/QUvE4WQYeJ8vA42QZeJwsgy0eJ59KxHYmDWZvueUWLF26FB9++CHmz5+Pdu3aYevWrfD39zdsI0Fq48aN1bpE5+fOnVOzfslkCyW9/vrrGD16dK3/DkRERERkw9PZjhgxQi0V+e9//2tY9/X1xYkTJ2ppz4iIiIjI3Jl00gRbIfW6L7/8cqm6XTI/PE6WgcfJMvA4WQYeJ8vA4wTz7GZARERERHS9mJklIiIiIovFYJaIiIiILBaDWSIiIiKyWCbvZmDtsrKyVAcGaSembzFGpiP9imWGuZICAgLQqlWrK7Y9ffo00tLS1Gxxbm5utbiXtkkmRNm3b59qjh0eHl7uNnI8Tp06hZCQkAp7RVdmG7q+97QDBw6oBu4NGjQo9bOkpKRyO8707NnzimnKpUe4/I3Vq1fvmg3RqeqkjWV2drb63Kno/UuOl2wnx7FOnTpGb0PGkSFLZ8+eVTOUNmnS5IpB4jI5lLz2Zfvk9+7d+4rHSkhIQGRkpHrvDAoKsr1DIgPAqGb89NNPOi8vL13z5s3V5aBBg3Spqal8uU1o8ODBuvr16+v69OljWGbNmlVqm8TERF3v3r11Pj4+uqZNm6rL3377zWT7bO3k9Z4xY4aubt26Ojc3N91TTz1V7nbvv/++ztXVVdeqVSu13ejRo3W5ublV3oaMExsbq5s2bZo6Ti4uLrqXX375im0WLFigc3R0LPX3JUvZ97033nhDPUbr1q3V8ZowYYKuoKCAh6Ya/Pjjj+p9Kzw8XL2+3t7eug8++OCK7Z599lnDMZDL//znP7ri4uIqb0PG+eqrr9Qxaty4sa5ly5Y6X19f3RdffFFqm3fffVfn4eFR6m/phhtuKLWNHI9p06aVOk4VvYdaMwazNeTUqVM6Jycnw3/OlJQUXYsWLXSTJ0+uqaekSgaz8gZ9NRIAde7cWZeZmWl4Q5HA6MKFC3yNa8CePXt0c+bM0SUlJek6dOhQ7hvx1q1bdXZ2drpVq1ap69HR0bqQkBDdSy+9VKVtyHjbtm1TQZG8lzVp0qTCYDYgIOCqj7N69Wqdg4ODbuPGjer6mTNndP7+/rq3336bh6cazJ49W72mevJFXPJWmzZtMty2cOFCFfTs2rVLXT98+LAKmubNm1elbch4r7/+ui4yMtJw/YcfflDvX7t37zbcJp898p54raDY09NTd+jQIXVd7i/HTZJptoTBbA35v//7P5XBKPkt9uOPP1ZZiOzs7Jp6WqpEMCvZBfmDj4qKuiLLkJycrD5oJbuhl5eXp7Kz8sZCNauiYHbKlCm6rl27lrrtueee09WrV69K21D1uFowK4Hp0aNHVfBTXlZ8zJgxuoEDB5a67bHHHlNf9qlmyJc6yYbr3XTTTbrbb7+91Db33nuvrkePHlXahqqXBKUls+jymdO2bVvdwYMHdcePH9fl5+dfcR85i3jvvfeWuk2Om3zW2RIOAKsh+/fvR+fOnWFnZ2e4rXv37qoukLOYmdY333yDBx54AO3bt1dTKO/evdvws8OHD6OoqAhdunQx3Obs7IwOHTqoY0qmIa99yWOi/3uSmrLExMRKb0M1Lzk5GbfddptaZGryt99+u9TPKzpOERERqsaTqpfUUcr//6ZNm17zGEgttL71fGW2oepz/PhxZGZmljpO4ujRoxg7dixuvPFGBAcHq8+vkio6Tvtt7POKwWwNvqHLwKKS9NflZ2QaU6ZMUW/s8oYcGxurgtnbb79dDRoqeWzKO3Y8bub998S/OdNr1KiR+tuSAXhnzpzBzz//jOeffx4LFiwwbFPRcZIASQaFUfUpKCjApEmT1CBWeZ+71jGQgUj6LxSV2YaqhyS5Jk+erILQoUOHGm7v2LGj+ls6duwYoqOj8eabb6pEzObNmw33y8nJKfc4yd+SLX3pYDBbQ5ycnNR/tJLkP50+00emcc8998DT01OtywjfuXPnqqBW/+Ygx02Ud+x43Mz774l/c6bXo0cPdRZDT7KzN998MxYuXGi4jcepdsgZpgkTJqiOEcuWLSv1/sW/J/P6wjFmzBiVZPntt9/g4OBg+NmQIUNUlwO9Rx55RJ1RXLRo0TU/rxwdHUudGbZ2DGZrSMOGDdXpzZL018u2syHTkW+w8uahPzZy3ER5x47Hzfz+nuTY6ds6VWYbqn3SzqnkcanoOLm7u1+RYSLjA9mJEydi69at2Lhxo8qYl1TRMZB2dvoAqTLb0PUHsnfffbdqGSnHSdrUVeXvSd7b5D4x5Rwn/WeZrWAwW0OkvuWff/5Rvd/05Ntxs2bNbO4/mbmQ02PyJl/S+vXr1W1t27ZV1+V0nAQ+y5cvN2wjfQClllaOKZmGvPZ//fVXqQyE/D3179/f0JuxMttQzfegLfs3J2c99H9f+uO0evVq9UFe8jgNHjxY9dCk61NcXKxKC/7++28VIJWtwdQfg1WrVqlt9eQ9r+R7XGW2IeMVFhaqWlgpy9m0aVO5yZKyf09SOrBnz54r/p5WrFhhKCmQ4yXXbe44mXoEmrWSnonS3qlnz56qNYqMJJVR8osXLzb1rtms06dPq2Py6aef6tasWaN6kkoboVGjRpXa7vvvv1dt1aRd1JIlS3QdO3bU9e3bV1dUVGSyfbf2v5UtW7aoRfpjjh07Vq0fOHDAsI30KZWejMOGDdMtW7ZM9/TTT6tjJO24qrINGU86E+iPk3SIkDaDsq5vCaQfRS0dJJYvX6775ZdfdAMGDFB/YxEREaX6CoeGhupuu+02tZ10MpAuL3v37uXhqQYPPvigztnZWfftt98ajpcsJdt1SScXOS7jxo1Tx0A6gUgv9BMnTlRpGzKevK7S8lFaaJU8TufPnzdsI31lX3vtNdVuUDrsyGeRvMfFx8cbtpG/LW9vb92kSZPUcbrnnnt0fn5+pR7HFtjJP6YOqK2VzDQlI3lltLzMACaF2yWLu6n2yYjpTz/9VI0cldNlUs8np3nK1hZJBuL7779Heno6evXqhZkzZ8LLy4uHrAbI4Lvhw4dfcXvLli3x1VdfGa7HxcVh9uzZ6pScHLvHHntMHZuSKrMNGefixYsYPXr0Fbd36tQJH330kVqXrPi8efNUpklIfd/jjz9+RflAVFSUem+Uzi5ymnT69OlXjMgm49xxxx2lzgjq3XnnnXjyyScN16WW9p133lED9eRs4VNPPYU2bdqUuk9ltiHjyGdPRkbGFbffe++9ePjhhw3vjZ988gm2b9+uxnh07dpVvad5eHiUus+xY8fw3nvvqdnCpMZWPq+aN29uU4eGwSwRERERWSwWKBERERGRxWIwS0REREQWi8EsEREREVksBrNEREREZLEYzBIRERGRxWIwS0REREQWi8EsEREREVksBrNERLXk119/RWxsrNm93jKXu0yBKftHRGRpGMwSEdWSCRMmYN++fWb1estsXTLb2hdffKEC2vL89ttvuHDhQq3vGxFRZTCYJSKyYTJts0yBKoHs/Pnzy91mypQp2LlzZ63vGxFRZThWaisiIisgp9H79OkDOzs7HDx4EL6+vujevTvs7bXv9YWFhVi8eDFuvPFGBAQEGO63fPlytGvXDo0aNSr1ODqdDocOHYKPjw969eqlHlfmsT9y5AjCw8PRoUOHcvdDspxyP39/f/To0UPdr6S8vDzs2LEDmZmZaNu2rXqs8n6P/Px8HDhwQM3HLvtXntTUVPVY8rv17NkTQUFBhp8tW7ZM7UedOnWwcOFC9fvJ/pS0atUqFBQUYNu2beoxXFxcMGrUqKvuQ0JCAnbt2gV3d3d07txZvc5lXWsb+d3l5/LcXbp0QWBgYAVHlYhsno6IyEa4uLjobr75Zl2TJk10I0aM0AUHB+uGDh2qKy4uVj/PyMjQAdDt2LGj1P3q1aun+/bbb0s9Tr9+/QyP4+3trbv11lt1L774oq5Zs2a64cOH6zw9PdX1ss8vz1e/fn21H/7+/mrbgoICwzY7d+7UhYaG6rp27ap+Jts8+eSTVzzOsGHDdOHh4bpRo0bpFi1aVO7vu2LFCp2Xl5euR48ean/d3d113333neHnU6dOVc/VvHlz3d1336379NNPr3iM6dOn65ycnHR9+vRR2zz44INX3Yc5c+bofHx8dDfddJNu4MCBav+XL19e6jGvtc3u3bt1AQEBar/lNZDnmD9//lWPLRHZLgazRGQzJAAbMGCALjc3V12PiorSOTs769auXVvlYHbQoEG6vLw8dX3jxo3qfhLc6QNTCSQlCExNTS11vxYtWhhuk+f38/PTffHFF+p6VlaWLiQkRPfll18a7nPhwgUV2MnjlXycbt26qe0rkp6eroL1V155xXDbxx9/rPPw8NDFxMQYbpNgUQLWq5HA89dff73itSy7D+vXr1e/T0REhOG2hQsXqmA1LS2t0tuMHTtWN2XKFMPPc3JydH/88cdV95GIbBdrZonIpkj9p5wqF2FhYWjcuDFOnjxZ5ceZPHkynJ2d1bqUGOgf29HR0XCbnCI/d+5cqfs9+OCDqixB//zjxo3DL7/8oq6vXr0aly5dgre3typ3kFP5cnpfTv9v3Lix1OM88MAD6hR9RTZs2IDk5GTMnDnTcNvDDz8MV1dXrFy5EtWh7D589913ajCZlHDIvsvvVVxcrEod5LbKbuPm5obIyEh1m5B9vvnmm6tln4nI+rBmlohsitSpliSBbW5ubpUfx8/Pr9RjVHRb2ccuW/8qgepff/2l1s+fP68CZOkeUJLUo0rQXVLdunWvun8SDIaEhJQKNh0cHNTzy8+qQ9l9kP2XYFwC8ZLuuusuQ+BfmW1effVV3H///QgNDUXXrl1VIPvoo4+qIJ+IqCwGs0REl+kHgkmmsCRjgt2KpKSkXHFdP7hJgjXJ5v7www9wcnK66uOUHTRWljxm2ecSkq2trsFUZfdB9l8eWwaTVaQy20jGeu3atWr/pXXY7Nmz8fvvv6sBYUREZbHMgIjoMsliyqj606dPG16Tw4cPq0xidVm6dKlhXYJmuS5dAcSQIUNQVFSEb7/9ttR9JMBNTEys0vNI5wLpirBmzRrDbXv37lVlD/rnqyxPT89KBfTDhg1TpRKSfS3p4sWLhi8IldlGJnHQZ7qlc8Irr7yi9l1eByKispiZJSIqYeLEiZg1axbS09NVMPj111+rms3q8s8//2D8+PEYMGCAKidISkoy1LVKCcA777yDxx57TNWPSsuqqKgodUr+o48+wg033FDp55GyhCeffFLV5D777LPqFP67776rfj9pR1YVcqp/3rx5qhWZBLYSYJbnoYceUu2+pF542rRpqg2YtO2SMoqjR4+qzHdltpH9lsB24MCBqgb5008/xe23337NbDUR2SZmZonIZowZMwb16tUrdZtkCmVAkt7//vc/vPTSSyrAkoBWBkvJwC59j9mKHufuu+9WNap6EnjJbSVP6cv95PEkM7pnzx60b98eu3fvRnBwsGGbGTNmYOvWrfDw8MCWLVvU48iEBiUD2fKevzwSGH/11VeG3rdyvWzWt3///ipovhqZHWzw4MEq4Pzzzz8r3AcJmCXr+v7776sgXH5H6T0rr6U+EK3MNosWLcI999yjBuZJUC+vyYIFC675+xKRbbKTlgam3gkiIiIiImMwM0tEREREFovBLBERERFZLAazRERERGSxGMwSERERkcViMEtEREREFovBLBERERFZLAazRERERGSxGMwSERERkcViMEtEREREFovBLBERERFZLAazRERERGSxGMwSERERESzV/wP535tdac2JtwAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "X_tr, X_va, y_tr, y_va = train_test_split(X, y, test_size=0.2, random_state=SEED, stratify=y)\n", "m = LGBMClassifier(n_estimators=2000, learning_rate=0.03, random_state=SEED, verbose=-1)\n", "m.fit(X_tr, y_tr, eval_set=[(X_tr, y_tr), (X_va, y_va)],\n", " callbacks=[early_stopping(100, verbose=False), log_evaluation(0)])\n", "print(\"best number of trees:\", m.best_iteration_)\n", "\n", "curves = m.evals_result_\n", "plt.figure(figsize=(8, 4))\n", "plt.plot(curves[\"training\"][\"multi_logloss\"], label=\"train\")\n", "plt.plot(curves[\"valid_1\"][\"multi_logloss\"], label=\"validation\")\n", "plt.axvline(m.best_iteration_, ls=\"--\", c=\"gray\")\n", "plt.xlabel(\"number of trees\"); plt.ylabel(\"log-loss (lower = better)\"); plt.legend();" ] }, { "cell_type": "markdown", "id": "6ff99a1c", "metadata": {}, "source": [ "Same picture as the depth plot in 03: training error keeps falling, while validation error bottoms out early and then rises.\n", "Past the dashed line, extra trees only make validation worse. The default LightGBM never checks this: it always builds 100 large trees (31 leaves each) at a 3× faster learning rate, far too much model for 400 houses." ] }, { "cell_type": "markdown", "id": "6624f0dc", "metadata": {}, "source": [ "## 3. Tuning each model (small, manual grids)\n", "For each model we try a few settings that go from \"flexible\" to \"restricted\", scoring each with CV:\n", "- **Logistic Regression** `C`: smaller = more regularised (simpler).\n", "- **Random Forest** `min_samples_leaf`: every leaf must contain at least this many houses.\n", "- **LightGBM** `num_leaves` / `min_child_samples`, **XGBoost** `max_depth`: tree size. Plus fewer, slower rounds and random row/column sampling.\n", "\n", "(Tools like **Optuna** search these automatically. For learning, a small grid is clearer.)" ] }, { "cell_type": "code", "execution_count": 4, "id": "5afb23ae", "metadata": { "execution": { "iopub.execute_input": "2026-10-23T06:13:52.664496Z", "iopub.status.busy": "2026-10-23T06:13:52.664496Z", "iopub.status.idle": "2026-10-23T06:16:00.147652Z", "shell.execute_reply": "2026-10-23T06:16:00.145206Z" } }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
modelsettingcv_acc
0LogisticRegressionC=0.10.7092
1LogisticRegressionC=0.30.7149
2LogisticRegressionC=1.00.7190
3LogisticRegressionC=3.00.7174
4RandomForestmin_samples_leaf=10.6928
5RandomForestmin_samples_leaf=30.7174
6RandomForestmin_samples_leaf=50.7214
7RandomForestmin_samples_leaf=100.7100
8LightGBMnum_leaves=40.7296
9LightGBMnum_leaves=70.7173
10LightGBMnum_leaves=150.7017
11LightGBMnum_leaves=310.6985
12XGBoostmax_depth=20.7296
13XGBoostmax_depth=30.7279
14XGBoostmax_depth=40.7213
15XGBoostmax_depth=60.7181
16CatBoostdepth=20.7280
17CatBoostdepth=40.7345
18CatBoostdepth=60.7263
\n", "
" ], "text/plain": [ " model setting cv_acc\n", "0 LogisticRegression C=0.1 0.7092\n", "1 LogisticRegression C=0.3 0.7149\n", "2 LogisticRegression C=1.0 0.7190\n", "3 LogisticRegression C=3.0 0.7174\n", "4 RandomForest min_samples_leaf=1 0.6928\n", "5 RandomForest min_samples_leaf=3 0.7174\n", "6 RandomForest min_samples_leaf=5 0.7214\n", "7 RandomForest min_samples_leaf=10 0.7100\n", "8 LightGBM num_leaves=4 0.7296\n", "9 LightGBM num_leaves=7 0.7173\n", "10 LightGBM num_leaves=15 0.7017\n", "11 LightGBM num_leaves=31 0.6985\n", "12 XGBoost max_depth=2 0.7296\n", "13 XGBoost max_depth=3 0.7279\n", "14 XGBoost max_depth=4 0.7213\n", "15 XGBoost max_depth=6 0.7181\n", "16 CatBoost depth=2 0.7280\n", "17 CatBoost depth=4 0.7345\n", "18 CatBoost depth=6 0.7263" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "grids = {\n", " \"LogisticRegression\": [\n", " lambda C=C: make_pipeline(StandardScaler(), LogisticRegression(C=C, max_iter=3000)) for C in [0.1, 0.3, 1.0, 3.0]\n", " ],\n", " \"RandomForest\": [\n", " lambda leaf=leaf: RandomForestClassifier(n_estimators=500, min_samples_leaf=leaf, random_state=SEED, n_jobs=-1) for leaf in [1, 3, 5, 10]\n", " ],\n", " \"LightGBM\": [\n", " lambda nl=nl: LGBMClassifier(n_estimators=300, learning_rate=0.03, num_leaves=nl, min_child_samples=10, subsample=0.8,\n", " subsample_freq=1, colsample_bytree=0.7, random_state=SEED, verbose=-1) for nl in [4, 7, 15, 31]\n", " ],\n", " \"XGBoost\": [\n", " lambda d=d: XGBClassifier(n_estimators=300, learning_rate=0.03, max_depth=d, subsample=0.8, colsample_bytree=0.7,\n", " random_state=SEED) for d in [2, 3, 4, 6]\n", " ],\n", " \"CatBoost\": [\n", " lambda d=d: CatBoostClassifier(iterations=500, learning_rate=0.03, depth=d, random_seed=SEED, verbose=0) for d in [2, 4, 6]\n", " ],\n", "}\n", "labels = {\"LogisticRegression\": \"C\", \"RandomForest\": \"min_samples_leaf\", \"LightGBM\": \"num_leaves\", \"XGBoost\": \"max_depth\", \"CatBoost\": \"depth\"}\n", "values = {\"LogisticRegression\": [0.1, 0.3, 1.0, 3.0], \"RandomForest\": [1, 3, 5, 10], \"LightGBM\": [4, 7, 15, 31], \"XGBoost\": [2, 3, 4, 6], \"CatBoost\": [2, 4, 6]}\n", "tuned = []\n", "for name, makers in grids.items():\n", " for v, make in zip(values[name], makers):\n", " mean, std, _ = cv_score(make, X, y, verbose=False)\n", " tuned.append({\"model\": name, \"setting\": f\"{labels[name]}={v}\", \"cv_acc\": round(mean, 4)})\n", "tuned_df = pd.DataFrame(tuned)\n", "tuned_df" ] }, { "cell_type": "code", "execution_count": 5, "id": "ba35ed23", "metadata": { "execution": { "iopub.execute_input": "2026-10-23T06:16:00.155450Z", "iopub.status.busy": "2026-10-23T06:16:00.155450Z", "iopub.status.idle": "2026-10-23T06:16:00.181475Z", "shell.execute_reply": "2026-10-23T06:16:00.180470Z" } }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
modelsettingcv_acc
17CatBoostdepth=40.7345
8LightGBMnum_leaves=40.7296
12XGBoostmax_depth=20.7296
6RandomForestmin_samples_leaf=50.7214
2LogisticRegressionC=1.00.7190
\n", "
" ], "text/plain": [ " model setting cv_acc\n", "17 CatBoost depth=4 0.7345\n", "8 LightGBM num_leaves=4 0.7296\n", "12 XGBoost max_depth=2 0.7296\n", "6 RandomForest min_samples_leaf=5 0.7214\n", "2 LogisticRegression C=1.0 0.7190" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "best = tuned_df.loc[tuned_df.groupby(\"model\").cv_acc.idxmax()].sort_values(\"cv_acc\", ascending=False)\n", "best" ] }, { "cell_type": "markdown", "id": "eb3b3822", "metadata": {}, "source": [ "- Every tree model improves as it gets **more restricted** (smaller trees / bigger leaves), up to a point.\n", "- After tuning, the models are all close together (within ~1–2%). Remember the noise level is ~±0.8%.\n", "- The tuned settings above are copied into notebook 06." ] }, { "cell_type": "markdown", "id": "62f87843", "metadata": {}, "source": [ "## 4. Re-testing a dropped idea\n", "In 04, `log_area` did nothing for the Random Forest, as expected for trees. Logistic Regression is *not* a tree, so a log can matter for it:" ] }, { "cell_type": "code", "execution_count": 6, "id": "452c9fea", "metadata": { "execution": { "iopub.execute_input": "2026-10-23T06:16:00.185740Z", "iopub.status.busy": "2026-10-23T06:16:00.185740Z", "iopub.status.idle": "2026-10-23T06:16:00.572550Z", "shell.execute_reply": "2026-10-23T06:16:00.571545Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "LogReg raw: 0.7190\n", "LogReg + log_area: 0.7206\n" ] } ], "source": [ "_tr, _ = basic_prep(*load_data())\n", "log_area = np.log(_tr.area).rename(\"log_area\")\n", "lr = lambda: make_pipeline(StandardScaler(), LogisticRegression(max_iter=3000))\n", "print(f\"LogReg raw: {cv_score(lr, X, y, verbose=False)[0]:.4f}\")\n", "print(f\"LogReg + log_area: {cv_score(lr, X.join(log_area), y, verbose=False)[0]:.4f}\")" ] }, { "cell_type": "markdown", "id": "ac949c0a", "metadata": {}, "source": [ "The difference is within noise, so it stays dropped. Still, re-testing old ideas when you switch model type is a good habit." ] }, { "cell_type": "markdown", "id": "7a33df56", "metadata": {}, "source": [ "## 5. The ordinal trick\n", "Classification treats \"budget\" and \"premium\" as unrelated labels, so calling a premium house \"budget\" costs the same as calling it \"mid\".\n", "But the tiers have an **order**. Alternative: predict a **number** (0–2) with a regressor, then round to the nearest tier." ] }, { "cell_type": "code", "execution_count": 7, "id": "eececfd3", "metadata": { "execution": { "iopub.execute_input": "2026-10-23T06:16:00.574595Z", "iopub.status.busy": "2026-10-23T06:16:00.574595Z", "iopub.status.idle": "2026-10-23T06:16:01.105974Z", "shell.execute_reply": "2026-10-23T06:16:01.104866Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "ordinal LightGBM (regression + rounding): 0.7239\n", "best LightGBM classifier: 0.7296\n" ] } ], "source": [ "rskf = RepeatedStratifiedKFold(n_splits=5, n_repeats=3, random_state=SEED)\n", "accs = []\n", "for tr_idx, va_idx in rskf.split(X, y):\n", " reg = LGBMRegressor(n_estimators=300, learning_rate=0.03, num_leaves=7, min_child_samples=10, subsample=0.8,\n", " subsample_freq=1, colsample_bytree=0.7, random_state=SEED, verbose=-1)\n", " reg.fit(X.iloc[tr_idx], y.iloc[tr_idx])\n", " pred = np.clip(np.round(reg.predict(X.iloc[va_idx])), 0, 2)\n", " accs.append(accuracy_score(y.iloc[va_idx], pred))\n", "print(f\"ordinal LightGBM (regression + rounding): {np.mean(accs):.4f}\")\n", "print(f\"best LightGBM classifier: {best.set_index('model').cv_acc['LightGBM']:.4f}\")" ] }, { "cell_type": "markdown", "id": "b0ecb35e", "metadata": {}, "source": [ "Compare the two numbers. If they're within noise, the trick doesn't pay off here, but it's always worth one try when the target is ordered." ] }, { "cell_type": "markdown", "id": "bd3283d4", "metadata": {}, "source": [ "## Submit the best single model\n", "Train on **all** training data (no fold held out), then predict test:" ] }, { "cell_type": "code", "execution_count": 8, "id": "397c2ab3", "metadata": { "execution": { "iopub.execute_input": "2026-10-23T06:16:01.108482Z", "iopub.status.busy": "2026-10-23T06:16:01.108482Z", "iopub.status.idle": "2026-10-23T06:16:01.684030Z", "shell.execute_reply": "2026-10-23T06:16:01.684030Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "best single model by CV: CatBoost depth=4\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "leaderboard: {'public': 0.6935}\n" ] } ], "source": [ "top_name = best.iloc[0].model\n", "top_setting = best.iloc[0].setting\n", "top_make = grids[top_name][[f\"{labels[top_name]}={v}\" for v in values[top_name]].index(top_setting)]\n", "print(\"best single model by CV:\", top_name, top_setting)\n", "final = top_make().fit(X, y)\n", "print(\"leaderboard:\", score(save_submission(test, final.predict(X_test), \"05_best_single\")))" ] }, { "cell_type": "markdown", "id": "0feb0305", "metadata": {}, "source": [ "Next: **06**. Instead of picking one model, combine them." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.10" } }, "nbformat": 4, "nbformat_minor": 5 }