Sales Forecasting with Python Random Forest on UK Retail Data

Introduction

Imagine you run a shop. Every month you need to order stock, hire staff, and plan your budget all before you know how much you're actually going to sell. This is the challenge every retail business faces. Getting it wrong in either direction is costly: too much stock wastes money, too little means missed sales.

This project solves that problem using machine learning. We use the Online Retail II dataset a real record of transactions from a UK-based online gift shop between 2009 and 2011, with over one million rows and build a model that predicts next month's revenue based on patterns in the past.

If you've never built a machine learning model before, don't worry. Think of it like teaching a calculator to spot patterns. You show it what happened in the past, and it learns the rules well enough to make an educated guess about the future. By the end of this post you'll understand every step of how that works.

The Dataset

Before writing any code, it's worth understanding the raw material. The Online Retail II dataset contains one row per product per invoice think of it like a detailed till receipt for every transaction the shop ever made.

One thing you'll notice straight away: there's no Revenue column. We have to calculate it ourselves using Quantity × Price. The dataset also has cancelled orders, missing customer IDs, and wildly unrealistic values that need cleaning before we can trust any analysis.

Project Workflow

The project follows a structured pipeline across seven stages. Each stage feeds into the next you can't skip ahead because each step depends on the one before it.

Step 1 Environment Setup

Every Python data science project starts with imports. These are the tools we're bringing into the project think of them like picking up your equipment before starting a job.

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.ensemble import RandomForestRegressor
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_absolute_error, r2_score, mean_squared_error
from sklearn.preprocessing import LabelEncoder
import warnings
warnings.filterwarnings('ignore')

sns.set_style('whitegrid')
plt.rcParams['figure.figsize'] = (12, 6)

print("All libraries imported successfully")

• pandas the main tool for working with tabular data (like a spreadsheet). We'll use it to load, filter, group, and reshape the dataset.
• numpy handles fast numerical calculations. Pandas uses it behind the scenes, and we use it directly for things like square roots.
• matplotlib & seaborn chart libraries. Matplotlib is the base; Seaborn sits on top and makes things look nicer with less code.
• sklearn (Scikit-learn) the machine learning library. It contains the Random Forest model, the train-test splitter, and the evaluation metrics all in one place.

Step 2 Data Loading & Inspection

The dataset lives in Google Drive and is loaded using Pandas. After loading, the first job is always to inspect what you've got shape, column types, and missing values.

from google.colab import drive
drive.mount('/content/drive')

df = pd.read_csv('/content/drive/MyDrive/sukh/online_retail_II.csv', encoding='latin-1')

print("Shape:", df.shape)
print("\nColumn names:")
print(df.columns.tolist())
print("\nData types:")
print(df.dtypes)
print("\nFirst 5 rows:")
df.head()

df.shape tells you how many rows and columns exist for example (1067371, 8) means over a million rows and 8 columns. encoding='latin-1' is needed because the file contains special characters like accented letters that the default UTF-8 encoding can't read. If you skip this, Python throws a decoding error before you even see the data.

print("MISSING VALUES")
print(df.isnull().sum())
print("\nMissing value percentage:")
print((df.isnull().sum() / len(df) * 100).round(2))

print("\nBASIC STATISTICS")
print(df.describe())

isnull().sum() counts how many blank cells exist in each column. Dividing by the total row count and multiplying by 100 gives you the percentage a useful way to judge how serious the problem is. describe() shows the minimum, maximum, mean, and quartiles of every numeric column in one go.

Step 3 Data Cleaning

Real-world retail data is messy. Rows that don't represent real completed sales will teach the model the wrong patterns. This step removes all of them and builds the Revenue column the model will eventually predict.

print("Shape before cleaning:", df.shape)

# Remove missing Customer IDs and descriptions
df = df.dropna(subset=['Customer ID'])
df = df.dropna(subset=['Description'])

# Remove cancelled orders (Invoice starts with 'C')
df = df[~df['Invoice'].astype(str).str.startswith('C')]

# Remove negative or zero quantities and prices
df = df[df['Quantity'] > 0]
df = df[df['Price'] > 0]

# Remove extreme outliers
df = df[df['Price'] < 500]
df = df[df['Quantity'] < 10000]

# Create the Revenue column
df['Revenue'] = df['Quantity'] * df['Price']

# Convert InvoiceDate from a string to a real datetime object
df['InvoiceDate'] = pd.to_datetime(df['InvoiceDate'])

print("Shape after cleaning:", df.shape)
print("\nMissing values after cleaning:")
print(df.isnull().sum())

• Cancelled orders any invoice starting with C is a refund, not a sale. The ~ symbol means "NOT", so ~df['Invoice'].str.startswith('C') keeps everything that does not start with C. Think of it as a filter that flips the selection.
• Missing Customer IDs rows without a customer ID are guest checkouts. We can't track their behaviour over time, so they're excluded.
• Outlier thresholds a price of £500+ or a quantity of 10,000+ almost certainly represents a one-off bulk wholesale order, not typical retail. If we include these, the model's sense of "normal" gets thrown off.
• Revenue derivation the dataset has no Revenue column, so we create it: Quantity × Price. This becomes the number we want to predict.
• Date parsing pd.to_datetime() converts the date string (e.g. "2010-12-01 08:26:00") into a proper Python datetime. Once it's a datetime, we can extract the month, year, day, and hour all of which become useful features.

Step 4 Exploratory Data Analysis

Before building any model, it's worth spending time looking at the data visually. EDA (Exploratory Data Analysis) is like reading a book before writing a review you need to understand what's there before you can say something meaningful about it. We ran four key analyses.

Revenue by Country

Grouping revenue by country and plotting the top 10 immediately shows that the UK dominates by a huge margin, with Germany, France, and the Netherlands as smaller export markets. This tells us the model is primarily learning UK domestic retail patterns.

country_revenue = df.groupby('Country')['Revenue'].sum().sort_values(ascending=False).head(10)

plt.bar(country_revenue.index, country_revenue.values, color='#1a237e')
plt.title('Top 10 Countries by Revenue')
plt.xlabel('Country')
plt.ylabel('Total Revenue (£)')
plt.xticks(rotation=45, ha='right')
plt.tight_layout()
plt.show()

.groupby('Country') collapses all rows for each country into a single row, then ['Revenue'].sum() adds up all revenue for that country. sort_values(ascending=False).head(10) sorts highest-first and keeps only the top 10 a pattern you'll use constantly in data analysis.

Monthly Revenue Trend & Seasonality

This is the most important chart for the forecasting task. Plotting revenue month by month over 2009–2011 reveals a clear pattern: every November and December, revenue spikes dramatically. That's Christmas shopping. This seasonal signal is so strong it becomes one of the model's most powerful features.

df['Year_Month'] = df['InvoiceDate'].dt.to_period('M')
monthly = df.groupby('Year_Month')['Revenue'].sum()

plt.bar(range(len(monthly)), monthly.values,
        color=['#e53935' if i in [11, 12, 23, 24]
               else '#1a237e'
               for i in range(len(monthly))])
plt.title('Monthly Revenue Trend 2009–2011 (Red = Christmas Peak)')
plt.show()

dt.to_period('M') converts each full datetime into just its year-month (e.g. "2010-11"), which makes it easy to group by month. The colour list inside color=[...] uses a Python list comprehension a compact loop that checks the bar's position and assigns red for Christmas months, dark blue for everything else.

Day of Week & Hour of Day

Weekdays generate far more revenue than weekends, and the busiest hours are between 11am and 2pm. This suggests the customer base is largely businesses buying gifts for their own clients they shop during working hours on working days, not on Saturday evenings like a typical consumer.

df['DayOfWeek'] = df['InvoiceDate'].dt.day_name()
df['Hour'] = df['InvoiceDate'].dt.hour

day_order = ['Monday','Tuesday','Wednesday','Thursday','Friday','Saturday','Sunday']
day_revenue = df.groupby('DayOfWeek')['Revenue'].sum().reindex(day_order)
hour_revenue = df.groupby('Hour')['Revenue'].sum()

# Day of week chart
plt.bar(day_revenue.index, day_revenue.values,
        color=['#e53935' if d in ['Saturday','Sunday'] else '#1a237e' for d in day_order])
plt.title('Revenue by Day of Week (Red = Weekend)')
plt.show()

# Hour of day chart
plt.bar(hour_revenue.index, hour_revenue.values,
        color=['#e53935' if h in [11,12,13,14] else '#1a237e' for h in hour_revenue.index])
plt.title('Revenue by Hour of Day (Red = Peak Hours)')
plt.show()

Customer Segmentation

By totalling up each customer's lifetime spend, we can group them into three tiers: New (under £1,000), Returning (£1,000–£5,000), and VIP (over £5,000). VIP customers are a small percentage of the total headcount but drive a disproportionately large share of revenue a classic pattern known as the Pareto Principle (roughly 20% of customers generating 80% of revenue).

customer_stats = df.groupby('Customer ID').agg(
    Total_Orders=('Invoice', 'nunique'),
    Total_Revenue=('Revenue', 'sum'),
    Avg_Order_Value=('Revenue', 'mean'),
    Total_Items=('Quantity', 'sum')
).reset_index()

customer_stats['Segment'] = customer_stats['Total_Revenue'].apply(
    lambda x: 'VIP' if x > 5000
    else 'Returning' if x > 1000
    else 'New'
)

print(customer_stats['Segment'].value_counts())

The lambda on the last block is a tiny anonymous function. For every row, it checks the customer's total revenue and assigns a label. It's equivalent to writing a full def function, just much more concise for simple one-liners like this.

Step 5 Feature Engineering

A machine learning model can only work with numbers it can't read a date string or understand what "Christmas" means. Feature engineering is the process of converting your raw data into numerical inputs the model can actually learn from.

Right now we have one row per transaction. The model needs to think about time in months, so the first step is rolling everything up to monthly totals, then creating 14 features that describe each month.

monthly_df = df.groupby('Year_Month').agg(
    Monthly_Revenue=  ('Revenue',     'sum'),
    Monthly_Orders=   ('Invoice',     'nunique'),
    Monthly_Customers=('Customer ID', 'nunique'),
    Monthly_Items=    ('Quantity',    'sum'),
    Avg_Order_Value=  ('Revenue',     'mean')
).reset_index()

# Time index features
monthly_df['Month_Number']  = range(1, len(monthly_df) + 1)
monthly_df['Month_Of_Year'] = monthly_df['Year_Month'].dt.month
monthly_df['Year']          = monthly_df['Year_Month'].dt.year

# Seasonal flags (0 or 1)
monthly_df['Is_Christmas'] = monthly_df['Month_Of_Year'].apply(
    lambda x: 1 if x in [11, 12] else 0)
monthly_df['Is_Summer'] = monthly_df['Month_Of_Year'].apply(
    lambda x: 1 if x in [6, 7, 8] else 0)
monthly_df['Is_Q1'] = monthly_df['Month_Of_Year'].apply(
    lambda x: 1 if x in [1, 2, 3] else 0)

# Lag features  what did revenue look like 1, 2, 3 months ago?
monthly_df['Prev_Month_Revenue']   = monthly_df['Monthly_Revenue'].shift(1)
monthly_df['Prev_2_Month_Revenue'] = monthly_df['Monthly_Revenue'].shift(2)
monthly_df['Prev_3_Month_Revenue'] = monthly_df['Monthly_Revenue'].shift(3)

# Rolling average  smoothed trend over the past 3 months
monthly_df['Rolling_3_Month_Avg'] = monthly_df['Monthly_Revenue'].rolling(3).mean()

# Drop rows where lag/rolling features are NaN (the first few months)
monthly_df = monthly_df.dropna()

print("Monthly dataframe shape:", monthly_df.shape)
print("Features created:", monthly_df.columns.tolist())

Step 6 The Random Forest Model

A Random Forest is an ensemble model instead of training one model and hoping it's right, it trains 200 separate decision trees and averages their predictions. Here's the intuition: imagine asking 200 different analysts to each predict next month's revenue, each one looking at a slightly different slice of history. The average of their answers is almost always better than any single analyst's guess. That's exactly what a Random Forest does.

Each individual "analyst" (decision tree) makes a series of yes/no decisions for example: "Is it a Christmas month? → Yes → Is last month's revenue above £500k? → Yes → Predict high." The "random" part means each tree sees a random subset of the features, which prevents them all from making the exact same errors.

features = [
    'Month_Number', 'Month_Of_Year', 'Year',
    'Is_Christmas', 'Is_Summer', 'Is_Q1',
    'Prev_Month_Revenue', 'Prev_2_Month_Revenue',
    'Prev_3_Month_Revenue', 'Rolling_3_Month_Avg',
    'Monthly_Orders', 'Monthly_Customers',
    'Monthly_Items', 'Avg_Order_Value'
]

target = 'Monthly_Revenue'

X = monthly_df[features]   # inputs  (14 columns)
y = monthly_df[target]     # output  (1 column  what we want to predict)

# Hold back 20% of months as unseen test data
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

rf_model = RandomForestRegressor(
    n_estimators=200,    # 200 trees in the forest
    max_depth=10,        # each tree can make at most 10 splits
    min_samples_split=2, # a node needs at least 2 samples to split
    min_samples_leaf=1,  # each leaf needs at least 1 sample
    random_state=42      # same random seed = reproducible results
)

rf_model.fit(X_train, y_train)

print("Training set size:", X_train.shape)
print("Testing set size:", X_test.shape)

X is the table of inputs (all 14 features) and y is the column we're trying to predict. train_test_split randomly assigns 80% of the months to training and 20% to testing. The model never sees the test months during training, so they give a fair measure of how it performs on genuinely new data. random_state=42 locks the random seed so the split is always the same making your results reproducible.

Evaluation & Results

Training a model is only half the job. The other half is measuring whether it actually works. We use three metrics, each answering a slightly different question about prediction quality.

y_pred = rf_model.predict(X_test)

mae  = mean_absolute_error(y_test, y_pred)
rmse = np.sqrt(mean_squared_error(y_test, y_pred))
r2   = r2_score(y_test, y_pred)

print(f"R2 Score:  {round(r2, 4)}")
print(f"MAE:       £{round(mae, 2):,}")
print(f"RMSE:      £{round(rmse, 2):,}")
print(f"Model explains {round(r2 * 100, 2)}% of revenue variation")

Feature Importance

One of the best things about a Random Forest is that after training it can tell you which features it relied on most when making predictions. This is called feature importance, and it's a great sanity check if the most important features make intuitive sense, you can trust the model more.

feature_importance = pd.DataFrame({
    'Feature':    features,
    'Importance': rf_model.feature_importances_
}).sort_values('Importance', ascending=True)

plt.barh(feature_importance['Feature'],
         feature_importance['Importance'],
         color=['#e53935' if imp > 0.1 else '#1a237e'
                for imp in feature_importance['Importance']])
plt.title('Feature Importance  Random Forest Model (Red = Most Important)')
plt.xlabel('Importance Score')
plt.tight_layout()
plt.show()

print("Top 5 most important features:")
print(feature_importance.tail(5)[['Feature', 'Importance']].to_string())

In this model, the lag features particularly Prev_Month_Revenue and Rolling_3_Month_Avg dominate the importance scores. This makes complete intuitive sense: the best predictor of next month's sales is what happened in the last few months. Seasonal flags like Is_Christmas also rank highly, confirming that the model has picked up the Christmas peak pattern we spotted in EDA.

Step 7 Making the Sales Forecast

Now for the payoff. To predict next month's revenue, we construct a single-row DataFrame with all 14 features filled in using the most recent real data, then feed it to the trained model.

# Pull the last 3 months of actual revenue for lag features
last_month_revenue   = monthly_df['Monthly_Revenue'].iloc[-1]
last_2_month_revenue = monthly_df['Monthly_Revenue'].iloc[-2]
last_3_month_revenue = monthly_df['Monthly_Revenue'].iloc[-3]
rolling_avg          = monthly_df['Monthly_Revenue'].iloc[-3:].mean()

# Work out what the next calendar month is
last_month_number  = monthly_df['Month_Number'].iloc[-1]
last_month_of_year = monthly_df['Month_Of_Year'].iloc[-1]
last_year          = monthly_df['Year'].iloc[-1]

next_month_number   = last_month_number + 1
next_month_of_year  = (last_month_of_year % 12) + 1        # wraps 12 → 1
next_year           = last_year if next_month_of_year != 1 else last_year + 1

# Use historical averages for the volume features (we don't know the future values yet)
avg_orders       = monthly_df['Monthly_Orders'].mean()
avg_customers    = monthly_df['Monthly_Customers'].mean()
avg_items        = monthly_df['Monthly_Items'].mean()
avg_order_value  = monthly_df['Avg_Order_Value'].mean()

next_month_features = pd.DataFrame({
    'Month_Number':         [next_month_number],
    'Month_Of_Year':        [next_month_of_year],
    'Year':                 [next_year],
    'Is_Christmas':         [1 if next_month_of_year in [11, 12] else 0],
    'Is_Summer':            [1 if next_month_of_year in [6, 7, 8]  else 0],
    'Is_Q1':                [1 if next_month_of_year in [1, 2, 3]  else 0],
    'Prev_Month_Revenue':   [last_month_revenue],
    'Prev_2_Month_Revenue': [last_2_month_revenue],
    'Prev_3_Month_Revenue': [last_3_month_revenue],
    'Rolling_3_Month_Avg':  [rolling_avg],
    'Monthly_Orders':       [avg_orders],
    'Monthly_Customers':    [avg_customers],
    'Monthly_Items':        [avg_items],
    'Avg_Order_Value':      [avg_order_value],
})

prediction = rf_model.predict(next_month_features)[0]
print(f"PREDICTED REVENUE: £{prediction:,.2f}")

• .iloc[-1] means "the last row" negative indexing counts from the end, so -1 is the last row, -2 is second-to-last, and so on.
• % 12 + 1 is modular arithmetic. When the current month is December (12), 12 % 12 = 0, and 0 + 1 = 1 which correctly rolls over to January. Without this trick you'd get month 13, which doesn't exist.
• Historical averages are used for order count, customer count, and item count because we genuinely don't know those future values at prediction time. Using the average is a reasonable approximation.

Interactive Predictor Widget

The final step turns the technical notebook into something anyone can use. Using ipywidgets, we build a simple UI with two dropdowns and a button no coding required to generate a forecast.

import ipywidgets as widgets
from IPython.display import display, clear_output

year_dropdown = widgets.Dropdown(
    options=[2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020],
    value=2012,
    description='Year:',
    style={'description_width': 'initial'}
)

month_dropdown = widgets.Dropdown(
    options=[('January',1),('February',2),('March',3),('April',4),
             ('May',5),('June',6),('July',7),('August',8),
             ('September',9),('October',10),('November',11),('December',12)],
    value=1,
    description='Month:',
    style={'description_width': 'initial'}
)

predict_button = widgets.Button(
    description='Predict Sales',
    button_style='success',
    icon='check',
    layout=widgets.Layout(width='200px', height='40px')
)

output = widgets.Output()

def on_button_click(b):
    with output:
        clear_output(wait=True)
        predict_sales_for_month(year_dropdown.value, month_dropdown.value)

predict_button.on_click(on_button_click)

display(widgets.VBox([
    widgets.HBox([year_dropdown, month_dropdown]),
    predict_button,
    output
]))

• widgets.Dropdown creates a clickable dropdown menu. The options parameter sets the available choices.
• predict_button.on_click(on_button_click) wires the button to our function. Every time the button is pressed, Python calls on_button_click automatically.
• clear_output(wait=True) erases the previous result before printing the new one. Without this, predictions would stack up on the screen with each click.
• VBox stacks widgets vertically; HBox places them side by side. Combining them gives you a basic layout dropdowns on one line, then button below.

Business Insights

Beyond the model metrics, the analysis surfaces insights directly applicable to retail operations things a manager could act on today, with or without the model.

• Christmas planning window. November and December consistently generate disproportionate revenue. The model's seasonal flags quantify this premium, giving procurement and staffing teams a data-backed basis for scaling up 6–8 weeks ahead rather than guessing.
• Revenue momentum matters. The dominance of lag features in the importance chart confirms that recent performance is the strongest predictor of near-future revenue. A sudden dip in the rolling average should trigger investigation, not be dismissed as noise.
• VIP customer retention is disproportionately valuable. A small number of high-spend customers drive a large share of total revenue. Losing one VIP customer has a much bigger impact than losing ten New customers.
• Peak hours drive operational scheduling. The 11am–2pm trading window identified in EDA is the busiest period. Customer service, warehouse dispatch, and logistics should have maximum capacity during these hours.

Potential Improvements

This project covers the core pipeline well, but there are several directions you could take it further:

• Try gradient boosting models. XGBoost or LightGBM are algorithms purpose-built for tabular regression tasks like this one. They often outperform Random Forest and are worth benchmarking against it.
• Use time-series cross-validation. The current 80/20 random split can accidentally put future months in the training set, which inflates performance metrics. Scikit-learn's TimeSeriesSplit always trains on the past and tests on the future a more honest evaluation.
• Add external signals. UK public holidays, Google Trends data for gift-related searches, or macroeconomic indicators (like consumer confidence) could improve accuracy, especially around unusual periods.
• Forecast at product-category level. Aggregating to total monthly revenue loses detail. A separate model per product category enables much more targeted inventory decisions.
• Output a prediction interval, not just a point estimate. Instead of saying "next month will be £480,000", say "we're 80% confident it'll be between £420,000 and £550,000." That's a more honest and actionable forecast for a planning team.

Conclusion

This project walks through a complete machine learning pipeline applied to a real retail forecasting problem. Starting from over one million raw transactions, we cleaned the data, explored it visually, engineered 14 meaningful features, and trained a Random Forest Regressor that predicts monthly revenue from patterns in the past.

The most important lesson from the feature importance analysis is simple: recent history is the best predictor of the near future. The lag features and rolling average dominated the model which is exactly what you'd expect from a stable, recurring business. Seasonal flags confirmed that the model learned the Christmas effect without us having to hand-code it.

If you're new to machine learning, this project is a good template to learn from. The same seven-step pipeline load, clean, explore, engineer, train, evaluate, predict applies to almost any forecasting problem you'll encounter: energy demand, website traffic, inventory replenishment, or staffing needs. The tools and thinking carry directly across.

The full notebook with all code, charts, and the interactive widget is available in the GitHub repository linked below.