futurera | open data, time series, forecasting

SUMMARY

-Performance

Performance for the forecast

We have an example of data points / observations with the forecast.
Forecasting model is SES (singel Exponential Smoothing)


#	1	2	3	4	5	6	7	8	9	10	11	12	13
actual	20	20	25	28	29	28	31	34	35	36	41	45
predicted-(ɑ=0.8)		20	20	24	27	29	28	30	33	35	36	40	44
predicted-(ɑ=0.2)		20	20	21	22	24	25	26	27	29	30	33	35

Initial forecast=20

Bias
Bias = ( ∑ (^predictedy_i - ^actualy_i) ) / n
^predictedy_i: predicted value i-th data point / observation ^actualy_i: true value for the i-th data point / observation n: number of data points / observations

step 1
sum of the difference between the predicted(ɑ=0.8) and actual values

= (20 - 20) + (20 - 25) + (24 - 28) + (27 - 29) + (29 - 28) + (28 - 31) + (30 - 34) + (33 - 35) + (35 - 36) + (36 - 41) + (40 - 45)

= 0 + (-5) + (-4) + (-2) + 1 + (-3) + (-4) + (-2) + (-1) + (-5) + (-5)

= -30

step 2
then averaging with n = 11

Bias_(ɑ=0.8) is (-30) / 11 ~ (-2.73)

Ww do this again with the predicted(ɑ=0.2)
Bias_(ɑ=0.2) is (-75) / 11 ~ (-6.82)

Both forecast are negative, so the bias(ɑ=0.8) is an underestimation. More so for ɑ=0.2 (see the graphs).

There are many ways to code the bias in Python (here are two possible versions).

def func1(actual,predicted):

differences = []
for i,a in enumerated(actual):

p = predicted[i]
differences.append(p - a)

bias = sum(differences) / len(differences)
return bias

def func2(actual,predicted):

differences = [p - a for p, a in zip(predicted, actual)]
bias = sum(differences) / len(differences)
return bias

Top

Prediction Direction Accuracy (PDA)
PDA = (( ∑ (Prediction Direction_i = Actual Direction_i) ) / n ) x 100%
The direction is calculated through the difference between the current_value minus the past_value. If the directions of the predicted and the actual is the same then the counter will be raised.

step 1a: differences actual
#	actual	direction	result
1	20
2	20	20 - 20	0
3	25	25 - 20	5
4	28	28 - 25	3
5	29	29 - 28	1
6	28	28 - 29	-1
7	31	31 - 28	3
8	34	34 - 31	3
9	35	35 - 34	1
10	36	36 - 35	1
11	41	41 - 36	5
12	45	45 - 41	4
13

step 1b: differences predicted (ɑ=0.8)
#	predicted	direction	result
1
2	20
3	20	20 - 20	0
4	24	24 - 20	4
5	27	27 - 24	3
6	29	29 - 27	2
7	28	28 - 29	-1
8	30	30 - 28	2
9	33	33 - 30	3
10	35	35 - 33	2
11	36	36 - 35	1
12	40	40 - 38	2
13	44	44 - 40	4

step 1c: results
#	actual	predicted	count
1
2	0
3	5	0	1
4	3	4	1
5	1	3	1
6	-1	2	0
7	3	-1	0
8	3	2	1
9	3	3	1
10	1	2	1
11	5	1	1
12	4	2	1
13		4

summation = ∑ (Prediction Direction_i = Actual Direction_i) = 1 + 1 + 1 + 0 + 0 + 1 + 1 + 1 + 1 + 1
= 8

n = 10

pda = (8 / 10) x 100% = 80%

Top

def func(actual,predicted):

pda_count = 0
for i in range(1, len(predicted)):

p = predicted[i] - predicted[i-1]
a = actual[i] -actual[i-1]
if (p > 0 and a > 0) or (p < 0 and a < 0):

pda_count = pda_count + 1

pda = (pda_count / (len(predicted) -1)) x 100
return pda