You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: watttime/evaluation/report_card_template.html
+8-8Lines changed: 8 additions & 8 deletions
Original file line number
Diff line number
Diff line change
@@ -273,14 +273,14 @@ <h2>Analysis of Historical Signal Data</h2>
273
273
"plot_distribution_moers",
274
274
plot_distribution_moers,
275
275
"2) Distribution of Historical Signal",
276
-
"A box and whiskers plot is helpful to sucintly examine the entire distribution of signal values. When studying this plot, look for how the median and interquartile range changes between model values. A dramatic shift could be caused by changes to the underlying grid (e.g. coal retirements), or new modelling features (e.g. considering curtailment). Any large changes should be explainable by the modeler."
276
+
"A box and whiskers plot is helpful to succinctly examine the entire distribution of signal values. When studying this plot, look for how the median and interquartile range changes between model values. A dramatic shift could be caused by changes to the underlying grid (e.g. coal retirements), or new modelling features (e.g. considering curtailment). Any large changes should be explainable by the modeler."
277
277
) }}
278
278
279
279
{{ tab_container(
280
280
"plot_heatmaps",
281
281
plot_heatmaps,
282
282
"3) Heatmaps of Signal Values Over Time",
283
-
"These heatmaps provide a succinct way to look for diurnal and seasonal trends in signal values. To achieve this purpose, the color values are normalized for each model seperately to make the rank order differences more apparent. Generally, we do not expect significant differences in the relative diurnal rank ordering of models without an explainable cause, such as changes to the power system or new modelling features."
283
+
"These heatmaps provide a succinct way to look for diurnal and seasonal trends in signal values. To achieve this purpose, the color values are normalized for each model separately to make the rank order differences more apparent. Generally, we do not expect significant differences in the relative diurnal rank ordering of models without an explainable cause, such as changes to the power system or new modelling features."
284
284
) }}
285
285
286
286
{{ tab_container(
@@ -293,8 +293,8 @@ <h2>Analysis of Historical Signal Data</h2>
293
293
{{ tab_container(
294
294
"plot_bland_altman",
295
295
plot_bland_altman,
296
-
"5) Bland Altman Plot for Bias Assesment",
297
-
"The Bland Altman plot is a useful tool for comparing the bias between two datasets. This plot is run on a random sample of 2500 datapoints. It contains three elements: 1) A scatterplot of the relative differences between the modeled point_times against the mean of both modeled point_times. We expect to see the y axis of the scatterplot randomly (uniformly) distributed across all means. If there is a trend, it indicates that one model has biases at certain value ranges that the other model doesn't hold. 2) The reference line is the mean delta between the two models. The magnitude and direction of this line indicate if one model has large systemic bias when compared with the other. 3) 95% Confidence Intervals denote the area that we would expect all datapoints to be within if both models had the same bias. Note that in some cases it is expeccted for bias to change between models (e.g. systemic changes to the grid). A suitable explanation should be sought out if bias is apparent."
296
+
"5) Bland Altman Plot for Bias Assessment",
297
+
"The Bland Altman plot is a useful tool for comparing the bias between two datasets. This plot is run on a random sample of 2500 datapoints. It contains three elements: 1) A scatterplot of the relative differences between the modeled point_times against the mean of both modeled point_times. We expect to see the y axis of the scatterplot randomly (uniformly) distributed across all means. If there is a trend, it indicates that one model has biases at certain value ranges that the other model doesn't hold. 2) The reference line is the mean delta between the two models. The magnitude and direction of this line indicate if one model has large systemic bias when compared with the other. 3) 95% Confidence Intervals denote the area that we would expect all datapoints to be within if both models had the same bias. Note that in some cases it is expected for bias to change between models (e.g. systemic changes to the grid). A suitable explanation should be sought out if bias is apparent."
"Most models allow for multiple fuel types to be marginal at a given time. Each fuel type is asigned a percentage of the total margianl fuel mix, and these will sum to 100%. Changes in signal value are primarily caused by differences in the marginal fuel mix, but can be affected by other things (such as modeled differences in carbon intensity or damages over time). In addition to fuel types like natural gas, coal, and oil being marginal, interchange can also be a marginal fuel type. Interchange can represent either imports or exports. This plot shows a time series of the marignal fuel mix. It is helpful to look for diurnal and seasonal patterns, along with differences between models. Users can pan to examine different time periods."
311
+
"Most models allow for multiple fuel types to be marginal at a given time. Each fuel type is assigned a percentage of the total marginal fuel mix, and these will sum to 100%. Changes in signal value are primarily caused by differences in the marginal fuel mix, but can be affected by other things (such as modeled differences in carbon intensity or damages over time). In addition to fuel types like natural gas, coal, and oil being marginal, interchange can also be a marginal fuel type. Interchange can represent either imports or exports. This plot shows a time series of the marginal fuel mix. It is helpful to look for diurnal and seasonal patterns, along with differences between models. Users can pan to examine different time periods."
312
312
) }}
313
313
314
314
{{ tab_container(
315
315
"plot_fuelmix_heatmap",
316
316
plot_fuelmix_heatmap,
317
317
"2) Heatmaps of Marginal Fuel Mix Over Time",
318
-
"These heatmaps show the average percentage of the marginal fuel mix for a given fuel type by month and hour. This plot is helpful for examining diurnal and seasonal trends in the marginal fuelmix that can be impacting the signal value. It is also useful to understand which hours of the day curtailment is occuring, indicated by the carbon_free fuel type which may be present."
318
+
"These heatmaps show the average percentage of the marginal fuel mix for a given fuel type by month and hour. This plot is helpful for examining diurnal and seasonal trends in the marginal fuelmix that can be impacting the signal value. It is also useful to understand which hours of the day curtailment is occurring, indicated by the carbon_free fuel type which may be present."
319
319
) }}
320
320
{% endif %}
321
321
322
322
{# Forecast Section #}
323
323
{% if show_forecast %}
324
324
<h2>Forecast Performance</h2>
325
-
<p>Our modeled signal values produce real-time or historical outputs. To achieve AER impact while guaranting an acceptable charge rate / duty factor it is necessary to use a forecast to schedule electricity consumption or charging ahead of time. Our forecast models use traditional machine learning techniques to predict what the signal value will be ahead of time. Our forecast models tend to perform worse at further time horizons from the time they are instatiated. As such, we examine the performance of our forecast models at predicting the signal value over a variety of time horizons.</p>
325
+
<p>Our modeled signal values produce real-time or historical outputs. To achieve AER impact while guaranteeing an acceptable charge rate / duty factor it is necessary to use a forecast to schedule electricity consumption or charging ahead of time. Our forecast models use traditional machine learning techniques to predict what the signal value will be ahead of time. Our forecast models tend to perform worse at further time horizons from the time they are instantiated. As such, we examine the performance of our forecast models at predicting the signal value over a variety of time horizons.</p>
326
326
327
327
{{ tab_container(
328
328
"plot_forecasts_vs_signal",
@@ -364,7 +364,7 @@ <h2>Forecast Performance</h2>
364
364
"plot_impact_forecast_metrics",
365
365
plot_impact_forecast_metrics,
366
366
"6) Estimation of AER Impact using Forecasts",
367
-
"These plots simulate a vairiety of AER scenarios for typical customer patterns including charging an EV during the day (3 hours in a 12 hour window starting at 8 AM), the night (2 hours in a 8 hour window starting at 9 PM), and running a thermostat (running for 30 out of 60 minutes throughout the day). Each of these metrics could capture a different source of impact (e.g. solar curtailment may only be accessible to EV day charging). The 'simulation' simply looks at the cleanest timem to use electricity based on the forecasted value, and evaluates the impact of this against the historical signal value as truth. We also show the impact captured if the forecast was perfect by displaying the maximum possible impact if a 'perfect' forecast were used. These metrics assume that the modeled values are accurate, and only quantify the degree of error caused by the forecast model. Units are normalized to be impact achieved per day (which may or may not be per window)."
367
+
"These plots simulate a variety of AER scenarios for typical customer patterns including charging an EV during the day (3 hours in a 12 hour window starting at 8 AM), the night (2 hours in a 8 hour window starting at 9 PM), and running a thermostat (running for 30 out of 60 minutes throughout the day). Each of these metrics could capture a different source of impact (e.g. solar curtailment may only be accessible to EV day charging). The 'simulation' simply looks at the cleanest time to use electricity based on the forecasted value, and evaluates the impact of this against the historical signal value as truth. We also show the impact captured if the forecast was perfect by displaying the maximum possible impact if a 'perfect' forecast were used. These metrics assume that the modeled values are accurate, and only quantify the degree of error caused by the forecast model. Units are normalized to be impact achieved per day (which may or may not be per window)."
0 commit comments