Tutorial 3: Mixed Integer Linear Programming

This example is based on the urban scale example model, but with an override. An override file exists in which binary and integer decision variables are triggered, creating a MILP model, rather than the conventional Calliope LP model.

Warning

Integer and Binary variables are still experimental and may not cover all edge cases as intended. Please raise an issue on GitHub if you see unexpected behaviour.

Units

The capacity of a technology is usually a continuous decision variable, which can be within the range of 0 and energy_cap_max (the maximum capacity of a technology). In this model, we introduce a unit limit on the CHP instead:

chp:
    constraints:
        units_max: 4
        energy_cap_per_unit: 300
        energy_cap_min_use: 0.2
    costs:
        monetary:
            energy_cap: 700
            purchase: 40000

A unit maximum allows a discrete, integer number of CHP to be purchased, each having a capacity of energy_cap_per_unit. Any of energy_cap_max, energy_cap_min, or energy_cap_equals are now ignored, in favour of units_max, units_min, or units_equals. A useful feature unlocked by introducing this is the ability to set a minimum operating capacity which is only enforced when the technology is operating. In the LP model, energy_cap_min_use would force the technology to operate at least at that proportion of its maximum capacity at each time step. In this model, the newly introduced energy_cap_min_use of 0.2 will ensure that the output of the CHP is 20% of its maximum capacity in any time step in which it has a non-zero output.

Purchase cost

The boiler does not have a unit limit, it still utilises the continuous variable for its capacity. However, we have introduced a purchase cost:

boiler:
    costs:
        monetary:
            energy_cap: 35
            purchase: 2000

By introducing this, the boiler now has a binary decision variable associated with it, which is 1 if the boiler has a non-zero energy_cap (i.e. the optimisation results in investment in a boiler) and 0 if the capacity is 0. The purchase cost is applied to the binary result, providing a fixed cost on purchase of the technology, irrespective of the technology size. In physical terms, this may be associated with the cost of pipework, land purchase, etc. The purchase cost is also imposed on the CHP, which is applied to the number of integer CHP units in which the solver chooses to invest.

MILP functionality can be easily applied, but convergence is slower as a result of integer/binary variables. It is recommended to use a commercial solver (e.g. Gurobi, CPLEX) if you wish to utilise these variables outside this example model.

Running the model

We now take you through running the model in a Jupyter notebook, which is included fully below. To download and run the notebook yourself, you can find it here. You will need to have Calliope installed.

Notebook

Calliope Urban Scale MILP Example Model

For more details on analysing input/output data, see the full urban scale example model

In [1]:
import calliope

# cufflinks allows for easy plotly plots to be 
# produced from a pandas DataFrame
import cufflinks

# We increase logging verbosity
calliope.set_log_level('INFO')
In [2]:
model = calliope.examples.milp()

# Note, we see the overrides that we have applied printed here, thanks to inreasing logging verbosity
[2018-06-04 12:03:20] INFO: Model: initialising
[2018-06-04 12:03:22] INFO: Override applied to model.name: Urban-scale example model -> Urban-scale example model with MILP
`techs.boiler.costs.monetary.energy_cap`:35 applied from override as new configuration
`techs.boiler.costs.monetary.purchase`:2000 applied from override as new configuration
`techs.chp.constraints.energy_cap_min_use`:0.2 applied from override as new configuration
`techs.chp.constraints.energy_cap_per_unit`:300 applied from override as new configuration
`techs.chp.constraints.units_max`:4 applied from override as new configuration
Override applied to techs.chp.costs.monetary.energy_cap: 750 -> 700
`techs.chp.costs.monetary.purchase`:40000 applied from override as new configuration
[2018-06-04 12:03:22] INFO: Model: preprocessing stage 1 (model_run)
[2018-06-04 12:03:24] INFO: Model: preprocessing stage 2 (model_data)
[2018-06-04 12:03:26] INFO: Model: preprocessing complete. Time since start: 0:00:05.478835
In [3]:
# Model inputs can be viewed at `model.inputs`. 
# Variables are indexed over any combination of `techs`, `locs`, `carriers`, `costs` and `timesteps`, 
# although `techs`, `locs`, and `carriers` are often concatenated. 
# e.g. `chp`, `X1`, `heat` -> `X1::chp::heat` 
model.inputs
Out[3]:
<xarray.Dataset>
Dimensions:                               (carrier_tiers: 3, carriers: 3, coordinates: 2, costs: 1, loc_carriers: 10, loc_tech_carriers_conversion_plus: 3, loc_techs: 26, loc_techs_area: 3, loc_techs_conversion: 2, loc_techs_conversion_plus: 1, loc_techs_export: 4, loc_techs_finite_resource: 9, loc_techs_investment_cost: 20, loc_techs_milp: 1, loc_techs_non_conversion: 23, loc_techs_om_cost: 9, loc_techs_supply_plus: 3, loc_techs_transmission: 10, locs: 4, techs: 9, timesteps: 48)
Coordinates:
  * loc_techs_area                        (loc_techs_area) object 'X3::pv' ...
  * loc_techs_export                      (loc_techs_export) object 'X3::pv' ...
  * loc_tech_carriers_conversion_plus     (loc_tech_carriers_conversion_plus) object 'X1::chp::gas' ...
  * loc_techs_om_cost                     (loc_techs_om_cost) object 'X1::supply_gas' ...
  * loc_techs_finite_resource             (loc_techs_finite_resource) object 'X3::demand_electricity' ...
  * costs                                 (costs) object 'monetary'
  * loc_techs_transmission                (loc_techs_transmission) object 'X1::power_lines:X2' ...
  * loc_techs_investment_cost             (loc_techs_investment_cost) object 'X1::supply_gas' ...
  * carriers                              (carriers) object 'gas' 'heat' ...
  * loc_techs_milp                        (loc_techs_milp) object 'X1::chp'
  * loc_techs_supply_plus                 (loc_techs_supply_plus) object 'X3::pv' ...
  * locs                                  (locs) object 'X3' 'N1' 'X1' 'X2'
  * techs                                 (techs) object 'power_lines' 'pv' ...
  * coordinates                           (coordinates) object 'y' 'x'
  * timesteps                             (timesteps) datetime64[ns] 2005-07-01 ...
  * loc_techs_conversion_plus             (loc_techs_conversion_plus) <U7 'X1::chp' ...
  * loc_techs_conversion                  (loc_techs_conversion) <U10 'X3::boiler' ...
  * loc_techs                             (loc_techs) object 'X3::demand_electricity' ...
  * loc_carriers                          (loc_carriers) object 'X2::gas' ...
  * carrier_tiers                         (carrier_tiers) <U5 'out_2' 'in' 'out'
  * loc_techs_non_conversion              (loc_techs_non_conversion) object 'X3::demand_electricity' ...
Data variables:
    resource_unit                         (loc_techs_finite_resource) <U5 'power' ...
    export_carrier                        (loc_techs_export) <U11 'electricity' ...
    resource_eff                          (loc_techs_finite_resource) float64 nan ...
    energy_con                            (loc_techs) float64 1.0 1.0 nan ...
    force_resource                        (loc_techs_finite_resource) bool True ...
    energy_cap_min_use                    (loc_techs) float64 nan nan nan ...
    units_max                             (loc_techs_milp) int32 4
    resource                              (loc_techs_finite_resource, timesteps) float64 -18.76 ...
    energy_cap_max                        (loc_techs) float64 nan 2e+03 ...
    resource_area_per_energy_cap          (loc_techs_area) int32 7 7 7
    lifetime                              (loc_techs) float64 nan 25.0 25.0 ...
    energy_prod                           (loc_techs) float64 nan 1.0 1.0 ...
    energy_cap_per_unit                   (loc_techs) float64 nan nan nan ...
    resource_area_max                     (loc_techs_area) int32 1500 1500 1500
    parasitic_eff                         (loc_techs_supply_plus) float64 0.85 ...
    energy_eff                            (loc_techs) float64 nan 0.98 nan ...
    reserve_margin                        (carriers) float64 nan nan nan
    cost_purchase                         (costs, loc_techs_investment_cost) float64 nan ...
    cost_om_prod                          (costs, loc_techs_om_cost) float64 nan ...
    cost_om_annual                        (costs, loc_techs_om_cost) float64 nan ...
    cost_energy_cap                       (costs, loc_techs_investment_cost) float64 1.0 ...
    cost_om_con                           (costs, loc_techs_om_cost) float64 0.025 ...
    cost_export                           (costs, loc_techs_om_cost, timesteps) float64 nan ...
    cost_depreciation_rate                (costs, loc_techs_investment_cost) float64 0.1102 ...
    distance                              (loc_techs_transmission) float64 10.0 ...
    lookup_remotes                        (loc_techs_transmission) <U18 'X2::power_lines:X1' ...
    available_area                        (locs) float64 900.0 nan 500.0 1.3e+03
    loc_coordinates                       (coordinates, locs) int32 3 7 7 7 ...
    colors                                (techs) <U7 '#6783E3' '#F9D956' ...
    inheritance                           (techs) <U29 'transmission' ...
    names                                 (techs) <U29 'Electrical power distribution' ...
    carrier_ratios                        (carrier_tiers, loc_tech_carriers_conversion_plus) float64 1.0 ...
    carrier_ratios_min                    (carrier_tiers, loc_techs_conversion_plus) float64 0.8 ...
    lookup_loc_carriers                   (loc_carriers) <U175 'X2::supply_gas::gas,X2::boiler::gas' ...
    lookup_loc_techs                      (loc_techs_non_conversion) <U35 'X3::demand_electricity::electricity' ...
    lookup_loc_techs_conversion           (carrier_tiers, loc_techs_conversion) object None ...
    lookup_primary_loc_tech_carriers_in   (loc_techs_conversion_plus) <U12 'X1::chp::gas' ...
    lookup_primary_loc_tech_carriers_out  (loc_techs_conversion_plus) <U20 'X1::chp::electricity' ...
    lookup_loc_techs_conversion_plus      (carrier_tiers, loc_techs_conversion_plus) object 'X1::chp::heat' ...
    lookup_loc_techs_export               (loc_techs_export) <U20 'X3::pv::electricity' ...
    lookup_loc_techs_area                 (locs) <U6 'X3::pv' '' 'X1::pv' ...
    timestep_resolution                   (timesteps) float64 1.0 1.0 1.0 ...
    timestep_weights                      (timesteps) float64 1.0 1.0 1.0 ...
    max_demand_timesteps                  (carriers) datetime64[ns] 2005-07-01 ...
Attributes:
    model.calliope_version:            0.6.2
    model.name:                        Urban-scale example model with MILP
    model.subset_time:                 ['2005-07-01', '2005-07-02']
    model.timeseries_dateformat:       %Y-%m-%d %H:%M:%S
    run.backend:                       pyomo
    run.bigM:                          1000000.0
    run.cyclic_storage:                False
    run.ensure_feasibility:            True
    run.mode:                          plan
    run.objective:                     minmax_cost_optimization
    run.objective_options.cost_class:  monetary
    run.objective_options.sense:       minimize
    run.operation.use_cap_results:     False
    run.solver:                        glpk
    run.zero_threshold:                1e-10
    calliope_version:                  0.6.2-dev
    defaults:                          available_area: null\ncarrier_ratios: ...
    allow_operate_mode:                1
In [4]:
# Individual data variables can be accessed easily, `to_pandas()` reformats the data to look nicer
# Here we look at one of the MILP overrides that we have added, the fixed `purchase` cost
model.inputs.cost_purchase.to_pandas().dropna(axis=1)
Out[4]:
loc_techs_investment_cost X2::boiler X3::boiler X1::chp
costs
monetary 2000.0 2000.0 40000.0
In [5]:
# Solve the model. Results are loaded into `model.results`. 
# By including logging (see package importing), we can see the timing of parts of the run, as well as the solver's log
model.run()
[2018-06-04 12:03:26] INFO: Backend: starting model run
[2018-06-04 12:03:30] INFO: Backend: model generated. Time since start: 0:00:09.890080
[2018-06-04 12:03:30] INFO: Backend: sending model to solver
[2018-06-04 12:03:33] INFO: Backend: solver finished running. Time since start: 0:00:12.857638
[2018-06-04 12:03:33] INFO: Backend: loaded results
[2018-06-04 12:03:33] INFO: Backend: generated solution array. Time since start: 0:00:12.964354
[2018-06-04 12:03:33] INFO: Postprocessing: started
[2018-06-04 12:03:34] INFO: Postprocessing: All values < 1e-10 set to 0 in unmet_demand
[2018-06-04 12:03:34] INFO: Postprocessing: Model was feasible, deleting unmet_demand variable
[2018-06-04 12:03:34] INFO: Postprocessing: ended. Time since start: 0:00:13.499920
In [6]:
# Model results are held in the same structure as model inputs. 
# The results consist of the optimal values for all decision variables, including capacities and carrier flow
# There are also results, like system capacity factor and levelised costs, which are calculated in postprocessing
# before being added to the results Dataset

model.results
Out[6]:
<xarray.Dataset>
Dimensions:                     (carriers: 3, costs: 1, loc_tech_carriers_con: 19, loc_tech_carriers_export: 4, loc_tech_carriers_prod: 21, loc_techs: 26, loc_techs_area: 3, loc_techs_cost: 20, loc_techs_investment_cost: 20, loc_techs_milp: 1, loc_techs_om_cost: 9, loc_techs_purchase: 2, loc_techs_supply_plus: 3, techs: 9, timesteps: 48)
Coordinates:
  * loc_techs_area              (loc_techs_area) object 'X3::pv' 'X2::pv' ...
  * loc_techs_om_cost           (loc_techs_om_cost) object 'X1::supply_gas' ...
  * costs                       (costs) object 'monetary'
  * loc_techs_investment_cost   (loc_techs_investment_cost) object 'X1::supply_gas' ...
  * carriers                    (carriers) object 'gas' 'heat' 'electricity'
  * loc_techs_milp              (loc_techs_milp) object 'X1::chp'
  * loc_techs_cost              (loc_techs_cost) object 'X1::supply_gas' ...
  * loc_techs_supply_plus       (loc_techs_supply_plus) object 'X3::pv' ...
  * techs                       (techs) object 'power_lines' 'pv' ...
  * loc_tech_carriers_con       (loc_tech_carriers_con) object 'X1::chp::gas' ...
  * timesteps                   (timesteps) datetime64[ns] 2005-07-01 ...
  * loc_techs_purchase          (loc_techs_purchase) object 'X3::boiler' ...
  * loc_tech_carriers_export    (loc_tech_carriers_export) object 'X1::pv::electricity' ...
  * loc_techs                   (loc_techs) object 'X3::demand_electricity' ...
  * loc_tech_carriers_prod      (loc_tech_carriers_prod) object 'X1::chp::electricity' ...
Data variables:
    energy_cap                  (loc_techs) float64 64.93 274.8 0.0 0.0 0.0 ...
    carrier_prod                (loc_tech_carriers_prod, timesteps) float64 116.1 ...
    carrier_con                 (loc_tech_carriers_con, timesteps) float64 -286.6 ...
    cost                        (costs, loc_techs_cost) float64 607.5 11.85 ...
    resource_area               (loc_techs_area) float64 350.0 0.0 0.0
    resource_con                (loc_techs_supply_plus, timesteps) float64 0.0 ...
    resource_cap                (loc_techs_supply_plus) float64 38.95 0.0 0.0
    carrier_export              (loc_tech_carriers_export, timesteps) float64 0.0 ...
    cost_var                    (costs, loc_techs_om_cost, timesteps) float64 7.165 ...
    cost_investment             (costs, loc_techs_investment_cost) float64 0.4472 ...
    purchased                   (loc_techs_purchase) float64 0.0 1.0
    units                       (loc_techs_milp) float64 1.0
    operating_units             (loc_techs_milp, timesteps) float64 1.0 1.0 ...
    capacity_factor             (loc_tech_carriers_prod, timesteps) float64 0.3869 ...
    systemwide_capacity_factor  (techs, carriers) float64 nan nan nan 0.0 ...
    systemwide_levelised_cost   (carriers, techs, costs) float64 nan inf nan ...
    total_levelised_cost        (carriers, costs) float64 0.03351 0.02533 ...
Attributes:
    model.calliope_version:            0.6.2
    model.name:                        Urban-scale example model with MILP
    model.subset_time:                 ['2005-07-01', '2005-07-02']
    model.timeseries_dateformat:       %Y-%m-%d %H:%M:%S
    run.backend:                       pyomo
    run.bigM:                          1000000.0
    run.cyclic_storage:                False
    run.ensure_feasibility:            True
    run.mode:                          plan
    run.objective:                     minmax_cost_optimization
    run.objective_options.cost_class:  monetary
    run.objective_options.sense:       minimize
    run.operation.use_cap_results:     False
    run.solver:                        glpk
    run.zero_threshold:                1e-10
    calliope_version:                  0.6.2-dev
    defaults:                          available_area: null\ncarrier_ratios: ...
    allow_operate_mode:                1
    termination_condition:             optimal
    solution_time:                     6.952948
    time_finished:                     2018-06-04 12:03:33
In [7]:
# We can sum operating units of CHP over all locations and turn the result into a pandas DataFrame
df_units = model.get_formatted_array('operating_units').sum('locs').to_pandas().T

#The information about the dataframe tells us about the amount of data it holds in the index and in each column
df_units.info()
<class 'pandas.core.frame.DataFrame'>
DatetimeIndex: 48 entries, 2005-07-01 00:00:00 to 2005-07-02 23:00:00
Data columns (total 1 columns):
chp    48 non-null float64
dtypes: float64(1)
memory usage: 768.0 bytes
In [8]:
# Using .head() to see the first few rows of operating units

df_units.head()
Out[8]:
techs chp
timesteps
2005-07-01 00:00:00 1.0
2005-07-01 01:00:00 1.0
2005-07-01 02:00:00 1.0
2005-07-01 03:00:00 1.0
2005-07-01 04:00:00 1.0
In [9]:
# We can plot this by using the timeseries plotting functionality.
# The top-left dropdown gives us the chance to scroll through other timeseries data too.

model.plot.timeseries()
In [10]:
# plot.capacities gives a graphical view of the non-timeseries variables, both input and output

# Note, because we fix unit size, CHP now has a maximum capacity of 300kW, 
# compared to 260kW in the non-MILP case

model.plot.capacity()