blue-marble
Using GAMS-bundled solvers
GAMS comes bundled with several solvers, e.g. CPLEX, which can only be invoked through GAMS itself. Using GAMS bundled solvers with Pyomo involves invoking the GAMS executable and passing the solver name as an argument. A variety of arguments ("Option"s) can be passed to GAMS itself, and some of them can be passed to the bundled solver.
GridPath has some behaviours that currently prevent use of GAMS bundled solvers. These are explored below.
@milindsmart, I'm doing some issue cleanup and don't know what to do with this one, as I don't have access to GAMS and can't work through the problem. If at some point you can document some of what you learned and perhaps add here from our discussion on Slack, I'd appreciate it.
Hi Ana, and sorry for the long delay, I was on a hiatus of sorts from GridPath due to some other projects. I'm definitely available now for some time.
With respect to this particular issue, I'll be happy to develop a PR since I have the GAMS instance to work with. Need your guidance on the best way to architect and modify this though. Let me know if this sounds okay to you. I'll also document this further (including Slack discussions) by tomorrow.
Hi Milind. Sounds great. There's no rush on this as you are the only user with this need right now, so it's really up to you. I'll keep this issue open.
The issues with using GAMS with GridPath are best framed in terms of getting parameters to GAMS and the solvers from the GridPath database. Before getting into it, I want to say that I'm not very happy about the aesthetics of the architecture involved in using GAMS with GridPath, seems like a lot of layering violations. It seems inevitable though with indirections like GP CSVs → GP database → GP tab files → Pyomo → GAMS → Bundled-solver... Let me explain
Pyomo-GAMS interface choice
Currently solver_options are meant to pass options directly to the solver. The way this works is, first, a Pyomo solver object is attempted to be instantiated from SolverFactory. The SolverFactory class uses the solver_name passed to it from GP and calls a specialised subclass such as GAMSSolver via a system of decorators and registration.
For GAMS, there is a choice of running GAMS through the shell, or through a Python API. This functionality is accessed via a keyword argument to SolverFactory:
https://github.com/Pyomo/pyomo/blob/3058f462d8069901e6c04651ae5b962b60f45cb7/pyomo/solvers/plugins/solvers/GAMS.py#L125-L135
GridPath currently does not pass any options to SolverFactory.
GAMSSolver-specific parameters
Another set of parameters that matter here are the GAMSSolver-specific parameters fed to solve() like logfiles, load_solutions and tmpdir. Thankfully, these are passed on properly as long as they are added to solver_options.
GAMSWriter parameters
A third set of parameters - anything in io_options - will be passed on by GAMSSolver to GAMSWriter which writes out a GMS file that is actually run by GAMS itself. This includes warmstart, the all-important solver (discussed next), skip_trivial_constraints, and mtype (model type - lp/mip/?). There too, we are mostly lucky since Pyomo passes any options that are not recognised by GAMSSolver into io_options, for GAMSWriter. Couple of parameters that give us trouble - solver and add_options.
solver parameter
GridPath takes special care to not allow one of the solver options to be solver. In addition, the solver command line argument is constrained to be the same as the solver chosen among the solver_options_id subscenarios. This aspect is seen in the below segment of code.
https://github.com/blue-marble/gridpath/blob/66560ab084e1e2f4800e270090d5efc8f6ff01a6/gridpath/run_scenario.py#L609-L645
I was able to run GridPath by loosening these constraints on the solver argument of gridpath_run.
add_options parameter
add_options consists of "option"s to be fed to GAMS that are not recognised by Pyomo at all (neither by base Pyomo, nor by GAMSSolver, nor by GAMSWriter). GAMSWriter tries to use it as an list of strings (actually just an iterable of strings), adding them to the GMS file, which is then run by GAMS. A practical example of a parameter I needed to pass is Option threads=4, which GAMS uses to enable parallelism in the solver.
This doesn't work with the solver_options framework in GridPath because any text in the value column is already implicitly a string, which does not allow itself to be considered as an iterable (except as a character-by-character iterable like any string).
Solver-specific options
Finally, some solver-specific parameters cannot be set in any other way than having a separate .opt file in the GAMS working directory. A prominent example is the lpmethod parameter used for choosing the barrier method. This cannot be set in any other way than a cplex.opt file, AFAIK.
To summarise, the following extensibility points are involved when trying to use GridPath with GAMS
| Extensibility point | Current status | Possible modification | Criticality/Necessity |
|---|---|---|---|
Pyomo-GAMS interface type (Shell/Python API) via solver_io argument to SolverFactory |
No current way of doing this in GP | This functionality applies to solver( interface)s other than GAMS as well, so perhaps another column in solver_options (maybe called solver_io) is called for |
Optional |
| GAMSSolver parameters | Works as intended | NA | Critical |
| Solver name (= "gams") | Currently tied to the chosen solver_options value |
Remove constraint | Critical |
| GAMS parameters (via GAMSWriter) | Also works since unrecognised options are passed properly to GAMSWriter (solver is currently tied to the solver name and solver_options value ; add_options doesn't work since current functionality doesn't allow it to be a string as required by GAMSWriter) |
Remove constraint on solver. add_options can become a list if repeated multiple times with different values for the same key |
Critical |
Solver-specific parameters (in a solvername.opt text file) |
No current way of doing this in GP | Custom python code or some shell hack | Intermediate |
But there's almost certainly a better way of doing this.