sci.op-research

>> > Let me return to the original problem which is pre-processing, not
>> > solver management during optimization. I happen to use MPL as my
>> > model manager. But the paradigm I've increasingly followed is
>> > separating data management from model management as much as I can. If
>> > you are into a data base to service the model for a penny, you are in
>> > for a pound. So you might as well leverage it's full capabilities
>> > because it's a lot easier to manipulate data on that platform than it
>> > is with GAMS or MPL.
>>
>> > My models can be intermediate sized, and when I need to support one
>> > with a data base, I use Access. I can generate almost every input
>> > vector in Access using the Query Builder including index sets and pass
>> > those such that MPL merely reads in the data/indexes and passes those
>> > on to the with very little manipulation formulation. And error
>> > checking is much easier this way.
>>
...
>> > SteveM
>>
>> But how would your paradigm cope when you have data coming in from
>> databases
>> of various types possibly scattered across many sites?
>> Bob Daniel
>
> Bob,
>
> Again, I don't run large scale models so do not have that
> requirement. When I was in the OR group at American Airlines long
> ago, we used SAS for data management of distributed legacy data bases
> until we got the model up and running. And then we handed data
> management over to the Sabre programmers. But those were also the
> MPSX days where we had to write our own MPS file generators.
>
> But with a query builder like Access, it's much easier for me to see
> incrementally how queries are building out. I mean wham, bam, back
> and forth testing queries with sample data sets. MPL is much less
> transparent in doing that.
>
> You are obviously much more experienced on that end. If you want to
> give us an idea of your approach for data management given current
> software capabilities, that would be great. Can you follow this with
> something brief?
>
> Thanks,
>
> SteveM
I'm not sure I can be brief without being facile. I'll try to make it not
sound like a commercial.

When we did Xpress' lp-model it was almost de rigeur to stick to the mantra
that separation of model and data was a Good Thing. We would have been
slaughtered by the academics had we not done this. But when we decided to
replace lp-model a few years ago we had long and hard thoughts about what
paradigm to adopt and talked to lots of our users.

Xpress-Mosel was the result of this, and it's radically different in that
it's a programming language with math programming things as first class
objects. We decided on this approach as lots of users wanted the ability to
do fairly rigorous data manipulation, comparisons and checking in the same
place as where the model was defined. Data coming from more than one
database, which is inherent in modelling using on-line data from Production,
Marketing, Sales, Inventory etc are a fact of life. And IMHO these have to
be reconciled at one central location in many models.

So Mosel had to have full programming capabilities. OK, yet another language
to learn but it's pretty easy (even an old FORTRAN man like me could pick it
up!) The programming aspect has other benefits we wanted too, like being
able to write complex optimization schemes e.g. decomposition, and be
extensible. Since it's compiled one can hide the intellectual property/
prevent naive users messing up the model.

Obviously the modeller has to have the ability to suck raw data out of, and
pump answers back to, external databases, so that was a must have. In the
old days a Report Writer was the final leg of the tripod, with modeller and
optimizer. Having a programming capability renders that obsolete.

I certainly wouldn't decry doing lots of stuff in Access/whatever. If one is
familiar with that then one can clearly be highly productive. As long as the
whatever isn't Excel ...

Bob
These are my personal opinions.