ENGAGE - Revision history

Genie-user at 14:11, 20 January 2009

2009-01-20T14:11:00Z

Genie-user: Wiki discussion of proposal to ENGAGE programme

2008-11-24T14:25:20Z

Wiki discussion of proposal to ENGAGE programme

New page

For those of you not involved in previous discussions - OMII and the
ENGAGE project may fund the GUI development part of our earlier proposal
to OMII to request support for open-sourcing GENIE. Some other aspects of
this may also go ahead with seperate OMII funding, but the ENGAGE project
is coming to it's end so getting a proposal together for the GUI work is
the most urgent part. Steve is Steve Brewer, who I expect most of you
know. He suggested gettign the ball rolling in the most expedient way
possible on exploring possible funding options - in an informal email
discussion. My slightly incoherent ramblings start here:

Dear Steve,

Thanks for an informative discussion on the phone earlier. I'm copying in
the people who have been party to the last few emails, in the hope that
they might chime in with useful info whenever they can, also Rupert Ford
and Graham Riley from Manchester who have been involved in BFG2
(http://intranet.cs.man.ac.uk/cnc/projects/bfg.php,
http://source.ggy.bris.ac.uk/wiki/GENIE_BFG) work with GENIE, which may
integrate nicely as part of the work we put into the proposal. Have also
copied in Andrew Price (SOTON e-science) who is a long-standing GENIE
collaborator and whose input and funded involvement we would also welcome.

GENIE (Grid-ENabled Integrated Earth-system model) is a modular, flexible
Earth-system model of intermediate complexity (EMIC), which is used
extensively within the UK and internationally in Earth-system science. One
important aspect of GENIE which makes it unique (or nearly so?) amongst
EMICS is the modularity which currently allows experienced users to plug
in new or different models of a particular Earth-system component or in
theory even to use a different 'glue' code for sticking all the components
together.

To enable less experienced users to access these aspects of the model, and
also to facilitate ease of running the model in its 'standard'
configuration, a user interface would be extremely valuable. As well as
saving end-user time in manipulating XML config files and running shell
scripts, for example, it will also broaden the community of researchers
able to use GENIE to answer their research questions as less technical
computing knowledge will be necessary. The facility to increase or
decrease the complexity of options available in the GUI to produce
teaching, or targeted research versions would aldo be valuable.

Steve - you suggested that the best way to go about formulating a good
proposal for ENGAGE was to have an 'ideal world' goal for Mid-May '09 -
the deadline for completion of work funded under the ENGAGE project - from
which we can work back to person-months, staff availability etc. SO here
goes!

I envisage a beta release of a 'scaleable' GUI, which can be easily
configured to work at any required level of complexity with the underlying
code (so for example, parameters would be at their default values if not
selected for control by the GUI). The GUI would also allow configuration
and selection/deselection of a series of currently available modules
(different atmospheres, land and ocean models for example) and the ability
for the model to run with the current standard 'GENIE-main' model glue but
also with the BFG2 option. This would require effort from the Manchester
team and would realise excellent the preliminary work already undertaken
by their group. Part of this would be to BFG-wrap a number (1 or 2?) of
modules not currently available to BFG-GENIE - although it would not be
possible to comprehensively cover all current (non-BFG)GENIE-available
models due to time constraints.

We would need to ensure that this GUI was future-proof - we already have a
coherent XML configuration system, so it simply has to be able to
dynamically expand to cope with more modules and parameter options. New
model glue codes and approaches would presumably require new GUI code
unavoidably.

The GUI would be released along with the GENIE source and presumably be
included in the GENIE subversion repository for download.

Platform options:

A preliminary effort has produced 'Aladdin' which is a basic GUI built in
Matlab, currently running only with the old non-XML configuration of
GENIE. This was intended for release within the GENIELab Matlab Toolbox
(http://source.ggy.bris.ac.uk/wiki/GENIE_GENIELab) - which includes
Toolboxes for running GENIE ensemble experiments on the National Grid
Service facilities, and integrates the OMII-supported GEODISE toolbox for
optimisation and tuning. For this reason, the Matlab approach is
attractive as Aladdin or its decendent will integrate with the
Grid-Enables aspect of GENIE. Although Matlab is closed-source, commercial
software, the Toolboxes can be open-source (i think) and freely available
and can be run as an executables using freely available Matlab libraries
(Sudipta - is this correct?) for those without access to the full
software. GENIELab has been realised through the hard work of Andy Prcie
at Soton, and Aladdin through the similarly heroic efforts of Sudipta
Goswami at UEA. Sudipta is probably best placed to start a step-by-step
plan of how the existing Aladdin might be developed into a beta release in
6 months time - what work needs doing etc.

Alternatively a platform-independent Java GUI is a possible option or
possibly a web-based one (which would have to run locally as well as on
remote servers, so would need to ship with its own web-server). My feeling
is that moving away from the existing base of work with Matlab is a 'phase
2' goal and that we should build on the work Sudipta has already done for
now, although this does depend on the specific requirements of the ENGAGE
funding wrt open-source / commercial software and also the manpower
available - Steve to confirm.

General comments

This is all very ambitios for what would most likely be a 6 month project,
and I'm sure it will need simplifying somewhat. However this depends a bit
on how / if we can split the work up into discrete chunks which can be
tackled at different institutions. Someone will of course have to perform
a technical project management / overview role and this might be best
acheived by Gethin if he's happy with that.

BFG2 integration is valuable because other models and EMIC components are
being wrapped in a compatible way, which would mean GENIE could integrate
e.g. Hadley CENTRE atmosphere models for 'free'. This would mean that the
GUI would have to be extra-clever to cope with reading the information for
a newly acquired BFG-wrapped model to get the parameters which can be
configured, then providing options to configure them. Not impossible, as
BFG presumably needs to know about all of these anyway. Such non core
GENIE modules would raise broader problems as well, such as integration
with XML configs so this is a future goal which would not be realised
within the short timeframe of this possible funding.

Most of the people currently involved have limited flexibility in their
workloads and are fully funded over the coming months. However, it may be
that they can juggle contracts around to fit some of this work in -
people, can you confirm what availability you could muster between 1/12/08
and 15/05/09? At most I'd guess we could possibly get 1 full-time
persons's worth of time out of the recipients of this email over the
fundable period. Steve mentioned that OMII have a good track record in
'matchmaking' projects with people with the necessary skills so we might
be able to bring some fresh blood in. Maybe Manchester have someone
looking for a job with them who might work well on the BFG side of GUI
intergration and module-wrapping?

In terms of short timescales vs lots of people and communications
problems, the current GENIE team have existing strong working
relationships and this would greatly strengthen the likely success of the
project.

Apologies all for the rather rambling nature of all this. Getting it
bashed out so that some discussion can get underway seemed important. I
feel a bit like I've just volunteered you all for something without asking
you first, so if you have better things to do with your lives / jobs over
the next few months don't be afraid to decline this possibility.

Look forward to the discussion getting started

Best regards to all

Martin

← Older revision		Revision as of 14:11, 20 January 2009
Line 1:		Line 1:
−	~~For those of you not involved in previous discussions - OMII and the~~
−	~~ENGAGE project may fund the GUI development part of our earlier proposal~~
−	~~to OMII to request support for open-sourcing GENIE. Some other aspects of~~
−	~~this may also go ahead with seperate OMII funding, but the ENGAGE project~~
−	~~is coming to it's end so getting a proposal together for the GUI work is~~
−	~~the most urgent part. Steve is Steve Brewer, who I expect most of you~~
−	~~know. He suggested gettign the ball rolling in the most expedient way~~
−	~~possible on exploring possible funding options - in an informal email~~
−	~~discussion. My slightly incoherent ramblings start here:~~

−	~~Dear Steve,~~
−
−	~~Thanks for an informative discussion on the phone earlier. I'm copying in~~
−	~~the people who have been party to the last few emails, in the hope that~~
−	~~they might chime in with useful info whenever they can, also Rupert Ford~~
−	~~and Graham Riley from Manchester who have been involved in BFG2~~
−	~~(http://intranet.cs.man.ac.uk/cnc/projects/bfg.php,~~
−	~~http://source.ggy.bris.ac.uk/wiki/GENIE_BFG) work with GENIE, which may~~
−	~~integrate nicely as part of the work we put into the proposal. Have also~~
−	~~copied in Andrew Price (SOTON e-science) who is a long-standing GENIE~~
−	~~collaborator and whose input and funded involvement we would also welcome.~~
−
−	~~GENIE (Grid-ENabled Integrated Earth-system model) is a modular, flexible~~
−	~~Earth-system model of intermediate complexity (EMIC), which is used~~
−	~~extensively within the UK and internationally in Earth-system science. One~~
−	~~important aspect of GENIE which makes it unique (or nearly so?) amongst~~
−	~~EMICS is the modularity which currently allows experienced users to plug~~
−	~~in new or different models of a particular Earth-system component or in~~
−	~~theory even to use a different 'glue' code for sticking all the components~~
−	~~together.~~
−
−	~~To enable less experienced users to access these aspects of the model, and~~
−	~~also to facilitate ease of running the model in its 'standard'~~
−	~~configuration, a user interface would be extremely valuable. As well as~~
−	~~saving end-user time in manipulating XML config files and running shell~~
−	~~scripts, for example, it will also broaden the community of researchers~~
−	~~able to use GENIE to answer their research questions as less technical~~
−	~~computing knowledge will be necessary. The facility to increase or~~
−	~~decrease the complexity of options available in the GUI to produce~~
−	~~teaching, or targeted research versions would aldo be valuable.~~
−
−	~~Steve - you suggested that the best way to go about formulating a good~~
−	~~proposal for ENGAGE was to have an 'ideal world' goal for Mid-May '09 -~~
−	~~the deadline for completion of work funded under the ENGAGE project - from~~
−	~~which we can work back to person-months, staff availability etc. SO here~~
−	~~goes!~~
−
−	~~I envisage a beta release of a 'scaleable' GUI, which can be easily~~
−	~~configured to work at any required level of complexity with the underlying~~
−	~~code (so for example, parameters would be at their default values if not~~
−	~~selected for control by the GUI). The GUI would also allow configuration~~
−	~~and selection/deselection of a series of currently available modules~~
−	~~(different atmospheres, land and ocean models for example) and the ability~~
−	~~for the model to run with the current standard 'GENIE-main' model glue but~~
−	~~also with the BFG2 option. This would require effort from the Manchester~~
−	~~team and would realise excellent the preliminary work already undertaken~~
−	~~by their group. Part of this would be to BFG-wrap a number (1 or 2?) of~~
−	~~modules not currently available to BFG-GENIE - although it would not be~~
−	~~possible to comprehensively cover all current (non-BFG)GENIE-available~~
−	~~models due to time constraints.~~
−
−	~~We would need to ensure that this GUI was future-proof - we already have a~~
−	~~coherent XML configuration system, so it simply has to be able to~~
−	~~dynamically expand to cope with more modules and parameter options. New~~
−	~~model glue codes and approaches would presumably require new GUI code~~
−	~~unavoidably.~~
−
−	~~The GUI would be released along with the GENIE source and presumably be~~
−	~~included in the GENIE subversion repository for download.~~
−
−	~~Platform options:~~
−
−	~~A preliminary effort has produced 'Aladdin' which is a basic GUI built in~~
−	~~Matlab, currently running only with the old non-XML configuration of~~
−	~~GENIE. This was intended for release within the GENIELab Matlab Toolbox~~
−	~~(http://source.ggy.bris.ac.uk/wiki/GENIE_GENIELab) - which includes~~
−	~~Toolboxes for running GENIE ensemble experiments on the National Grid~~
−	~~Service facilities, and integrates the OMII-supported GEODISE toolbox for~~
−	~~optimisation and tuning. For this reason, the Matlab approach is~~
−	~~attractive as Aladdin or its decendent will integrate with the~~
−	~~Grid-Enables aspect of GENIE. Although Matlab is closed-source, commercial~~
−	~~software, the Toolboxes can be open-source (i think) and freely available~~
−	~~and can be run as an executables using freely available Matlab libraries~~
−	~~(Sudipta - is this correct?) for those without access to the full~~
−	~~software. GENIELab has been realised through the hard work of Andy Prcie~~
−	~~at Soton, and Aladdin through the similarly heroic efforts of Sudipta~~
−	~~Goswami at UEA. Sudipta is probably best placed to start a step-by-step~~
−	~~plan of how the existing Aladdin might be developed into a beta release in~~
−	~~6 months time - what work needs doing etc.~~
−
−	~~Alternatively a platform-independent Java GUI is a possible option or~~
−	~~possibly a web-based one (which would have to run locally as well as on~~
−	~~remote servers, so would need to ship with its own web-server). My feeling~~
−	~~is that moving away from the existing base of work with Matlab is a 'phase~~
−	~~2' goal and that we should build on the work Sudipta has already done for~~
−	~~now, although this does depend on the specific requirements of the ENGAGE~~
−	~~funding wrt open-source / commercial software and also the manpower~~
−	~~available - Steve to confirm.~~
−
−	~~General comments~~
−
−	~~This is all very ambitios for what would most likely be a 6 month project,~~
−	~~and I'm sure it will need simplifying somewhat. However this depends a bit~~
−	~~on how / if we can split the work up into discrete chunks which can be~~
−	~~tackled at different institutions. Someone will of course have to perform~~
−	~~a technical project management / overview role and this might be best~~
−	~~acheived by Gethin if he's happy with that.~~
−
−	~~BFG2 integration is valuable because other models and EMIC components are~~
−	~~being wrapped in a compatible way, which would mean GENIE could integrate~~
−	~~e.g. Hadley CENTRE atmosphere models for 'free'. This would mean that the~~
−	~~GUI would have to be extra-clever to cope with reading the information for~~
−	~~a newly acquired BFG-wrapped model to get the parameters which can be~~
−	~~configured, then providing options to configure them. Not impossible, as~~
−	~~BFG presumably needs to know about all of these anyway. Such non core~~
−	~~GENIE modules would raise broader problems as well, such as integration~~
−	~~with XML configs so this is a future goal which would not be realised~~
−	~~within the short timeframe of this possible funding.~~
−
−	~~Most of the people currently involved have limited flexibility in their~~
−	~~workloads and are fully funded over the coming months. However, it may be~~
−	~~that they can juggle contracts around to fit some of this work in -~~
−	~~people, can you confirm what availability you could muster between 1/12/08~~
−	~~and 15/05/09? At most I'd guess we could possibly get 1 full-time~~
−	~~persons's worth of time out of the recipients of this email over the~~
−	~~fundable period. Steve mentioned that OMII have a good track record in~~
−	~~'matchmaking' projects with people with the necessary skills so we might~~
−	~~be able to bring some fresh blood in. Maybe Manchester have someone~~
−	~~looking for a job with them who might work well on the BFG side of GUI~~
−	~~intergration and module-wrapping?~~
−
−	~~In terms of short timescales vs lots of people and communications~~
−	~~problems, the current GENIE team have existing strong working~~
−	~~relationships and this would greatly strengthen the likely success of the~~
−	~~project.~~
−
−	~~Apologies all for the rather rambling nature of all this. Getting it~~
−	~~bashed out so that some discussion can get underway seemed important. I~~
−	~~feel a bit like I've just volunteered you all for something without asking~~
−	~~you first, so if you have better things to do with your lives / jobs over~~
−	~~the next few months don't be afraid to decline this possibility.~~
−
−	~~Look forward to the discussion getting started~~
−
−	~~Best regards to all~~
−
−	~~Martin~~