[esp-r] Re: Results Module, pre-scanning results databases

[Jon Hand]

The speed of results recovery is a function of the available memory in the computer
and how 'fresh' the results file is.  If a simulation has just been run and there is more
memory available than the size of the results file then the scanning task will
use the data in the memory buffer rather than going to disk.

I have worked with results files ~2GB with a machine with 3GB of memory and
the pre-scan might take 5 minutes but if it the results file was from an earlier
session the time grew.

I have code that detects likely long pre-scans and skips the pre-scan but that
has yet to get into the development_branch.  

For now the easiest fix is to ensure that you have a significant block of
free memory when you start simulation / results recovery tasks.

-Jon Hand
________________________________________
From: esp-r-bounces at lists.strath.ac.uk [esp-r-bounces at lists.strath.ac.uk] On Behalf Of Achim Geissler [achim.geissler at intergga.ch]
Sent: 23 May 2012 20:54
To: Justin DeBlois
Cc: esp-r at lists.strath.ac.uk
Subject: [esp-r] Re: Results Module, pre-scanning results databases

Dear Justin

sadly, this is normal behavior for res. Somewhere around 0.5 to 1 GB of results data size it becomes very slow. Possible workarounds are using the IPV feature and not doing full year simulations but only "typical period" simulations. Also, alternatively, you could try the xml output - run time may increase, but results are dumped in a text file (drawback is that you need to pre-define which results you want or redo the calculation).

Best
Achim



On May 23, 2012, at 5:28 PM, Justin DeBlois wrote:

I am working with fairly large results sets, and it has become very time consuming to open them in the results module.  It takes several hours to get through the "pre-scanning database" step, but the res process is only using about 1% of the CPU and 15MB ram.  I have tried with the Linux version in Cygwin and Ubuntu.  Is this normal, and is there any way to speed it up?  Thank you,

Justin

--
Justin DeBlois
Mechanical Engineering
University of Pittsburgh

_______________________________________________
esp-r mailing list
esp-r at lists.strath.ac.uk<mailto:esp-r at lists.strath.ac.uk>
http://lists.strath.ac.uk/mailman/listinfo/esp-r

achim.geissler at intergga.ch<mailto:achim.geissler at intergga.ch>