Wintermute Engine Forum

Wintermute Engine => Technical forum => Topic started by: Art on January 10, 2006, 01:23:57 PM

Title: Saving and loading takes too looooong
Post by: Art on January 10, 2006, 01:23:57 PM
Hi all. I'm preparing release of a game. And i've got a problem. After going through the game, saving and loading takes more and more time. For example one of our testers says that it takes 19!!! minutes to save and 4!! to load.
This happens after you get 6th level. If i force game to start from 6th level at once, saving and loading is ok.
What can it be? May be with time engine saves to much information? If so is there any way to reset (forget) games states when new level is started ???
Any ideas ? ???
Title: Re: Saving and loading takes too looooong
Post by: Mnemonic on January 10, 2006, 03:19:22 PM
19 minutes is definitely not normal. In the past similar problems were tracked down to either a script error or a problems with the internal Array class. Are you using the Array class? Or some frequently used for() loop?
Title: Re: Saving and loading takes too looooong
Post by: Art on January 10, 2006, 03:32:27 PM
Yes I do both :(
Title: Re: Saving and loading takes too looooong
Post by: Mnemonic on January 10, 2006, 03:41:01 PM
1) If you are using the WeatherGod script, by any chance, make sure you're using the latest version. The original version was causing this kind of problem.
2) If you are using Array, try to eliminate it. It shouldn't be a big deal, since the variables work as arrays anyway. You will only need to track the items count in a separate variable.
Title: Re: Saving and loading takes too looooong
Post by: Art on January 12, 2006, 04:40:40 PM
1. I do not use this script. And what is this? Anything interesting and usefull? ;)
2. I really use arrays, and it's important for me to use them. It would be very difficult to program my current logic without them, couse all my dialogs are based on array of strings. At this moment i've implemented my own save and loading functions. Saving occurs automatically in the end of each level, and loading just moves to the first scene of the level user wishes to load. So states are not saved.
Title: Re: Saving and loading takes too looooong
Post by: adonf on April 23, 2007, 03:43:58 PM
Hi

We're having the same problem, except that saving "only" takes 7 minutes, but it could be because we have more powerful PCs nowadays ;)

We dont' use the Array class or the WeatherGod script. It takes longer and longer to save as the player progresses in the game, this 7 mn figure I gave was after playing though 30 scenes or so. The first half of the saving bar goes very fast, and the last half takes most of those 7 minutes (divided into 3 steps)

There are two reasons I can think of :

Maybe we have too many game states in script that are never freed (all our states are grouped as member variables of several global variables, these members are initialized only when we need to so their number goes up as the player progresses)
Or maybe we shouldn't save every entity's state automatically (in Scene Edit we check "Remember nodes states" and "Including sprites" in all scenes) but save only the states that we need in script variables.

I'd like to have your opinion if we're on the right track before we start making major changes to our scripts.

Thanks,
 Olivier
Title: Re: Saving and loading takes too looooong
Post by: metamorphium on April 23, 2007, 08:45:25 PM
don't you have any recurring object definition without disposing of them?


Title: Re: Saving and loading takes too looooong
Post by: Mnemonic on April 24, 2007, 07:00:55 AM
I don't think variables or scene states can normally cause this. It must be some major leak. One user encountered the problem after placing something like:

global SomeWindow = Game.LoadWindow("blah");

to base.inc. So basically a new window got loaded whenver some script got executed. Check if you don't have some similar pattern in your scripts. Secondly, it's possible you hit some undiscovered leak in WME. The problem most likely occurs in some periodically executed script (like the game_daemon.script in WME demo). Do you have many such scripts running? You could try disabling some of them to find which one is responsible for the leakage.
Title: Re: Saving and loading takes too looooong
Post by: adonf on April 24, 2007, 10:37:18 AM
Oh wait, we are using Array.  :-[

Does this still create leaks ?
Title: Re: Saving and loading takes too looooong
Post by: Mnemonic on April 24, 2007, 05:00:06 PM
Unfortunately I wasn't able to sufficiently replicate the issue so I'm afraid there's still some trouble buried inside the array class..
Title: Re: Saving and loading takes too looooong
Post by: adonf on April 24, 2007, 05:42:17 PM
Thanks to your suggestions we managed to bring the save time from 7 minutes down to 3 or 40 seconds (depending on whether we saved before or after playing a given minigame): we were creating a window on each line of speech and never released it, that was easy to solve once we knew where to look.

The 40 seconds saving time was a little more tricky: some scene entities are created dynamically in this minigame. Originally we used Scene.CreateEntity, when we switched those calles to Layer.AddEntity the saving time went down to 20 seconds

And then we managed to go down to 4 seconds after the minigame by replacings calls like this:
Code: [Select]
layer.AddEntity(ar[k] + "node" + nodeindex);to this
Code: [Select]
var name=ar[k] + "node" + nodeindex
layer.AddEntity(name);
where ar is an array of Strings created by the String.Split() function.

I'm giving you all these details not to show off but to let you know about possible leaks in the scripting system.

Now we're very satisfied with our 150 times improvement in saving time.
Thanks again for your help.

Olivier
Title: Re: Saving and loading takes too looooong
Post by: Mnemonic on April 25, 2007, 09:55:56 AM
Ok, thank you for the info, adonf. And I'm glad you were able to figure out the primary source of the troubles.
Title: Re: Saving and loading takes too looooong
Post by: Mnemonic on April 25, 2007, 02:22:55 PM
Perhaps the debugging console might display all currently loaded windows/entities/actors so that the developers would notice they're loading stuff and not unloading it...
Title: Re: Saving and loading takes too looooong
Post by: metamorphium on April 25, 2007, 06:03:36 PM
that would be good indeed!  ::rock
Title: Re: Saving and loading takes too looooong
Post by: adonf on April 26, 2007, 09:30:27 AM
Yes, that would be very useful.

By the way, we found more problems since my previous, optimistic post. we'll be doing some tests today but I suspect that dynamic creation of entities is causing some problems.
Title: Re: Saving and loading takes too looooong
Post by: adonf on April 26, 2007, 11:37:00 AM
So we wrote some code that increases the saving time from a handful of seconds to one minute after it's run:
(scroll down, the interesting part is at the end)

Code: [Select]
var rand;
var ar = new Array();
// initialize string randomly
for (var k = 0; k <= 20; k = k + 1)
{
            rand = Random(1,3);     
            switch(rand)
            {
                        case 1:
                                    ar[k] = "G/500";
                                    break;
                        case 2:
                                    ar[k] = "B/1000";
                                    break;
                        case 3:
                                    ar[k] = "R/100";
                                    break;             
            }
}
 
// Change i to change the save time
// i = 500  --> 1mn
// i = 1000  --> 3mn
for (var i = 1; i <= 500; i = i + 1)
{
            for (var j = 0; j < ar.Length; j = j + 1)
            {
                        var ch = new String(ar[j]);
                        var tab = ch.Split("/");
            }
}
Game.Msg("GO SAVE!!!");
Game.LOG("FINISH!!");

If we comment out the lines with new String and Split the saving time does not change.

We're going to work around these functions but I hope this can help you track these leaks. Oh and we did the same kinda thing with dynamic entities and it didn't change a thing.

Cheers,
Olivier
Title: Re: Saving and loading takes too looooong
Post by: Mnemonic on April 26, 2007, 12:27:29 PM
Thanks! I'll look into it.
It should be easy to work around by not using array, but a simple object variable and track the length separately, but I'm sure you know that.
Title: Re: Saving and loading takes too looooong
Post by: Mnemonic on April 28, 2007, 05:00:09 PM
All right, I plumbed the leak (it was actually in the String.Split() method). The fix is available in the latest WME build (http://dead-code.org/download/download_latest.php).

Thanks again for providing the repro code.
Title: Re: Saving and loading takes too looooong
Post by: adonf on May 02, 2007, 10:03:21 AM
and thanks to you for fixing it  ;D