unloading textures

edited April 2011 in Help request
hey
I'm running into some memory problems when running my game on the ipad.

I've now profiled the game inside the simulator. there the memory problems will of course not occur.

one thing I noticed is that the used memory will always grow, but never decrease.
it's like this:

in the game menu: 30MB
inside a level: 200 MB
back in the game menu: 200MB

it's not a leak. after starting as this amount will not increase after starting the level again.
it'S just like the textures are not unloaded. so if I start different levels they will all contribute to the overall memory consumption, even though they might not be active.

is this intended?
if so is there a way to clear the currently unused textures?

I know that I have to do something against my memory consumption generally ;)
currently I'm packaging all the textures of the animations into huge power-of-two textures. however I guess this will not be enough. I already though about extending orx with support of compressed textures.

Comments

  • edited April 2011
    did you clear all configs that not use in the level?
    and are you sure you delete the object in the game when switching to the game menu?
    as far as I know, the textures are categoried by its filename and its deleting is based on the reference counter in the structure.
  • edited April 2011
    As laschweinski pointed out, textures will be released as soon as nothing reference them anymore (usually you need to make sure you deleted all the objects using those textures).
    You can go through all the textures using the orxStructure_GetFirst/GetNext accessor and check their reference counter.
    You should never try to delete something that you didn't explicitly create as its legal owner will probably call the delete itself at some point and you'll crash/assert.
    You can also parse all the objects to see what graphic/texture they're using and verify you don't keep objects alive that you thought were destroyed.

    As for compressed texture, it's something I'd like to add for iDevice plugins at some point but I never had the need so far as I've never used more than 14MB of video memory at once on those devices.
  • edited April 2011
    I tested my game in linux today, unfortunatelly, I also encountered tdomhan's problems
    I modify orx lib code to print all deleting info, I found that one of bitmaps which is a role bodies' atlas picture won't be deleted.
    in debug mode, there is a warning said that:
    "Animset is locked, can't remove all anims."
    the animset is locked therefore the pictures contain the anim textures. how to unlock it?

    then I remove all animation. and test it again.
    Still, the memory of the process in the system monitor won't go down, it seems that some resources still won't be removed from memory.

    I am sure that I delete all config section since I get the sectionCount is 1

    Thanks for your help.
  • edited April 2011
    yeah I'm as well pretty sure that everything is deleted.

    in orxDisplay.m I changed the creation of the Image to
    [none[UIImage alloc] initWithData:poTexData]
    
    as the other initializer will do some caching itself. which is unnecessary as orx will do some caching(or rather reuse textures) anyway.

    that said it doesn't really change the behavior in a noticable way(memory wise). but as those problems are also present on the linux platform the cause obviously lies somewhere else.

    I just ran the app again on the iPad and I get the same message that complains about locked animsets.
  • edited April 2011
    from menu to game it increase 5MB
    while game to menu no any difference in memory at most time
    then go to game again, sometimes, it increase 0.2MB
    the game scene it totally the same.

    that's my complement of this issue.
    or is it related to dlmalloc's running strategy?
    dlmalloc lib request the memory but won't be released. then it remove them from system afterwards at some right time.
    therefore memory shows in the monitor won't decrease.
  • edited April 2011
    Ah yes, animset. I'll check why it doesn't get unloaded properly then. Do any of you have a test project I could use for debugging as I'm not on my computer and have no other sources than orx itself?

    I'm pretty sure we can rule out dlmalloc as being the source of the problem.

    I didn't know for UIImage, thanks for pointing it out. Can you send me a diff of your changes?
  • edited April 2011
    i would send you the code tomorrow ,but do not show them to others now for uiimage just replace it by the code he highlight
  • edited April 2011
    No worries, I keep code/data that is sent to me private unless stated otherwise.
    If you feel more comfortable I can sign a NDA too.

    I guess a release is need somewhere for the UIImage isn't it? I don't have access to a mac right now so I'd feel more comfortable with a diff so as to not break the plugin compiling. :)
  • edited April 2011
    with my method there is a release needed. I would send you a diff, but right now I'm still in the process of modifying orxDisplay.m and I don't want to send you a messed up version ;)
    I'm implementing compressed textures btw
    it's already working, beside the resulting texture being flipped vertically :D
  • edited April 2011
    Oh great, two things at once! I'll merge it when you're done, no hurry.

    Welcome to the fabulous world of upside-down coordinates. ;)
    The most fun I had was when I was grabbing the screen content and was trying to reapply it via a shader as I would with a regular texture. I had to rewrite all coordinate-related parts in the display plugins at that time as I previously took the short path of using an upside-down viewport. Bad bad choice. ^^
  • edited April 2011
    the PVRTexTool by imagination technologies will by default already flip the compressed texture vertically :/

    lesson learned so far: compressed textures are awesome in regard to the memory consumption, but are pretty much useless for animations. the artifcats will cause some really irritating flickering. however they seem to be great for static images.

    now I'm now considering to load my textures as GL_UNSIGNED_SHORT_4_4_4_4. couldn't tell any difference after converting some pictures and it would save me halve the memory.
  • edited April 2011
    Sounds good for all those format supports. Are you doing it for iDevices only or also for the other plugins (well, not the PVRT, but the other formats)?

    Also, I did a blind try at fixing the animset issue. I haven't tried it at all but I believe in my lucky star. ;) Let me know if it helped with your problem or not.
  • edited April 2011
    it is ok for animset now
    all bitmap have been removed.
    but the memory still could not decrease
    the similar problem also occur in windows.
    at the beginning the size is 16Mb
    after enter the game it increase to 19MB
    then I switch the view again and again the size of memory almost remains at 19MB, every time it slightly increase 0.1.
  • edited April 2011
    Mmh, I'll check to see if I can find something, but it might have to wait for me to come back home.
    Also, can you try not using dlmalloc in the orxMemory.c file and see what happens with the default memory allocator, just to be sure?
  • edited April 2011
    I guess we really need that profiling module soon then. I'll try to see if I can do anything about it soon. I'll keep you informed.
  • edited April 2011
    During my shower I got a revelation! ^^

    Those memory changes you notice (going up but stabilizing after a while) comes from the use of orxBANKs.
    An orxBANK will allocate memory chunks when it's out of free slots and will never free them but will instead reuse the free slots when needed, becoming then very efficient as no memory allocation will actually take place.

    I'll add a function to compact memory banks to your convenience that you can call between levels when you want to claim all the previously used memory. I could do it automatically but this "garbage collection" might impact performances at the wrong time, whereas if you call for the collection manualy that shouldn't be a problem. What do you think?
  • edited April 2011
    I did some profiling on my testbed (orxBounce, included with orx's source).
    When simply having the basic particle spawner and the four walls, orx internally holds on ~361KB of allocated memory in ~300 distincts allocations. The total memory used, including orx and its external dependencies is ~40MB.
    After spawning physical objects for about a minute (some sprite based, some text based) with a max of 500 physical objects at once, the memory peaks (and remains at):
    - orx: ~700KB in ~360 distinct allocations
    - total, including dependencies: 44MB

    After collecting unused bank segments with orxBank_CompactAll(), we go down to:
    - orx: ~361KB in ~300 distincts allocations
    - total, including dependencies: still 44MB

    I guess the next step will be to try to monitor what's so expensive in the dependencies.
    The total video memory used for the textures, including power of two rounding if/when needed, is exactly 156KB + 128KB for the internal font.
    We're still under 1MB in peak which means that the OS needs + external dependencies still account for over 40 MB (ie. 40 times more than what orx internally uses in this case).

    External dependencies are: GLFW, Box2d, SOIL, OpenAL-soft, libsndfile, stb_vorbis. This test isn't using any audio resources so I expect the corresponding external dependencies to have pretty low memory consumption. My guess is that on alternative platforms/plugins, such as iDevices, results might be pretty different but I don't have the required hardware to try it right now.

    Any thoughts?
  • edited April 2011
    how do you use the orxBand_CompactAll I tried to call it in the end of switching the scene. But it seems no effect.
    the memory still remain no change.
  • edited April 2011
    Banks will be freed but that won't change your memory use as you can see in my previous post.

    Two reasons for that: dlmalloc won't give the memory back to the OS by itself unless someone calls dlmalloc_trim() and secondly and most importantly it's not orx that uses the most memory but the external dependencies. I unfortunately have no control on how they allocate/free their memory. Box2D is using a high watermark policy, which means it'll allocate more memory if needed but will never free it (as stated by its author, Erin Catto, on Box2D forum). I don't know for the other dependencies.

    If you have any idea, please let me know.
  • edited May 2011
    a high watermark policy is fine with me. I just feared that some graphics would still be using memory, even though they were not used anymore.
    but if the memory is going to be reused for other graphics it's totally fine with me.
  • edited May 2011
    As far as texture go, the memory gets freed at the dlmalloc level (and I can force it to release them back to the OS).

    What worries me is all the memory that is not directly allocated by orx. I'll try to see if I can modify the dependencies to hook their allocation back to orx so as to monitor them more closely.
    Not sure when I can do that though...
  • edited May 2011
    I forgot that I wanted to sent you the patch for implementing the compressed textures.
    attached you will find the first part. I don't you if it fits your style of code.
    your code is always so clean etc, that don't want to change it and mess it up ;)

    anyway I added some part to check whether the right extension is present or not. then I check for the file ending, if it ends in PVR i will load the specific compressed texture. If compressed textures are not supported it will try to load texture that has the same file name but ends in PNG. For loading the texture I added the PVRTexture class that can be found in the sample project from apple. I just modified it slightly to support the orx smoothing.

    I hope that my code is not too ugly and this helps somebody else as well.

    I also wrote something about creating the compressed texture here:
    http://orx-project.org/wiki/en/orx/tutorials/community/tdomhan/compressedtextures
  • edited May 2011
    the forum wouldn't let me upload text files. attached you will find a zip containing the patches :D https://forum.orx-project.org/uploads/legacy/fbfiles/files/compressedtexturespatches.zip
  • edited May 2011
    Thanks a lot for the patch!

    Don't worry about the coding style, I can easily adapt it if need be. ;)

    I'll integrate this in the trunk either tonight or during the week end and it'll be part of the next release candidate.

    Thanks again!

    PS: I read your wiki post about it but I think the first sentence is lacking a word or two. :)
  • edited May 2011
    hey,
    I just wanted to give a heads-up that I will add support for PVR textures in ARGB_4444 format and also support for repeatable textures, if they are not compressed. I guess I can drop the mipmapping support, because it's not relevant for orx, anyway?
  • edited May 2011
    Hi!

    Sounds like a plan. I wanted to support different pixel format on computer as well but I wasn't sure how to do the interface part, especially for changing the texture values on the fly.

    As for repeatable textures, how do you see that working, code and config wise?

    And yes, mipmaps are not very relevant for 2D as you said.
  • edited May 2011
    Hi!

    I couldn't apply this patch as there's a version mismatch apparently: svn is trying to fetch a revision 278 (?) from the trunk before patching and if I use the latest from trunk it tells me the lines don't match. Would you mind sending me another patch for that?

    I updated the pre-multiplied issue but I wrote the code directly (same version issue with the patch). I did it from windows so it might not work/compile on iPhone. If that's the case please let me know but I should be able to test myself on iPhone before the end of the week end!
  • edited June 2011
    mhh yeah, I forgot that I have the orx sources in my own SVN repo.

    I will do some more code cleanup and testing and then upload it to trunk directly if that's ok.
  • edited June 2011
    Normally that'd be fine but the trunk is currently in release candidate state till the final release of 1.3.
    So if you don't mind I'd rather review the code before it gets there. I'm using a branch for new dev on the anim system myself. :)
    If you send me a diff from made from the latest version on the trunk I should be able to merge it with no difficulties.
  • edited June 2011
    alright, I guess I can create patches that refer to the orx SVN.
    until when would you need them?

    I would create them next week during the holidays, after adding some stuff and cleaning up a bit, if that's ok with you.
  • edited June 2011
    That's totally fine. There's no worries. I didn't get much feedback on the release anyway so it can wait. :)
  • edited June 2011
    I guess many are anyway using the SVN version.
  • edited June 2011
    You'd be surprised, I found out that usually people are reluctant to sync the SVN repository. :)

    Anyway, it's not like orx had hundreds of users...
  • edited June 2011
    Alright attached you'll find a patch for a current SVN version of ORX.
    In the meanwhile I've rewritten the whole code for loading pvr textures.
    I no longer use the PVRTexture class that was written in objective-c and part of an example by apple.
    In this step I also included the whole loading code into orxDisplay.m of the iphone plugin.
    Is there any other platform supporting PVR texutres? If so, it could easily be moved somewhere else. The only iphone specific code is the byte swapping, but I hope it shouldn't be to replace that.
    the rest has been "orxified". meaning I have replace ios/objective-c API stuff with calls to the orx API. (e.g. using orxFile)

    The code also performs a check whether the require opengl extension is present. if not it will fallback to a file named the same, but ending in "png".

    it get's a bit trickey if you want to support multiple platforms with the same config files, while using PVR on the one and PNG on the others. In this case you need different config files. (Or the more hackish version: changing the loading code on one architecture to rename the incoming file name)

    Beside the support for normal compressed architecutres I also added support for ARGB4444, ARGB1555 and some others. The reason for that is that the PVR compression is not very suitable for comic style textures, because of the artifacts it causes. However reducing the bit depth is quite a good method to save memory. Furthermore even if using ARGB8888 the loading time should be reduced, as the raw image data can be transfered to the video memory rather then first decoding/uncompressing a like, as it is in the case of PNGs.


    so long,

    Tobi https://forum.orx-project.org/uploads/legacy/fbfiles/files/orxDisplay.gz
  • edited June 2011
    Thanks for all this, Tobi! :)

    I'll try to do the integration as soon as possible. I don't know if PVR is used on other platforms, but if you're looking for the OGL extension before using it, I might as well add it to the other plugins.

    Lastly, I'm ok with doing on-the-fly file renaming to try loading different textures. That's what happens with plugins with the whole debug/release thing. That sounds like an easy way to support different formats on different platforms. Maybe the suffix could be changed in config to give flexibility to the user? But I'm not sure this is really needed.
  • edited June 2011
    Didn't get much coding time last week end, sorry. I'll try to integrate it next week end! :)
  • edited June 2011
    I'm in the process of merging your code but the patch only contains changes for orxDiplay.m, no other files. As it looks like you're adding stuff to the orxView class, my guess is that I'm missing at least one file in the patch. :)
  • edited June 2011
    Ok, the code's in and I assumed the only missing part was the boolean for compressed texture support, from the orxView interface.

    I haven't tried the code myself, so if you find any mistake I made, please feel free to fix it directly on the svn. :)
  • edited June 2011
    Last question: I've noticed you don't do any vertical flip nor alpha-demultiplication when loading a PVR texture, is that normal (ie. will it be consistent with regular textures)?
  • edited June 2011
    ok nice :D

    when "orxifying" the pvr loading code I incoporated everything in that one file, so that should be fine. (so a png and a pvr created with pvrtextool can be used interchangably)


    I don't do any alpha demultiplication because it's never premultiplicated in the first place.

    I don't do any y-flipping, because the PVRTexTool, that creates the pvr files, will by default store y-flipped images.
  • edited June 2011
    tdomhan wrote:
    I don't do any alpha demultiplication because it's never premultiplicated in the first place.

    Ah yep, going directly through OpenGL ES calls.
    I don't do any y-flipping, because the PVRTexTool, that creates the pvr files, will by default store y-flipped images.

    Good to know. Is there any other way to produce PVR images that wouldn't do the Y-flipping? If so, a note on the wiki might be helpful.

    Let me know if I made some mistakes when integrating your code. :)
  • edited June 2011
    I mentioned something in the wiki. Also added some note to the PVRTexTool now.

    although I'm anyway wondering if anybody finds this article in the wiki ^^
    maybe we should include the (useful) arcitles of others somehow in the whole wiki structure.(most of mine are just notes to myself :/ )

    I don't know when I'll find some time for integrating the new version of orx. but I'll let you know if I encounter any bugs after integrating.
Sign In or Register to comment.