Zonja Capalini

Splitting Megaregions

One of the most interesting developments in Opensim lately have been megaregions. Megaregions are non-standard “big” regions created by combining several standard, 256×256 meters regions. For example, you can combine four standard regions in a square to get a megaregion of 512×512 m. Megaregions are very convenient because of several reasons: for example the usual lag in region crossings disappears completely (since you are in the “same” region), and the same is true of vehicles. On the other hand, megaregion support is still experimental, and a number of things that work in normal regions don’t work or work only partially in megaregions. For example, you cannot define parcels outside of the SW region (this is due to the way parcel maps work), and therefore you can’t have independent music streams (for example) when x > 256 or y > 256.

Diva Canto’s distribution of Opensim directly supports megaregions. You can also migrate an existing non-megaregion installation to a megaregion configuration, if you follow certain precautions. Migrating from a megaregion configuration to a non-megaregion configuration is also possible — you change the setting that implements megaregions, “CombineContiguousRegions” to false, and it works as expected.

OAR support for megaregions is partial: since technically all the objects in the megaregion are stored in the SW region, when you take an OAR of the SW region you get all the objects of the megaregion but only the terrain for that same region — this means that you cannot fully rebuild a megaregion from an OAR, which is a pity (you will be missing the terrain files for all regions except the SW one).

Splitting megaregions

In this post I’ll describe some experiments I made with an OAR manipulation tool I developped following a suggestion of Stefan Andersson. The tool takes as its input a megaregion OAR and splits it into several non-megaregion OARs, creating one OAR for each region that constitutes the megaregion and has objects in it. This is so because megaregions may have holes, and when a possible region does not have objects there’s simply no information in the megaregion OAR about whether a it’s a region with no objects (for example, a sea-only region) or a hole in the megaregion.

The tool

The tool works in a very simple way: the OAR contents is first optionally extracted, and then it iterates over each object in the megaregion, extracting the x and y coordinates from the XML2 file and calculating the position of the corresponding region in the grid (relative to the SW region). Then a copy of the object with new non-megaregion coordinates is written to a temporary directory called ‘objects.i.j’, where ‘i’ and ‘j’ are the grid coordinates offset + 1 (for example, the SW region will have i = j = 1, the region to its east i = 2, j = 1, the region to its north i = 1, j = 2, etc). Finally, each of these directories is temporarily renamed to ‘objects’ (the standard directory name for objects in a OAR), and the ‘tar’ command is invoked to produce a file called fname.i.j, where fname is the name of the original OAR file, or ‘region’ if this name is unknown.

For example, using the tool against a 2×2 megaregion OAR called ‘test.oar’ will produce 4 OARs: test.1.1.oar (the SW region), test.2.1.oar (the SE region), test.1.2.oar (the NW region) and test.2.2.oar (the NE region). The current version of the tool has two problems: the first one is that it does not attempt to perform asset splitting, that is, the original megaregion OAR contains the assets (i.e., textures, scripts, object contents, etc) for all the objects in the megaregion, and the splitted OAR files will contain a copy of all the assets in the megaregion (instead of only the assets corresponding to the objects in the splitted region); this can be manually corrected by “normalizing” the OAR file: load it into an empty region and save a new OAR.

The second problem cannot be corrected, and is due to the (already mentioned) fact that megaregions do not contain terrain data for regions different from the SW region. Due to this fact, all the OAR files produced will contain the same terrain file, that is, the terrain file corresponding to the SW region (would it be better if I simply didn’t include any terrain?). This is not so much of a problem as it appears, if we can have access by other means to the missing terrain files: we can manually load the terrain in the target region and later load the oar with the ‘–merge’ option, or we can first load the oar and then the terrain.

How should megaregion terrains be stored in OAR files?

It is very probable that a future version of the OAR file format will include terrain information for all the regions in a megaregion. There are two ways to store such information: one is to store a big r32 file with the terrain data for the whole megaregion, and another one is to store the individual terrain files for each region. Both approaches have advantages and disadvantages: storing the terrain for the whole megaregion is ideal if we want to do some kind of online editing of the terrain (this can also be done if we have the single-region terrain files, but we have to aggregate the terrains first, which is cumbersome); on the other hand, storing the individual terrain files is ideal for megaregion splitting tools like the one presented in this article. The ideal solution would be to include both the megaregion terrain and the individual region terrains, and define semantics for the OAR load process: for example, “use the megaregion terrain if it is present, else use the individual region terrains”.

Implementation and technical details

To save coding time, I’ve implemented the tool as a new command called “SPLIT” in version 0.2 of my ZOE OAR editor. You open a megaregion OAR file with ZOE, type “SPLIT”, and that’s it — in some seconds you have the new OAR files. I’ve also updated ZOE so that it now depends on Cygwin being installed (instead of using 7z like in version 0.1) — this means that ZOE should run unchanged under Linux (but I can’t test that since I don’t have access to a Linux machine). This also means that ZOE should be called using “rexx zoe oarname” instead of “zoe oarname” (Cygwin bash does not understand the latter). To summarize, here’s what you’ll need:

  1. Open Object Rexx.
  2. (Only for Windows:) Cygwin (tested with Cygwin 1.7.1-1).
  3. ZOE version 0.2, which you can download here.

Other minor changes in ZOE 0.2 are that the “zoe ??” and “zoe (c)” special calls have been eliminated (please refer to the documentation for ZOE 0.1 for details).

February 13, 2010 - Posted by | OpenSim, Tech News, ZOE | , , ,

10 Comments »

  1. Hi there. I’m in the midst of trying to revert my current Megaregion sim setup back to individual regions & your ZOE tool sounds like just the ticket! Unfortunately, I can’t seem to get anywhere with using it & am probably going about it incorrectly. I have installed Rexx, Cygwin, and have downloaded ZOE 0.2. I’ve tried opening Rexx and entering rexx zoe myoar.oar & nothing seems to happen. The instructions for ZOE 0.1 make is sound as if it should be a standalone program, yet it does nothing from a standard command prompt, nor does rexx zoe myoar.oar when issued from the cmd prompt. I’m also not sure just when/where to use the SPLIT command, since nothing seems to actually be running. If you could get back to me with a quick “dummies” guide to getting started, I’d really appreciate it. Thanks!!

    Comment by Kenneth Rougeau | September 13, 2011

  2. I’m afraid ZOE is a quite old program, which I do no longer maintain, and I’m not sure it will work with the current versions of OAR files.

    Anyway, if everything is installed properly, when you open a cygwin command prompt rexx should be available; try to type “rexx” at the command prompt, and it should give you an informative message; else something is not installed correctly,

    You may need to put zoe.rex in the same directory as the oar file, and cd to that directory.

    Finally, the “split” command is a subcommand of ZOE — ZOE is a line-mode interactive program that accepts commands by itself. When you get ZOE running, you can type “help” (iirc) and it should offer you a list of the available commands.

    HTH,

    Zonja

    Comment by Zonja Capalini | September 14, 2011

  3. Thanks Zonja,

    I hadn’t realized I needed to be withing Cygwin. Sadly, it leads to the same effect. Issuing the rexx command gives me the programs usage info, but rexx zoe just returns me to the command prompt, as does rexx zoe myfile,oar, or any other combinations I could think of to try. Ah well, it was certainly worth a look :) Thanks again! I suppose I’ll just have to rebuild my world from scratch.

    Have a great night,
    – Ken

    Comment by Kenneth Rougeau | September 14, 2011

  4. To me this sounds like a path problem. Please try to write “rexx \zoe \oarfile”, e.g. “rexx c:\my_oars\zoe c:\my_oars\megaregion.oar” to see if this solves the problem.

    Comment by Zonja Capalini | September 14, 2011

  5. Thanks Zonja, I really appreciate your assistance. Unfortunately, it looks as if it just doesn’t function anymore. No real biggie, it just would have made life a smidgen easier haha.

    I tried executing the command as
    rexx c:\cygwin\home\myname\zoe c:\cygwin\home\myname\myoar.oar
    which resulted in a “program not found” error.

    Since Cygwin uses a Unix style interface, I tried it with forward slashes instead of back, which did not error, but did nothing else either.

    I think I have a non-ZOE workaround figured out & will be experimenting this afternoon. My assumption is that, if I load up the megaregion OAR into the new sim with megaregions enabled, then turn the “combine contiguous regions” setting off, and them individually backup each area to a new oar, that it might work just fine. As it is, i can turn off “combine contiguous regions” and the sim works fine, however a huge amount of “not found” errors appear on the console & are likely not only creating a HUGE log file, lol, but may be an indication that the whole mess is unstable…

    Wish me luck (and thanks again!!!!),
    – Ken

    Comment by Kenneth Rougeau | September 14, 2011

  6. I played with megaregions for my Condensation Land world, and had to revert to classical regions, because of the instability, including tons of error messages in the logfile. But that was a long time ago, maybe now the whole experience is smoother, I really can’t say. I hope your solution works! :-)

    Comment by Zonja Capalini | September 15, 2011

  7. Thanks again Zonja :) I’m actually switching back because the fixed NPC code doesn’t work properly on megaregions (just the root region), and since AI experimentation is my primary reason for fiddling around with the sim, I’ve gotta fix it haha.

    Sadly, my attempted workaround only led to massive errors and eventual stack overflow. I’m poking around in the XML files within the OAR and it looks like I should be able to fix things by manually altering the X &/or Y coordinates, then repackaging related region files into a new OAR, which I think was essentially what your script did.

    Wish I knew why it won’t work for me. I tried it on my Linux server with the same results – no output, just returned to the command prompt. It’s probably an update in Rexx itself that broke it since the OAR file format hasn’t really changed any. I’ll try grabbing an older version and see if I get anywhere.

    Otherwise, I’ll try to wrap my head around the math involved and see if I can’t script something that will fix the coordinates, etc. I’ve got your ZOE script as a basis & I figure the calculations have to be in there :)

    Have a great day & thanks again!
    – Ken

    Comment by Kenneth Rougeau | September 15, 2011

  8. It’s all very simple, the only nuisance, iirc, is that the same data appears in several places and has to be changed in all these places, lest you end up with a corrupted or inconsistent OAR, due to the nature of the serialization format. Otherwise it’s plain XML, and easy to process. In case it may help you, let me point you to my (very incomplete, and probably outdated) documentation for the XML2 format: https://zonjacapalini.wordpress.com/2010/01/25/partial-documentation-for-xml2/

    Comment by Zonja Capalini | September 15, 2011

  9. Awesome! I’ll go take a look :)

    I tried downgrading ooRexx to an older version (3.1.2) with no change in results. Not sure just how old that package is, but I tried it with both Zoe 0.1 and 0.2 and had no output at all. Ah well, was worth a shot. I’m off to look over the XML2 data. Thanks!

    Comment by Kenneth Rougeau | September 15, 2011

  10. You’re welcome ;) I never had problems myself with ooRexx, but I’ve heard that some other people have. Never been able to reproduce the problem, tho. Good luck! :-)

    Comment by Zonja Capalini | September 15, 2011


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: