Zonja Capalini

Reverse-engineering OARs: Understanding parcel maps

Format 0.2 Opensim Archives (OARs) include, since Opensim version 3c271b, a “landdata” directory that stores parcel data. For each parcel in the region there is a file in XML format in the directory. Amost all objects in Opensim are associated with a unique uuid that identifies them, and parcels are not an exception to that rule. Files in the “landdata” directory use the uuid of the parcel as their filename and have a “.xml” extension.

What’s inside each one of these .xml files? Let’s take a peek at a sample one:

    <?xml version="1.0" encoding="utf-16"?>
    <LandData>
      <Area>6256</Area>
      <AuctionID>0</AuctionID>
      <AuthBuyerID>00000000-0000-0000-0000-000000000000</AuthBuyerID>
      <Category>-1</Category>
      <ClaimDate>1255170099</ClaimDate>
      <ClaimPrice>0</ClaimPrice>
      <GlobalID>bfbf7bd4-7d7b-4a35-aba0-c352201cddcf</GlobalID>
      <GroupID>00000000-0000-0000-0000-000000000000</GroupID>
      <IsGroupOwned>False</IsGroupOwned>
      <Bitmap>AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
       AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
       AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
       AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
       AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
       AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
       AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
       AAAAP//fwAAAAAA//9/AAAAAAD//38AAAAAAP//fwAAAAAA//9/AAAAAAD//38AAAAAAP//f
       wAAAAAA//9/AAAAAAD//38AAAAAAP//fwAAAAAA//9/AAAAAAD//38AAAAAAP//fwAAAAAA/
       /9/AAAAAAD//38AAAAAAP//fwAAAAAA//9/AAAAAAA=</Bitmap>
      <Description />
      <Flags>505454667</Flags>
      <LandingType>0</LandingType>
      <Name>Arrabal Tango Club</Name>
      <Status>0</Status>
      <LocalID>4</LocalID>
      <MediaAutoScale>0</MediaAutoScale>
      <MediaID>00000000-0000-0000-0000-000000000000</MediaID>
      <MediaURL />
      <MusicURL>http://xx.xx.xx.xx:8000</MusicURL>
      <OwnerID>69af4428-cec0-48d8-a0d9-c05268facca5</OwnerID>
      <ParcelAccessList />
      <PassHours>0</PassHours>
      <PassPrice>0</PassPrice>
      <SalePrice>0</SalePrice>
      <SnapshotID>00000000-0000-0000-0000-000000000000</SnapshotID>
      <UserLocation>&lt;0, 0, 0&gt;</UserLocation>
      <UserLookAt>&lt;0, 0, 0&gt;</UserLookAt>
      <Dwell>0</Dwell>
      <OtherCleanTime>0</OtherCleanTime>
    </LandData>

That’s the xml file for Arrabal Tango Club in Condensation Land — I’ve only obscured the music URL and wrapped the part between the <Bitmap> and </Bitmap> tags, which originally used only one line, with no intervening blanks. This string encodes the shape and position of the parcel in the region. How do we know that? Well, the parcel map has to be stored somewhere, and it has to be quite large, so the only candidate is the “Bitmap” string :-) Most probably this long string is a text encoding of binary data.

Indeed this is the case: the string is using the Base64 encoding (you can find that quickly by browsing the Wikipedia). After decoding the string, we get a binary chunk of exactly 512 bytes. I checked that all the terrain files have strings of exactly the same length, so this number had to mean something.

But — wait: parcel granularity in Opensim (and Second Life) is 4 meters — this means that parcel selection works in minimal units of 4 meters. Since a normal region is 256×256 meters and 256/4 = 64, this means that, parcel-wise, each region is like a 64×64 checkboard. But 64×64 = 4096, so this is a 4096-square checkboard. And, significantly, 4096/8 = 512 — so it might be that our binary chunk is representing the parcel map in the following way: let’s number the squares in their natural order (i.e., start by putting a one on the SW square and continue horizontally; when a line ends, go up one line and continue), and then construct a bit string consisting of a 0 for each square which doesn’t belong to the parcel, and a 1 for each square that does belong to it.

To check whether my hypothesis was working, I wrote a small program to visualize terrain files using a command prompt — I’d write periods instead of zeros to make the output more legible. Here’s the first result:

0708 - Map for Arrabal Tango Club, wrong version

(I have only rescaled the image to make it square) This looks not bad, since Arrabal is really in the NW part of the region and is (roughly) square, but there is something wrong in the image: there’s a strange column of dots in the right part of Arrabal that should not be there. Interestingly. the number of 1s before the first dot is exactly 16. Could this have something to see with an endianness problem? I modified my program to order-reverse every 8 bits in the original chunk, ran it again, and — voilà!:

0709 - Map for Arrabal Tango Club, right version

Now this looks pretty right! :-) Let’s take an aerial shot of (a local copy of) Condensation Land (the property lines can be seen if you look hard enough :))

0710 - Condensation Land, property lines showing

and superimpose it with a (conveniently  alphaed, and positioned to fit exactly the island borders) version of the last map:

0711 - Photo + map

This looks as if it’s fitting perfectly! Now it should be pretty easy to implement deletion of objects by parcel in Zoe: since for each object we already have its X and Y coordinates, to evaluate whether the (root prim of the) object fits in a certain square, all we have to do is to divide X and Y by 4, truncate round both values, calculate Y*64+x, and use that value as a bit-level index into our binary chunk to determine whether the object is “inside” or “outside” the parcel:

0712 - Zoe - Deleting all objects in a parcel

Here’s the result of saving the OAR and loading it:

Reverse-engineering OARs: Understanding parcel maps

This looks pretty neat! :-) It’s been fun to do the research, it has all taken some few hours (including writing this post), and the next version of Zoe will include deletion by parcel name.

December 18, 2009 - Posted by | OpenSim, Tech News, Tutorials, ZOE | , , , , ,

6 Comments »

  1. ¡ Me encanta este estilo hard!. Como la buena ciencia-ficción. Te felicito por tu profunda investigación en un tema del que no se encuentra absolutamente nada de forma accesible para el usuario.
    Un gran aplauso xiqueta ;-)

    Comment by Alberto Navarro | December 18, 2009

  2. Gracias! :-) La verdad es que hago estas cosas para relajarme de RL, que a veces es un poco heavy ;-) Y si de paso a alguien le sirve, pues mejor :-)

    Comment by Zonja Capalini | December 18, 2009

  3. Es como dice Albert: de ciencia – ficción… Realmente, un trabajo impactante. Más si se tiene en cuenta de que es en tus ratos “de ocio”… Bravo !!!

    Comment by Ludmilla | December 19, 2009

  4. Gracias! :-) Es que cuando estoy nerviosa, pelearme con los bits me calma :-P

    Comment by Zonja Capalini | December 19, 2009

  5. […] not specifically stored anywhere; and the parcel a SceneObject is in is not stored anywhere, but it can be calculated from the parcel information with relative ease. These are two good candidates for our OAR-specific […]

    Pingback by Using SQL to manipulate OARs « Zonja Capalini | January 1, 2010

  6. […] in megaregions. For example, you cannot define parcels outside of the SW region (this is due to the way parcel maps work), and therefore you can’t have independent music streams (for example) when x > 256 or y […]

    Pingback by Splitting Megaregions « Zonja Capalini | February 13, 2010


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: