Reverse-engineering OARs: Understanding parcel maps
Format 0.2 Opensim Archives (OARs) include, since Opensim version 3c271b, a “landdata” directory that stores parcel data. For each parcel in the region there is a file in XML format in the directory. Amost all objects in Opensim are associated with a unique uuid that identifies them, and parcels are not an exception to that rule. Files in the “landdata” directory use the uuid of the parcel as their filename and have a “.xml” extension.
What’s inside each one of these .xml files? Let’s take a peek at a sample one:
<?xml version="1.0" encoding="utf-16"?> <LandData> <Area>6256</Area> <AuctionID>0</AuctionID> <AuthBuyerID>00000000-0000-0000-0000-000000000000</AuthBuyerID> <Category>-1</Category> <ClaimDate>1255170099</ClaimDate> <ClaimPrice>0</ClaimPrice> <GlobalID>bfbf7bd4-7d7b-4a35-aba0-c352201cddcf</GlobalID> <GroupID>00000000-0000-0000-0000-000000000000</GroupID> <IsGroupOwned>False</IsGroupOwned> <Bitmap>AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAP//fwAAAAAA//9/AAAAAAD//38AAAAAAP//fwAAAAAA//9/AAAAAAD//38AAAAAAP//f wAAAAAA//9/AAAAAAD//38AAAAAAP//fwAAAAAA//9/AAAAAAD//38AAAAAAP//fwAAAAAA/ /9/AAAAAAD//38AAAAAAP//fwAAAAAA//9/AAAAAAA=</Bitmap> <Description /> <Flags>505454667</Flags> <LandingType>0</LandingType> <Name>Arrabal Tango Club</Name> <Status>0</Status> <LocalID>4</LocalID> <MediaAutoScale>0</MediaAutoScale> <MediaID>00000000-0000-0000-0000-000000000000</MediaID> <MediaURL /> <MusicURL>http://xx.xx.xx.xx:8000</MusicURL> <OwnerID>69af4428-cec0-48d8-a0d9-c05268facca5</OwnerID> <ParcelAccessList /> <PassHours>0</PassHours> <PassPrice>0</PassPrice> <SalePrice>0</SalePrice> <SnapshotID>00000000-0000-0000-0000-000000000000</SnapshotID> <UserLocation><0, 0, 0></UserLocation> <UserLookAt><0, 0, 0></UserLookAt> <Dwell>0</Dwell> <OtherCleanTime>0</OtherCleanTime> </LandData>
That’s the xml file for Arrabal Tango Club in Condensation Land — I’ve only obscured the music URL and wrapped the part between the <Bitmap> and </Bitmap> tags, which originally used only one line, with no intervening blanks. This string encodes the shape and position of the parcel in the region. How do we know that? Well, the parcel map has to be stored somewhere, and it has to be quite large, so the only candidate is the “Bitmap” string :-) Most probably this long string is a text encoding of binary data.
Indeed this is the case: the string is using the Base64 encoding (you can find that quickly by browsing the Wikipedia). After decoding the string, we get a binary chunk of exactly 512 bytes. I checked that all the terrain files have strings of exactly the same length, so this number had to mean something.
But — wait: parcel granularity in Opensim (and Second Life) is 4 meters — this means that parcel selection works in minimal units of 4 meters. Since a normal region is 256×256 meters and 256/4 = 64, this means that, parcel-wise, each region is like a 64×64 checkboard. But 64×64 = 4096, so this is a 4096-square checkboard. And, significantly, 4096/8 = 512 — so it might be that our binary chunk is representing the parcel map in the following way: let’s number the squares in their natural order (i.e., start by putting a one on the SW square and continue horizontally; when a line ends, go up one line and continue), and then construct a bit string consisting of a 0 for each square which doesn’t belong to the parcel, and a 1 for each square that does belong to it.
To check whether my hypothesis was working, I wrote a small program to visualize terrain files using a command prompt — I’d write periods instead of zeros to make the output more legible. Here’s the first result:
(I have only rescaled the image to make it square) This looks not bad, since Arrabal is really in the NW part of the region and is (roughly) square, but there is something wrong in the image: there’s a strange column of dots in the right part of Arrabal that should not be there. Interestingly. the number of 1s before the first dot is exactly 16. Could this have something to see with an endianness problem? I modified my program to order-reverse every 8 bits in the original chunk, ran it again, and — voilà!:
Now this looks pretty right! :-) Let’s take an aerial shot of (a local copy of) Condensation Land (the property lines can be seen if you look hard enough :))
and superimpose it with a (conveniently alphaed, and positioned to fit exactly the island borders) version of the last map:
This looks as if it’s fitting perfectly! Now it should be pretty easy to implement deletion of objects by parcel in Zoe: since for each object we already have its X and Y coordinates, to evaluate whether the (root prim of the) object fits in a certain square, all we have to do is to divide X and Y by 4, truncate round both values, calculate Y*64+x, and use that value as a bit-level index into our binary chunk to determine whether the object is “inside” or “outside” the parcel:
Here’s the result of saving the OAR and loading it:
This looks pretty neat! :-) It’s been fun to do the research, it has all taken some few hours (including writing this post), and the next version of Zoe will include deletion by parcel name.