Grider: Prototypying the next generation Hypergrid

Hypergrid

I’ve already blogged (here and here) about hypergrid, a virtual world architecture that allows the federation of Opensim grids (or “worlds”), and in particular teleports between grids. Using hypergrid, you can teleport, for example, from OSGrid to my grid, the Condensation Land Grid, or between OSGrid and Sciencesim, or between Sciencesim and the Cyberlandia grid, etc. Wagner James Au reported in the New World Notes blog about a demo run by Tom Murphy at the Metaverse University in which hypergrid teleports were shown between grids located in different parts of the US. Hypergrid is developing very fast, and once it’s stabilized and distributed widely, it will implement the vision of a world-wide network of connected grids, where big corporate or educational grids will coexist with small, home-based grids. And thanks to the hypergrid teleport, all these grids will appear as a single world like Second Life is today — hence the name “hypergrid” :-)

Here’s a sample video showing hypergrid teleports in action:

In this article I’ll discuss some details of the current hypergrid architecture, some of the security risks involved, and how a new, more secure hypergrid architecture (sometimes called “safe hypergrid” or “hypergrid 2″) is being developed.

Taking a look at the Opensim console

If you own an hypergrid-enabled Opensim installation, try the following: log in, and teleport from one region to another. You will see a lot of information scrolling through the Opensim console, but if you look at the logs, you will find something similar to the following (I’ve trimmed the timestamps and the full module name to make the information more legible):

[LOCAL INVENTORY SERVICE]: Requesting inventory for user 17fff378-c1ad-4b42-ae8a-dcb644a5ceaf
[LOCAL INVENTORY SERVICE]: Received inventory response for user 17fff378-c1ad-4b42-ae8a-dcb644a5ceaf containing 17 folders and 32 items


What are these lines, and what do they mean? Well, the Linden viewer (and therefore all the current SL and Opensim-compatible viewers, since they all are, to some extent or another, based on the Linden viewer) assumes that the region you are in will cache (i.e., make a copy of) all your inventory, so that when you interact with your inventory (for example, when you search for an item, wear some clothes, remove an attachment, etc.), this interaction is handled by your current region, which then forwards the request to the inventory server, if needed.

Why is this so? I really don’t know — all I can do is speculate. Maybe the Lindens designed Second Life in this way in the hope of minimizing the number of interactions with the overloaded inventory servers, or maybe there’s some other reason. The fact is that the Linden viewer works this way: when you teleport to a new region, all your interactions with your inventory are handled by this new region (in many cases you won’t even notice it, because all your inventory will have downloaded and will be cached by the viewer anyway).

A problem of trust

This mechanism works well in Second Life, where all the servers are owned and controlled by Linden Lab, and therefore trusted, i.e., you don’t expect a Second Life region (or, better, the server that runs the region)  to act against your interests, or the interests of Linden Lab. Of course you can find an island run, or occupied, by griefers, and then the avatars in the island might harass you. But the idea that the server itself may act against you is unthinkable in the Second Life grid.

Let’s now try an hypergrid teleport, and look at the Opensim log. We willl find lines similar to the following:

[HGStandaloneInvModule]: Processing request for inventory of 17fff378-c1ad-4b42-ae8a-dcb644a5ceaf
[HGStandaloneInvModule]: Sending back inventory response to user 17fff378-c1ad-4b42-ae8a-dcb644a5ceaf containing 17 folders and 32 items

Apart from the fact that the module issuing the message has changed, nothing else seems to be different. Now that we are in a foreign region (i.e., in a region outside of our home grid), let’s wear something — the foreign region will honor the request, ask for the item to our grid’s inventory service if it does not currently have a copy of it, and once it receives the requested item, it will send it to our viewer. Now let’s purge an item. The mechanism is the same: the region will ask our home inventory service to purge the item, and then send us an acknowledgement, so that the item will dissapear from our inventory.

What about trust in this case? Well, we are in a foreign region, right? This means that we are in a grid that is not our home grid. Who can tell us that the foreign grid is not operated by pirates, instead of by nice people? Since our inventory is being copied to the foreign region, how can we know that the foreign region operators aren’t using a modification of Opensim that copies all our inventory to their grid, full rights and everything, so that they can use it themselves, or resell it as if it was their own?

Obviously this would be a very bad thing. However, it becomes quickly apparent that in an open intergrid situation we can’t really assume that we know anything about the region we are teleporting to — Indeed there is a situation much worse than our inventory being simply copied: imagine that the region we are teleporting to is run not by thieves, but by evil griefers. Since our home inventory service has to honour purge requests sent from the foreign region, the region itself can send a set of purge requests and erase all our inventory!

Cristina Lopes indeed programmed an experimental region (called “Do Not Come Here” for reasons you’ll quickly understand) that does exactly this: it erases all your inventory, as a proof of concept of the security problems of the current hypergrid implementation. That’s why if you look at the list of public hypergrid nodes in the Opensim wiki you’ll find a cautionary text that says “For the time being, and until the security concerns are addressed, we advise you to be careful about who you link to.”

The solution: grider, and the safe hypegrid

What’s the solution to this problem? Well, at first glance it seems very easy: write a modified version of the Second Life viewer that interacts with the inventory by talking directly to our home grid inventory service instead of using the region we are in as a cache. This way, we avoid all the risks and the hypergrid becomes safe.

The problem with this approach, now, is not technical, but legal: the Second Life viewer was released as open source using a GPL license. Basically, this means that if you make a modification to the viewer, you have to make your modification public, and your modification has to have itself a GPL license or a license compatible with GPL. On the other hand, Opensim is using a BSD license; this means that everybody can take the Opensim source code, modify it (or not), brand it with another name, and sell it.

In practice, this creates the following situation: no person that has looked at the code for the viewer can develop for Opensim, because this would automatically change the license of Opensim code to GPL, which is incompatible with BSD.

But we have just said that we need a modification of the viewer, and the lead architect for hypergrid is Cristina Lopes, who is a core Opensim developer, and therefore she cannot look at the viewer code!

To overcome this problem, Cristina Lopes is developing Grider. Technically, Grider is a proxy: a program that sits between the viewer and the internet, intercepts the viewer requests for inventory, and translates these requests into the safe hypergrid (or hypergrid 2) protocol. This can be written without looking at the viewer code, only examining its behaviour, which avoids the license conflicts. If you look at your Opensim.ini (for a sufficiently advanced revision), you’ll find a section named [Hypergrid] with a single line that says

safemode = false

and the following comment “Keep it false for now. Making it true requires the use of a special client in order to access inventory”. This “special client” is Grider.

How does it work, in practice?

Grider is an early prototype, a proof-of-concept more than a finished product, so that when it becomes widely available it will most probably be very different from its current incarnation. Anyway, at the moment of this writing, Grider starts by asking for the path to your favourite viewer (say, the Second Life viewer, or the Hippo viewer), stores it for subsequent runs, and then starts that viewer. For the time being, a hack has to be used here: in the “First Name” field, you have to write your first name, followed by a period, followed by your second name (that would be “Zonja.Capalini” in my case); in the “Second Name” field, you’ll write the name of the destination grid, for example “osgrid.org”, or “condensationland.com:9000″ (the port defaults to 8002 if it is not specified); finally, you’ll write your password as usual in the password field.

Remember that we are still using our viewer, but we have started Grider, that silently sits between the viewer and the internet. Since we have specified our avatar name and the grid we are connecting to, Grider knows where to log in, and the address of our home grid. From now on, every time we interact with our inventory, Grider will intercept the viewer’s instructions and translate them into the safe hypergrid protocol (e.g., it will talk to our grid’s inventory service, not to the region we are in). Very simple. We won’t notice anything different, but we’ll be safe!

A glimpse at the future

Hypergrid 2 decouples inventory handling from the regions, so that you can be in one grid and interact with your inventory, located in another grid. This seems to be the trend of thought in the development of Opensim: decoupling user authentication (i.e., log in services) from inventory management, region and asset management, messaging, etc., as to make all the services as independent as possible.

So that we can imagine a future in which we log in to the metaverse using our Google account, teleport to a region hosted by Amazon Cloud Computing services, have our friends managed by AIM, and store our inventory in our own PC, for example. And we’ll still be able to communicate, meet, build together, share, etc., with our friend, who may be using the same providers as we are, or completely different ones. We’ll be able to get almost (or completely) free, light-use regions from some providers, or host them ourselves in our home computer, or even in our laptop, and meet with some friends there, or hold meetings with some few people; on the other hand, several companies will offer “premium” regions, capable of holding hundreds of avatars simultaneously. We’ll have our inventory at home, or hosted by a provider, or both things at the same time. Companies selling inventory hosting services won’t necessarily be the same companies that sell regions, or user authentication services. We’ll have several avatars, as we now have several mail addresses; we’ll use some avatars for some purposes, and other avatars for some other purposes, as we currently do with email addresses. And we’ll be able to buy two copies of a nice dress with one of our avies, hosted at A.com, and send one copy to another of our avies, hosted at B.com… The possibilities are mind-boggling now (although most probably all of this will look ridiculously simple in a few years :-)) — and Hypergrid 2 is a first step in this direction, towards this exciting future.

The Openspace fiasco: six months later

The Openspace fiasco

Six months ago, I blogged about the Openspace fiasco and how it made business customers and creative residents equally angry. 

In case you’ve not read the post, let me make a quick summary: I work for a company which we will call “C”; C owned a standard sim C1 and three openspaces C2, C3 and C4; at the same time, I was the owner of the Condensation Land archipielago (the Condensation Land, Condensation North, Condensation Beach., Condensation South and Condensation Southwest sims); all the islands were Openspaces, except Condensation Land. When the unjustifiable increase of 66% in the price of Openspaces was announced, C’s executives asked me to start researching for an alternative to Second Life, because they had lost their confidence in Linden Lab. I was also forced to look elsewhere for my own islands, because most of my tenants were not able to afford the price increase, got fed up, and left Second Life.

In this post I’ll describe the alternatives that were considered, the decisions that were taken, and what we learned in the process. Although the experience has been bitter in many aspects (having to abandon land which you have carefully created and helped to make beautiful is awful; Second Life customer service is a disaster; etc), the final results are quite interesting. I’ve though to share them, in case somebody can find them of interest and profit from our experience.

Dynamic hypergrid links: the new Metaverse

Would you like to have your own SL-like micro-world in-house, to be able to make backup copies of your islands and your inventory at will, to be able to clone your avatar with all your inventory into your friend’s micro-world when needed, to be able to pack a whole island and share it with friends or with the world, to be able to use fantastic buildings and landscapes others have created, to be able to add an island instantly, to decide how much computer power you’re going to allocate to your regions, to be able to run 10+ islands with 20+ avatars in a single computer, …?

“Sure,” you’ll say, “all these things are very nice, but this has to come at a price, right? Everybody will be isolated in their own micro-world, and what’s the fun of that?”

Up to recently, that was it. Either you were in a “big” world (Second Life, Open Life, OSGrid, etc), or you had your own micro-world, and then you were isolated in it. This started to change when Christa “Diva” Lopes, from the OpenSim dev team, implemented the first phase of the Hypergrid concept, what we can now call static hypergrid. I blogged about it one month ago; here’s an older article from Vint Falken; of course it all started with Justin Clark-Casey’s seminal post “Could there be a future without big grids?”.

With static hypergrid, grid owners decide whether their grids will be accesible from outside (i.e., if they’ll run their grid in hypergrid mode) and if so they inform other grids; other grid’ operators then use a simple command to link their grid to the first. This architecture gets complemented with the possibility to use prebuilt .xml files to link several (potentially thousands) of new regions at once with a single command.

That’s a great architecture, but it puts the power of jumping from one grid to another into the grid owner’s hands. And it’s good that it is so, because the foreign regions appear on the origin grid’s map, and if I were a grid operator I’d love to be able to control what appears on my map. However, it has several disadvantages for the end-user: it precludes the possibility to jump to an arbitrary grid; if grid A links to grid B and grid B links to grid C but grid A does not link to grid C, you have to jump to B to get to C, which is preposterous; in general, the major problem is that you’re limited in where you can teleport to by your grid owners. Of course SL works that way, but SL is not an open grid, at least for the moment. And if OpenSim has to become the 3D web, you have to be able to teleport, to jump, anywhere. The very same notion of your ISP determining which web pages you can visit and which you can’t visit is against the fundamental idea of the internet.

Ideally, what I’d love to be able to do would be something similar to the following: assume that I’m working in my soon-to-be-opened OpenSim Condensation grid, I get tired of building, and I decide to teleport to my friend’s Ludmilla grid (note: to another grid, not to an island inside my grid) to see whether she’s available to go exploring, so that I tp to her world, and, lo!, there she is; I greet her, and we both tp to OSGrid (the free OpenSim grid) and start exploring from there… Impossible? No! Here’s a video that demonstrates exactly that:

 

Hypergrid, or how to teleport between WORLDS using OpenSim

[Update 20090106: Corrected information about names after hypergrid tp, added two captures]

HyperGrid

This post presents a user experience with Hypergrid, a technology included in OpenSim that allows teleports between virtual worlds (or grids), and includes also a short tutorial so that you can try it yourself, link your virtual world with your friends’ worlds, etc. It assumes a minimal working knowledge of OpenSim, at the level described in my previous post. Yes, yes, I know, after a “Installing OpenSim” post there should come another titled “Saving our database” or “Our first Grid” — but I’m doing that for fun, ok? :-)

Why Hypergrid matters

Hypergrid represents a radical change in the model for 3D online OpenSim-compatible virtual worlds. Up to now, we’ve been used to big, monolithic worlds (like Second Life or OSGrid), modelled under the metaphor of a flat, grid-like world (hence the term grid). The world is made of pieces called islands or regions which occupy a place in the grid determined by two integer coordinates, and these in turn determine the visibility between islands. Thus, if island A is at (1000,1000) and island B is at (1001,1000), A and B share a border: the east of A is the west of B.

This “big world” model has several problems: big worlds don’t scale well, the data is in the hands of the service provider, not the customer, and it’s easy to for the service providers to scam their customers. Besides, if OpenSim has to become the 3D web, one doesn’t understand clearly why having some islands (= having a web) should imply having neighbours or occupying a position in a vaster plane — there’s no notion of “neighbour web” in the WWW.

Enter HyperGrid. HyperGrid allows us to work with a completely different model, what we could call a “small world” model. Everybody can have a world, a grid, in the same way that everybody can have a web. You, your community, your employer, whomever. And, as webs are linked using HTTP links, so grids are linked using HyperGrid. Easy, no?

Therefore HyperGrid matters. Indeed, it matters a lot. A hypergrided universe is a universe in which you can have your own world, take your own backups, create your own contents and save it to your HD, invite your friends… and still not be isolated.

To be fair, the technology has its problems. For example, DRM as we know it today cannot work, and there also are some security concerns. Anyway, these problems will most probably be tackled, one way or another.

Installing OpenSim in Windows XP with MySQL.

Update 20090201: Included instructions for Windows 64 bit as reported by Xen Zerbino (Thanks! :-)).

Update 20090123:Fixed a bug in OpenSim.ini detected by Alpha Runningbear. (Thanks!)

Update 20090105: Reworded step 6 to eliminate some tech-talk, added a fall-back for initial log-in, and describe the shutdown procedure in as per Vint Falken’s suggestions 1 and 2 (thanks! :-)).

Note 1: OpenSim is considered to be alpha software. This means that many things you expect from your daily use of Second Life don’t work in the same way, or simply they don’t work at all. Development of OpenSim is very active, tho, so that we can only expect OpenSim quality and features to better with time.

Note 2: This is not for the faint of heart. :-) You’ve been warned! :-)

Note 3: All the information presented here was valid on January the 3rd, 2009. I’ll correct errors that are brought to my attention, but this post should not be taken as a substitute of the official OpenSim wiki.

The purpose of this post is to present a short, comprehensive, do-it-yourself tutorial for the installation of OpenSim in a Windows XP machine, using MySQL as a back-end for persistence. I’ve tried to write the tutorial in such a way that you can understand it even if you’ve got no previous exposure to OpenSim or to database concepts; however, some familiarity with the operating system is required, in particular you’re assumed to know how to 1) open a system prompt; 2) open a text editor and create a file (the “notepad” application will suffice). A step-by-step procedure follows; you’ll be asked to download some files; these files implement OpenSim and MySql.

“We believe this is fair”, or how to make business customers and creative residents equally angry

“We believe this is fair”
M. Linden, in A Letter to Second Life Residents

Let me tell you a story. I work for a company, let’s call it “C”. C had two full sims in Second Life, C1 and C2; C1 was mainly used for education (a very few hours a week, maximum 10 avies), and C2 was used to hold vanity offices for C’s top executives. At the same time, I had two normal sims, Condensation Land and Condensation Beach, which I was renting to some SL and RL friends and to some of C’s employees, losing a small amount of money every month in the process, but it was all so fun that I really didn’t care very much.

Then came the announcement and later the price change: you could have an OpenSpace, 1/4 of a sim for 1/4 of the price, this seemed fair, allowed finer control of the investments, to landscape more beautifully, to make places less congested, etc.

[As an aside: I don't think you can "abuse" a program. You can abuse a child, your husband or a resident, but not a program. Think about "abusing Excel" -- it's simply ludicrous.]

I met with C’s executives, and decided to do the following: C2 would be converted into four openspaces; three of them would be put around C1 and would hold, apart from the then current offices for C’s executives, more vanity offices for C’s personnel; one would be sold to me and become part of the Condensation Archipielago.

C thus paid US$ 250 to Linden Lab for the conversion, 40 hours of my time = € 4000, and was unable to use C2 during the two months it took for the Lindens to process the ticket, but had to pay for it anyway (= 2 * US$ 300); when you convert an island to an openspace you can’t keep the terraforming or the contents, all is wiped in the process.

C had invested or lost 250+4000*1,2+2*300 = 5650 US$, and had sold one void (US$ 250), but had now 256K sqm instead of 128, everything was nicer and ampler, and everybody was happy.

On my side, I received the new island (Condensation South), for which I paid US$250 to C and US$ 100 to Linden Lab, and I started to convert Condensation Beach to a void. Condensation Beach was hosting one of my best customers, my RL friend and fellow flickerite Ludmilla Writer. Since the conversion to a void wipes the island and Ludmilla is an active resident who can’t be without a proper house, I ordered another void (Condensation SouthWest), for which I paid US$ 250 more, terraformed it, and helped Ludmilla to move temporarily to it while her former place of residence was converted. Then I paid US$ 250 more to have Condensation Beach converted, waited two months more, during which I could not use a full sim I was paying for (US$ 600), then got four openspace sims; one was Condensation Beach, another was Condensation North, and the other two I sold in the market, for which I got US$500. Then I helped Ludmilla to terraform her Island, terraformed the other three islands, and gave my tenants nice terrains on the new islands. Most of my tenants never log in, but they kept paying me because it was not expensive and it was nice and fashionable to have a small piece of land in Second Life. I have not made an exact calculation of the amount of time I invested in terraforming the Condensation archipielago, but it’s for sure more than 100 hours; let’s put it down 100 hours for the sake of the calculations.

Of course it was my free time, but if I had been working this would have costed € 10,000 = S$ 12,000.

Ludmilla, who’s not a rich person IRL by the way, was making a big effort to be able to pay for her own island, but she was so proud of it that she spent a lot of L$ in buying trees and stuff to make it beautiful anyway.

So I had invested or lost 250*3 + 300*2 + 12,000 = 13,100 US$ and had earned US$ 500 by selling my two remaining islands, but everybody was happy: the new islands were wonderful and although I had put a huge amount of time in terraforming them, everything was so beautiful and my friend Ludmilla was blooming.

We then threw out a party for the opening of Ludmilla’s island:

Since Ludmilla’s island was an openspace, we made the party in Condensation Land, which is a full sim, so as not to stress the lesser sims (yes we were caring!).

A month later Linden Lab announced the price changes and the protests started: