Using Ogre3D’s OpenGL renderer with the Oculus Rift SDK

Hello there!
The process of getting a scene rendered by Ogre to the Oculus Rift is a bit envolved process. With a basic conaissance of Ogre, and trials and error while browsing the Ogre wiki, Documentation and source code itself I got the thing runing each time Oculus changed the way it worked.
Since we are in the version 0.8 of the SDK, and that 1.0 will come with probably not much change in this front, I think I can write some sort of guide, while browing my Ogre powered VR game engine, and tell you the story of how it works, step by step.

I’ll paste here some code with explaination. It’s not structured into classes because I don’t know how you want to do. I don’t use the Ogre Application framework because I want to choose myself the order where things happen


Before we dive: some warning and considerations:

This post is only documenting how I made it work for my projects. There’s probably a cleaner way to implement it, but this is what I did to make something that run.

This is using Ogre 1.9 and the Oculus SDK 0.8.0.0-beta. For more recent versions of theses libraries, this may not apply.

Also, I’m using Ogre’s RenderSystem_GL and not RenderSystem_GL3+. I probably should use the GL3+ but I started using the old one with the fixed pipeline. It shouldn’t change the way that woks fundamentally. I’m getting a bit more familliar with OpenGL Core-profile programming, and I think I will update my engine to the GL3+ renderer. This probably need to use Ogre RTSS to generate shader program for the old fixed pipelin function, or to write GLSL form the ground up. And I didn’t want to mess with the OpenGL pipeline here.

Last warning before we go: They are some direct GL call that only GL 4.3 (glCopyImageSubData). You need your graphics card and graphics driver to support OpenGL 4.3+ (And If you have hardware that is pwerfull enough to use the Rift, you will. We also need to include and use glew to make OpenGL call.


So, I started making a little game engine for Virtual Reality (and I mean by that : just for messing with an Oculus Rift devkit). And since I didn’t wanted to get my hands too much dirty, I avoided writo,g graphics code directly, and Ogre seemed a nice solution for “not having to do everything myself while only using free software and wirting my own C++ code” so I started with that.

I followed along the differents changes made by Oculus on the way (they have a tendency to break their own API each time they can). From the time that it was just an extension of your desktop to the whay they do it now.

Long story short: In the few latest SDK revisions, they introduced something called the Oculus Compositor. The user have to install a “runtime” that contains several drivers and other goodies, including a background service that talk to the rift. They also worked with GPU vendors (Nvidia and AMD) to integrate a more direct way to access a screen at low level through their drivers (without having to hack into Windows’s graphic stack). And they implemented something called”Direct Driver mode” that doesn’t show the rift as a regular computer screen anymore.

They idea is simple : The Oculus runtime talks to the rift and your application has no direct access to it. You get every information you need from the runtime (head/eye position, size of display, etc.) and you submit frame content to update.

The Oculus Compositor has “layers” you can put stuff on. Some layer are made for images and 2D HUD (to render them at a higher) resolution, and some are for your 3D environement ( ovrLayerType_EyeFov ). This is the only thing we are interested in today.

The Oculus Rift screen in in front of both user’s eye. The left part of it is for the left eye (same goes for the right part). The goal is to put there a distorted image that match the inverse of  the lenses distortion to fill the user’s feild of view with the desired image. The lenses have a really high maginfication, but this introduce heavy distortion and chromatic aberation that have to be corrected. It has to be rendered for the imediate position/orientation of the player.

The distortion calculation and the chromatic aberation correction is done by the Oculus Compositor itself (int he past, they gave the shaders code to do it). We just need to put our rendered scene at the good position/orientation and feild of view.

The feild of view of a camera is basically definded by what we call a projection matrix. The Oculus SDK can calculate each eye’s projection matrix. We obviously need to create two cameras to render the scene for two eyes.

The tricky part is to give the rendered image to the Oculus Rift compositor. The way it’s intended to work is to request a render texture form the Oculus Compositor, and render to it (we do what we call “RTT” rendering (Render To Texuture)).

The only problem is that there’s no easy way to make ogre use a texture that hasn’t been created by it’s own TextureManager. The work-around is to render on an Ogre internal texture and to copy the result to the Oculus texture. Hopefully this can be done directly on the GPU memory without costing much processing power.

To get information about what Ogre is doing behind the scene, we will need access to the RenderSystem_GL in our code, this means two important things:

  • The program will not run with another RenderSystem “plugin” initialized
  • We need to include the headers of the RenderSystem’s componant we need to access

So, we will assume (or we will load manualy) that it’s RenderSystem_GL that is the RenderSystem used by Ogre, and so we will permit ourself to do crazy things like this :

Here, for example, we cast the TextureManager to a GLTextureManager. Since we use the RenderSysem_GL, the instance of the TextureManager that is instanciated is, in fact a GLTextureManager (yay! polymorphism!)

So, I’m not giving a peice of code that works “out of the box” to do it. I have a slightly older implementation somewhere on GitHub that I will have to update. But I will explain what to do to acheive it, like step by step. Without actually structuring the code into good looking, usable and correctly named classes…

Everything regarding the Oculus SDK that is used here can be found here: https://developer.oculus.com/documentation/pcsdk/latest/

The first thing we want to do is to initialize the Oculus SDK. The C++ SDK is presented as a library you have to link to your code to use. This library is mostly in good old C (and not C++) and call everything that belongs to itself with the prefix “ovr” (for “Oculus Vritual Reality”), and is called “LibOVR”.

Some components (all the maths stuff actually) have C++ classes for them. They are englobed into a namespace called “OVR”. I will assume that the directive using namespace OVR; has not been called so I will write OVR:: where it’s needed.

So, we need a bunch of varaibles and objects to hold what we get from libOVR. We will declare everything usefull here.

Next, we can do the initialization:

(MessageBox is a function from Windows.h . The main cause of ovr_Create failing is that there’s no Oculus Rift attached on the system. You can tell the oculus configuration utility to use a “debug hmd”. It will then accept to initialize like there was a DK1 or DK2 plugged on USB and HDMI.

At this stade, we can display any information about the current connected headset. Here’s an example:

If you want to directly get position/orientation information from the headset you can even do :

A “pose” is the a term used to call the user’s point of view on the VR scene. It contains information like it’s head position and orientation.

Now, let’s talk about manually initializing Ogre. I’m not a fan of the Ogre’s application framework, and I prefer to do stuff my way. That’s why I decided to show you the steps you need to render to the rift with Ogre’s OpenGL renderer and not organizing this code into classes.

We need to include Ogre and some other componants form the Oculus library at this point:

We need to store pointers to Ogre’s objects:

note: everything with the “debug” prefix in it’s name is only used to show the mirrored texture form the HMD to the window. The program doesn’t render to the window it creates but to a texture (We will do RTT rendering, and we need to setup a texure as a render buffer, and to put camera viewports on it)

We also need to a bunch of variable to comunicate with the Oculus Rift :

A few things here are present as an array of two elements. To make the code a little nicer, I propose to declare an enume that assocate “left” to index 0 and “right” to 1

We can now start initializing Ogre :
//Create the ogre root with standards Ogre configuration file

Next we create a window. We still want a window so it’s easy to get events from the system (if you use OIS for example, you want to pass it the HWND of the window and keep it focused). The window itself is not usefull for rendereing for the rift, but displaying the debug (mirrored) view from the headset on it is usefull during developpement or demonstration. (Since only one person can wear the headset, and you probably won’t want to put it and remove it each change you do on your code). The Oculus can provide an OpenGL texture with a copy of the Rift content called the mirror texture. We will use it on our render window

Ogre keeps every object on the 3D inside a tree (graph with no cycles) called the Scene Manager. We need to create at least one scene manager to do anything usefull with Ogre. There are multiple types. I manually loader the “OctreeSceneManager” plugin earlier, so it’s time to use it!

Since I want to use my window as a debug output, I will create another scene manager, add a Quad on it with a 16:9 aspect ratio, put the mirrored texture on it and render it to the window with an orthographic projection and no lighting what so ever.

We can create the render camera. We also add a “Camera Controll Node” that you can expose to move the base PoV of the scene :

 

Now we need to create OpenGL textures inside Ogre that will match the render buffer created for us by the Oculus SDK. Just before that, I would like to init Glew to be able to access OpenGL functions

Now we know we need to ask the Oculus Runtime to create for us the rendering texture set. It’s an array of texture we have to write our render output on. We will put both eyes on the same texture, so we will create one large enough for each eye.
We want the maximum coverage of the user’s FoV so we will ask for it

We will use the “bufferSize” to create the texture

We create an equivalent texture inside of Ogre. We will give it the name “RttTex” just to make it easier to fetch it back

(Note taht we are sure we called the createManual() implemented by Ogre::GLTextureManager)

Now, we need to get the OpenGL TextureID of this texture. The Ogre::GLTexture has a “getGLID()” method that return the GLuint we want:

We can also add your render viewport to this texture

Same goes for the mirror texture

We can also configure the our debuging output the same way we did by adding a viewport from the debug camera to the windo. We can also set the mirror texture to the debug plane’s material

Now, we can configure the oculus compositor to know what we put on the texture. We will create a single layer with what we render in 3D on it. If you want to put text or other things on top of it, you can add more layers, according to the Oculus documentation.

Now, everything is in place. For each frame of the game you need to

  1. Increment the index of the Oculus Texture Set
  2. Get window event, if you don’t pump window messaging, Windows will think your program is not responsive and will ask to close it after some time;
  3. Update the position of cams[left] and cams[right] accordign to data from the oculus SDK
  4. Updtate the two viewports on the render texture (vpts[left/right])
  5. Copy the rendered image to the Oculus Texture Set
  6. Submit the frame
  7. Copy back the mirror texture to Ogre
  8. Update the Window

I tend to want to separate the tracking update to the render part since I may want to use the Oculus Position/Orientation inside my gameplay.
For updating the camera :

And for actually render and submit the frame :

And… voilà…

It’s not that complicated but it is a fairly long chunk of code to acheive this

This code can be find on this repository (once I update it) https://github.com/Ybalrid/ogre-oculus-opengl. If you have any questions, there’s the comment section below. 😉

Comments

Leave a Reply

Your email address will not be published / Required fields are marked *

*