OpenXR News: Converging on a Widely Accepted VR API Standard

[Music] all right good morning everyone welcome to the open X our panel my name is Yvonne I'm CEO of sin 6 cents X makes special-purpose HMDs you know arcades military medical and so on we also created the OSP our software platform which was useful in in open X R and we have a great pound for you here today to discuss open XR a the path that we're taking on converging on a standard for AR and VR so let me introduce our all-star panel for you it's Center from Colorado State University at six foot five right we've got Joe playing for valve right now okay Mason plays for Google pulpit driana for oculus and Nick I think it's okay that I put you as power forward for

epic games the reason we're excited about this panel is that first all all five of us are very active in the day-to-day formation of the standards whether it's the you know weekly conference calls or the face to faces and so on Nick and Kay also have formal roles Nick is the chair of the open XR working group in K is the spec editor which does all the grunt work in merging the standards as it goes together I would like to start by just explaining what is the standard what are we trying to achieve what is the problem that we're trying to solve and then open it up fairly quickly for questions from the audience don't worry if you don't have questions I have plenty of questions for my colleagues here so the I like to say that you know great works

of art I don't know if open XR will be a great work of art but great works of art starts from misery if you think about if you think about Mozart or run go or Tolstoy you know they had the pickled lives and then they sort of poured their misery into into their work so open XR was born out of some misery the misery is that if you don't have a standard for VR and AR then every time a game developer wants to support multiple platforms they have to learn multiple api's and then keep chasing these api's as they evolve if there is no standard then a hardware vendor if you make eye tracking or an hmd that's not mainstream or sun or some other peripheral then you basically have to sort of beg the engine companies or the content providers to support your hardware and the biggest misery of all is the end users who are

not really sure if they should commit to a particular piece of hardware a particular solution because they don't know how long it's gonna last now standards seem to be sort of boring but think what would have happened if there was no standards for gasoline on cars you know if there were four gas stations and Chevy gas stations and BMW gas stations and you'd have to drive miles and miles just to find the right gas station for you that that would be a problem so the fundamental issue that we're trying to solve is to say well without a standard we're wasting all this time that could be spent on creativity on on reaching new heights in VR and just you know grunt work of making sure that my game works with his device and so on and so on and the situation today is you've got you've got most games work through

engines then they work through various runtimes whether it's steam or oculus or Svea or daydream and so on and then some of these runtimes support multiple devices some of these runtimes support just a few devices and there's just dismiss so what we're trying to do in the open XR working group which is part of the Chronos organization Cronus's the organization that brought you opengl and now Vulcan and others is to standard two layers one is an application interface which says okay this is how applications talk with VR runtimes and then a device interface which basically says these is this is how the runtimes talk with the devices and if this works out the way we intended to work then when you write a game or another VR VR air experience you'll be able to run it across multiple devices very easily and then on the flip side if you have a new piece of hardware a peripheral in hmd or

something like that then you could basically connect it to existing runtimes and run existing applications and we believe that this will also help drive user adoption because they're not gonna fear of well if I buy this device will recon don't really work on it next year so in terms of sort of a an architecture or you know market tech chure marketing architecture you basically have two open interfaces one is the application interface and the other is the device interface and in the middle there's a vendor specific runtime so and we'll ask the panel a little bit later you could envision an oculist runtime that implements open XR or valve runtime or or Google runtime and so on and basically the devices would connect the runtimes through a well-defined interface and they would pass you know ro poses controller and peripheral state the runtime would do whatever they want

with it and then pass it on in a normalized way to the application the application would then figure out what to do if it what's the action that the user is taking and what to draw pass it back to the runtime and then eventually it would get to either the display or the supported devices so that's that's sort of the dream the turns out that the pain that we the suffering that we spoke about was felt by a lot of companies and so here are some of the companies that have sort of publicly committed to the standard that have agreed to put their logo in support of the standard so you see of course all the panels the oculus and in valve in Google and epic and Sensex and also you know Toby and Sony and Intel and LG and and so on and so on so plenty of companies are working on the standard are supporting the standard and of course we invite all

of you individually and collectively to join us in making sure that the standard is a good one and that we do it quickly so I think they're sort of I'd like to start off with some initial questions you know what when you know when we'll be ready what's in it what's not in it but one challenge that we have with the standard is because the inner workings of the standard are not public yet there are some things that we can say and some things that we cannot say so let me ask Nick the workgroup chair the first question he he promised to be my shield during this conversation so what is it that I should not ask what is it that we're not gonna be able to talk about today so basically we can talk about our individual companies or individual opinions on these matters but as far as declaring something is or is not in the standard above and beyond what was already shown on the slide we can't legally talk about yet we're excited to

share it all but there are some IP licensing implications and whatnot that require us to be pretty cagey about the exact specifics of the standard but I think we have a diverse enough panel up here that we can safely say what each of our companies cares about and that I think will be a good indication of the kinds of problems that we're thinking about with the standard itself so you're saying the standard doesn't have a head tracking no head tracking at all it's not important you know we're tracking no head tracking no display interface just you know just a generic middleware connect your ERP to something else it's just an API to implement for fun very good thank you all so ok so for the record Nick cannot confirm that the head tracking is going to be in the standard but very good so the the other thing that I wanted to ask and maybe Joe you can take this one when you look at the logos of the companies you know two companies that don't show up there in the at least in the logo slide are Microsoft and

Apple which is why I'm using a Chromebook today what will they be or are they part of the standards and what do we think about you know their products and and the standard so Microsoft and Apple are two examples of companies whose logos are on that slide there are other companies whose logos are on that slide too and whether or not you're on the slide doesn't necessarily say anything one way or the other about whether or not you're involved because our companies are involved that are involved but haven't chosen to publicly announce their involvement everything we do as we're building the spec is under NDA what what we so it they don't have to announce that they're involved and and that's just fine they can contribute to the standard contribute to the IP pool and be part of this process without without having a be a marketing thing what I will say is that everybody is welcome to join us so if any of you in this room want to come participate in this destination process you can individually join Chronos your company

can join Chronos and you can do what we did all week which is we had two full-day meetings in Chicago on Monday and Tuesday so you can come and join us at those join us on the once a week calls and and start working on the spec so if you want to work on this you're welcome to it encouraged you so so joint open X R and see the world right because we were in Vancouver and in Seattle and in Chicago and next time we're in Taipei and and the sona's long yeah you're right so it's okay you know you you're doing a phenomenal job in merging all the requests to the standard but could you tell us when the standard will be ready so and again this is a thing that we we can't exactly say when it will be ready we can say that we released our statement of work in January of last year and we can say that previous standards have taken between 12 and 18 months from that point down so so 12 to 18 months from from January that's right okay so so let's assume a standard is ready and it's

i defied and it's now made public so let's sort of go around the table and maybe I'll start with Paul will your company support OpenX our standard will will oculus support the open XR standard or you just in it to travel I'm just in it for the Taipei trip this year oculus and intends to fully support the standard to the extent possible and any extensions that exist that are that are reasonable and rational will also support oculus as you know has both a mobile and a PC division and we intend to support it to the extent possible on both sides of that okay and Nick will epoch support the standard yeah absolutely we were quite keen on it because one of the the biggest things that people ask is you kind of alluded to in the beginning is people have either a new device or a new device that's looking for content and the

process of bootstrapping a plugin in the engine right now between the engineering developments and the QA time and kind of the maintenance of that especially well api is tend to be pretty flexible and malleable at this time and changing it was kind of a giant pain in our ass so that's why we really wanted to try to get everybody to a standard so basically i'm starting to write a open XR ue4 plugin as we speak and trying to keep up with the header although I would say ass why do we what oh yeah that one's fine that's one okay so okay well Google support open XR standard so we haven't made a public statement about that and I am a programmer so but what I can say a point to you is the fact that Google is extremely committed to open standards we have chrome we have Android we have we've already open sourced some of our audio stuff for VR too so we we think a standard makes sense it completely makes sense to have a VR standard and we'll be making decisions going forward so in York you're also the API lead for

daydream so I think you're much more than a programmer that's true that's true that don't think there are people above my paygrade to get to make this so the other guy in the panel that his job title says programmer is Joe so will valve support open XR well in my case there are no people above my paygrade no we don't we don't really have those things but but I intend to write code that causes open XR to be supported DVR so okay so that so that's good then and just for the record census will also support open XR but let's sort of go back the other way so let's assume that I'm a developer right now and I'm using open VR you know the valve API for for VR and now there's open XR so will I have to redo my applications will they open VR calls be deprecated will they coexist I mean how do you see that happening going forward well I imagine Paul has an answer of his own but in in my mind we kind of see open XR is sort

of the next generation of over VR we always kind of expect it open VR to morph into an open standard in some way and it turns out this is the way that it's happening so I don't know exactly when that will happen or exactly what the transition will look like probably both will be supported for a period of time but eventually we think everything will be open XR based on our site Paul do you want to as well the answer for oculus is practically identical our long-term goal is to dump both our mobile and our PC api's and use open XR exclusively it they'll of course be a transition period which will be some undetermined amount of time it has to do with backwards compatibility with applications that are in development or applications that still need the old API but D while the applications written to the older API will need to continue to run indefinitely the publishing of an SDK for new applications start development on will eventually switch to

an open X our only API yeah that's a good point we there there are thousands of VR apps out there that are targeting Oakley VR right now and they're not gonna stop working that backward compatibility is important and it will be maintained indefinitely and Nick will open xrb essentially just another target that you could publish to from epic yeah we know we kind of go through great pains to abstract the platform differences as much as possible between the vendors right now so our hope is that we can start bootstrapping this open XR plug-in and then people can just use that and then as steam in valve or an oculus sorry steam and steam and the steam valve get up and running that transition should be abstracted behind all the frameworks and stuff in the engine anyway so we hope on the application side it's basically people just wake up one day and have access to a lot more platforms in unity is also in the working group right I know they're not

on the panel today the other do you know what their plans are I cannot speak for competing engine companies but I should hope that since they have contributed to the meeting area similar mindset and and and yes I mean OS we are API will also evolve to support open XR ok I wanted to ask you about mobile it was it was brought up and by the way feel free to answer the previous question as well but is it really possible to create a standard that works both on desktop and mobile I mean after all there are all these sort of pretty substantial differences between what the device can do and what's important for the device how do you see that happening so just on the previous question let me say that we also as well as daydream our team supports earth VR and we also have tilt brush and several other products that run on steam and on oculus rift so if these guys are supporting open XR I'm

gonna be writing support for open X it's just so we can keep that going so so in terms of that you know I'm sure that's probably gonna happen and so in terms of mobile yeah there are definitely challenges on mobile Android has a very different permissions model then desktop does for example I cannot on Android load a third-party DLL that's just not a thing I cannot go into system and load a third-party DLL because that would be completely insecure and it would break a droid security model so um there's some differences between our process and our our implementation and gear VRS implementation so we do all of our stuff cross process on day dream so our compositor runs in a completely different process and that means some things have to be different so we have to pass images across process boundaries and we have to pass all of this data and that that can be challenging and it can be challenging on a mobile platform I mean instead of dealing with with how many watts of GPU we have ones of watts of GPU power so that's challenging to it and it's really cool what we've made

happen and the hardware companies that we work with are amazing and they do amazing things but there are different challenges and and sometimes I feel like half my job is trying to half of graphics programming is really moving data from one place to another place and so a lot of the challenges that we face are in terms of that and they're just different on mobile fundamentally different than they are on PCs and you know now I realize that because all of you are writing code then the more panels we have the slower the standard is going to evolve so maybe we cut it short right thanks for coming to the extent you can tell us what happens in the work group in a situation like mobile where there may be a set of different requirements or a subset of people who are particularly interested in a particular topic whether it's mobile or eye tracking or you know ear tracking or you know something else yes so there's kind of a couple different paths depending on how much interest there is for mobile I mean that's a large portion

of the market share right now so we have representatives from Google we have representatives from a couple of the other mobile guys Samsung and Qualcomm in there and really what we try to do is make sure that this the core standard doesn't preclude either desktop or mobile concerns and there's been a lot of very vocal support for various issues on on mobile to make sure that those are covered and we haven't precluded them from doing what they need to do to meet the memory model or performance characteristics when there are kind of more esoteric concerns that may maybe aren't as broadly applicable Cronos has what are called extensions and there's various degrees of extensions so if you're familiar with Vulcan there's khr extensions which we kind of consider to be ones that are potentially very broadly adopted but maybe not applicable to everybody there's ext extensions which are we expect a couple people to implement but we still want to kind of standardize because we believe that there's something some value in standardizing those and those eventually can be promoted up to khr extensions if they become more widespread and then there's

vendor specific extensions so if you know a particular GPU vendor or a particular phone vendor wants to do something that's very specific to their platform they can do that and still work within the same ecosystem so between forming technical subgroups remote mobile concerns or using extensions where applicable we try to make sure that everybody's supported so we followed the Google activity this week I wasn't sure if Kay and Joe are gonna be representing the same company pretty much I guess not but but Paul you you guys have both a mobile product and a desktop product and you have also represented it from the mobile side how do you see the so the difference is between mobile and desktop as it relates to open XR the oculus we have as I was saying before we have a gear VR group and we have our PC group and for the last few years we've worked together to to talk about our shared interests and so but before even getting here we had a lot of ideas about how an API needed to

work to support both mobile and PC and a lot of that stuff has continued into the evolution of this of the open XR API and our mobile guy Cass Everett is he was there Chicago last week or actually was earlier this week sitting right with us and we think that this it's entirely possible to support both at the same time we felt that a couple years ago I think that the way that the API is evolving is is looking good to solve that we need to make sure that the mobile side has a has as much influence as the PC side sometimes in our in our meetings there's a little bit of a you know some of the talk sometimes focuses on PC centric stuff when mobile and standalone devices are a major part of the future of er especially in coming

years I think we're getting there I think but you know Kay can speak a little bit for for the mobile side with respect to that I can speak for our mobile side to some degree but I would say we're this is looking like it's gonna work out fairly well okay I'm gonna ask one more question and then open it up for the audience I have two notes here from the conference organizers one is that they would prefer that people who have questions come up to the microphone so that your questions can be heard on their recording and the second request is that you make sure to give five stars to all the presenter and the moderator in the evaluation that you get towards the end of this so if anyone has questions you feel free to start lining up next to a microphone but let me ask one last question for now so we spoke of a Microsoft and Apple and it may be that they'll support the standard

may be that they're part of the effort you know we sort of can tell but what if they're not well open XR being able to run on a Microsoft hmd who wants to to take that so let me let me take that you know answer my own question I think that if we do our job well enough and it's an open standard a well defined device interface then there will be someone whether it's Microsoft or a hacker someplace that very very quickly writes and open XR plug-in device plug in for a Microsoft device and it may be sometimes that they may have to use some you know backdoor techniques or maybe they'll have to figure out some undocumented api's but I think these will pop up super quick I mean we've seen it in our experience with OS VR that there's a

a set of peripherals a set of devices that we supported inherently but then people all over the world started writing plugins for their favorite game engine or favorite device from you know from 20 years ago from 2 days ago and if you have an open enough standard in these devices and that support just pops up that's that's really a market force right you you you have a standard that's widely adopted enough in the market and people who make devices are going to insist that people who are writing software support the standard people who write software are going to assist the people who run time support the standard and and that pressure is difficult to resist once it reaches a certain level of critical mass so our goal here is to get to that critical mass so that the question isn't really whether some of this company or that company or this product of that product supports the standard it's more that it would just be unheard of for them to not support the

standard like yeah like who's gonna make a thumb drive that isn't support USB this is the main sense so I think we need to get to that point with with this standard and probably other standards so that we're making it easy for users and software developers to understand exactly what's gonna work with what and I think that's one of the reasons that the Cronus process tries to be so open that everyone is welcome to curate that to create that critical mass so it will be almost impossible not to support it if you have a new device or a game engine and so on so turning it over to the audience who's got the first question and if other people have questions feel free to you know line up behind the first please I've got to hopefully the first one is easy is this as C API or C++ API I don't know if we can say for

sure but it's a C API so I know there are various officials ordering line officials C++ wrappers but other open Chronos standards is that something that you guys likely to provide or it's not a focus right now okay so the other one I guess is a little bit more general with a lot of these cross platform type api's you can either have something that goes for the lowest common denominator and everything else has extensions or you can try and have something that covers everybody by including features that aren't available everywhere like you know not all platforms have motion controllers so our motion controllers a core thing or are they an extension I'm just wondering if you can talk to how that balance is being dealt with let me take that and then Nick maybe you can answer as well I mean I think that what we've what I've seen is the philosophy inside the working group is if a particular feature

is supported by a large enough group of vendors then we feel that there's enough experience around the table to say okay could we define a standard for that other things that are less common at least less common at the moment tend to go more into either extensions or even you know vendor specific extensions so if you have a unique piece of innovation that only your company provides and you could still tie in to the standard then this it becomes more popular it could work its way towards the Cronus extension or even a core feature yeah I think part of the governing philosophy there's you know we're kind of targeting devices that are either on the market or soon to be on the market and not precluding anything there but trying to build our our core feature set for around devices that will be largely supported so motion controllers are pretty prolific right now so I think it's safe to say that we will have support for them but one of the important considerations I think is how we make this that won't be outdated in two years when

something else comes along and I think there's already kind of precedent for that I mean no SVR has semantic paths which are a pretty big part of it and that's kind of an interesting way to future-proof yourself against it you know changes in the hardware trying to bootstrap new devices while maintaining an API that provides enough functionality that you can actually make good use of them so I think there's ways to kind of satisfy both of those those things but it as a general rule of thumb if it's out or coming out it's probably in the majority of devices support it's probably part of the core standard I think Nick just confirmed that retinal implants are not supported by the standard at this time it'll be an extension so I kind of think about it from the other direction which is which is what does an application need to have available to it so that it can be delivered to the standard and not require vendor specific extensions to actually work on other platforms and that list is not not as long as you might think and so the this that's a set

of things we're really focusing on getting in the standard especially of 1.0 some of some some things you might expect to be in that set may end up as extensions right away or maybe you're not quite understanding yet depending on what you know how commonly applications need them but but really it's all about getting to the point where an application can be delivered against where a single binary will run against against the Occulus runtime or the stevia runtime or whatever without needing to target a specific vendors SDK and part of our discussion in the working group at least this week has been around the MVP you know how do we get something that's useful enough to be in core but then doesn't take us forever because I think that seems to be a consensus that there's a need a desire to have to standard out thank you yes please have you guys all engaged with the browser teams so or is this purely a native API so the question was with

regards to the browser teams and for instance you know we didn't mention web PR as sort of an engine okay maybe you want to make sure we have a pretty substantial web PR team and leave announce what we're doing with low PR and definitely we have our web BR group - participating in open SR we want to make sure that they have what they need their needs are a little different because they're running in chrome and they have some requirements like they want to be able to make queries without draining the batteries and so um these are kind of important to them and we want to make sure that they're included and they've been involved since day one and though the next hour sander itself you know and Mozilla's logo was on that on that logo she an existing runtimes I mean proprietary you know Valve oculus OS VR they run with what they are today so I think it's safe to assume that we would like you know an open XR runtime to run within the browser as well thank

you yes please hi a great discussion based on the vendor list in the panel it seems like VR is kind of like a corn and maybe the majority of what the standard would be about for now like when do you see mix reality augmented reality sort of joining more so we just shipped we just shipped our AR core a couple of weeks ago so we are moving I mean in that direction it's extremely important to us it's it's so we're gonna make sure that that the expertise that are we have the tango team at Google who's been doing AR for years we're going to make sure their expertise gets leveraged in and that we're doing things that that will moving forward that we'll be able to work with that and Nick maybe you can also comment on the X and open XR yeah I mean it was intentionally made X to not preclude AR we would really like there's a lot of overlap right tracking and display a lot of the API itself is is very transferable and we would love to there is a our representation on the

group right now we would love to get more a our representation I'm not gonna call anybody out but you guys know who you we would love to make sure they saw somebody with a t-shirt here yesterday yeah this one company that should probably involve there's several at several in this room in fact I companies out here especially San Francisco which is kind of a hotbed for that sort of thing but you know we want to make sure that our tracking and our display in our layers and stuff like that are all able to handle both the the case where you kind of didn't pass through or you're doing a are or you're doing VR and I think we're pretty good there but it always helps to have proof points and expertise right because there's the Devils in the details with a lot of a lot of this stuff so thanks anyone else want to add to to the em our discussion I mean the one thing I would add is is that VR devices are shipping today in quantity and their devices are still a little bit further out at least for head head born err devices so I from

from our point of view it's important for us to focus and get the first version of this back out and that probably will be mostly about VR as a result but beyond that you know what we're trying to do for a our vs. VR and what devices are supported and what platforms are supported and whether or not you do things in a in the IPC style or the in process style on mobile what we're trying to do in all these cases is is build the API in such a way that it's that it's agnostic to that that the API can support either one and doesn't require that you go one particular way or another because the whole point of having an open standard is that you have an interface where two different implementations from two different companies probably at two different times come together and on either side of that boundary you just go hog-wild and you do whatever you think is best there's a lot of experimentation and appens on both sides of that boundary so we're trying to define the boundary to

be as flexible as possible while still making it so that those things can interoperate yes please when the standard gets released what do app developers need to do to certify that their application actually adheres to the standard and will you be giving them a stamp that or logo that they can attach to say yes I've actually met the standard do you want to answer that Nick and form insisting it's kind of there's two different sides to that there's the hardware and runtime manufacturers and what they have to do generally we haven't announced what our specific plans for open XR are but if you look at volcán that's probably a pretty good precedent so on the hardware side if you're writing kind of driver layer stuff though usually a process for conformance testing where we have a sample set of tests that it has to pass to prove that it faithfully reproduces the functionality in the API applications themselves don't really have that same sort of barrier there's of course validation that you're using the the API right but the you're pretty much free to do with whatever you want

with that but from the hardware and runtime side being certified to be over the next hour compatible is basically gated by that passing of the conformance test so if every runtime supports open XR how do you get differentiation between the vendors why would the oculus runtime be better than the valve runtime or will one be better than the other because because Paul's will a better programmer to be obvious you want to well the ideally the API allows some differentiation in the way that the runtime works that you know runtimes that while they're essentially accomplishing the same thing getting a you know VR display in front of the user how they do it I think differs between runtimes and hopefully the API and standard will allow some some flexibility there if anything just to allow somebody to go and do something do

some new great thing and not be impeded by the API and doing it you know we have some technology at oculus you know ASW if you've heard of that and our goals is to make it so that the API effectively supports ASW but also allows anybody else to implement their own ASW as well on top of that API Joe do you want to talk to that that's right I mean there'll be a lot of diversity runtimes and support for things like ASW is one of them support for different devices the user experience model the way that you know room setup happens like all sorts of different things can be points of competition and that I think that list is going to grow and one of the benefits of having an extensible API is the foundation is that you can add those pieces on to the the core API in a consistent way and if it turns out that that we add support for retinal

implants let's say to pick one that's definitely not 1.0 and we do it with a valve extension and oculus add support for retinal implants and they do with an oculus extension and they're not exactly the same but they're kind of similar then we can come together and we can work on on making on ext or a khr extension that supports retinal implants and and then it's just part of the standard and we both have an implementation of that and it would it would work across the boundaries but but as the technology is coming up we don't have to we don't have to hash it out in a conference room in Chicago we can work within our companies figure out what the right answer is from our point of view and then once we've got that out of the market in some way then we can we can start to negotiate to figure out what the common functionality is so surprisingly when we do the word cloud of this bound implants will be much bigger than I thought to begin with but one thing I wanted to ask about the standard is whether it also includes a standard for content acquisition meaning

the connection to the App Store or do you feel that that's gonna remain so the vendor specific Kay do you want to address that I'm not sure it's kind of our current API does not really contain a any sort of interface for connecting to stores so I know that we would move in that direction but maybe we would in the future I think currently it's kind of independent I know uh gear VR currently has a manifest flag so when you when you create a give your application you have to add a manifest bag that connects to believe it connects to the gear VR store but I don't think it's something that we do currently and I don't know I can't obviously can't comment on gears plans if they're gonna continue doing that herb but I think it's it could happen it's not in our current API so we'll see the Microsoft but the microphone is still available for questions but in the meantime I wanted to ask the panel what

was to you is the most or a contentious point in the discussions about the open XR standard and on the flip side what was the most positive surprise that you had in these discussions who wants to go first I can speak for the latter question pretty easily and I would say the answer to that is that things are going fairly smoothly there's quite a bit of agreement on things I the a lot of these questions like Joe brought up the eye implant situation and how how in the future that's something that you know when that comes around we'll want to converge on that but in a sense all the API development that's happening right now is a lot of that same discussions already though the discussions around tracking around how does how displays are identified and how [Music] you know how input is happening a lot of it is us agreeing on things and realizing that there's a lot of

commonality but in our needs and as a result that I think the API is coming together pretty cleanly and pretty smoothly and fairly rapidly without a whole lot of there's not a lot of bickering there it doesn't happen like you hear about and in other places sometimes I definitely concur on that it's been a great experience everyone is I mean we have a lot of respect at Google for the other for oculus we have respect for like about them the work that they do for sure and just none for epic prize they respect for unity turnoff it was really nice to see that coming back though but I think if I was going to say the most contentious point I would I would probably raise what the audience member over there asked which is about what should be in core and what should be in an extension because I think that that is definitely like things that we think should be car and maybe oculus doesn't think it should be an extension so I think there's always a

lot of if something is important I mean you can see on the list of people who are involved in open X are many companies have something that they work on that they feel is extremely important and it is important often for the future of yaar for the future of AR even and so but making a decision as to whether we can functionally get that into core and make that out and like get 1.0 out and a reasonable time frame is always a hard decision and things have to get pushed into extensions that we don't want to push or that some people it's really important to them so I think that that's that's hard and and I I concur I mean going into the first meeting we didn't know what the atmosphere is gonna be like whether it's gonna be sort of my way or the highway or is it gonna be hey we all have sort of the same we all want the VR and AR market to grow to make that grow we need a standard so let's sort of put our best ideas together and see if we can make something good happen on that subject so if you have a someone

in the audience whose company's not part of the open XR and they have an IP concern you know are they giving up their IP if they contribute to the standard maybe you can talk to that Nick sure um you don't necessarily have to give up your IP when you do when you go into the group discussion they have different rules on the the IP if you attend a work group meeting basically there are IP rules in place to make sure that company X can't put something into the standard that they have a patent on that would force everybody that adopted the standard to have to license that patent so there are provisions around there that say if you do put something a contribution forth into the standard everybody in the group everybody waives the right to claim patentability on that so your contributions are but it doesn't mean that you can't have secret sauce that you keep to yourself and they're not you can still make a runtime that does something if you had a patent and completely hypothetical a singer in her space work for some reason that was a patented technology probably not but retinal implant retinal

implants if you had a patent on retinal implants you could still implement that in the run time and not have to give anything towards the the common IP zone you just couldn't make something that into the standard that you have a patent on if that makes sense and how about the low-level graphics API I mean so will this work with DirectX will this work with metal I mean will this run on a Mac will this support Vulcan so I can't we can't answer any of the specific questions but here's what I can say we support DX 10 11 12 Vulcan GL on PC we support metal on OSX we support GL and Vulcan on Linux and and we expect all those things to be supported in our API going forward so if this becomes our API then I would expect all those things to be supported I don't know if you have anything to say about about mobile I want one more thing I

want to say is you've always continuing to ask questions because none of you are standing at the mic please go ask questions we're happy to you know answer whatever whatever you're you're wondering about absolutely yeah go ahead is there a group dedicated to documentation and examples or is this much more of a reference for the hardware vendors to build drivers towards documentation what's dead I mean you've you met programmers before right no um the one of the goals is Kay is doing a marvelous job editing the spec and the spec is trying to be as verbose and self documenting as possible so it has the the header in there but it also has little snippets of example code the intended usage what is the expected income and outcome and the kind of you know API contract for that so there is that there is talk about whether or not we want to release a full sample

implementation of it and of course anybody that adopts it is free to make an open-source implementation of the standard if they like so that's kind of one of the ways we're looking to try to leverage that sort of stuff but we're not ready to announce anything specific but we certainly don't preclude anybody from releasing source to it so one of the things that's a little bit different between mobile and PCs is the thermal management right you know melting phones and so on does the standard address some of the thermal concerns we can't exactly say what the standard addresses are not but we currently have an API actually Android has an API - what's the state performance mode API where you can you can switch and manage your thermal performance it's it's certainly important for the future of VRS and the future of AR that we do that I mean and it's not just I mean if you heat up so the consequence of heating up your phone it's kind of an interesting because the hotter your phone gets the less the less good it is at conducting so the more

power it needs so you're kind of once you get hot you're getting hotter at an exponential rate because the more hot the hotter you get the more power you need and so the hotter you get again and then you're hotter again and you need more power so this is something that we need to manage and that all VR on mobile has to manage so any of us and everyone so I would expect something I mean we already have stuff in Android that's not going away so and I'd expect you there to be more in the future and I say good I was switching the slide back just to make sure that Qualcomm is on the logo list so they are active in the group and you'd think that it's a mobile processor company that's one of the things they might care about I was gonna say as kind of a middleware engine provider one of the biggest challenges that we see is people developing for mobile on one hand it's one of the most accessible platforms because a lot of people have the cell phones that are compatible and it's kind of low cost to get into you don't have to have the whole PC but it's also one of the most challenging because of the thermal

constraints and we see most most people that fail at making mobile VR fail because they heat up the devisor aren't able to optimize within the mobile perf and thermal requirements so that that's one thing I would love to see I mean in the industry as a whole not necessarily even within the standards group more attention towards educating people on how to Manesh thermals what are the best practice is what happens it because when you tell people about you know thermal runway runaway most people don't know that when you get hotter you start using more power and then you in turn kind of have this exponential curve and simple less is like that once you tell people it makes sense but to me that's also partly a matter of Education not standardization and by the way we're about ten fifteen minutes from the end of this session and you should be receiving the email survey by now we're shooting for six stars but please so we support both five and touch at the moment and we found that the biggest headache is bought in both of them is less about API differences and more about how do we map equivalent actions between quite different controller

designs so I'm just wondering if the API tries to say like this is a grip button or this is a grab control versus this is button one button two and you just have to go through each platform and map those two based on your own kind of design decisions let me take that then maybe Joe you can add to that later so so one of the things that we have in OS we are you know open source VR is the notion of an abstraction layer where we basically allow you to define controllers and then map them into say fire button or a trigger button or teleport button and then abstract that from the game so a game could say I want to use the teleport button and then at some lower level the device manufacturer says well I think this particular yellow button is really going to work well for teleport or for a particular game and so we think this is one of the good features of course we are and as far as census is concerned we're certainly

pushing to have something like that in in open XR and Joe since you have no opinion then here's my non opinion so we we we think that this is a pretty important issue to solve and it's not it's it's not even just limited to I have a single application about my single application to support oculus touch and the vive controllers and up coming knuckles and the windows mr controllers like you want all those to work in the same application and the differences between them are you know this button as this or the dead zone on the joystick needs to be this on this kind of controller and it's different when you have a trackpad and like that those those sorts of changes and the sorts of things you would have to do to adapt your game are important to address as a as a challenge as an issue we we also think that it's important to from the hardware vendor side to be able to add support for new controllers there

are 2000 VR VR only games on Steam games some of our games some of them are not applications I mean and most of those are never going to ship another update and it would be it would be a tragedy if they don't support knuckles when it comes out so we think that it's important to to build things in such a way that that future Hardware can be supported both because we think that it's it's not we're not done integrate innovating on hardware and figuring out what the right input mechanism is but also because it makes a market for interesting new hardware devices that are application specific and because it enables accessibility for people who may not have the ability to use from the standard controller and all these are things that were that we're very interested in solving as part of the standard and that's not saying anything at all the actual solution will be and the other side of this is community

contributions right because you can say well the game the game developer thought that this is the right combination to do mapping between actions and buttons but now the community figure out or some eSports champions figured out that if he does this other thing and maybe he wants to publish his key mappings to VR so we I gave I gave a talk about getting tf2 up and running on the dk1 back in like 2014 and what I said at that time is something I still believe which is that we don't know what mouselook is yet for VR we don't know that it's waz D plus you know rotating the camera around whatever the equivalent is for VR for teleporting for reloading mechanics for moving around for picking things up and manipulating objects like all those those basic verbs are still experimental right now and so as part of figuring

that out we have to try a lot of different things and that might mean that there's some disagreement because there was disagreement when mouselook was coming up as the as the way to do things so having the community be able to communicate with each other and share configurations about you know this is this is how I think it should go no I think this should go this way like that's important and it's it's a big area of interest for us which the steam controller system that we've that we have to support a bunch of different games and push different controllers on Steam is sort of in that direction trying to figure out how to how to make that ring that remapping lis a thing that can come from the community and not just from the game yeah I just think on that note it's not as simple as mapping button to button because survive we tend to find you don't want to hold ban the grip buttons if you're using those who grabbed you it's better to press and release whereas with the touch it's very much designed that you hold the grip button the whole time you're holding the object so you can't necessarily just do a button to button mapping to tour all these things which and it's not

necessarily buttons I mean even on a desktop interface you can achieve you can close a dialog box or you can answer yes no question with a mouse click or a keyboard shortcut or touch on the screen or maybe you know tomorrow with the hand or a head gesture so I think there may be a need for a mechanism to say whatever the input mechanism is we want to map it into some kind of action beyond the simplistic one-to-one mapping between a button and an action thank you next question please I was continuation a little bit on the last one how much it into the user space application does this go for example with the steamvr plugin for unity determining between left and right controllers you just kind of get the controllers and you do some sort of like position determination or with leap motion you have gesture gestures do you all are you looking to also go into that area as well Nick do you want to answer this unity question and first of all I'd recommend an engine change but in all

honesty what we wanted that's kind of the perfect Avenue for extensions for things that come along that maybe aren't broadly adopted across all hardware like gestures we would probably surface those via an extension where applications still have a standardized ways to get that data whether it's coming from a leap motion or Google solely or any other thing that takes gesture recognition is an input and then service that up there are kind of common interfaces for routing things to the applications especially surrounding the input and we would probably make sure that there's a good mapping to there so that it fits into the ecosystem well and you can just check for the existence of that in your application and if it's provided then you can hook into those same way okay so from the engine level me developing an application towards Unreal Engine change at all Wow do you got oh the Unreal Engine guys implementing it yeah we would basically do the piping up through our our input layer that we have in the engine and make sure there's a strong mapping between that so they the hope is the end

user if you're using an engine middleware be a Unreal or unity um the abstractions that are already in place would just be wrapped they were gold yeah I think one thing I can say about this is that the last couple years we've already been working with unity and unreal and other engine developers and a lot of our goals and what we what we've done in in the design are based on two years of experience or more of dealing with these kind of problems so we have a lot quite a bit of experience and understanding a lot of these use cases and our goal is to address all the ones that are irrelevant and of course join the group and help us figure it out next question please I have a couple things actually as we all know the the VR user comfort and testings are pretty important you know components of the VR development so I'm just wondering what are what kind of considerations are you having those two area what are you making what are you doing

to kind of make it easier for these beer application be tested across different platform because Alton are different hardware and so on what kind of tools you're thinking and in terms of comfort again what kind of how are you designing the API to make sure that there are tools to kind of measure and issue some kind of a basic comfort all right so the our philosophy about that with Stevi are and with working the HTC on the vehicle is that we should build a platform where the platform itself is capable of being comfortable that you can that you can build an experience that that does not get people sick that doesn't doesn't cause any motion sickness and from there it's up to the application of elders there are there are people who are trying things that will get some people sick and won't get other people sick there's people trying things that where they're actually deliberately setting out to induce motion sickness and that's

fine and though this experimentation that's happening on some on top of the platform but the platform itself is trying to be built in such a way that its supports comfortable experiences and I think that the I I think that I guess I'm sort of applying that same philosophy when I'm participating in discussions about open XR the API basically is irrelevant for comfort because you can build a comfortable system and if the tracking system is good enough and the objects are good enough and the application is friendly to user comfort then the user will be comfortable for testing I think it's a little easier with a single standard API because you can use the same binary against multiple runtimes time for one more question and then there and then after that a quick question to make about is there a mechanism to preview at some point the standard before it gets published for people maybe to provide input if they're highly interested but you want to take the audience question first please so

related to that question is the standard body defining any kpi's at each of the layers for both device as well as for the middleware and applications in terms of testability or I mean I think I think the goal is that you would be able to take say a steamvr or an oculus store application replace the proprietary runtime if the open XR runtime and then at the very least get the exact same experience on the native device and then beyond that start getting very similar experience on devices that the game was not initially designed to use but maybe others have different view on this yeah I mean it's kind of the same with other like graphics API as if you were a graphics card manufacturer and you wanted to support Vulcan you can make a really bad implementation of Vulcan and it can be really slow and non performant but the hope is because there's an open standard and ostensibly other companies

will be implementing runtimes and drivers for this that people would naturally grab it toward gravitate towards the best api or implementation of the api so we're really focused on making sure that we're not precluding anybody from making the most performant application in terms of latency and turnaround and you know where we block them and where we don't block them and make sure that that's very standardized and well-defined so that applications can theoretically write to a specific behavior pattern from a runtime and the runtimes can in turn expect the same from an NIC last question and maybe 20 seconds for the answer on auditing or being able to get a previous two standard yeah so before we released the standard the aside from joining the the Chronos organization which if you join Chronos you're have access to any of the working groups o open XR is just one of them you can additionally we're looking at starting a technical advisory panel where we invite people to take a look that might be either implementing runtimes that aren't part of Kronos or people that are implementing applications on top of the layer can

preview portions of it in order to solicit feedback so that's has to be voted on and ratified by the working group for each member so if you're interested in it you can contact me through the Chronos open XR working group chair email that's listed on the Kronos org website under open XR and let me know you're interested in that can add you to the list and then when we go through and ready to open up that tech advisory board we can invite on a case-by-case basis so Nick Paul K and Joe thank you for joining me today and thank you all for [Applause]