Workflow Driven Tools Design

[Music] hi everyone i'm vesa parkane i work as a senior tools program at remedy today i'm going to talk about how we improve the overall design of our internal tools framework during the past few years so few words about remedy first you might don't know us from games such as max payne allen wake and quantum break we're based in espo finland where and we currently have about 150 employees we're working on two games p7 which is our own ip and on our first fps single-player campaign for crossfire 2. so it's always important to know the reasons behind decisions leading up to a design so i'm going to start with a bit of background of the problems we faced after that i'm going to dive into what i actually mean by work floater and design on and how we build our current tools framework are in that concept so let's start by looking at what we had when the products of quantum break started here's our world editor or wet as we call it in all of its mfc glory

it is a monolithic application originally built for alan wake and it has been our most important tool ever since but the tool has become quite outdated not only due to its ui but also due to the technical depth which has caused several big problems one of the biggest problems is probably something that is familiar to most of you changing the simple things regarding opening a level as well as exporting it making the iteration speed unbearable due to the monolithic design uh the code had become so tangled that it was virtually impossible to determine what kind of effects even seemingly small changes would have it heavily uses single tones so modifying the state in one panel could cause undecided side effects in all of them possibly causing a lot of regression and because all the code was so tightly coupled it meant that even the most basic tests would have required initializing the whole application which makes any kind of test difficult to write and slow the run so we did not want to continue using mfc just imagine that so we decided we're going to build the

new tools with something more modern and ultimately ended up with wpf as a lesson learned from wet we wanted the new tools to be completely plug-in based based uh to prevent the new call from becoming a similar monolithic mess and because we were building tools from scratch we wanted to reuse existing functionality as much as possible by hooking the new tools both of the engine and web so this is a high level architecture we came up with for our dot net tools and the plugin system there are shared csr libraries that can be used in different applications there are different plugins providing different back panels and background services and finally there is a simple booth track strapper executable that loads plugins for a given workflow based on a configuration file for example for mesh editing workflow we loaded asset browser property editor and live editing functionality for timelines we also need curve editor and timeline editor and for screen by la workflow we had completely different plugins so let's have a look at probably the simplest workflow we had during quantum

break editing mesh properties here we have the asset browser and a property editor to edit the current selection you can see that the contents of the panel change when the user selects an asset and if the user wants to view multiple messages simultaneously he can open another property panel which by default follow the same selection so the user must use the lock feature to lock one of the panels while having the another panel to continue to follow selection like simple stuff and for comparison here's a quick look of what the timeline editing looked like during quantum break it required a few more panels and the thing to note here is that the asset browser panel is actually the only one that that's common for the messenger workflow the property panels are completely specific to timelines as well as well as the hierarchy and the timeline panel i'll get back to that soon so all in all the approach to use.net tools was a success for quantum the development speed improved and thus we could ship better tools with better ux it also

made it easier to look make them look significantly better and the plug-in framework allowed easy unit testing of individual systems so naturally we wanted to continue building new functionality on top of our wpf tools and ultimately enable us to replace web but there was major design for with this plugin based loosely called design due to how the panels communicated when the selection changed in the asset browser panel it sent the message through a multicast system called the mediator to avoid tight coupling between the panels the immediately forwarded the message to any panels that were listening to a given message and those panels were responsible for determining how to handle it in the case of the property editor it was responsible for deciding if it should change the selection keep it or clear it and you could have any number of panels in different states handling the same messages so let's have a look at a concrete exam example of the problem that occurred when we added new panels for mesh editing workflow to allow users to edit physics definitions

first of all you can see that it's very cumbersome to open new panels individually if they don't have button be open yet and when the user opens the panels they are empty because they haven't received any selection changed messages and now user has to manually reselect the mesh to get some content to the new panels and now the user can finally select the node in the fbx so let's take this even a bit further now the user wants to edit the materials as well let's see how that goes so there's a yet another panel let's drag it there we change the color and now we want to check out the texture

as well yay we got another panel and and as last step now the user wants to check out some other material using the asset browser so he searches for for a specific material selects it and boom the mesh selection is gone because he forgot to use the lock so you're just looking at pumping with the panels yeah just looking at the video is painful enough so you can probably imagine our users going something like that so that and that's just the tip of the iceberg there are plenty of other problems as well the loosely coupled design and the subsequently growing number of panels cause a mental load that was simply too big to manage it was simply too easy to make mistakes

you could argue that the design also gave the users quite a lot of flexibility in terms of how they set up their workflows but that would essentially mean that you need a team of super users and not mere mortals it was also very difficult to read information on the screen when the number of panels increase here you can see that it's not immediately obvious that there are three separate assets here mesh material and a texture also in this version some some panels did have locks and source control statuses while the others did not it didn't feel right that we would duplicate the same information for all the panels involving the same mesh if we introduced locks to all of them the users could really suit themselves in the food by mixing locked states on different panels representing different parts of the same asset all the panels had business logic inside them because they had to work individually due to they lose a couple of design here are three examples of panels that resemble each other but is having different business logic causing us

having to maintain three different versions the property editor followed asset selection handle riding to disk and as well as source control operations physics definition editor follow the selection of fbx hierarchy and also handle saving and source control operations and then there were quite a few editors that simply followed some other selection like timeline events or notes in a node graph got several general problems one of the first for example was that if there were two panels editing the same data with only one of them writing data to the disk closing the one writing the data would essentially mean that your data never got saved and user could just keep on editing and never realize that his changes are not there yeah we had around 10 different variations properly at one point and it was very confusing that the users because they didn't behave similar to each other this also meant that the development costs got higher after adding new variations because in the worst case

scenario some changes had to be made in each of them and each of them had to be tested it also virtually prevented end reuse of the business logic because it was tied to the ui and adding more panels just added more complexity here is an example of us adding a behavior tree editor if we just start with one panel for editing the graph and one for editing and selection everything works relatively fine but one of the first requirements we got from the users was that they want to be able to edit multiple multiple crafts simultaneously so how would we make that work technically in case of two graphs should the node property panel listen to both of them what what about if we have two node properties panels should the users manually hook them up with specific graph and the situation would get even worse if the users would want to have some completely different asset types open simultaneously as the number of of panels would just keep on increasing so something had to be done the problems and successes during quantum gave us the basis for creating design goals for reworking the architect

architecture most importantly we need the better structure for the application so we're not solving the same problems in multiple similar use cases context sensitivity we wanted our users to be able to use the tools right away just like an asset and they'll say relevant things this was also a technical goal it should be easy to determine in the code what the current state is what has been changed and so on we wanted to keep the flexibility of the plugin system so we could easily modify the set of the panels and services that we showed the users we were easily able to unit test them and in that addition we wanted to be able to have games specific plugins and finally we should be able to reuse the ui controllers as much as possible to provide consistent look and feel so it was very clear that the main observation main observation was that the biggest mistake was providing the business logic inside the panels that apply rules to specific workflows we clearly needed something to glue

together different parts of the workflow and this is from where the term workflow driven design comes from the application should model the workflows in some way and that should be responsible for setting up all the relevant things for the users by configuring the panels enabling the background services and exposing the workflow specific commands by reusing the same panels in multiple workflows by injecting any custom behavior to them we can signify improve the code reusability as well as the user experience and most importantly this design allows the domain experts like ai or animation programmers with intimate knowledge of the specific workflows to concentrate on modeling the behavior of the workflow this improves things significantly as the tools team can concentrate on building the framework and the ui controls while the domain experts who work closely with the users for the actual business logic it also means that experts rarely have to dive into any wpf details as those are handled in the panels and

controls provided by the framework lowering the learning curve to build new tools so with these points in mind we started concepting the ui to get a clearer picture how we could design the architecture of the new system let's look at a video of a recent build of our tools which is quite close to the original ui design we had here you can see three different node editors built by ai rendering and animation teams you can see that the tools are now context aware selecting an asset opens new tab with relevant panels inside them and you can see that the source control status is now shown clearly in the top right corner now we're use using the same property editor everywhere let's go back to the mesh example one more time now you you get all the relevant panels open when the message mesh is opened you can open a material

you can easily browse through the different materials and if you go to the texture you can brush through those as well so let's move the high level technical design from the previous video we can easily identify the following concepts the panels the document hosting the panels and the data that is being read and written a naive implementation would look something like this panels are assigned to some document and then the document handles reading and writing the data however there are two major problems now because the panels need to know about the specific documents they are going to be they're going to be used in we would not be able to reuse them on other workflows which is against our design goals also we're mixing low-level concepts like reading and writing data with the document which is clearly an ui concept making it difficult for us to use that code in other systems that might not even have an ui

the solution for the first problem is relatively simple we use dependency inversion to abstract out the document with an interface to prevent coupling with a specific implementation as for the second problem we need to create a new concept to abstract out the details about the data we decided to call that a context it handles all the details about the data like where it's coming from how it is saved as well as tracking any changes to summarize the concepts on a very high level the document is the ui layer describing how things are edited it works as a clue between the panels commands and the data the context on the other hand handles all the details of what is being edited it extract out the details from the ui allowing us to use ui components with all kinds of different data let's look at the two main cases how all of this is put together the simpler one is the configured panel to always show the same piece of data in the same panel

for example in the case of meshes both the mesh properties and mesh viewer are configured to always show the same data of the same asset property spans always shows the full contents of a metadata file while the viewer panel shows the mesh itself the fbx is read only while the metadata is editable let's look at how the panel is configured to behave this way first we need to configure the context to load the data we use some string id to identify the piece of content and then we pass in a method that loads it loading happens nicely so if there's no panel requesting that data it's never read from the disk and now that the context knows how to read the file we can configure the document to host the property editor panel we use the same id that we use to register content and to let the panel know which piece of data it should request and show when it's opened the second case is a bit more involved now the document will contain a panel

that changes its content based on user interaction let's have yet another look at the mesh editing and especially the physics definition properties panel when the user changes the selection inside the document using either the fbx panel or meshview panel the physics definition panel reacts it also note that the selection is synchronized between all three panels so we start again with the context knowing how to load the data this is configured exactly the same way as in the previous example but now there is a small problem we need to have an internal selection we cannot add a property to the mess document because that would cause a tight coupling between the panels and the mesh document implementation it would also make it more cumbersome to extend if the number of internal selections would increase so the solution in this case was to introduce a concept of components which is very similar to components in an entity component system we register a component responsible for requesting the data and maintaining the selection now instead of depending on specific

document the panels can depend on a generic selection provider component now the only thing we need to do is to hook up the panels again because the selection provider is such commonly used component we made it simple to use we can just tell a panel the follow-up selection with the string id and it automatically starts loosing the selection provider and we tell the fx hierarchy to load the fbx data and use the given selection provider to set the selection the last problem we had to solve was tracking down the changes from multiple sources like from several different files or from database it ended up being really simple because with the context we already have a single point of entry to all the data and we can easily track all the changes without having to know any details about the data itself by using the observer but pattern by aggregating a single stream of all the changes happening in each different piece of content that have been registered to it we can easily hook up not only panels that track the changes

but additional systems as well for example we can catch each change to undo stack or live at this system just by listening that single stream so let's review the concepts one once more the document is used used to configure the ui part of the workflow while the context is used to read write and track the data used in the workflow so let's have a look at the final example our screenplay workflow that we created quite recently and which really starts to show the benefits of this design here we have the screenplay hierarchy and the screenplay editor the really cool thing here is that the data is actually stored in the database instead of on disk and the document context system abstracts it out so we can reduce the panels like the property panels as well as other systems like undo redo here the user wants to use the dialog lines on a cinematic so he uses an asset browser to open a timeline on disk and then he just tracks the dialog unit

on it and that automatically creates tracks for each character and new events for each dialog line the user also wants to make some modifications to the screenplay and this causes the corresponding event in the timeline editor to be highlighted because the duration changed user can now export the latest screenplay data and then re-time the event to match the source data and i think this really validates this design because there are two different workflows screenplay writing and building cinematics working almost seamlessly together and within the same ui with the same controls even though the data is stored and handled completely differently so here are the biggest benefits we've seen so far our productivity significantly increased it allowed other teams to work on their specific workflows and thus we've managed to add more new workflows to our dotnet tools during the past two years than we did during quantum break

context sensitivity significantly reduced the fumbling with the panels now that the panels are grouped inside the documents the state is clear and reusing the same panels allows to have better and more consistent ux as for the take away thinking the whole have a high level goal for your workflow high level design before diving the details and solving like problems one by one clearly separate different layers of the application so you can reuse test reuse and test the code and last and most importantly promote reusability and empower other teams i strongly believe that as a tools programmer the first and foremost responsibility should be to empower other people may that be either end users or other programmers thank you [Applause]

so questions thank you for your talk uh really interesting stuff i was wondering how long did you uh did your team work on the refactoring or just uh cutting out the mfc and just switching and how did you sell it to the management well we are still using the mfc application but this is just replacing the framework of the the net tools the loosely coupled one and it took about half a year about two people working full time to get like all the bits and pieces of the framework up and running and basically we just sold it by saying that we're spending now half a year and then we're gonna save like three years later on

hi um great talk you mentioned the selection provider i was curious can you elaborate more on how things connect to um these sorts of providers like how they know which provider they want to listen to yeah so like i said it's it's not really just the selection program it's kind of this component system that you can kind of have any kind of component there but in the case of selection provider we just made it work with with the same string id so it just you give a panel the string id and then you say that it should follow a selection and it just met just ask looks up a selection private component for from the document using that string id and automatic users there thank you

i guess nothing else thank you you