Emotional Neural Style Transfer: Expressing Character AI Emotion through Paintings

[Music] hi i'm edgar handy from squaring japan so today i would like to share you about our research on application of styles transfer for expressing character emotion um so for today's topic for today's talk we will divide it into four sections in the beginning i will talk about the overview about using painting to express ai emotion then we will jump into the rule-based emotional emotional mechanics that ultimately control how the neural style transfer behaves then finally we will talk about the machine learning system and how it is being implemented and this would be the the main beef of this the main dish of this talk so uh before jumping into the main section i would like to give a little bit introduction i'm a machine learning engineer working in ai divisions foreign

japan before i joined square enix i was involved in aiml researchers on cognitive developmental science and reinforcement learning on games so we just started a new division called ai division uh back in january and we are responsible for all researchers and square enix in within the scope of ai such as machine learning improving workflow for for game development and then reaching game contents and today's uh presentation and this whole project would not have been possible with the help of all people that are listed here so i would like to say thank you beforehand and now let's jump into the overview so first i would like to summarize the whole presentation in under one video this is a very short video and here i would like to introduce you to a character ai called pino that you see in the middle this is our experimental testbed for this experiment uh pinot is a character ai equipped with emotional system that you can see on the

panel on the left side pinot learns how to interact with it with his surroundings by interaction with the player you as a player control the vary that you see in near near pino's legs to encourage pinot uh making interaction with the world and you can give feedback and through these interactions you will see the emotion and mood system weights all over the place and finally pinot will show you what it feels by you by painting uh its surroundings and this is will be done with neural cell transport so let's take a look at how this works if you see here you can move around the theory and you know interact with the world you can give feedback through the fairies and this will affect uh the emotion and mood system as you see everything is growing over there and people in the idle time can decide uh hey i want to show you what i feel but rather than giving you those uh weird panels pinot will occasionally okay i would like to paint something that i see

we'll put it on canvas and we will start a machine learning system in the background to do style transfer and it will be the painting and this is the painting of the rubber ball that you see at the on the floor so we will get into more detail about this and the full version of the video later on so this summarizes everything about what we're going to do so now let's jump into the uh talk about paintings so um generally when we want to express character ai emotion in games uh it has been limited to these triad facial expression vocal insulation and body gesture and we are trying and in square enix we are trying to explore uh a new way to let our ai express their what they feel in many variations possible and we see painting as one of the options so paintings has been a natural medium for us human to express what we feel what we interpret what uh to our surroundings about our surroundings and we just we would like to apply the same

concept to the ai to give a new way for ai to interact with the player and these whole the system is implemented using neural style transfer which we will jump into the detail later on so before we talk about the machine learning system i will have to i would have to um explain you about the rule based system of the ai called emotion mechanics as i mentioned before pinot has a emote has an emotion that controls the style transfer emotion this emotion is defined inside this ai is defined by uh emotions and mood which is a short-term feeling and the long term feeling uh the talk in depth about this topic has been done by my colleague watching back in gdc 2019 if you have any interest in this please refer to his talk so we want to put everything together but we need a model first we need to mod that we need the model for the emotion

and we we applied foc modified occ model as you can see it's on the diagram on the right side occ model this is a modified version of this and um the what the ai feels is this is is affected by three aspects first is the consequence of an event that the all a observe whether whether they are pleased or displeased towards that whether the ai is approving or disapproving towers uh an action of the other agent or another ai or whether the ai likes or dislike an object and all this is all the emotion that is implemented in this game is which is within the red box that you see at the bottom of the diagram multiple of these emotions can be activated at the same time we will take a look at this mechanics later now we have to move we have to model the mode system for this we also use a modified system called modified path model that you can see in the diagram in the middle it consists of two axes

and it this we can divide this with the two-dimensional space into four quadrants the first is exuberant docile afraid and hostile where exuberance corresponds to all positive x's and afraid corresponds to all negative x's now we have in the we have the model for each for the individual system for emotion and moon for to get the final system we have to put everything together as you can see on the diagram on the right side we put all the implemented emotions inside the two-dimensional mode space and you can see a red dot in the middle that is the current mode value of the ai so the current value ar of the ai is very is dynamic it changes over time and it changes according to the activated in more activated emotion for example if the whole uh emotion at the top left corner of the space is activated then the uh the red dot will gradually move tower scope and if multiple of them activate at the same time then we will have to compute the resultant movement

so this mod value will be sent to the machine learning system later to define its final results now we have the emotion system done let's take a look at the neural style transfer now this is the machine learning uh topic which will be the main dish of today's talk i think most people here already know what neural style transfer is because it's very popular lately it's been going for years uh but please let me give introduction short introductions with just in case neural styles transfer is a machine learning technique for you to do style to do stylization on a given arbitrary content image as you can see on the left side is my dog you put them inside neutral network and you set the reference style that you want to use for example an abstract painting that you see at the bottom and the result is on the right side so for our use cases there are four types of implementations that we we do that we did the first one is basic style

transfer we on that which we only transfer one style you already saw that in the previous slide in the previous slide and then the next one is blending style transfer mask style transfer and color control style transfer we will jump into one into it one by one however due to the limited amount of time that we have today uh we have to skip the technical detail on the color control style transfer but i will get into the overview just a little bit blending style transfer as the name suggests it's a technique for you to blend multiple styles two or more into one content image as you can see in the gray box for ease of comparison i also uh put the basic style transfer inside the blue box so you can see what the difference what a difference when you do only one style transfer and the blending of the two styles but okay the example is only two styles but you can actually do more for mass style transfer um it's similar to the blending style strength for however we use a soft mask to define uh

to localize two style regions each in each region you can also do multiple blend of styles for example in our use case we only have two regions uh the bright region and dark region and you can see that the bright region corresponds to the blue style and the background the dark regions corresponds to the land of multiple of two styles so uh this finally this is color control style transfer which is the one that we will skip the detail for today um this is a technique where you do uh do style transfer however you can control the degree between the content image or the style image color as you can see at the bottom 100 correspond to basic style transfer zero percent correspond to uh style transfer however you would like to maintain you uh you maintain the um the content image color but if you notice you still retain the uh the style image uh touch so it looks like a painting of a dog

also you can also do blending with this however it would be the um it would be the outside it will be outside of today's topic to implement the style transfer uh we use a uh architecture based on the previous work called a learn representation representation for artistic style this architecture is extremely simple it consists of only a bunch of convulsions assembling and residual blocks to get into a little bit more detail inside we can see feature extraction layer down sampling layer of the feature maps then we do bottleneck bottom computations with residual blocks after that finally we up sample the feature maps back into the pixel space and you get the final results of the style transfer the stylization computation is mostly done inside the orange block called conditional instance normalization or i will call it sim zooming in the scene we see two operations normalization and i think transformation of the given feature maps

and this is done individually for each channel the iphone transform here is basically just a scaling and shifting operations with a learnable fm parameters and all these learnable even parameters are unique for each style so we have to learn them one by one now in order for you to blend the styles assume that we have two styles that that corresponds to two even parameters gamma one beta two gamma one beta one gamma two and beta two if you want to have seventy percent thirty percent of the blend of the styles then what we do is just compute the context combination of them by weighting uh each each um its parameters and then sum them over and you will get the new gamma and beta which corresponds to the blend of the seventy percent and thirty percent of the styles now in order so in order for you to do the mask style transfer um we

we have to we have to upgrade the sim module into something called mask conditional instance normalization now this is almost exactly the same as before however we take an additional input which is a soft mask to define the style localizations area zooming in to get into the detail mass conditional instance normalization consists again of two operations normalized and i think transformed but this time with the mask inside there we give the soft region mass as you can see on the right diagram what we do here first is we divide the feature maps on each channel into two regions so we define the region region one as all feature maps pixels that correspond to the mask value greater than 0.5 and the rest of them into the region two after that we do normalization for each region we do not mix them and finally we do everything transformation with a new parameter with

new parameters called mass shift and mass scale parameters and we i will have to remind you that because each region has its own style we have a thin parameters that is unique for each region now there's a little bit more detail um if you're an artist you would like to okay i don't want to look at this feel free to go to sleep a little bit and i will go a little bit to the more engineering aspect of this one um in order for we to do fm transformation on each feature map because it's a 2d pen source we can imagine that each parameter gamma 1 and gamma 2 um is also it's also a tensor gamma 1 is a i think trans is an f in parameter correspond to region 1 gamma 2 is an athlete parameters correspond to the region two now in order to make in order to blend this into a localization into a stylized localization so it's a localized stylization uh we first use the original mask to do

element-wise multiplication into the uh tensor of the scale of the scale parameter then we create an important mass do and then we again we do element-wise multiplication to the uh fn parameter of the second region and then we sum them up the final result is the mass scale parameter that we can apply to the whole feature maps to get the mask stylization so if you if you realize this is basically a complex combination as we did before for spiral blending but this time we are doing a blending but for uh special blending it is the uh stylized the style localization and finally we also do the same thing for shift parameters basically we define again original mask and inverted mask it's element wise multiplication we sum them together to get the final mask c parameters so we follow the same training setup as the previous paper to model uh to train the

model so uh if you would like to know the detail how to train this model please refer to the original paper because it will give you all the awesome stuff to do this it's very it's extremely simple um finally the mod well the model is optimizing vgg 16 perceptual loss which is a very common optimization strategy for style transfer and we train all of this inside tensorflow now we already know the emote the rule based emotion module we have the neural style transfer if we combine all of this you can still get the diagram in the middle you have emotion module that outputs the mood value where we can compute the style which or what we call the style of in parameters to compute the either the style localization or the blend of this multiple styles and then we send all those into the style transfer module and you get the final results on the right side now we have uh we have uh we have explained the

the motion module and the start transfer module we get everything so we would like to take a look at the again at the video uh the full version of the video of the previous video to look at the more detail on what's happening as before i said before uh pinot is trying to learn how to interact with the world us sofia you who control the theory can encourage whatever you can do so as you can see here people just hit the ball and it is a bad thing to do we can give a feedback to filipino that's a bad thing this will in the background affect pinot's behavior towards that goal later but that's the out of the scope of today's topic and according to and according to what he knows doing and you the user interaction that you give through the theory it will affect what you know fails and you see the mood system down there it's it's waiting over time [Music] now in the idle time kino

would like to show you what it feels rather than showing you the um the panel that you saw before you know we'll show it by painting what it sees for example the very rubber ball that they just ate before and we do and it it is implemented using the style transfer that we saw we saw before where the penalty emotional system affecting the final results of the appropriate style being applied to the canvas so just like any other experiments there are something that went bad and when when good uh for this uh for this research however there are a little bit difficult difficulty on this one because r is a subjective matter um it's really hard to say whether which which style is actually being represented properly inside our neural network for example for me i think the top uh the top pictures are good and the bottom pictures are bad but that's not really the case for other people so um in the

end we have to choose the appropriate we have to choose the appropriate style for each emotion pass it in the neurals test it inside the neural network see if it's if it's bad then we will throw it we will throw the style if it's good then we will adopt the style so it's really hard to find um like a set of hyper parameters that can grant you the best uh the best representation for each for all the style that you want to use now we have them we have to explain the machine learning system uh we would like to see how it's actually is being implemented inside the game the game that you saw before is implemented inside ue4 and uh since it's before we have to implement the style transfer using c plus first we have to construct the style transfer pipeline as you can see in the diagram here they start from the camera view which corresponds to what the ai sees they come from the camera view you get the texture that will be convertible to pixels and you can also do the objection

uh you can also do objective section to generate the soft mask for emotion module as we have seen before we can compute the style weights get all these three things send off the middle style transfer module and you get the final stylization results finally we apply back the results into texture and you see what you see what was drawn on the canvas execute this system we have two choices obviously whether we want to use cp or gpu so we have to wait which one is better for uh this problem first let's take a look at gpu when we're trying to implement this e4 does not have um gp inference support for tensorflow model so logically thinking we can try to tensor we can try to implement tensorflow gpu inside ev4 however due to the complexities of the gpu library of tensorflow it is way too expensive to do and we also try on the next runtime that one is easier to be implemented for gpu

inference however it's not what it is not portable to consoles due to the lack of cuda support um we can also try to write a gpgp core share code for a console but we find that it's way too expensive for such a small experiment and finally even though we can do everything that we want with the gpu and consoles uh for our use case the gpu budget would be prioritized for graphics and uh it's not really a good idea for us to use the gpu and into the into the table and when we are looking at the cpu uh we try to implement the tensorflow lite so tensorflow lite is a sub library of tensorflow uh that you can do cpu inference it was actually dedicated for microcontrollers and an arm architecture but yeah you can we can try to adjust the source codes inside ev4 to do that so and tensorflow light is also very small so there are

only fewer source sources to be adjusted such that it can compile nicely cpu even though people say it's it's slow uh it's fast enough for a use case because we can put animations to buffer the computation time um we can also and then by using cpu we there's no resource conflict with the gpu to do graphics rendering and finally which uh we can also port them into console for x86 cc for an american arm architecture which is something that we really want to do so finally we cross out the gpu and we go with the cpu options now to fit everything inside ue4 we have to take we have to take necessary source codes from tensorflow lite library put them inside ue4 as a plugins however this is not going to compile well so what we have to do is first adjust a lot of source codes try and error trying to make sure that e4 and the tensorflow lite uh would compile together nicely after days of experiment on trying out this we

finally get everything running well and finally we have a nice plug-in that where we can do the api call from the game quotes to read the uh the tensorflow format model and do the cpu inference and after all the spins for laughing around a week we finally get this this is the in-game ui of the style transfer on the left side you can see the camera view of the ai that's what pino is looking at and the corresponding mask after we do the objective section the final result of the child transfer is on the far right side and the blend is shown in the middle where you can see the top panel corresponds to the style implemented at the background it only consists of 100 of one style and the bottom panel corresponds to the styles the blend of styles that is applied to the box and for this experiment um this uh this style corresponds to the

emotional uh to the mood value that that is shown in the middle diagram so i would like to emphasize here is that this is a testbed where we can try multiple styles so uh for the one that we are looking where we are showing you right now is just like it's just an example of how we can try to blend multiple styles by you by playing around inside this mood space so in the in at the end of the day we will still have to uh choose which style is better for the uh to express the appropriate emotions of the ai so to summarize today's talk um we explore uh paintings with style transfer as a new way to express ai emotions the ai emotion uh the ai emotions itself is modeled using modified occ and pad model the appropriate styles of the style transfer is selected according to the mode value of the ai and we also introduce four basic uh four use cases that we did in this experiment which is basic style transfer blending

mask and color control style transfer finally we implemented the whole system use insight ue4 using c plus but interface from tf live for cpu inference so before we close the presentation for today i would like to share you one i would call a weird experiment that we did we tried something uh something called g buffer style transfer and this is inside ue4 these style transfer leverage uh leverage is the game engine g buffer and we do start transfer into the each component of the g buffer before the final image rendering so let's take a look at the basic struct transfer at the top panel you see in the middle that's the ue4 sim and if we take a picture of that and we do basic style transfer as usual treating it as an image you get the results on the right side but if you look at the bottom where we do the g buffer cell transfer that's the current that's the same uefa scene however what we do is that we access the g buffer components we apply that correspond the yellow styles into each

of its into some of the g buffer components and what you get is the result on the bottom right corner which looks really fascinating so thank you very much for coming to my sessions uh if you have any extra questions later please feel free to contact me by either email twitter or linkedin i also tweet a lot about machine learning that will be useful for game development if you're interested please check it out and thank you very much for the for coming to the session [Music] you