Sunday, October 27, 2019

Dungeon Siege - high resolution textures (ESRGAN)


I spent several days upscaling Dungeon Siege textures using AI image upscaling ESRGAN. The initial plan was to just do Castle Ehb, but after I got great results on grass textures I got carried away. The end result is a texture pack that is 900 MB in size and contains 500 textures. Since the algorithm increases width and height of texture by 4, this means 16 times increase in surface and file size. The areas I worked on are Farmlands, Stonebridge, Glacern (only buildings and items) and upper parts of Castle Ehb. A lot of time went into this, since there was a lot of experimentation at the beginning and a lot of bottlenecks slowing the whole process down. After all the work it is still just around 15-20%(if not less) of the textures that have been enhanced.

You can see the final results in game in the video below (watch in fullscreen at 1080p to see the difference more clearly) or you can see direct comparison of textures in the next section. You can download the texture pack from ModDB.



***

HOW IT WAS MADE

Dungeon Siege modding tools can be found here:
https://sites.google.com/view/chickengeorgemods/modding

I used this tutorial to setup ESRGAN:
http://alphagarg.blogspot.com/2019/01/esrgan-neural-network-ai-for-upscaling.html 

Various upscaling models can be found here:
https://upscale.wiki/wiki/Model_Database

The vast majority of textures were upscaled using the MISC model. This model gives great results on wood, stone, bricks, grass mixed with dirt, etc... Below are several direct comparisons of the textures themselves. In top-left corner is the original texture in its original size, they are 128x128 or 256x256 pixels. Then left is that texture upscaled 4 times using linear filtering like in the game and on right is the new ESRGAN texture.

Example of MISC model working well on complex textures containing ground, wood and grass.

MISC model gives wood textures a lot of new fine detail.

Stone and bricks get their usual roughness of the surface (MISC).

MISC model sometimes gets carried away. Some stone wall textures become too rough and grainy and outlines of individual rocks become less defined. That is why I sometimes used the MANGA109 model as well. This model gives "cartoony" results and is not good for realistic textures, but its advantage is that it enhances the lines. So what I would do is; take the MANGA109 result, increase the brightness and contrast by just few percent to further enhance the lines, then reduce the transparency/opacity in GIMP to about 60% and copy that texture over the MISC result. This results in more defined lines and also reduces the roughness of the original MISC result. This was used on some rock and brick walls and some wood floor textures.

 
MANGA109(left), MISC(right) and final mixed version in the center.

The MANGA109 model gives great results on various carpets. The problem with this model is that it was trained on JPEG images, so it can produce some artifacts and noise, but this noise is actually good for carpets, since they are not a smooth surface to begin with.

MANGA109 model was great for carpets.


UPSCALING FAILURES

The quality of results depend on the quality of the upscaling model used and the size and quality of the texture itself. There is a lack of specialized upscaling models. By specialized I mean models trained on specific texture types like wood, stone walls, grass, leaves, ground, carpet, old architecture, windows, etc... This will probably improve over time when new upscaling models get made.

Many of the textures in Dungeon Siege are very low resolution, which means the scaling algorithm does not have enough information to recognize specific patterns so it can enhance them and add new detail, so it just creates a mess. There is nothing that can be done about it, unless an artists makes new textures. For example Dungeon Siege maps are full with various vegetation, but no model gave good results for those.

This carpet texture is too low resolution and upscaling gives poor result (MANGA109).

The algorithm fails to recognize the rock surface under the grass and creates a mess (MISC).

GROUND model creates good grass, but also turns ground/dirt to grass. 

Swamp textures lack any clear detail, so the upscaling gives poor results.


MANUAL ENHANCEMENTS

Sometimes you need to help the algorithm a bit. When the result is too grainy and rough you can smoothen it out a bit by denoising the original texture in GIMP before the upscaling process. If the results are a bit blurry then adding a bit of HSV noise to those areas in the original texture can help sharpen those areas in final upscaled texture. This works best with wood or stone textures. In some cases (for example castle marble textures) I had to manually straighten out some lines on the original texture and also repeat the uspcaling process on a downscaled upscaled texture and also straighten the lines manually again. Sometimes you also have to be creative and use elements from another texture to enhance a different one.

Fixing texture by adding straw manually.

Algorithm made the face on this texture unrecognizable, so I used a face from another texture.


BOTTLENECKS

There are many things slowing down the whole process:
- Since I don't have a Nvidia GPU I have to do the upscaling on my CPU which is a very slow process. For a 128x128 texture it takes around 50 seconds and for a 256x256 more than 3 minutes on a i5-4690k.
- Original textures are in a custom RAW format and only available converters are from RAW to BMP and PSD, and from PSD back to RAW. The ESRGAN upscaling program does not work with PSD. So I need to convert original RAW files to BMP, then upscaled results from PNG to PSD, and finally PSD to RAW.
- Another problem is that textures have an alpha channel which is lost during the BMP conversion. So for each upscaled texture I have to manually add an alpha channel. For textures that are partially transparent, like windows or spider web, I also have to extract the alpha channel, upscale it separately and then add it back in.
- The things already mentioned under "Upscaling failures" and "Manual enhancement".

FUTURE UPDATES

I don't know if I will upscale the whole game, after all this work I need a break from this. It takes too much time and some textures give poor results like forest, jungle and swamp floor textures. IMO for those I could resize the final results by half, so the flaws are less noticeable and textures would be at least a bit sharper. I think good results could be achieved for snow and desert terrain textures. I think icy caverns don't need to be upscaled, since ice is supposed to be a bit blurry. Some dungeons, like Wesrin Cross, also have very poor textures that lack detail so the algorithm doesn't really do much good.

Monday, September 2, 2019

Fixing a GPU by baking it in an oven - It actually works !

Back in May I was starting to have issues with my GPU. The monitor would suddenly turn black and few seconds later the computer would freeze. It would happen very rarely and after reconnecting the GPU and the cables everything would be fine. I thought it was some loose contact somewhere. But then one day after it happened I couldn't boot into Windows and in BIOS the screen was full of artifacts. My i5-4690k has an integrated GPU, so I could easily confirm it was my GPU, a Sapphire Dual-X R9 270X OC (2 GB) from 2013. It is an old card, but still good enough for me. This happened while I was playing Claw, a 2D platformer from 1997... not a very honorable death for a 2013 card.

I was already looking at the second-hand market and contacted few people to buy a used RX 570, when I read about people baking their GPUs in an oven and fixing it. Since the GPU is 6 years old, out of warranty and obviously broken, and I have an old oven in the garage that is barely ever used, I decided to give it a chance.

I preheated the oven to 190°C, put the GPU board inside and baked it for 8 minutes. Then I opened the oven slowly and let it cool down inside. People warned about being careful not to cool down the GPU too fast, because it could cause cracks in the solder. Also be sure to open all doors and windows, because it will smell of solder.

Since I didn't have thermal paste, for a quick test I put it in the computer without the cooler, I just put a case fan to cool it and was monitoring the temperatures. To my big surprise the artifacts were gone and I could boot to Windows again. Then I started Furmark and it would crash right at the start, even though the temperatures were still low. Luckily after getting thermal paste and mounting the cooler it worked fine. My guess is there is still some bad contact between the GPU chip and the board, but by mounting the cooler and adding pressure the contact is now stable.

It has been three months already and it still works. I have played and finished few games and even ran some benchmarks. Only once did I get that black screen, about a month ago, but it didn't repeat again.

Friday, October 12, 2018

1 000 000 views on YouTube !

About a month or two ago my YouTube channel reached 1 000 000 views ! When I uploaded my first few videos in 2010 I remember being excited reaching 100 views. I guess if I enabled monetisation I might have made a bit of money, but that was never the goal, I just wanted to share what I was working on.

My YouTube channel

Around 80% of those views are from The BloodCrafter video, a Minecraft joke made in 2011 that often went viral over the years. I made that thing in one afternoon, while other projects like Serious Sam 2D and Quake 2D, which took months to finish, have far less views, but still a respectable number.

To celebrate these 1 000 000 views I have prepared... nothing. Even this blog post took me few months to finally write. Everything is in a vegetative state. The Broken Mug Engine and the Quake 2D remake in that engine are not legally dead, but progress is extremely slow and it will never be as polished as I would like it to be. I fix a thing or two every few months when I'm bored of doing other things. The original campaign can be finished, but there are bugs all over the place, mostly because all new features I would make, I would leave unfinished and the project grew to a size that is very hard for me to manage. Also it doesn't help that the forums where I have been posting updates about these projects over the years have been shutting down left and right, I will have no one to share this with, but strangers.

Sunday, May 14, 2017

Chrome and Chrome SpecForce - Visual Fidelity (unofficial patch)

Another year, another patched old game(s). This time it is Chrome and the expansion/prequel Chrome SpecForce. All I wanted to do is play in 1080p with a FOV that would not give me headaches. Then I discovered Java source code that came with the games. Fix here, fix there, and is basically a whole patch that should be usable to others too.

The goal of this "patch" is to enable the games to be played in widescreen resolutions, ultra widescreen, multi monitor setups and in 4K and above by fixing text rendering and FOV issues and properly scaling some HUD elements. Another goal was to increase the visual fidelity of the game by forcing the game to render highest quality assets even when they are far away from the player. Only assets that will actually be changed are the HUD map textures. No gameplay changes have been made. Jackfuste from WSGF helped fix all the FOV issues and I used HUD map textures for Chrome from Chrome Widescreen Mod(Chrome HD Fix).

I have uploaded it to ModDB and there are more details and a list of changes there and in the included change log file. I wont post it all again here. Detailed instructions are also included.

md5:d4acfc676f0308ef65cea2ce18647199

Here is a short video comparison:


Here are some before/after screenshots (move the slider):

Saturday, March 11, 2017

Skeletal animation editor - ghosting

I added ghosting to my animation editor, so it is much easier to make animations and adjust individual frames, since I can see previous(red) or next(blue) frame of animation too:


I can also render and preview all frames at once:


Monday, March 6, 2017

Simple animation interpolation

I added simple animation frame interpolation, with basically single line of code, to make the animations smoother:

current_angle = last_frame_angle + (time_elapsed_since_last_frame/frame_time)*(next_frame_angle-last_frame_angle)

Left is the original animation, right is with interpolation between frames, both slowed down:

Monday, December 19, 2016

Box2D ninja rope


I recently played a game called BEEP which had a grappling hook mechanic. The game used Box2D, so I wanted to figure out how they did it. After figuring out a combination of revolute and prismatic joints is the best solution, I also wanted to make a ninja rope like in Worms games, where you can move up/down the rope, stand on the rope like it is a pole and the rope wraps around the terrain. I didn't manage to make it perfect, but it kind of works.


Swinging

1. Rope joint method

The easiest method is to just create a new rope joint connecting the player and terrain body and Box2D will handle the rest. There are two problems with this method:

a) Since it is a rope nothing is stopping the player to move closer to the point where rope connects to the terrain. The rope just stops the player from moving further away than what is defined with joint's maximum length (SetMaxLength()) variable. This makes it impossible to stand on the rope.
b) The player can't move up or down along the rope. The joint's maximum length can be changed after it is created, but that will either; try to move the player "by force" upwards if it is changed to lower value even if something is blocking the way(might even just teleport it instantly), or the player will just fall down a bit if the rope is changed to be longer.

2. Distance joint method

Alternatively a distance joint can be used instead of a rope joint. The difference is that the player body will not be able to move closer to where rope connects to the terrain or fall down, since the joint always tries to maintain the same distance. This is actually closer to the ninja rope behavior in Worms than the rope joint option and you can stand on the rope using this method. SetLength() can be used to "move" the player along the rope, but similar to the rope joint it tries to push the player to new position even if there is something in the way.

3. Chain method

This is also a simple method, but requires a lot more work. Simply create many small bodies between the player and the terrain body and connect everything with revolute joints. The problems:

a) The rope is a bit hard to control, moves around too much and can stretch. In games like Worms or Bionic Commando the rope isn't behaving very realistically, more important was giving the player intuitive control over it.
b) You can't move up/down the rope or stand on it.

The positive thing about this method is that the rope can wrap around terrain without any additional coding.



Tips:
1) For some reason joints are more stable if the bodies connected have higher density value, but the downside is that heavier rope is harder to control.
2) Enable joint motors and set torque to very small value and set the speed to zero, so the chain stops swinging sooner.
3) Connect the terrain and the player with a rope joint to prevent the chain from stretching too much. The max length should be same as the length of the entire chain or just a tiny a bit longer, depending how much do you want to allow it to stretch.
4) Since all the chain bodies would by default have zero linear and angular velocity when created, it would slow down the moving player when initially created. I don't know how to calculate the initial values of each link based on player velocity and stuff, but simply giving all the bodies the same velocity as the player at the moment of creation works good enough.

4. Prismatic and revolute joint method

The best way to make Worms like ninja rope using Box2D is using a prismatic joint on a rotating body:
a) With a prismatic joint the body is limited to moving/sliding along an axis and that is exactly what we need.
b) Using the joint's motor we can make the attached body move up/down without breaking the physics simulation and if some object is in the way and blocking the movement it will behave normally (unless we set the motor torque to an insanely huge value).
c) Using a motor with a high torque we can fix the players position on the rope, so the player can "stand" on the rope, just like in Worms games.

The rope will have three parts:

1. Rope/hook (long thin triangle) - a body which is connected to terrain using a revolute joint, so the whole thing can swing.
2. Slider (rectangle) - a body connected to the hook using a prismatic joint, so the player can move up/down the rope using the joint's motor.
3. Rotor (yellow circle) - Usually in Box2D games the player body has a fixed rotation, so it always stays upwards. We can't connect it directly to the slider, because with fixed rotation it wont allow connected bodies to rotate freely. There needs to be a body between the player and slider connected to both with a revolute joint, so the player can maintain the fixed rotation, but the rope can still freely rotate. If you have a player that can rotate freely, this isn't needed and can be attached directly.



Wrap around terrain

It is important to note that only one segment of ninja rope is fully active at any given moment. Basically the swinging part will work as described above in the method 4, and the rest of the rope are just static points with a line rendered between them.




a) How to detect when a corner is hit and split the rope ?

One method is to use the rope/hook part from method 4. It needs to be very long and thin, and needs to be a bullet body. When bullet bodies collide in Box2D you get the exact point of impact in PreSolve() callback function which you can use to determine where the rope needs to be split.

- The rope needs to collide with the terrain, but shouldn't bounce of the terrain, we just need the collision point. So in the PreSolve() callback the contact needs to be disabled with contact->SetEnabled(false).
- The rope should not be connected directly on the collision point, because we don't want the tip of the rope/hook to constantly collide around that point while rotating and cause problems. The connection point needs to be a bit outside of the terrain body (see image above).

When existing rope collides all the rope objects are deleted and the rope is recreated in that collision point. Like mentioned previously; only the last part of the rope is active. The previous connection points are still needed to render the entire rope and to know when to reconnect the parts.

The downside of using this method is that when the player is moved all the way up the rope, you have all this extra mass bellow him (the player in not the center of mass) that can cause unwanted movement.
The solution is to make the rope/hook body very small, so the center of mass is always the player, and use some other method to find out where the rope hits the terrain.

b) How to reconnect it all again ?

Simple; if you swing clockwise and hit something and split the rope, you will have to reconnect only when you swing in the opposite direction and past that point. Use the revolute joint's speed to check the direction(negative is clockwise, positive is counter-clockwise), and you can check if you past that point using a cross product formula:

http://stackoverflow.com/questions/1560492/how-to-tell-whether-a-point-is-to-the-right-or-left-side-of-a-line

Two last rope points is the line you need to cross, and the point you check is the current player position. If you crossed the line you remove the last rope point and recreate the rope in the last of the remaining collision points. While doing all this splitting and reconnecting you need to keep in mind the total length of the rope, and change the rope/hook body length accordingly.

 
You can try it out for yourself using the link above. I also uploaded the source code for the rope handling. Not sure how useful it will be, since I just copied it from my game/engine, so don't expect to just copy/paste it into your code and think it will work. Think of it more as a sign of goodwill. Don't forget to read the included README.

Sunday, October 23, 2016

Random #7 - attack of the view bots

This blog seems to be under "attack" by bots or something. For the last month I'm getting ten times more daily views than the average was. It started on September 30th. Every few hours the statistics shows a spike of 30 new views, and every day the same thing. The views are coming from USA, operating system is Macintosh and the browser is Chrome, but it shows no referring site or anything to pinpoint the source. It has made the statistics data completely useless.