Merge pull request #212 from goldenx86/oct20

yuzu progress report October 2020

site/content/entry/yuzu-progress-report-oct-2020/index.md

@@ -0,0 +1,205 @@
++++
+date = "2020-10-25T12:00:00-03:00"
+title = "Progress Report October 2020"
+author = "Honghoa"
+coauthor = "GoldenX86"
+forum = 324017
++++ 
+
+How's it going, yuz-ers? Here work continues. This month's report offers you mythological hammers dropping, Vulkan and OpenGL stability fixes, and some news on the continued fight to make Super Mario 3D All-Stars render.
+
+<!--more-->
+
+## Project Mjölnir
+
+### Part 3 - Smite as hard as you desire
+
+[Morph](https://github.com/Morph1984) strikes again with [part 3 of `Project Mjölnir`](https://github.com/yuzu-emu/yuzu/pull/4866) - a major rewrite of yuzu's `Human Interface Device (HID)`. This concludes months of hard work that also involved the contributions of [Rei](https://github.com/its-rei) (who designed the new user interface) and [jroweboy](https://github.com/jroweboy) (who started the original HID rewrite that later Morph continued).
+
+Just as [three different gifts](https://en.wikipedia.org/wiki/Mj%C3%B6lnir#Origins_in_the_Prose_Edda) were given to the Nordic gods by the dwarves (including the iconic hammer of Thor), this project was divided into three parts focused on different aspects of emulating the input control of the Nintendo Switch.
+
+The first part was centered on overhauling the user interface, as well as modifying and fixing errors in the backend that handled the mapping and detection of controllers. These changes allowed yuzu's implementation of input to behave more closely to the hardware, improved the user experience, and also paved the way for the subsequent improvements in the following PRs.
+
+The second part took care of implementing the `Controller Applet`: a service used by games to communicate and set up the kind of controller input they can accept, how many players will use them, what character they will control, etc.
+
+And now, at last, comes the third part of this project: introducing bug fixes, `Controller Profiles`, and an overhaul to the implementation of `Rumble` realized by [german77](https://github.com/german77).
+
+{{< imgs
+	"./load_profile.mp4| Loading a profile for a Sony DS4 controller"
+  >}}
+
+`Controller Profiles` had been a placeholder since Part 1 of `Project Mjölnir`. They allow users to set up the configuration of their gamepads and save them in a file, which can be loaded and marked with a distinctive name. This is particularly useful for people who own more than one type of controller, since they store all button and axis information, as well as the port and engine used to read the input data. Changing between one configuration and another becomes as simple as clicking a dropdown menu and choosing the desired profile, and all the settings will then be applied automatically. Users now can create a profile by simply clicking on the "New" button and writing a descriptive name to save it.
+
+[Modifier Buttons](https://github.com/yuzu-emu/yuzu/pull/4809) was also another feature that wasn't working well since Part 1. A `Modifier Button` allows the user to change the range of an axis on the fly by pressing it, which is particularly useful when said axis is mapped to digital buttons instead of an analog stick. This is because digital buttons are interpreted as a "virtual axis" that is being pushed from the minimum value (0%) to its maximum value (100%) every time they are pressed. With this change, it is now possible to set the maximum value of this axis to a value lower than 100%, effectively limiting its range. Useful for walking or sneaking!
+
+Next on the list of fixed bugs, we had a [controller reconnection issue](https://github.com/yuzu-emu/yuzu/pull/4816). Sometimes, a controller instance was being pushed back into memory right after being disconnected. This faulty behaviour was a bug that prevented the controller from being recognized by yuzu when it was reconnected by the user again. [Morph](https://github.com/Morph1984) implemented a simple guard to avoid this and solved the problem.
+
+For some games, vibrations didn't work properly, or at all. To understand why this was happening, we will need to explain a bit of what’s going on behind the curtains of abstraction, so please bear with us here.
+
+`Rumble` in the Nintendo Switch is physically generated by a pair of identical [Linear Resonant Actuator motors](https://en.wikipedia.org/wiki/Linear_actuator) that can emit a wide range of frequencies on the low and high spectrum, independently from each other. This contrasts with the approach used in other controllers, which typically have low frequency [Eccentric Rotating Mass (ERM) motors](https://en.wikipedia.org/wiki/Rotating_unbalance) at the left side and a high frequency ERM at the right side; depending on the desired frequency band at which one wants to make those controllers vibrate, it is possible to send signals to one or the other and thus accomplish low and high frequency rumble. The Nintendo Switch also characterizes vibration in such a way that it's possible for games to distinguish between the left and right motors of different players. This is because each device has an identifier handler that is used to send the correct vibration commands to the right place.
+
+Prior to this PR, the functions that processed this handler information weren’t coded yet, or they were stubbed and returned incorrect information. Even with those limitations, this exceeded the capabilities of the [SDL library](https://www.libsdl.org/) (an [API](https://en.wikipedia.org/wiki/API) that yuzu uses to communicate with the hardware of the input devices), which was designed to work with generic gamepads that do not present any of these unique features of the Nintendo Switch. As a result, the rumble wasn't being processed correctly. This resulted in inaccurate behaviour, such as the lack of vibration for the controllers of players other than player one, since this handler information wasn't available for SDL to distinguish where to send the vibration signals other than for "controller 0". In order to make vibrations work as intended, [Morph](https://github.com/Morph1984) investigated how this information was characterized and implemented the functions to process these handlers, so that the vibration data could be sent to the appropriate devices and players. The addition of these functions also fixed a number of games, such as `Xenoblade Chronicles: Definitive Edition`, `Xenoblade Chronicles 2`, `Hollow Knight`, and many others that previously didn't exhibit rumble at all.
+
+Previously, some people using certain types of wireless controllers reported in-game performance losses whenever their gamepads rumbled. The root of this issue lied in the rate at which the vibration states were being updated to these controllers, so [german77](https://github.com/german77) implemented a solution to reduce the rate to 50 Hz and filter out some of the vibrations. But it was later reported that some vibration effects were missing in games such as `Super Smash Bros. Ultimate` due to this filter, so an alternative solution was needed. For this reason, [Morph](https://github.com/Morph1984) implemented `Accurate Vibrations`, a toggle option (off by default) that switches between a full implementation and one with the effective vibration rate reduced through heuristics. If this option is disabled, the vibration rate is limited to 100 vibrations per second, excluding vibrations with an amplitude value of 0 (which is considered as the "not vibrating" state). This rate was determined to be safe through testing done by [Morph](https://github.com/Morph1984) and [Ren](https://github.com/renA21).
+
+{{< imgs
+	"./configure_vibration.png| Vibration Configuration Window"
+  >}}
+
+These changes were enough to fix the stutters experienced by these users. In order to toggle `Accurate Vibrations` on, the user should navigate to `Emulation > Configure... > Controls` and click on "Configure" under the "Vibrations" tickbox. They can also change the intensity of the vibrations for each player, if desired.
+
+Furthermore, an additional check has been implemented when initializing a vibration device to determine if a controller is apt to rumble or not. It consists in sending vibrations with an amplitude value of 0 and reading the value sent back from the controller. While this approach works fine for the GameCube Adapter, SDL returns an incorrect value if two vibration signals with identical amplitudes are sent to the device. To work around this problem, yuzu first sends a vibration with an amplitude of 1/65535, then one with amplitude 0, and then reads the value returned from the device. This check is extremely important, because it helps in implementing the necessary functions that tell the games if the vibration device is properly mounted and capable of rumble, and it also makes it possible for yuzu to disable rumble capabilities in the backend automatically if a controller doesn't support it.
+
+On top of these changes, work has been done so that anyone can plug in their favorite controller and, if it supports vibrations, all configuration related to it will be applied automatically. In the case that the controller doesn’t, however, rumble will be disabled on the backend.
+
+{{< imgs
+	"./mapping.png| Manually mapping an analog stick"
+  >}}
+
+It is worth noting that, in case the automatic configuration process fails, users can map their analog sticks manually by clicking on any of the axis mappings. This solves issues with certain controllers that have their axis mapped as digital buttons.
+
+`Project Mjölnir` has come a long way since it started. Major parts of yuzu’s original code for input were rewritten, tested, and ironed out. Vast parts of the input frontend and backend were overhauled in favour of a more user friendly interface, aiming to make it easy to configure your controllers in the most intuitive way (and many of these configurations are also being applied automatically). Much of the effort done by the devs is easily perceivable, but there’s also a lot of work that went into making things run perfectly behind the scenes. Hours of investigation, lining up the details, and polishing every addition. Indeed, our devs gave it their best, just like the dwarves in the Eddas, to present you with this gift. The Nordic gods debated which among the three presents they received was the best, and they concluded the hammer was, without any doubts, the most precious of them. It is our most sincere wish that you will accept this hammer from us, and smite as hard as you desire!
+
+As a small sidenote: In the Edda, the dwarves happened to ask for Loki’s head as compensation for the gifts. But don’t worry, we won’t go that far with you! *(Yet)*.
+
+## Back to the Pokéfuture
+
+A highly requested fix, and understandably so. Pokémon has long been a franchise that uses time as a game mechanic. In yuzu, this feature was not working as intended with `Sword and Shield`. What would normally be saving, closing yuzu, and loading in later expecting time to naturally pass, ends up as if nothing happened, rendering several events (like berry respawns, changing weather, Pokéjobs, different Pokémon spawns, or just simple time of day changing) useless. We don't have a proper fix to naturally progress the time yet, but for now, thanks to the work done by [bunnei](https://github.com/bunnei), it is now possible to [bypass this problem by changing the clock from yuzu's settings during gameplay.](https://github.com/yuzu-emu/yuzu/pull/4792)
+
+You can find the setting in `Emulation > Configure... > System`, enable `Custom RTC`, and then enjoy time travelling!
+
+{{< imgs
+    "./rtc.png| Try to avoid changing the divergence value, Doc!"
+  >}}
+
+## [Greetings, Professor!](https://www.youtube.com/watch?v=xr-aIARiea4) Fixes to report!
+
+[Rodrigo](https://github.com/ReinUsesLisp) has for us a couple of stability improvements for `Fire Emblem: Three Houses`, only involving Vulkan and assembly shaders (GLASM) for now.
+
+On the Vulkan side, [managing primitive topologies as fixed state](https://github.com/yuzu-emu/yuzu/pull/4782) fixes regressions introduced by recent Nvidia drivers, but it has been reported to also help AMD Radeon users quite a bit too.
+
+On the GLASM side, and benefitting our Nvidia users the most, [implementing robust buffer operations](https://github.com/yuzu-emu/yuzu/pull/4807) like it is done in GLSL helps make the game playable when using this very useful shader implementation.
+
+{{< imgs
+    "./fe3h.png| I have something to ask of you (Fire Emblem: Three Houses)"
+  >}}
+  
+Crashes will still be experienced in specific chapters of the game, like Ch. 15 and Ch. 17. For those cases, we can offer you a workaround:
+
+1. You need to download one of the following old builds: [yuzu 66](https://github.com/yuzu-emu/yuzu-mainline/releases/tag/mainline-0-66) or [yuzu 41](https://github.com/yuzu-emu/yuzu-mainline/releases/tag/mainline-0-41). 
+2. Set it up as a standalone version by extracting it and creating a `user` folder where `yuzu.exe` is located.
+3.  This folder will replace `%appdata%\yuzu`, so you need to extract a copy of your keys there, along with any save data you want to transfer to get past the crashes. Make sure to also copy your `nand` folder so you can transfer all DLC and updates to the standalone yuzu build.
+4. Play the game, expect tons of glitches and pretty bad performance for such an old build. Save after progressing past the crashing area and transfer that save to the latest `Mainline/Early Access` version you are using, and continue playing regularly from there.
+
+## Emuception
+
+[bunnei](https://github.com/bunnei) seems to have a personal vendetta with `Super Mario 3D All-Stars`. He has:
+
+- [Implemented the nvhost_as_gpu::FreeSpace service.](https://github.com/yuzu-emu/yuzu/pull/4783) 
+- [Implemented the BufferQueue::CancelBuffer service.](https://github.com/yuzu-emu/yuzu/pull/4784)
+- [Stubbed IManagerForApplication::StoreOpenContext.](https://github.com/yuzu-emu/yuzu/pull/4793)
+- [Reworked the program loader to better handle multiple processes.](https://github.com/yuzu-emu/yuzu/pull/4799)
+- [Fixed crashes when trying to launch subsequent games.](https://github.com/yuzu-emu/yuzu/pull/4832)
+- [Improved GPU synchronization.](https://github.com/yuzu-emu/yuzu/pull/4869). 
+
+The last one also improves stability in games like `Luigi's Mansion 3`. The only thing missing to get `Super Mario 3D All-Stars` rendering is the `Texture Cache Rewrite`.
+
+## NVDEC improvements
+
+Work on video decoding continues, thanks to [epicboy](https://github.com/ameerj). Some games like `Onimusha: Warlords` and `Ni no Kuni` had an incompatible `BGRA8` format in our first NVDEC implementation. Now they render correctly.
+
+{{< imgs
+    "./ninokuni.png| Working menu rendering (Ni no Kuni)"
+  >}}
+  
+Unfortunately, a select few games like `FINAL FANTASY XII THE ZODIAC AGE`, `The Legend of Zelda: Link's Awakening`, and `Onimusha: Warlords` currently have problems with memory addressing, causing incorrect video decoding and various graphical glitches (flashing/flickering). While there is a workaround in place in Early Access builds, this fix is not stable enough to be considered mergeable. Once a more elegant solution is found, we will let you all know.
+
+{{< imgs
+    "./onimusha.png| Happy moments (Onimusha: Warlords)"
+  >}}
+  
+And last but not least, NVDEC is now in Mainline! Anyone can now enjoy the in-game videos in yuzu. As the tradition for all merged Pull Requests, [Lioncache](https://github.com/lioncash) didn't take long to [clean it up.](https://github.com/yuzu-emu/yuzu/pull/4837)
+  
+## Non-mythological input changes
+
+We have some great changes made possible from the work of [german77](https://github.com/german77). Thank you!
+
+To start with, [buttons that are impossible to use on certain input configurations are now blocked](https://github.com/yuzu-emu/yuzu/pull/4742). For example, if an emulated Pro Controller is changed to a left Joy-Con, the right stick, A, X, Y and Z will no longer be accessible to the games. No cheating!
+
+Up next, some devices have access to an accelerometer, but lack a gyroscope. While this setup is basically useless for full motion, it can be used to detect simple shaking motions, [so we don't see why we shouldn't add support for that.](https://github.com/yuzu-emu/yuzu/pull/4757)
+
+And finally, for our traditional fighters out there, we have some improvements for GC adapters. [Adapters can now be hotplugged, basic rumble support was added, and compatibility with more vendors was improved.](https://github.com/yuzu-emu/yuzu/pull/4781)
+
+A new challenger approaches! [FrogTheFrog](https://github.com/FrogTheFrog) tuned up [the shake values](https://github.com/yuzu-emu/yuzu/pull/4727) to improve compatibility with the rare `Steam Controller`. Wouldn't having Portal and Portal 2 on the Switch be awesome?
+
+## Bug fixes and improvements
+
+[bunnei](https://github.com/bunnei) [changed the values for `Multicore`, `Asynchronous GPU`, and `Assembly Shaders`](https://github.com/yuzu-emu/yuzu/pull/4805) to be enabled by default. This will help provide a much better experience out of the box to all users that haven't changed those values already. Some other advanced settings like `Asynchronous shaders` are not altered by this change as they still require further testing and bug-fixing.
+
+By [implementing CUBE textures and fixing an issue with arrays,](https://github.com/yuzu-emu/yuzu/pull/4766) [Rodrigo](https://github.com/ReinUsesLisp) made the `Pikmin 3: Deluxe Demo` playable. This will most likely help in making the final game playable too.
+
+{{< imgs
+    "./pikmin3.png| Thanks GG for the capture! (Pikmin 3: Deluxe Demo)"
+  >}}
+  
+In a minor change to Vulkan, [Rodrigo](https://github.com/ReinUsesLisp) [changed the way GPU devices are sorted in yuzu,](https://github.com/yuzu-emu/yuzu/pull/4765) helping pick the best GPU according to yuzu's current compatibility and needs.
+
+{{< imgs
+    "./vklist.png| A developer's system, gotta render 'em all!"
+  >}}
+
+The current method follows these rules:
+
+- Priority will be given to the GPU vendor following the `Nvidia > AMD > Intel` order.
+- Dedicated hardware will take priority over integrated. For example an RX 570 will have a higher priority than an Intel UHD630.
+- The device name will be considered, a GTX 1650 will be selected over a GTX 1060. Most of the time, newer architecture generations offer better performance benefits than pure raw performance.
+
+In the previous progress report, we mentioned that we removed a blacklist on AMD GPUs for the `VK_EXT_extended_dynamic_state` Vulkan extension. Turns out, RDNA1 GPUs crash to the desktop while using this extension, so [Rodrigo](https://github.com/ReinUsesLisp) (while using your writer's PC) had to [manually add a new blacklist only including current Navi based GPUs.](https://github.com/yuzu-emu/yuzu/pull/4772) One would expect that a reported extension is tested before releasing the drivers that start supporting it... The 20.10.1 driver version from AMD seems to revert to an older Vulkan version lacking this extension, so the Red Team is aware of the issue.
+
+[epicboy](https://github.com/ameerj) changed how `Asynchronous Shaders` [determines the amount of threads to use,](https://github.com/yuzu-emu/yuzu/pull/4865) increasing it for higher thread count CPUs. Past the 8 threads mark, one shader thread is added every two CPU threads.
+
+| CPU Threads | Shader Threads | Example CPUs |
+| :-----------: | :--------------: | ------------ |
+| 1 - 7 | 1 | R3 3200G, i5-9400 |
+| 8 | 2 | R3 3100, i3-10100 |
+| 12 | 4 | R5 3600, i5-10400 |
+| 16 | 6 | R7 3700X, i7-10700K |
+| 32 | 14 | R9 3950X, i9-9960X |
+| 128 | 62 | Threadripper 3990X |
+
+[Kewlan](https://github.com/Kewlan) is back again, fixing another important UI bug. This time, yuzu will [no longer ask for a profile if there is a single one created.](https://github.com/yuzu-emu/yuzu/pull/4817) The devil is in the details.
+
+[Morph](https://github.com/Morph1984) [added a check to see if a directory is a `nullptr`.](https://github.com/yuzu-emu/yuzu/pull/4785) This fixes save file creation in `Hades`. However, more work is needed to get the game playable.
+
+With the demo just released, [Rodrigo](https://github.com/ReinUsesLisp) started the work to get `Hyrule Warriors: Age of Calamity - Demo Version` playable. By implementing [FCMP immediate](https://github.com/yuzu-emu/yuzu/pull/4853) and [texture CUBE array shadows](https://github.com/yuzu-emu/yuzu/pull/4854) the game starts to render.
+
+{{< imgs
+    "./aoc.png| Where are my glasses? (Hyrule Warriors: Age of Calamity - Demo Version)"
+  >}}
+
+&nbsp;
+
+We personally want to thank [Lioncache.](https://github.com/lioncash) He's not mentioned much in the progress reports due to the technical nature of his work, but he is without a doubt one of the top contributors to the project. Thank you Lion, for the patience you have, and the huge help you give us.
+
+## Future changes
+
+There are some important core fixes incoming: `Local Wireless` emulation continues to progress well, we have more plans for `NVDEC` progress on their way, and there is now a roadmap of what will follow the `Texture Cache Rewrite` once it's finished, but we will tell you more about that later.
+
+As an early gift, here's a preview of the Texture Cache Rewrite's progress:
+
+{{< imgs
+    "./tcr.png| "
+  >}}
+
+And that's all folks! Thank you so much for reading, and see you all in the next progress report!
+Thanks RodrigoTR and GG for the pics!
+
+{{< imgs
+    "./burgy.png| Image by https://twitter.com/JamesTurner_42"
+  >}}
+
+&nbsp;
+<h4 style="text-align:center;">
+<b>Please consider supporting us on [Patreon](https://www.patreon.com/yuzuteam)!<br>
+If you would like to contribute to this project, check out our [GitHub](https://github.com/yuzu-emu/yuzu)!</b>
+</h4>