diff --git a/site/content/entry/yuzu-nvdec-emulation/banner.png b/site/content/entry/yuzu-nvdec-emulation/banner.png new file mode 100644 index 00000000..228b9ae9 Binary files /dev/null and b/site/content/entry/yuzu-nvdec-emulation/banner.png differ diff --git a/site/content/entry/yuzu-nvdec-emulation/index.md b/site/content/entry/yuzu-nvdec-emulation/index.md new file mode 100644 index 00000000..0ea12b85 --- /dev/null +++ b/site/content/entry/yuzu-nvdec-emulation/index.md @@ -0,0 +1,212 @@ ++++ +date = "2020-09-29T23:20:00+05:30" +title = "New Feature Release - NVDEC emulation!" +author = "CaptV0rt3x" +forum = 309130 ++++ + +Hello, yuzu fans! +Tired of broken cutscenes and having to mash your controller buttons in hopes of skipping them? Well, look no further! +Thanks to the efforts of [epicboy](https://github.com/ameerj), yuzu can now play (most of) your favorite in-game cutscene videos. +Jump right in to find out more! + + +  +{{< youtube usvAZosD16o >}} + +# NVDEC - What is it? + +For a long time, one of the most noticable missing features of yuzu was support for NVDEC.
+What is [NVDEC](https://en.wikipedia.org/wiki/Nvidia_NVDEC)? +NVDEC stands for **Nvidia Video Decoder** and is a feature in Nvidia GPUs that performs video decoding. +And since the Switch has an Nvidia Tegra X1 SoC, it too makes use of its NVDEC module to offload all CPU intensive video decoding tasks to the GPU. + +NVDEC supports video codecs like `MPEG2`, `VC1`, `H264 (AVC)`, `H265 (HEVC)`, `VP8`, and `VP9`, but Nintendo only exposes `H264`, `H265`, `VP8` and `VP9`. +Most games seem to use `H264` because of its wide support and efficiency ratio. +But many first party games are known to use `VP9` too - eg. Super Smash Bros. Ultimate, Pokémon: Let's Go, Pikachu/Eevee! + +With NVDEC support, users no longer have to rely on old save files or crazy button mashing to bypass broken cutscenes. +Grab the latest yuzu Early Access build to try it out now! + +  +{{< youtube XsXpxZE2rcc >}} +  + +# Development + +The Switch uses NVDEC along with VIC (Video Image Composer) in the GPU for video decoding. +Games are exposed to higher level APIs by Nintendo which abstract the configuration of these modules. + +NVDEC and VIC work together to decode incoming videos and the process usually goes like this: + + - Allocate memory + - Place encoded video frame data in known memory location + - Decode and place frame data into GPU memory + - Clear intermediary allocated memory + +Although this seems fairly simple, implementing it was quite the task. +[epicboy](https://github.com/ameerj) began working on this by implementing the service/command calls to NVDEC and VIC. +It was decided to use the [FFmpeg](https://github.com/FFmpeg/FFmpeg) library to decode the frame data once we knew its codec (`H264` or `VP9`). + +  +{{< youtube EGDodmeKGWY >}} +  + +### VP9 + +`VP9` was the biggest challenge as its decoding required knowledge on the different attributes of a specific frame. +Each frame carries two headers and raw frame data with it. +The headers contain the information as to which previous frame the current frame refers to or depends on. +Frames usually refer to previously decoded frames, along with transformations to be done on various parts of those, to compose new frames. +This is done in an effort to reduce the memory footprint of each frame. + +Here is where things got complicated. +On the Switch, these headers are parsed by the Nvidia library in the games; therefore, NVDEC never recieves this data. +But for us to decode the frames using FFmpeg, we need to provide it with the full frame - Two headers and raw frame data from NVDEC. +And since NVDEC hardware doesn't expose all of the data of the two `VP9` headers, [epicboy](https://github.com/ameerj) had to manually compose these headers from the provided information. + +[epicboy](https://github.com/ameerj) took an interesting approach to this problem - he buffered two frames in advance. +As the frame data from NVDEC holds some data on previous frames, knowing two frames in advance was useful. +But this wasn't sufficient as a few games resulted in issues with this approach. +We are currently researching these edge cases to properly fix it for all games. + +  +{{< youtube xHYYntEB05o >}} +  + +### H264 + +`H264` was relatively easier compared to `VP9`. +Each `H264` frame contains two headers and raw frame data with it. +Unlike `VP9`, `H264` doesn't have different headers for each frame but has same headers for entire video. + +[ogniK](https://github.com/ogniK5377) had, long ago, already implemented `H264` support in his experimental branch. +[epicboy](https://github.com/ameerj) based his work off of [ogniK](https://github.com/ogniK5377)'s and fixed a bug in it which caused distorted videos. +It turned out that [ogniK](https://github.com/ogniK5377) was using incorrect dimensions when writing the pixel location. + +Currently NVDEC operations are synchronous, but are asynchronous compatible. +This means that in the future, NVDEC operations can be made asynchronous thus yielding even better performance. +Also, thanks to the FFmpeg library, we have access to hardware accelaration for faster decoding. +And in the future, this will allow yuzu to offload video decoding to the host GPU (user's GPU). + +  +{{< youtube ImXim7BXl0E >}} +  + +### VP8 & H265 + +The Switch officially supports `VP8` and `H265` too, along with `VP9` and `H264`. +However, we are yet to see games make use of these codecs for in-game videos and hence support for these codecs remains unimplemented for now. + +# Testing + +As development work progressed, there were a lot of challenges and issues with games. +**Link's Awakening**, for example, wasn't providing the proper memory locations to write frame data to and +the NVDEC/VIC service/command calls were inaccurate as well. +This led to data corruption when we wrote frame data to wrong memory locations. + +  +{{< youtube 4Lz1NqZsYlA >}} +  + +[epicboy](https://github.com/ameerj) and our testers spent tens of hours testing various games in yuzu to make sure any minor issues were ironed out. +This rigourous testing also brought to our attention more games that exhibit weird edge cases. +Over the next couple of weeks, we plan to fix these bugs and make our decoding even more accurate. + +Huge shoutout to our testers for testing and compiling these lists. + +{{< message "Games that work" >}} +The following games were tested by our team and work well with minor occasional glitches: + +* 1-2-Switch +* AI: The Somnium Files +* Animal Crossing: New Horizons +* Attack on Titan 2 +* Atelier Ryza: Ever Darkness & the Secret Hideout +* Bloodstained: Ritual of the Night +* Bulletstorm +* Children of Morta +* Dark Souls Remastered +* Deadly Premonition Origins +* Devil May Cry +* Devil May Cry 2 +* Devil May Cry 3 +* Disgaea 5 Complete +* Donkey Kong Country: Tropical Freeze +* Dragon's Dogma +* Dragon Quest Builders +* Dragon Quest XI:S +* Dragons: Dawn Of New Riders +* Fairy Tail +* Final Fantasy IX +* Fire Emblem: Three Houses +* Fire Emblem Warriors +* Go Vacation +* Hollow Knight +* Huntdown +* Hyrule Warriors: Definitive Edition +* Indivisible +* Just Dance 2020 +* Kirby Star Allies +* LEGO City Undercover +* Mario Kart 8 Deluxe +* Marvel Ultimate Alliance +* Monster Boy and the Cursed Kingdom +* ŌKAMI HD +* Paper Mario: The Origami King +* Pokkén Tournament DX +* Psyvariar Delta +* Resident Evil +* Resident Evil 0 +* Resident Evil 4 +* Resident Evil 5 +* Resident Evil 6 +* Resident Evil Revelations +* Resident Evil Revelations 2 +* Ring Fit Adventure +* Rune Factory 4 Special +* Shantae and the Seven Sirens +* Star Ocean First Departure R +* Steins;Gate Elite +* Super Mario 3D All-Stars +* Super Mario Odyssey +* Super Smash Bros. Ultimate +* TLOZ - Breath of the Wild +* Tokyo Mirage Sessions #FE Encore +* Trials of Mana +* Xenoblade Chronicles Definitive Edition +* Yu-Gi-Oh: Legacy of the Duelist +{{< /message >}} + +{{< message "Games with known issues">}} +The following games are known to have issues, and we are actively working on fixing them: + +* Alien: Isolation +* Ni no Kuni: Wrath of the White Witch +* Onimusha: Warlords +* Persona 5 Scramble +* Resident Evil 0 +* TLOZ: Link's Awakening +* The Legend of Heroes: Trails of Cold Steel III +* Final Fantasy VIII Remastered +* Pokémon Let's Go +* Pokémon Sword +{{< /message >}} + + +# Fin + +As always, we would like to remind users that the features released on Early Access are still being worked on. +Hence not all games might behave in the way we want them to. +If you come across more games (other than the ones mentioned above) that encounter bugs or issues, feel free to +reach out to us on our [Discord server](https://discord.com/invite/u77vRWY) and share your findings. + +See you next time,
+ - yuzu development team! + + +  +

+Please consider supporting us on [Patreon](https://www.patreon.com/yuzuteam)!
+If you would like to contribute to this project, check out our [GitHub](https://github.com/yuzu-emu/yuzu)! +

\ No newline at end of file diff --git a/site/content/entry/yuzu-nvdec-emulation/summary.png b/site/content/entry/yuzu-nvdec-emulation/summary.png new file mode 100644 index 00000000..41efd529 Binary files /dev/null and b/site/content/entry/yuzu-nvdec-emulation/summary.png differ