Xbox Cloud Gaming is getting mouse and keyboard support and latency improvements

Illustration by Alex Castro / The Verge

Microsoft is preparing to add mouse and keyboard support to its Xbox Cloud Gaming (xCloud) service that streams Xbox games to TVs, PCs, mobile devices, and more. The software giant teased the addition earlier this year, and now it’s encouraging game developers to get ready for mouse and keyboard support and some big latency improvements on Xbox Cloud Gaming soon.

“Xbox has been supporting keyboard and mouse for a few years now, and we’re working on adding it to streaming for PC users,” explains Morgan Brown, a software engineer on Microsoft’s Xbox game streaming team. “But you can start adding it to your game right now and your console keyboard and mouse users will appreciate it. It will light up in streaming once we’ve finished adding it.”

Microsoft Flight Simulator boss Jorg Neumann previously teased that the addition of mouse and keyboard support on Xbox Cloud Gaming could appear this summer. As Microsoft is encouraging developers to start thinking more about mouse and keyboard support for Xbox games streamed to PC, it’s likely that we’ll start to see this show up soon.

It will allow Xbox Cloud Gaming users to stream Xbox games, not PC ones, using a mouse and keyboard. We could see games like Sea of Thieves, Minecraft, Halo Infinite, and even Fortnite all support mouse and keyboard through Xbox Cloud Gaming. The list of Xbox games that support mouse and keyboard is still relatively small, though. It will be particularly useful when Microsoft expands the Xbox Cloud Gaming library later this year.

Image: Microsoft
Xbox developers will get new APIs to improve stream latency.

Alongside mouse and keyboard support, Microsoft is also offering developers more ways to improve streaming latency in their games. Microsoft has been working on a new Display Details API, which can save up to 72ms of latency overall. This is achieved by using Direct Capture, which reproduces hardware features in software to eliminate the wait time for VSync and double or triple buffering, and even the scaling needed for TVs.

Scaling and artifacts all add extra latency to game streaming, and many games already support Direct Capture to improve their performance on Xbox Cloud Gaming. Latency can drop to as low as 2-12ms, compared to 8-74ms through the traditional display pipeline. There are some limitations, though. Direct Capture only supports a maximum resolution of 1440p, and doesn’t support dynamic resolution or HDR just yet.

The resolution limitation won’t be an issue for most game developers right now, as Xbox Cloud Gaming scales games down to 720p on mobile and 1080p on PC and the web. Microsoft does eventually expect to support higher resolutions, but there’s no timeline on 1440p or 4K support for the new Xbox TV app. “That’s something that we expect will change over time, based on different devices, network conditions, and improvements to the streaming stack,” explains Brown. Tools will be available for developers soon to test their games and find out how to support Direct Capture.

Image: Microsoft
Direct Capture improves streaming latency in Xbox Cloud Gaming.

The latency improvements are key for game streaming services like Xbox Cloud Gaming, and as Direct Capture shows, it’s not all about just reducing network latency. Nvidia launched its RTX 3080 GeForce Now tier last year, with impressive latency improvements. Nvidia built its own Adaptive Sync technology, which varies the game rendering to match a synchronous monitor and allows GeForce Now to sync streamed games to any 60Hz or 120Hz monitor.

Nvidia’s Adaptive Sync also reduces some buffering between the CPU and GPU on the server side, and the end result is some impressive latency improvements over what’s available from Google Stadia or Xbox Cloud Gaming. Nvidia even claims to beat an Xbox Series X running locally at 60fps thanks to its 120fps GeForce Now support.