I love co-pilot. Throughout the pandemic my fiancé and I used the feature to play different sections of single player games as though they were multiplayer. I don’t think either one of us would consider ourselves to have a disability.
I used to work on games at Microsoft and they put their money where their mouth is when it comes to accessibility. I had spent multiple days in the accessibility lab interviewing and understanding how to make games easier for people with various disabilities. One of my managers encouraged me to test my features using the controller with one hand and farther away from the screen where bad font decisions were more obvious.
Two things I remember in particular from that time at Microsoft:
* The claim that making games better for people with disabilities makes them better for everyone
* The claim that most people will have a disability (not necessarily permanent) at some point in their life
I feel that those two lessons have permanently impacted my design lens.
> The claim that most people will have a disability (not necessarily permanent) at some point in their life
I think it was Microsoft that came up with a great infographic illustrating that disabilities aren't a black/white thing. It broke things down by permanent vs temporary vs situational.
Something like:
- Touch / tactile - arm/hand amputation (permanent), arm in a sling (temporary), new parent cradling a baby (situational)
> most people will have a disability (not necessarily permanent) at some point in their life
I've got an eye infection when I was a teenager. It has strongly tinted my vision white, so for a couple of days I could only see outlines of big objects highlighted by skylight.
From this experience I found I could still walk my usual routes without wandering off the road, as I could recognize familiar telephone poles, trees, I even remembered all the ruts in the road.
Yes, it's absolutely wonderful. I hope to never need it, but I'm so happy it exists for everyone that does.
It also shows why controller compatibility between consoles might be more important than one would think: the "native" controllers might just be inaccessible. If you want to play on a Switch, you can use an adapter [0] to use the Adaptive Controller. Similar products exist for the PS4, I believe. Hopefully someone reverses the protocol DualSense speaks with PS5 sooner rather than later and can make an adapter for that as well.
PSA: You can use the MAGIC-NS to connect multiple controllers at the same time while they appear to the Switch as one controller. This basically is co-pilot mode.
There’s a V1.32 note in red on that page that says it’s now compatible with the PS5, but I’m not sure what that means exactly. It’s hard to say from the product page what this thing does.
> Hopefully someone reverses the protocol DualSense speaks with PS5 sooner rather than later and can make an adapter for that as well.
Sony recently[1] mainlined its DualSense 5 driver to the Linux kernel, so reversing the protocol shouldn't be difficult. There previously were a couple of open-source 3rd party drivers for older DualShock controllers - I use a DS 4 controller for all Steam on Linux games and it works very well: most games recognize it as a PlayStation controller, display the correct button symbols on help screens and QTEs
Unfortunately this isn't the complete story. Being able to use a DS4/DualSense with a PC or other device is a different beast from being able to emulate a DS4/DualSense for a PS4/5.
The console actually authenticates the controller using some secret [0]. I find it very doubtful that Sony has implemented this authentication in their Linux driver and released the necessary secrets.
Unless this procedure is a part of a standard that I'm unfamiliar with that it may still need to be reversed for PS5/DualSense. We also need the secrets or use the workaround with a legitimate controller I referenced in a different comment [1, 2].
IIRC Sony puts DRM in their controllers, there's a chip with a key on it that Sony only gives out to selected partners. I'm not sure how 3rd party companies did it without the chip but afaik the method is not open source.
This is true, unfortunately. A common way around that is that you must connect both the controller you want to use and an original PS4 controller to the adapter. The adapter will forward all DRM/handshake messages to the original controller, but forward the input from the desired one.
Unless Sony is now signing every message I see no reason why this approach couldn't be used on the PS5/DualSense. That said, my cursory search didn't reveal a working PoC.
PS4 Controllers can be connected via BT to phones and via cable to PC with almost all functionality. There's an extension port on the controller but I haven't seen anything that plugs into it. Also don't know how easy it is to connect your own controller to the PS4 but there is an "ext" port on the controller that is reportedly a USB2 port but a proprietary shape.
You probably aren't remembering correctly: the protocol for DualShock 4 (PS4 controller) was reversed - there's a working userspace driver[1] that I have used for years to play games on Linux using a DS4 controller, connected via Bluetooth. More radically, Sony recently published an official Linux driver[2] for DualSense 5 - none of these require a "DRM chip" or additional hardware to use.
More interesting is that whilst the original PS3 controllers had a full Bluetooth stack (meaning you can connect them to an Android device or Linux and they work perfectly fine), the bootleg clones only implemented the spec enough to get them working with the PS3. I didn't even know this until I tried to connect my old PS3 controller to my Pi running retro emulators, and all the controller would do is input to bottom right on the right analog stick. Same on Android.
It was a big source of internal pride when I was at Microsoft. As far as tech companies go, Satya’s version of Microsoft really does give a damn about people, including it’s employees. Great place to work.
Microsoft has done a lot of things that they didn't "need" to for the sake of accessibility and I think that's worth celebrating. Particularly when it comes to something as simple as this I think there's no reason why it can't exist on every other console. As mentioned in other comments, their adaptive controller is also great.
This co-pilot mode sort of reminds me of Starcraft: Brood War where there is a cooperative mode where two people can control the same set of buildings and units. The amount that each player contributes to the game can be as variable as you need it to be, as far as one player not doing anything at all. Or if you were both skilled, it could simply augment what you could do. One person could be making sure you're spending resources and building the base while the other focused on controlling the units fighting the enemy, for example.
Overall I think the video games industry is very very behind in accessibility, though. Even some relatively simple accessibility options are absent from some of the biggest games. I believe Ubisoft has made great strides to make their video games more accessible to everyone and there are some notable ones such as The Last of Us 2 [1] and Celeste [2] that go above and beyond to ensure that anyone that wants to play the game can.
Accessibility can be expensive to implement and gaming isn't really considered an essential, even in terms of entertainment, but I think it's worthwhile to try and encourage even smaller developers to try to make their games more accessible.
I am terrible at video games. It's not what my brain does well so I miss out on playing computer games in general, I'm more interested in board games or cards. Not long ago I was waiting for laundry to be done with a friend who had a Nintendo Switch on her, and we started a game of Mario Kart. The entire time we played, maybe 15 mins, I thought wow, I'm better than I think! Then after the game I told my friend and she says, the whole time she was racing behind me to kill the bad guys so I could enjoy playing. What a beautiful gesture!
I don't think another player can actually be that helpful in Mario Kart, but it does have extremely strong steering assistance, which can be enabled/disabled in the pause menu. Not trying to be rude by comparing you to a small child, but the main benefit is that little ones can join in and have fun while they're not quite co-ordinated enough to steer properly.
If you wanted to get more into gaming, a Switch would be a good choice. There are lots of games which are mechanically accessible (still not trying to be rude here) to six-year-olds, while still being a ton of fun.
Mario Kart: Double Dash was actually a lot of fun with two players in a kart. A friend of mine who couldn't steer to save his life took over the weapons while I focused on driving. The best part is that the second player could make their character throw their weight left or right to shove the kart sideways and save you from an obstacle, or a tight turn taken a little too quickly.
Not quite true :) I’ve played a lot with my 2 youngest daughters and the AI adapts to the fastest player. So if you hang behind the “worst” player, it will become very easy.
Subsequently, if you race full-pull the other person will be in a permanent #12 spot. They will receive a lot of star and bullet power ups though ;-)
No offence taken! It's not so much the coordination but that I just don't care about playing. My sense of being entertained is different than what computer game play provides, so I don't put any effort into learning the moves. So in relation to the topic here, it's nice if I can hang out with friends who love games while I don't need to invest effort to join in on occasion, as I could get quite a lot out of copilot mode.
Most people when they say they are terrible at video games mean they are bad at reflex games. I recommend you don’t discount entire sub genres of non-real-time video games you might enjoy. Depending on your taste that can be anything from tactical/turn-based RPG’s to sports management games.
my youngest is 2.5, a survivor of bacterial meningitis at birth, the results are tragic. she has soft tone cerebral palsy and cortical visual impairment, imagine being stuck in a kaleidoscope for life.
as part of my journey, trying find answer and heal. i stumbled across the fact that Microsoft's current CEO Satya Nadella son Zain is a legally blind quadriplegic with cerebral palsy. his book refers to the first few years of his sons life as "a dark place". Satya is fairly older than me, I admire him very much.
I have no doubt that it bring great joy and comfort to Satya that he's been able to drive Microsoft to be a leader in accessibility. Most great efforts have great stories. This is one of them.
We did - I remember playing Doom with my friend as a kid - one of us would do the walking, another would do the shooting, on different peripherals. The idea is pretty fresh on the consoles, however.
That reminds me that when USB came out (or came to our, my parents' computer) I thought if only we got a new mouse, I'd be able to plug both in, getting two cursors, and be able to cheat at solitaire. Heh.
One game I particularly remember is Super Mario Galaxy 3D on the Wii. It had a two player mode where the second player didn't control Mario, but instead had the ability so interact with the environment and player in limited ways to help. I wonder why there aren't a lot of other games that do that.
Game design is hard and asymmetrical game design is incredibly hard. Game developers can hardly make symmetrical cooperative systems that properly take advantage of the multiplayer aspect. Borderlands, for example, doesn't add unique systems that take advantage of the fact that there are multiple players. There's no unique mechanics to bosses or enemies. There's not really interesting synergies that you can take advantage of by the different classes.
When games do coop well, it's usually an inherent part of the game. If you can make the assumption that there is at least two players playing together, you can create unique and interesting ways for the two players to interact on screen. "It Takes Two" is a recent game that was designed explicitly for two players and you can immediately tell the difference from it to a game that cooperative modes were an afterthought to the solo mode.
Games requiring more than one player immediately alienates a lot of players and limits your audience a lot, so that's probably why pure coop titles are extremely rare.
As for asymmetric titles like Mario Galaxy where the second player is not essential, it's pretty hard to make the gameplay satisfying for the secondary player while not impeding the first player or making it feel essential to the gameplay. The "little brother" mode costs a lot with little perceived benefit for the developers.
> As for asymmetric titles like Mario Galaxy where the second player is not essential, it's pretty hard to make the gameplay satisfying for the secondary player while not impeding the first player or making it feel essential to the gameplay. The "little brother" mode costs a lot with little perceived benefit for the developers.
I'm not sure how frequent this is, but when I was little we (me and my brothers) had limited time to play, so we would play our time and watch each others, sometimes completing long missions in Ground Control by playing one after the other. When you look at it this way, having something to do instead of just looking at the screen (which was already plenty of fun) is huge.
That's a very touching story. Although most of us don't work in the consumer hardware business, many of us work on frontend web design. This post serves to underscore the importance of accessibility in our designs.
My 3 year old is very interested in playing video games, but her hands are too small to hold a standard controller and reach everything. We've been playing Carto a bit on gamepass (no enemies, moving around, no timing constraints).
Just out of curiosity I checked, and co-pilot seems to be available on Windows too (within the Xbox sphere at least). I will try that out, looks like it could have potential for us.
If you haven't had this discussion with him already (and you think you can comfortably have this chat), he may already have some ideas that he hasn't thought to mention.
Otherwise, one suggestion would be controllers for quadriplegics. There are a few options out there, afaik most are controlled with your mouth but there are also gaze-tracking options. I believe the XBox accessibility controller is compatible with some of them.
I could have that talk with him. But i don't want to start it with no options or no suggestions.
Him and his wife don't have it easy ( they used to go walking in the mountains a lot, so the declining mobility in recent months are ... Depressing for them, although they don't really complain. I just notice it in the conversations when it comes up)
Can't really make any hardware recommendation, but as this post is about games, I would suggest trying out more puzzle like games like "The Witness" or "Outer Wilds". Despite being single player, I played both of them in tandem with my wife and I feel like it was a much better experience than playing alone as there was always something one of us would have missed or would get stuck on.
Well, it's game hardware for "disabled", so this is one of the best threads similar to my question. Where people could have the same interest or know about something already.
If he can comfortably move his head, might I suggest something VR related? Most of the first VR games that came out for mobile (i.e. using your phone as a VR screen) couldn't utilise a controller so they used head movement for controlling things. Usually you had e small crosshair in the middle with which you can select things by simply holding a position. I sadly can't remember the names of the games I tried but you might find them (or newer ones for newer headsets) through google. I know it's not a complete substitute but if they miss mountain walks, there are VR experiences (often videos) which simulate that. Might be interesting to try. I'm not aware of it but there might even be multiplayer ones, so that his wife can join him.
Platinum Games and some other studios are now in the habit of putting "easy automatic" (not always using that term) modes in, where you can set the game to automate some or all of its inputs. If you turn all the assists on in Toylogic's "Nier Replicant" for example, the game basically plays itself during combat but you still have full control so you are effectively orchestrating the action on screen instead of mashing the buttons. It's nice for people revisiting the game (it's a remaster with updated content) who don't want to do all the combat again. For some titles, this is a great accessibility feature since you can turn it on only for particular mechanics that are hard for you to keep up with (dodging, aiming, etc) and still play "the rest" of the game.
Increasingly I think we're discovering that there's a class of temporary accessibility issues that can get in the way of using a website or video game - you have trouble focusing because your kids are distracting you, you can't remember what your current objective is because the last time you played was a week ago, etc. It's great for games to provide assistance in those cases too.
You're already overthinking it with AI and stuff; there's games (especially on mobile) with minimal input requirements, think flappy bird, runner games, some of the racing games, etc. Even Epic's first major mobile title, Infinity Blade, played more like a slow interactive story than something needing lots of control input.
I mean I won't claim they're fun, just that they're easy to play.
Your argument would also refute the whole co-pilot experience itself. It is not about "there are simple games, go play that". It is more to enable them to play anything.
I think anything that can be co-piloted so smooth that other player is not annoyed, can be also simulated by an AI.
It is not really overthinking it. It is taking the learning and bringing it to a wider audience.
We're not anywhere close to just having a general purpose game playing AI for the current gen (though there is that Atari playing one)
But there are a lot of affordances devs can make in individual games. Someone else mentioned Celeste and Last of Us, but Microsoft's Forza series has a bunch of options, like brake assist or turning assist, that can take a lot off your plate.
Only from the perspective of the 'inferior' player - 'co-pilot' seems fine for competitive multiplayer, since as far as other players are concerned you could have been performing better, if you were only the better of the two of you, 'unimpeded' by the one being assisted.
Many games have "assist modes" of some kind. Sometimes as part of a difficulty setting, sometimes you have fine-grained controls, especially in racing games.
Usually, when these modes are enabled, there is some indication that they are used, and you can't enter ladders with them, to prevent cheating.
I used to work on games at Microsoft and they put their money where their mouth is when it comes to accessibility. I had spent multiple days in the accessibility lab interviewing and understanding how to make games easier for people with various disabilities. One of my managers encouraged me to test my features using the controller with one hand and farther away from the screen where bad font decisions were more obvious.
Two things I remember in particular from that time at Microsoft:
* The claim that making games better for people with disabilities makes them better for everyone
* The claim that most people will have a disability (not necessarily permanent) at some point in their life
I feel that those two lessons have permanently impacted my design lens.