You said that you valued ease-of-use. I feel like that's easier said than done. Could you tell us about the trial and error you went through to achieve this?
Tokunaga:
The more we focus on making it easy to use, the more things we as developers need to consider. For example, not only did we want a built-in microphone, but we also wanted it to be used without any setup. However, implementing a built-in microphone is actually quite difficult. If we don't make any adjustments, the microphone would naturally pick up sounds from the game, such as music. But if those aren't removed, then the most important sounds, the players' voices, might be drowned out. Especially in handheld mode, controller sounds, such as buttons being pressed and rumble, would also get picked up, so those need to be removed too. On the other hand, in TV mode, when players are far away from the console, their voices need to be picked up without them having to shout. This means achieving two conflicting concepts: removing the loud sounds coming from the TV, while picking up the faint voices of players sitting far away from the console.
Ono:
We kept asking you the impossible, Tokunaga-san. Like removing as much noise as possible that isn't someone's voice, but leaving in the sound of people clapping.
Tokunaga:
We installed a high-performance audio chip dedicated to voice processing and fine-tuned it relentlessly until it only picked the sounds we wanted it to. (Laughs)
Ono:
You have my eternal gratitude. (Laughs) One time I was asked, “Is it just one person's voice we need to pick up, or is it everyone if it’s being used by a family? It'd make our lives easier if you could narrow it down to one or the other”. To which I answered, “It’s not an either-or question. It’s both”. (Laughs) After all, we wanted to take various factors into account, like how far the sofa is from the TV and where players are sitting on the sofa. Tokunaga-san once told me to please stop being so vague, but if you're going to say “no setup required”, then it needs to be able to handle various situations.
What about the speakers? Did they also require some trial and error?
Ono:
To hit our goal of “no setup required”, we worked hard to balance the speaker's volume. Since the game and voice chat sounds play together out of the same speaker, we worked with the sound team to determine which one should stand out more. Even without manual setup, voice chat stands out more when someone's talking, and when no one's speaking, game sound will be more prominent. There's a natural transition between the two.
Tokunaga:
The microphone automatically detects which sounds are essential and adjusts the audio balance of voices so they're comfortable to hear. So, for example, if you switch from TV mode, which picks up voices at a distance, to handheld mode, which picks up voices nearby, the voices won't suddenly get louder for the person on the other side of the voice chat, and you can continue using the feature smoothly.
Ono:
We regard the ability to seamlessly transition between TV mode and handheld mode while using voice chat as one of the unique features of Nintendo Switch 2. If you’re using voice chat in TV mode and a family member wants to use the TV, you can just remove the console from the dock and continue to voice chat as-is in handheld mode.
Tokunaga:
You can take it anywhere in handheld mode, so you can now, for example, voice chat and play games while lying on your bed.
Wow, it looks like you're live-streaming.
Ono:
Precisely. In reality, it's just a video chat with friends, but the way it’s presented makes it look like YouTubers streaming their gameplay. I reckon it would be fun for friends to play together as if they're live streamers.
Tokunaga:
You may have seen PC and smartphone cameras that hide backgrounds. But for game consoles, the background needs to be hidden while the console is also using processing power to run the game, which is a real challenge. However, I think that whether the background is visible or not also affects safety and security, so we kept on fine-tuning it to use the least amount of processing power to show only the players. Finally, we achieved something we could be confident in.
Ono:
The background is properly hidden even with multiple people, not just when you're playing alone. I don’t think I’ve seen that very often.
Festival vibes?
Tokunaga:
Yes. Whenever someone came up against a challenge during development, various people would gather around and say, “Are you having trouble? Let me help”, then we’d all work together to solve it. As mentioned earlier, GameChat is a combination of various technologies. And since it has to run alongside a game, if too much processing resource is spent on any part of GameChat, it could affect the game's performance. Therefore, we had to fine-tune all the technologies involved. That necessitated the involvement of people from various teams, but everyone was so reliable and said, “If it’s for GameChat, we'll make it work!”. (Laughs) Everyone likes solving problems with technology, and they have fun doing it. Those festival vibes lasted more than two years. And with more than 100 people involved, what a festival it was. (Laughs)
Chiedi allo sviluppatore, parte 17: GameChat - Capitolo 2