Source: New Smart Element
Image Source: Generated by Wujie AI
The game version of "Westworld" has become a reality.
YouTube blogger Art from the Machine officially released Mantella, a new AI Mod that can resurrect NPCs in "The Elder Scrolls V".
The project caused a sensation on Reddit as soon as it was released.
By using ChatGPT, as well as the text-to-speech tool xVASynth and the speech recognition model Whisper, the game's AI agents are given consciousness and can engage in natural conversations.
It is worth mentioning that the Mantella Mod supports over 1000 NPCs and more than 20 languages.
Netizens have expressed their desire to change the rules of the game.
Many players have long been tired of monster hunting and leveling up in the game.
It is imaginable that after installing this Mod in VR mode, there will definitely be players spending their time chatting with NPCs in "The Elder Scrolls" all day long.
It's impossible to level up, it's so hard to fight monsters. The only way to pass the time in the game is to chat with NPCs.
The NPCs in the game all have their own stories, speak well, and players love chatting with them in the game.
Some netizens even feel that this Mod is not just a game, but the beginning of a new era of human-computer interaction.
1000+ NPCs Infused with "Soul"
Over the past few months, Mantella has been undergoing public testing, and the author finally released this Mod on the 19th.
In simple terms, this Mod integrates all the NPCs in "The Elder Scrolls V: Skyrim" into ChatGPT GPT, allowing the AI to bring the NPCs, who used to only move back and forth, to life.
All of these 1000+ NPCs can directly interact through voice, and each of them has their own unique background.
NPCs can remember the content of previous conversations with the player, as well as their own location, the in-game time, and the player's actions in the game, such as picking up items.
The dialogue content of NPCs is generated by ChatGPT, and the in-game voice implementation is done by an AI tool called xVASynth.
Let's see how NPCs interact with players.
Player: How much are these cheeses?
NPC: These cheeses are our specialty here, they cost 10 gold coins. Do you need anything else?
Player: McDonald's.
NPC: I'm sorry, there is no McDonald's in "The Elder Scrolls".
Player: I took away your wife.
NPC: I don't believe it, I'm not married, let alone have a wife, you must have mistaken…
What's even more interesting is that NPCs will humorously respond to your dialogue.
NPC: If you are implying that you assume you are my wife or you have no wife, I'm sorry to hear that news.
In fact, using AI to make game characters more vivid, NVIDIA is also making progress.
Remember, at this year's COMPUTEX, NVIDIA introduced a new custom AI model foundry service - Avatar Cloud Engine (ACE) for Game.
In a cyberpunk-style ramen shop scene, players can press a button to speak with their own voice, and the shop owner Jin will respond.
Jin is an NPC character, but his responses are generated in real time by a generative AI based on the player's voice input.
Jin also has realistic facial animations and voice, all matching the player's tone and background story.
The creation of these lifelike characters in the game uses a real-time artificial intelligence model rendering tool called Nvidia ACE.
NVIDIA stated that these in-game characters are not pre-set. They have a typical task provider NPC type.
Technical Introduction
The Mod maker has constructed a "group activity" NPC technical framework through ChatGPT—xVASynth—Whisper.
Whisper can recognize the voice input from the player through the microphone and convert it into text, then call the ChatGPT API to respond to what the player said in text.
Then, through xVASynth, the text responses generated by ChatGPT are converted into in-game voice that matches the characteristics of the game characters, allowing direct voice interaction with the player.
And the entire process is almost cost-free, only requiring a small fee for calling the ChatGPT API. It costs just a few cents to play for a day.
xVASynth
https://www.nexusmods.com/skyrimspecialedition/mods/44184
It can generate in-game NPC voice lines that match a specific voice from the game.
xVASynth uses Neural Speech Synthesis to specifically generate voice dialogues for NPCs in the game. It is based on a model trained separately based on the voice data of characters in the game.
It supports text-to-speech conversion (TTS) from text or direct audio input for voice conversion (V/C).
With this tool, users only need to provide a short segment of specific voice material as a template to directly generate voice content that matches the template style.
Mantella was completed using the framework of ChatGPT generating NPC dialogue content + xVASynth converting it into in-game voice.
xVASynth's voice conversion from text allows users to control many details of the voice, such as the pitch and duration of individual letters, energy, emotion, and style, to highlight the character's emotions and emphasis.
The use of neural speech synthesis technology allows it to produce natural voices, something that traditional methods based on concatenated existing data find difficult to achieve. This also means that it can generate entirely new voice content beyond what voice actors have already recorded.
This generated voice will not be a "mechanical" AI-transcribed audio, greatly enhancing the realism of NPCs and the immersion of game players.
What's even more impressive is that it supports 28 languages and can switch output between multiple languages using the same text prompt, greatly facilitating the production of multilingual versions of games.
To facilitate users in handling the thousands of different game voices, it also includes a built-in 3D voice embedding visualization tool.
This 3D visualization UI is also generated by AI, allowing users to color-code the voice based on the attributes of the game's NPCs, such as gender, occupation, and so on, freeing users from the traditional way of controlling voice through a timeline.
xVASynth is now available on Steam, allowing game developers and players to use most of its features for free.
Whisper
In order to complete voice interaction, NPCs not only need to speak themselves, but also need to be able to recognize and understand the player's voice communication.
The Mod developer used Whisper, a speech-to-text AI tool released by OpenAI.
OpenAI has collected over 680,000 hours of multilingual and multitask supervised data from the internet to train Whisper.
The use of such a large and diverse dataset has made Whisper highly adaptable to accents, background noise, and proper nouns. In addition, it can transcribe and translate in multiple languages.
Whisper uses a simple end-to-end architecture, implementing speech recognition through a Transformer encoder-Transformer decoder.
The input audio is divided into 30-second blocks, converted into mel-spectrograms, and then passed to the encoder.
The decoder is trained to predict the corresponding text content and is mixed with special tokens to indicate that a single model can perform tasks such as language recognition, multilingual speech transcription, and English speech translation.
Download and Installation
Requirements
Hardware: There are currently no minimum requirements, but there have been reports of Mantella crashing when running a modlist of 2000 mods. Mantella requires a certain amount of hardware allocation to run successfully, and it may crash if this is occupied by other hardware-intensive mods.
Storage: When installing all voice models, the Mod requires about 17GB of space. Unzipping the voice models requires a total of about 32GB.
Compatibility
It has been confirmed that Mantella can be used with FUS (pointing skyrimfolder to Skyrim), Librum (pointing skyrimfolder to overwrite/root), and Wildlands (pointing skyrim_folder to Wildlander/SKSE) Wabbajack mod lists.
If you have installed the unofficial Skyrim SE edition (USSEP), Mantella needs to be loaded after this mod.
Note: Since Mantella will access and write to the "Skyrim" folder, if you have "Skyrim" stored in "Program Files", Mantella may not work properly. Please make sure to store it outside of this folder (e.g. C:\Games\Steam).
Unzip the Mantella folder.
MantellaSpell.zip
The installation method for this compressed file is the same as for other MODs. If you have never manually installed a module before, there is a disc icon in the upper left corner of the user interface of Module Manager 2, where you can point to the MantellaSpell.zip compressed file for installation.
For Vortex, you can drag the compressed MOD into the Vortex panel.
xVASynth
- Download xVASynth through
Steam (https://store.steampowered.com/app/1765720/xVASynth/) or Nexus (https://www.nexusmods.com/skyrimspecialedition/mods/44184).
Download Skyrim voice models trained by xVASynth for all or any characters you may encounter. You must manually download through the Nexus Mods page, or use Nexus Premium for automatic downloads, as xVASynth includes the Nexus Premium API.
Store the compressed files in a folder under the "Optional" section at https://www.nexusmods.com/skyrimspecialedition/mods/44184?tab=files.
Open xVASynth and drag all the compressed voice model files into the sound panel. Wait for the installation to complete.
If this method is not suitable for you, you can also manually unzip the models into the correct xVASynth folder (xVASynth\resources\app\models\skyrim). After unzipping, you can delete the compressed voice model files.
For specific operating steps, please refer to the video.
User Discussion
After trying it out, users have expressed that it is very good, the voice is just right, and the immersion is overwhelming.
Perhaps the most groundbreaking Mod in the history of "The Elder Scrolls"!
This user has followed this Mod for a long time and believes that the Mod has directly changed "The Elder Scrolls: Skyrim" into a different game, and perhaps the way all games interact will change in the future.
Can't wait to enjoy it right away!
References:
https://www.reddit.com/r/singularity/comments/15vgk38/mantella_mod_bring_skyrim_npcs_to_life_using_ai/
https://www.nexusmods.com/skyrimspecialedition/mods/98631
免责声明：本文章仅代表作者个人观点，不代表本平台的立场和观点。本文章仅供信息分享，不构成对任何人的任何投资建议。用户与作者之间的任何争议，与本平台无关。如网页中刊载的文章或图片涉及侵权，请提供相关的权利证明和身份证明发送邮件到support@aicoin.com，本平台相关工作人员将会进行核查。
