Creating a visual novel with AI
As a professional software developer, it’s impossible to ignore the impact large language models (LLMs) are having on the industry. I toyed with the first versions of ChatGPT ⤴ back in 2022/2023 and, apart from it being a novelty and a really good chat bot, I wasn’t very impressed at the time. Especially given how often it hallucinated and gave blatantly incorrect responses as if they were correct. GitHub’s Copilot ⤴ was promising, and it did help generate boilerplate code and fill gaps based on prompts. However, I started to get really impressed when I started testing Claude ⤴.
Claude is an LLM trained specifically for writing code. I’ve been using it for a few months now to see how far it can go, and I’ve been also checking how other people are using it. Despite my initial skepticism a few years back, I do believe AI can be a huge help for developers, and I believe people who don’t know how to use it will be at a disadvantage.
I like to do small, pet projects to learn new things. To test and practice using Claude, I decided to create a very simple visual novel game in HTML and javascript, and wanted to see how quickly I could get that done. Long story short, in around 2 hours I had a very simple thing running. You can check it out here ⤴.
Development
For this experiment, I used the Cursor IDE ⤴ with the Claude 3.5 Sonnet model.
So this was my initial prompt for Claude:
I want to create a visual novel game that runs in a web browser using javascript. Create an HTML file and JS script to begin.
Note that I didn’t even specify what a “visual novel game” is. This was Claude’s response:
I’ll help you create a visual novel game that runs in the browser. I’ll start by creating the basic HTML structure and JavaScript files. The game will need a way to display text, characters, backgrounds, and handle user interactions.
This blew my mind. Claude already “knows” what a visual novel type game is, and it generated an HTML file with elements to display a background, characters, a dialog box and text, alongside a main game script in javascript. And, incredibly, it worked. I asked it to write some example scenes, and it came up with this:
Not too bad, right? Despite the weird flickering broken background image, it runs just fine. I love how it generated incredibly cliché names: Sakura High School, Mei Tanaka and Yuki Sato. These are probably the most generic Japanese names around 😂 in any case, it’s a perfectly serviceable setting.
After that, I decided to generate some images to give the “game” more visual appeal. I really don’t know how to use those image generation AIs, so I just went with one that I already knew, Dall-E, which now is openart.ai ⤴ apparently. I just used free credits and generated a blue haired, shy meganekko to be Mei; a brown haired, more assertive type to be Yuki, the class rep; an outside school shot to be the scenario-setting shot; and a classical classroom scenario. I’ve put more details about the image generation at the end of this post.
After adding these images to the project, I talked with Claude to make small improvements, like not doing fade-ins and fade-outs between each scene if the background didn’t change, moving characters to the left or to the right, small things like that. I also wanted to add some personal lines at the end, so I added some lines for the Developer character at the ending.
This is the final result:
Final remarks
I’m pretty satisfied with this small experiment.
If I had to code everything by hand, I’d probably easily spend half a day or more doing everything. I’d have to think first about how to show the images in the game screen, how to handle user input, how to draw a dialog box at the bottom, how to show the character portraits, etc etc. Each of these steps is not hard, but it demands time and some research. It’s code that’s at the core of a visual novel game, but it’s not the most important part, which should be the story. So I was amazed at how Claude basically generated all of that base code in one go, from the first prompt.
One of the metaphors I read that I thought made a lot of sense is, “AI is like a mecha for your brain”. It enhances your capabilities. It doesn’t always generate usable or correct code, but it does an awesome job at generating boilerplate code and generating simple things that you shouldn’t have to spend time doing. I know I can draw a dialog box at the bottom of the screen with rounded corners, so I shouldn’t have to spend time actually implementing it and remembering/researching all the related CSS properties. It’s also great to discuss different ideas and approaches about how to do something.
I believe that AI will bring a huge productivity boost to many, if not all, areas of knowledge. I’m not sure of all the societal impact it might have, but as a software developer, I’m really impressed by how far it has come along. And it will only get better/more scary from here.
Extra: Image generation
I clearly have no idea how to properly use these image generation AIs. I just mess around with prompts, models and pick whatever I think’s closer to what I wanted. Sometimes I try to be more specific with the prompt, but I’m not sure how that influences the actual generation. For example, for the developer character, I just wanted a black cat with classes, like, an actual cat, not a catgirl. I clearly failed miserably because I only got one image like that. The generated catgirls were cute though, and I really liked this one:
Image prompts I used:
Character: Mei Tanaka
anime, girl, detailed, dark blue hair, glasses, shy, very detailed, visual novel style, school uniform, transparent background, body from waist up, calm expression
Character: Yuki Sato
anime, girl, detailed, brown long hair, assertive, extrovert, very detailed, visual novel style, school uniform, transparent background, body from waist up, smiling
Her uniform doesn’t even match Mei’s, it’s almost the opposite color 😂
Scenario: outside school
scenario, visual novel scenario, background, outside school, no people, blue sky, few clouds, peaceful, outside school grounds, school gate, Japanese school, no characters, empty
I didn’t like this one, it doesn’t look like a Japanese high school at all. But I think it’s the less bad one I was able to get.
Scenario: classroom
scenario, visual novel scenario, background, inside school, classroom, Japanese classroom, no people, student desks, blackboard, daylight, class starting
This classroom looks really weird, with the desks misaligned and all the way by the wall with the black board. Not to mention all the blurred gibberish written on it.
Chacater: cat
animal, cat, black cat with glasses, upstanding, anime style, yellow eyes, thick rimmed glasses, alone, transparent background
Resources
- You can play the final game here ⤴, from the comfort of your browser.
- All the code and assets for this project are in my GitHub account ⤴.
- You can also find the full chat transcript with Claude in the GitHub project here ⤴ (generated by the Specstory Cursor extension - I didn’t know Cursor didn’t have a native way of exporting its chats, which I found a little disappointing)