Sound/Video Assignment #2: "Sound Fall" a collaboration with Jahnavi Shah

Playable Build: https://moawling.itch.io/sound-fall

Password: itp

Image Description: A screenshot of Jahnavi's and my game, "Sound Fall". The screenshot features a bright blue skybox with pixelated clouds and a straightforward walkway. Within the view of the camera, are various floating signs with text on them. The signs are lined up in a linear fashion. The signs texts are in order from closest to farthest:

Sign 1: I remember, not with clarity, but with feeling.

Sign 2: If I wait, this feeling can surround me.

Sign 3: These feelings are different, but similar.

Sign 4: I can fall too.

For our second assignment for Gabriel Barcia-Colombo's Sound and Video class, we were assigned to groups to design sound walks through Unity. I was interested in navigating how linearity manifests in game logic and the dimensional qualities of sound that can disrupt linearity. Together, with my partner, Jahnavi Shah, we designed a sound walk that explored the concept of memory.

Our initial vision for this game was to have various cubes that carried a "sound" from the player's memory. We were hoping to have various "sound cubes" that could be triggered through collision or other physics elements. Time permitting, we were hoping that we could even have sound cubes that spawned randomly and continuously.

Making a 3d experience in Unity was completely new for me as most of my game development experience has been through 2d engines. Thankfully, my class provided us with various pre-fabricated scripts for the audio triggers and first person controls.

One of the most frustrating problems that defined this project was figuring out box collision. For our concept, it was important for the cubes to fall from above and onto the main plane. I toggled with various rigid body, box collision, and plane collision settings for hours. For some reason, even the most promising solutions just weren't working for me! After burning through a couple of hours trying to trouble shoot this problem, we decided that the cubes falling through the plane would actually become part of the sound walk itself.

Because we had trouble figuring out the plane+box collision and figuring out how to trigger scenes the end design became one of meticulous placement and timing. We created multiple game object cubes, and timed their falls by their placement along the Y axis. The longer we wanted an object to take to drop, the higher the object had to be placed.

Image Description: A screenshot of our game in Unity. There are three windows of interest, the 'Scene' window, the 'Game' window, and the 'Inspector' window. The 'Scene' window shows all of the game objects within a scene. The 'Game' window shows the perspective of the main camera in the game. The 'Inspector' window displays various components that determine the quality of a game object.

In the Scene window, there are various blue dots, which represent various game objects in our project. These game objects are depicted far above the main plane of the game, as depicted by all three windows.

We used various planes to operate as floating signs to provide guidance on how to navigate the game conceptually and mechanically.

Image Description: A screenshot of our game. One of the floating signs is featured prominently, it reads "If I wait, this feeling can surround me". Nearby the sign is a lightning/thunder themed sound cube, which is falling down close to the sign.

For my part of the project, I was really excited to figure out how to render various pixel art textures, materials, and meshes.

I started with rendering the material to render a custom skybox (Unity manual linked). I rendered a pixel art cloud pattern in another program and imported the assets to create a new material.

Image Description: A screenshot of the Inspector window in Unity. The inspector window is showing the settings for a Skybox shader. The shader is made of 6 different pixel art cloud images.

I was hoping that I could use the same 6-sided shader to create 6-sided meshes, but unfortunately, this didn't work for the game object cubes that came with unity. I decided to use Blender to create some meshes to create assets inspired by video game tiles. Unfortunately, I couldn't figure out how to export the mesh into Unity, but I was able to use a repeating pattern for each sound cube, even if I couldn't create a unique mesh.

Image Description: A screenshot of Blender. On the left side is the unwrapped cube with a grassy tile pattern. The right features a rendered image of a cube with the texture.

While there are various things that didn't work out with this project, I feel like these limitations shaped the project conceptually. As a pixel artist, I believe that constrictions aren't restrictive, but definitive! I think that our team found unique workarounds to each problem that we encountered.

Of course, I do hope to figure out Blender and box collisions one day. Although, I look forward to learning how to coast without that knowledge as well.