← Back to blog
Technical · · 25 min read

When The Music Follows — The Results

last updated: 2025-10-07

I – Setting the Stage

I.1 - Introduction

When The Music Follows is an experimental project, as part of “The Regeling Experiment” from Stimuleringsfonds Creatieve Industrie, exploring how certain approaches to interactive music can heighten immersive stress in games. My goal was to learn what kind of music techniques and technical systems could contribute to that. So I ended up pretty much working with some music segments that were manipulated during gameplay. For example, to morph from a neutral piece of music into something more ominous, mostly through DSP (Digital Signal Processing) techniques, using MetaSounds in combination with custom C++ nodes. Together this played the key role in achieving these modular elements as part of the music system.

Early experiments made one thing clear; audio alone wasn’t enough to achieve the intended emotional impact. Visuals, even abstract ones, play a key role in reinforcing the experience. So I tried my best within my skillset to build something that feels like a small but hopefully immersive experience to help give context to the audio experience. This way, I hope to set a certain mood and give some context to the sound, while not building a perfect-looking level, as that isn’t my expertise. Still had fun and learned a lot in terms of programming and more during this journey, that definitely awakes eagerness into what’s next.

As a result I ended up designing a basic blockout level with a short gameplay loop, that I could use as a playground to put the music system with it’s audio features to the test. This blockout level has a few core mechanics, mainly revolving around entities, that are able to posses certain objects, while also using abstract colours for environment elements and creative use of shaders for certain gameplay moments. The player had to just walk from room to room, and the rooms could shift based on interactions. During the experience the music becomes reactive on entities that move and progress based on interactions of the player.

This article won’t dive too deep into the technical realm for now, but rather dive into some of my findings with some examples and elaboration, to conclude the experiment. A more thorough breakdown will be done separate from this conclusion and project on a later moment, where I aim to have isolated breakdowns of certain features in action.


I.2 - Motivation

The project is grounded in a shared passion for music, audio, interactivity, storytelling, horror, and the psychological effects of sound on the user. Many games stick to pre-composed stems or simple reactive music systems using traditional methods. I wanted to push further and explore how music could evolve dynamically in real-time, responding directly to player behaviour in relation to entities and other game elements.


I.3 - Goals

Before starting the experiment, I wrote down a few research questions I wanted to explore. I’ll revisit these at the end of the article to hopefully be able to find interesting ways to achieve these goals.

Main question: How can interactive music that responds to player actions heighten immersive stress in a game context?

Sub-questions:

  • Which adaptive music techniques practically influence player experience?
  • In what ways can music specifically raise stress/tension?
  • How does real-time player–music interaction shape stress perception?
  • What psychological effects emerge from adaptive music on stress/immersion?

This made me write down the following goals to help build the prototype:

  • Create an interactive music and modular system using Unreal Engine 5 MetaSounds.
  • Develop music that transitions from neutral to extreme in tension, using mostly DSP effects.
  • Explore how Modulation & Digital Signal Processing (DSP) of sound can elicit stress.
  • Test player experience and gather feedback.

I.4 - Music Approach (Content and Technical)

The musical foundation draws inspiration from synthwave, evolving into something darker with industrial and darksynth influences. The music was structured using both vertical and horizontal re-sequencing. I didn’t want to dive too deep into these techniques, as most of the game audio world already know about it, especially with how easy it already works with Wwise. This allowed me to focus more on experimenting with what I would see as a fresh or innovative approach in music, this is why I went more for the DSP route, to manipulate the audio, and offer unique perspective on the possibilities that appear.

Content – The WaveAssets Prepared

The music was built from a small set of prepared music stems that could be recombined and manipulated inside MetaSounds. This allowed me to create alternative mixes based on states with also taking the 2D vs 3D behaviour in consideration. Allowing for each entity to carry it’s own instrument element, while also sending a summed signal to the radio.

Vertical Layering

  • 3 instrument layers: Bass, Harmony, Drums
  • Each layer had two versions: a neutral one and a more intense variant (intensity1 and intensity2)
  • Layers could play together as a full track or be isolated for dynamic control in both 2D and 3D realm.

Horizontal Structure

  • 2 main sections (A & B) and 1 outro
  • Each section had its own harmonic structure and sometimes a different tempo
  • Transitions between sections were triggered by gameplay progression

In total, this setup produced (6 + 6 + 2) 14 audio files. Each file included an exit marker so MetaSounds knew when to seamlessly loop or trigger the next segment via OnCuePoint Triggers. This let the system keep the musical tail ringing out while starting the loop again, or jump smoothly to the next section depending on the active MusicPhase state.

The horizontal/vertical outputs were then routed to multiple DSP modules. These were modulated in real time by LFOs or gameplay parameters, giving the music flexibility to feel reactive and alive. Although the content itself was limited, the dynamic manipulation through MetaSounds opened up various possibilities in how it can be experienced by the user.

Technical

I had in mind to treat the audio content as source to be manipulated and morphed into new sounds using DSP modules prepared in UnrealEngine. I wanted to approach every element used in a modular way, by using MetaSoundPatches in the MetaSoundSource and ActorComponents in blueprints, this allowed for experimenting with a lot of elements, and decide what to use or not, but it also kept things organised and easier to modify.

Inside the MetaSoundSource, The WavePlayers went through various dynamic processing chains that were often MetaSoundPatches with a intensity mix. After that they were sent to bus writers, which allowed me to send the signal to a Source Bus that allows to play the music on a 3D position within the game, but also allowed me to process each signal on it’s own, while also giving it it’s own bus source. This allowed me to play with different presets of output based on the same music source, that could live on on it’s own or do something unexpected when close together, giving interesting results.

MetaSounds Music Structure Flowchart

This is a rough flowchart outline of how it ended up inside the MetaSoundsSource Patch, it might not 100% correspond with how it currently is, but this pretty much sums up it’s core principles in how I started setting up the system in MetaSounds:

Flowchart Music Structure


I.5 - Key Techniques for Immersive Stress

After doing research and mainly just experimenting with what could work. I ended up making a list of techniques or principles that should be visited during the experiment to potentially help achieve my goals:

  • Real-Time Audio Modulation: Using pitch, tempo, reverb, distortion, volume shifts, and granular synthesis to transform the sound so it evokes emotions that a normal playback would not.
  • Unpredictability: Unexpected or sudden musical changes or processing that only happen under certain conditions that could provoke an sense of alert when active.
  • Silence & Minimalism: Musical silence or near-silence used strategically to provoke anticipation and sense of building up unease. Is able to make users cautious that something is about to happen.
  • Syncing with Visual Cues: Visual glitches using changes in shader and distortions react to or trigger audio events. Using purely audio quickly makes it feel like something is missing, even a slight visual change in camera/lighting/shading is something that can slightly beef up or to give context to that moment.
  • Chaos vs Order: Using for example the GranularWavePlayer, it allows to morph from pretty neutral normal music to something very chaotic, almost like a swarm of terror that could play a role in alarming the user to keep running. While when things are in order it could be a moment to restore some of your breath or within to transition to something eerie.
  • Contrast: Like chaos and order, contrast relies on moving between neutral and extreme states, either gradually or suddenly, to create stress. By combining pre-baked content with real-time DSP modulation, the music can shift in loudness, pitch, and texture to provoke a stronger emotional response. For example, lowering the tone, increased intensity, or adding chaotic dissonant layers makes moments feel more stressful, especially when set against an earlier calm passage. Without contrast, players quickly adapt to constant chaos and the effect is lost. Finding the right balance of contrast is an iterative process that must be applied carefully.

II – Building the Systems

II.1 - Behind the Scenes: Triggers & Parameters for MetaSounds

A range of in-game triggers controlled the music system with elaboration in what it did in this experience.

State Change Events

  • OnEntityMovementStateUpdated: Based on movement type it would influence the sound, because the emitter would be moving chaotically, normal or standing still, but sometimes it was also used to trigger new audio cues:
    • Hazardous: Mixes in granular synthesis effects when entities become dangerous, while also taking proximity of entity in consideration for affecting certain parameters in the granular synth, affecting the amount of grains and filtering. (close is chaos / far is some room for breathing)
    • FollowTarget: Allows for the entity to follow user which could become a big chaos when all hugging close together. This is also inspired by the Weeping Angel principle from Doctor Who — it would only follow you if you don’t watch. It would become frozen when visible.
    • Idle: The entities 3D source would become static as they stop moving and just hover around their current position, making the audio function as a regular audio source.
    • OwnRoute: The entity would move randomly around it’s spawn location, with it’s 3D source attached, adding a sense of movement. This causes parameters to go from 0 to 1 based on players proximity.
  • OnEntityPossessedStateUpdated:
    • Possessed: Entity could possess a object that would then float around. This is where I updated the 3D emitter with a new preset as SoundSource, to give a feeling of being possessed. Sometimes I also added some additional static sound. Which was a sample I prepared in loop to mix in with the DSP Effects being applied to the music layers. (e.g. the TV’s that played music stems with static.)
    • Unpossessed: Would disable the sound for the TV’s as it’s not possessed anymore and change the RadioDSP-Preset back to default (from Eerie or Stutter).

There also were some states (enumerators), which allowed me to trigger various GrainPreset(s) in MetaSounds, using the GranularWavePlayer. The MusicPhase state allowed me to cycle between music segments (e.g. intro/main/bridge/outro, etc.). Each music cue had it’s own MetaSoundSource that used the music system as patch for reusability, which I also used to test multiple music versions during the early prototype phase.

MSS Music Preset: This is what you can use as template to replace audio to play different cue with the music system

Trigger Events

  • OnOverlapTriggerbox: Triggers audio events when the player enters specific zones, which could be a new music segment or triggering a preset to alter the sound gradually. With the option to trigger once, multiple times or only during certain gameplay stages.
  • OnChangeGameplayStage: Advances in music stages, triggered by entering the portal. This allows to do exactly the same as other triggers, but using it as a more building up structure to start with something small and creating increments in intensity or introducing changes in music that could make feel certain things more trippy.
  • OnInteractItem: Triggers new audio cues when interacting with key objects that could transition to new music segment or introduce a short stinger, basically whatever feels right to trigger.
  • OnDimensionEnter: Transforms the current music layer into an extreme variation, but also enables ProximityBased events on entities using GranularDelay. This in essence just fades all the existing music layers with it’s higher intensity variation from intensity 0 to 1

Each system was built to send parameters to the music manager using triggers, enumerators, booleans, float, and integer values. This manager was giving the values to the MetaSoundsSource. Allowing for a single place of controlling the complexity of the music system.

BP_MusicManager: All the events prepared to be accessed from anywhere

Continues (float) parameters

Often sending 0-1 clamped float values from various ranges (eg. in 0-4500 unreal units) to the BP_Music_Manager that then sends signal to MetaSounds. Used globally to directly alter the music and ambiance. Due to time limitation and iteration, not all float values were set up to work with gameobjects, but rather used as simple trigger, triggered by events to set a certain fixed updated value. I always try to use some form of interpolation inside the MetaSoundSource itself by default.

GameObject defined (continues):

  • SplineProgress: is based on player position and used for tension build-up, or to sometimes modulate the audio when walking trough hallways.
  • ProximityEntity: is able to mix in various DSP effects based on player distance from object and music intensity
  • ProximityRadio: Sends 0-1 float values (clamped) based on proximity with radio, allowed for mixing away the room tone as you got closer to radio to allow for focus/silence when it’s not playing yet.
  • Psyche & Terror: Entity allowed option to add their own preset in which DSP modulation preset to use, similar to ProximityEntity but by a longer proximity. Each preset modulates multiple layers in parallel using sometimes 3+ instances of pitch shift, random LFO’s, waveshapers and more in unique ways.

Event or State Defined (single):

  • GrainIntensity: Triggered by Hazardous state; mixing in the GranularWavePlayer
  • PitchShiftIntensity: Was able to lower pitch of 2D Music Output, with additional warble option that could be turned on/off.
  • Disturbance: Triggered by events; mixed in a harmonic reverb with extremely long tail, to give a eerie character to the output
  • MusicIntensity: Controlled by triggers and glitch effect to very shortly or during certain gameplay moments to be either set to 0.5 or 1. At 1 mixing in the intensity2 layer at maximum volume.

Triggers

Triggers were used to start both gameplay and utility-based events within MetaSounds, allowing sound to react dynamically to what happened in the game.

Gameplay Triggers

Gameplay triggers like TriggerStingerGlitch and TriggerStingerDamagePlayer were used to launch short bursts of DSP-Stingers. These triggered a rapid chain of glitch, reverb, and distortion effects shaped by ADSR envelopes with different settings based on the current music output, creating sudden, high-impact moments that enhanced tension.

Utility Triggers

Utility triggers were designed to control the behaviour of DSP effects, enabling or disabling certain parameters for additional precision. For example, the pitch warble trigger (TriggerStartWarble) added subtle modulation to the PitchShift effect, giving the sound a slightly unstable or “off” quality. Another example, the hall entry trigger (TriggerHallEnter), adjusted MetaSoundSource settings when entering specific areas. These triggers essentially functioned like macros, automatically applying predefined changes to shape the audio experience in real time.


II.2 - Technical Audio Features

The following features were built into the system to create a responsive and immersive audio experience, designed to work closely together, often utilizing unpredictability, proximity-based solutions, and real-time adaptation. While they were developed specifically for this psychological horror prototype, it doesn’t have to solely be applied to this genre, but can also be used in other genres, by customizing the settings to fit with your context. Of course the source also plays a key factor into how the output ends up sounding, and not every source works well with the DSP techniques used.

Some of the Audio Features

  • Runtime DSP Effects — A combination of real-time effects were used to manipulate musical elements based on gameplay context. These effects are modular and dynamic, which allowed for building tension or creating otherworldly transitions.
    • Shimmer Reverb: A reverb using a combination of multiple comb filters with various frequencies creating an echo chamber by using large feedback values, and all-pass filters to add the diffuse reverb character. Before that the signal has a pitch shift that allows to apply a shimmer-like effect. In this project I mainly made things sound extremely low and long with sometimes -15 semitones in pitch to give that feel of eeriness and reverb tail allowing for over 10 seconds.
  • Glitch/Stutter: Essentially random LFO’s with presets that repeats tiny pieces of audio that are sometimes of waveshaper affected signal, noise, bit reduction, pitch, reversed sound.
  • Radio DSP Presets: A small bank of presets for the (3D-source) radio, combining noise/LFOs/waveshapers and optional pitch modulation: Default (slightly broken/hi‑passed), Stutter (radio stutter), Eerie (pitched, “possessed” feel). Some presets also changed based on proximity of entity.
  • Granular Delay: Imagine recording tiny “grains” of your audio and playing them back at different times and pitches — like having multiple mini tape loops running simultaneously. Used to give a feeling of entity proximity by alternating the music in 2D when overlapping with entity.
  • DSP Stingers — Quick audio bursts that temporarily trigger DSP Processing via ADSR — a combination of glitch effect, shimmer reverb, distortion. Used to simulate musical ‘jump scares’ or ‘collisions’ without requiring extra samples. Light on resources and highly effective in psychological or thriller genres.
  • 3D vs 2D Audio — Blended positional (3D) and ambient (2D) music to blur the line between diegetic and non-diegetic sound. A portion of the music was routed to the 2D bus and another to the 3D bus, allowing different processing based on proximity and modulation.

  • Granular Wave Player — A modular granular synthesizer that evolves textures over time. Capable of both chaotic swarm-like effects and more pad-like textures. Especially powerful when paired with reverb to smooth transitions. Can be CPU intensive.

  • Horizontal & Vertical Resequencing — Managing how musical elements transition between segments and intensity layers in response to gameplay and tension levels. The system follows a structured A/B/C layout tied to gameplay states, but can also add or remove layers in real time through additional parameters.

  • Spline Progress Follower — Allowed audio changes based on the player’s position along a spline. Used in the final hallway to gradually shift music into a more chaotic state.


II.3 - Designing the Demonstration Level

The Blockout

Inspired by the concept of liminal spaces, where rooms appear, disappear, or feel endless and strangely familiar, and by games like Layers of Fear, which use shifting environments to distort perception, the goal was to create an eerie and unstable atmosphere that amplified psychological tension while giving context to how the music transforms. To achieve this, I designed a looping portal system that teleports the player back to the same room repeatedly while subtly changing the layout or even loading an entirely new level, making familiar spaces feel suddenly unfamiliar. This constant sense of spatial uncertainty suggests that reality itself is slowly altering around the player, blurring the line between what is real and imagined. This approach created a useful playground for experimenting with different DSP settings and combinations, while keeping the evolving music as the main focus of the experience.

Entities

To reinforce the idea that something is slightly unsettling, I wanted to play with the idea of the music sources being ‘possessed’ but not purely in a paranormal kind of way, but also embracing concepts of psychological horror supported by environmental and sound elements. These entities were placed throughout the level, each with different behaviour in both movements and audio results.

Some of them moved, others were stationary, and each reacted to the player differently or depending on which gameplay state the user would be the behaviour would change. Certain entities carried instrument layers of a more intense musical variant. Others modulated the sound with stutter, pitch shifting, or distortion the closer you approached them.

This dynamic behaviour was able to add both visual and auditory tension. Each entity had its own “audio signature,” making them feel like extensions of the music system itself. They weren’t just props in the environment; they were living elements of the soundtrack, actively contributing to the feeling that something was watching, reacting, and following you.

III – Outcome

Main question: How can interactive music that responds to player actions heighten immersive stress in a game context?

Of course the answer to this is subjective and there a tons of approaches to go for, but from this experiment that was mainly technical, the answer is that stress can be heightened most effectively by combining musical content with rich contrast to use with real-time morphing cycling trough various DSP techniques. When music morphs from calm → eerie → chaos through DSP and reacts immediately to player proximity or actions, it shifts from being neutral background presence to an active force, making players potentially feel observed, pursued, or unsafe. The effect is amplified when supported by visual cues and gameobject behaviour and when silence or minimalism creates space for tension to build.

Supporting insights:

  • Adaptive techniques that influence experience: Real-time DSP effects, granular processing, 3D vs 2D behaviour, and stingers were most impactful.
  • How music raises stress/tension: Through unpredictability, contrast, distortion, and abrupt or eerie transitions, music provokes urgency and unease.
  • Real-time interaction shaping perception: Immediate response to player input or location turned music into a reactive force, leading players to feel followed or pressured.
  • Psychological effects: Attempted to heighten the effect of paranoia, caution, and moments of accelerated movement showed music influenced both emotion and behaviour.

III.1 - Challenges

  • Visual Reinforcement: Solely relying on sound wasn’t effective enough. Abstract visual elements like colour shifts, glitches, and shape distortions were added to help ground the tension. Sound also needs to work with context, without any context it could feel empty and meaningless.
  • Randomization: Unpredictability and randomisation is powerful, but also hard to debug and should be considered only when it makes sense for the use case. A seeded random setup would have allowed consistent “random” experiences across tests. For a commercial project you would have to find a better balance to have more control in the potential chaos.
  • Balance between prebaked and runtime content: This experiment made me do a lot of things on the technical side due to scope but also to discover the possibilities, which didn’t set my focus on creating content from DAW enough. I think if there was more time I would have found a better balance, because not all things make sense to do on the engine side and it can also raise the CPU cost drastically.
  • Latency: Using various DSP with music elements in parallel (2D and 3D combined) can add latency to source, so whenever making a transition between normal and heavily processed, you could in some occasions hear it. Best would be to use similar signal flow to match latency for both sources, or use certain DSP where sync doesn’t matter as much.

III.2 - Missed Opportunities

  • Silence: I leaned toward abstract representations of silence (extremely low volume or slow movement), but true silent moments might have created a stronger contrast.
  • Environmental Audio: Limited time meant most focus went to music. Non-musical sounds (like abstract textures or drones) could’ve improved the experience.
  • Non-Diegetic Music Support: If I had more time I would have also written music elements to support certain moments in a non-diegetic way, that work seamlessly with the flow of diegetic music.
  • Level Design: While I focused on audio, game and environment design were minimal. A more polished world or even a simple tension-breaking puzzle could have elevated the entire experience.
  • Music Genre Choice: Starting from Synthwave with elements from Darksynth and Experimental music made sense for testing, but I’m pretty convinced that if I had used something acoustic or jazz-inspired with singing, it might have created a stronger emotional contrast when shifting into horror.

III.3 - Results & Conclusions

Method

To evaluate how adaptive DSP-driven music influenced player stress and immersion, two public builds (0.2.6 and 0.3.0) of the prototype were playtested. In total, 16 participants completed the experience (88% completion rate). Data was collected through an online feedback form.

Participants were asked to rate key aspects of their experience on a 1–5 scale, focusing on the relationship between music and emotional response:

Core questions:

  1. “The audio felt immersive.”
  2. “I experienced stress or tension during the game.”
  3. “The changes in music made the experience more intense.”
  4. “The changes in music made me feel the need to move faster or be more cautious.”
  5. “The music made me feel like something was watching or following me.”
  6. “Unexpected changes in sound made me question what was real or imagined in the game.”
  7. “The audio manipulated my sense of control or safety during the experience.”

Alongside these numerical scores, participants could elaborate through open-ended questions, such as:

  • “What moments or elements made you feel stressed or tense?”
  • “What do you think was missing to create more tension or immersion?”
  • “Did the choice of prepared music fit the DSP systems?”
  • “Which audio system or technique felt most effective?”

Build 0.2.6 – First Public Build

  1. Immersion: 3.7/5 — audio experience felt convincing for most players.
  2. Stress/Tension: 2.9/5 — tension was present but not experienced by everyone.
  3. Music Intensity: 3.6/5 — the system could successfully increase perceived intensity.
  4. Behavioural Response: 3.7/5 — some players reported moving faster or being more cautious.
  5. Presence & Following: 2.6/5 — feelings of being followed was weak, but present for some users.
  6. Reality Questioning: 2.6/5 — questioning reality were weaker.
  7. Control & Safety: 3.1/5 — audio manipulated the sense of control or safety for some users.

Weak spots: This build struggled most with weak contrast, missing gameplay/visual context, and technical roughness.

  • Contrast too weak — players wanted stronger shifts between calm/peaceful and extreme/chaotic states.
  • Visual/gameplay support missing — lack of clear goals, vague objectives in rooms, and neutral environment weakened immersion.
  • Silence underused — constant music made the experience predictable, less scary.
  • Aesthetic mismatch — music approach sometimes felt too neutral or not fitting for horror.
  • Technical issues — lag in later rooms, GPU/CPU heavy, sudden loud spikes breaking immersion.
  • Randomization confusion — hard to replicate events, leaving some players unsure if what they heard was intended.

Interpretation: Build 0.2.6 proved the concept, but also revealed where contrast, pacing, and visual & gameplay content alignment needed strengthening. Techniques like Runtime-DSP-Effects such as the stutter/static elements and DSP-Stingers were highlighted as especially effective.


Build 0.3.0 – Refined Iteration

  1. Immersion: 4.6/5 — participants consistently found the audio more engaging.
  2. Stress/Tension: 3.7/5 — with 50% rating 4/5 or higher (up from 33% in Build 0.2.6).
  3. Musical Intensity: 4.7/5 — nearly all players agreed the music transformations heightened intensity.
  4. Behavioural Response: 4.6/5 — participants more strongly reported moving faster or acting with caution.
  5. Presence & Following: 2.6/5 — feelings of being followed was weak, but present for some users.
  6. Reality Questioning: 2.7/5 — slightly improved but still very weak.
  7. Control & Safety: 3.7/5 — the audio contributed to a stronger sense of control and safety manipulation for more players.

Weak Spots: This build still had usability issues (unclear mechanics, controls), continuity problems, and room for more carefully managed contrast.

  • Audio continuity issues — occasional abrupt cut-offs at doors, some players noticed looping becoming predictable.
  • Contrast still improvable — though better than 0.2.6, a few still wanted stronger shifts or moments of calm vs chaos.
  • Balance of threat — some felt there was no “real” danger, others wished for more jump scares; a few thought vague threats were actually more effective.
  • Performance — some players noted frame drops on mid-range GPUs/laptops.

Interpretation: Build 0.3.0 showed clear gains in immersion, music intensity, and stress response. Techniques like GranularWavePlayer chaos-order shifts and DSP-Stingers were highlighted as especially effective.


Overall Takeaways

  • Contrast matters: Order-to-chaos transitions (granular, distortion, stutter, stingers) created the strongest tension spikes.
  • Immersion improved iteratively: Refinements in Build 0.3.0 lifted both averages and consistency.
  • Stress is subjective: Not all participants reacted strongly, but the proportion who did grew across iterations.
  • Room for growth: Silence, environmental audio, and clearer gameplay stakes could strengthen immersion further.
  • Experience: Even though this was an audio experiment, people still talked about missing visuals/gameplay and more, which was expected, but the things related to audio gave me a clear idea in which features have potential.

Conclusion

The experiment shows that interactive, DSP-driven music can contribute to immersive stress in a psychological horror context, especially when contrast and unpredictability are combined with quick and slow modulation of pitch, stutters, distortion, reverbs in both parallel and series.

It also shows it’s weaknesses and the level of complexity it brings. The best would be to find a right balance between the dynamic engine side generated and pre-baked content side, maintaining strong control while also being able to provide flexibility in dynamics that can be helpful in innovative ways to achieve immersive stress. The content side could have been given more attention to be better fitting with the context, but it feels like some features have some true potential when executed with more care.

In a game it would probably be best to use a system like this for special occasions, when the user enters a strange room for example and a ton of things are about to happen. While in other rooms it uses a more traditional approach in vertical/horizontal music.

Some people can also see use cases where this system could be used for interactive audio visual experiences using detection sensors to detect visitors, to make the digital experience more physical as well.


III.4 - Future Plan

I’m open to talk about this project in future talks, demo showcases, or workshops. If you’re curious about dynamic music systems, DSP techniques, psychological audio design, or how to get more out of MetaSounds, feel free to reach out.

I will also prepare a more in-depth breakdown and continue experimenting with certain features, since I already have ideas for comparable but different use cases. This could eventually grow into a video series under a new project title, as this project is now finalised. I still plan to do some minor refactoring to make the system more modular and organised. If I use this system or parts of it again in the future, I want it to be future-proof, customisable, and prepared in a way that sections of it can function as a plugin.

III.5 - Special Thanks

Thanks to everyone who helped testing, especially; Joost Hendriksen & Michel Rumpff Derksen for being extremely helpful in different stages of the project, Maksym Kokoiev for helping with building the GranularSynth, Joost van Dongen & Indra Saputo-Provaas for allowing me to present during Unreal Get Together, Unreal Source Discord for trouble shooting, Dan Reynolds & Brian Michael Fuller for making helpful YouTube videos!

Of course this experiment wouldn’t be made possible without Stimuleringsfonds Creatieve industrie!


References and Related Reading

adaptive-audiounreal-enginemetasoundsmusicdspresearch