Firstly, real-time music apps on mobile platforms face latency issues. Think of latency as the time it takes from making an action like pressing a key on a piano synthesizer app to realizing the result i.e. hearing the sound from the speakers. Ideally, you want this time to be as short as possible, but it’s not always that easy.
Secondly, reproducing musical instruments through control interfaces on 2D displays comes with limited haptic feedback. Basically, users don’t have the same exact physical feeling of pressing a key, turning a knob, moving a slider, etc.
Thirdly, many mobile devices are constrained when it comes to audio inputs and outputs. They do not have all the ports needed to work with advanced audio setups.
With real-time music apps, the recommended latency is something below 20 milliseconds. Low latency is usually achieved through a smaller buffer size (the amount of samples the audio API requests from the audio app with each call). However, smaller buffer size implies heavier CPU computation, eventually overworking the CPU and producing audio glitches.
Ultimately, it’s about striking a decent balance. Choose a buffer size that’s not so big that it produces high latency, but also not so small that it requires a lot of computational power, which can be achieved by precise benchmarks when you have access to all the devices targeted by your app, or by a quick runtime benchmark at the start of your app.
To avoid audio glitches, you need to provide continuous valid audio data on time at every single call of the audio API. When working on your real-time audio code, the code executed on the audio thread, you need to
Apps can harness tactile interfaces to extend the way they manipulate sound through touch. You can also use external controllers, for example through standard interfaces like MIDI, to bring back physical interactions and musical gestures to mobile music apps.
Modern mobile devices support most USB audio interfaces, allowing developers to extend the audio I/O capabilities of their mobile apps and interact with external audio hardware.
Before DevBreak is a series of live, online, tech talks. Each session features a senior tech expert from an innovative company, who demonstrates how they solved major programming challenges in their business. This series is part of DevBreak, the ultimate 2-day tech festival organised by talent.io.
talent.io is Europe's leading selective recruitment platform which has over €10m fund raised and is over 170 employees strong. We help great companies build great tech teams, in the simplest way: selected companies apply directly to vetted candidates. We cover most tech roles (software engineers, data scientists, product managers, DevOps engineers, CTOs). Our platform is open to permanent positions as well as freelance assignments, both on-site and remote.
