Let’s explore the possibilities of sound synthesis in browsers. We’ll explore the basics and, as a practical example, create a Yamaha DX7 synthesizer emulator.
Web Audio API
Browsers allow you to call JavaScript objects to control and create sound. Documentation: https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API
The API provides components for creating and modifying audio signals. These components can be connected together, and their properties can be changed on a schedule.
Hello World!
Let’s look at a simple example, something like the standard “Hello World!” for programming languages.
Example HTML page code with explanatory comments.
<html><button onclick='start();'>beep</button> <!-- create a button--><script>function start() {let audioContext = new AudioContext();//create the main objectlet when = audioContext.currentTime + 0.1;//start timelet beep = audioContext.createOscillator();//beep objectbeep.frequency.value = 440;//frequency of note Abeep.connect(audioContext.destination);//send sound to the outputbeep.start(when);//start soundbeep.stop(when + 1);//stop sound after 1 second}</script></html>
Open the page https://mzxbox.ru/fmsynth/beep.html in your browser and listen to the sound signal.
Sound envelope
If you pluck a guitar string or press a piano key, you’ll notice the difference between the natural sound and the computer-generated sound. The key’s sound increases in volume at the moment you press it and gradually fades over time. In synthesizers, this effect is achieved by adjusting the ADSR envelope:
More — https://en.wikipedia.org/wiki/ADSR
Let’s expand on our previous example and add an envelope. Here’s the component connection diagram:
Example code with comments
<html><button onclick='start();'>beep AHDSR</button><script>function start() {let audioContext = new AudioContext();//create the main objectlet when = audioContext.currentTime + 0.1;//start timelet beep = audioContext.createOscillator();//beep objectlet envelope = audioContext.createGain();//gain node to set envelope of soundbeep.frequency.value = 440;//частота ноты Ляenvelope.gain.setValueAtTime(0, when);//0 at beginingenvelope.gain.linearRampToValueAtTime(1, when + 0.05);//gradually increase to 1 in 0.5 secenvelope.gain.linearRampToValueAtTime(0.5, when + 0.2);//reduce to 0.5 in 0.2 senvelope.gain.setValueAtTime(0.5, when + 0.99);//setup last volume valueenvelope.gain.linearRampToValueAtTime(0, when + 1);//reduce to 0 at endenvelope.connect(audioContext.destination);//send final sound to outputbeep.connect(envelope);//send beep sound to envelope cahngerbeep.start(when);beep.stop(when + 1);}</script></html>
Open the page https://mzxbox.ru/fmsynth/envelope.html in your browser — the sound will be smooth, louder at the beginning and quieter at the end.
Modulation of sound signal
Audio modulation is the process of modifying a carrier signal using a modulator signal. More information can be found at: https://en.wikipedia.org/wiki/Modulation
Amplitude modulation is a type of modulation in which the variable parameter of the carrier signal is its amplitude.
Frequency modulation is a type of analog modulation in which the modulating signal controls the frequency of the carrier wave. Compared to amplitude modulation, the amplitude remains constant.
Amplitude modulation
Sound from a modulator connected to the gain parameter of a node whose input is connected to a carrier:
Code example
<html><button onclick='start();'>Amplitude modulation</button><script>function start() {let audioContext = new AudioContext();let when = audioContext.currentTime + 0.1;let carrier = audioContext.createOscillator();let modulator = audioContext.createOscillator();let result = audioContext.createGain();let level = audioContext.createGain();carrier.frequency.value = 500;modulator.frequency.value = 4;level.gain.value = 0.5;//reduce to 0.5result.gain.value = 0.5;//shift to 0.5modulator.connect(level);level.connect(result.gain);carrier.connect(result);result.connect(audioContext.destination);carrier.start(when);modulator.start(when);carrier.stop(when + 2);modulator.stop(when + 2);}</script></html>
The generator produces a sine wave signal [-1; +1], but the volume should change as [0; +1] — therefore, additional transformations are required.
Open the page to listen to the sound https://mzxbox.ru/fmsynth/amplitude.html
Frequency modulation
Connection mix:
Code example
<html><button onclick='start();'>frequency modulation</button><script>function start() {let audioContext = new AudioContext();let when = audioContext.currentTime + 0.1;let carrier = audioContext.createOscillator();let modulator = audioContext.createOscillator();let level = audioContext.createGain();modulator.frequency.value = 4;level.gain.value = 200;carrier.frequency.value = 300;carrier.connect(audioContext.destination);level.connect(carrier.frequency);modulator.connect(level);carrier.start(when);modulator.start(when);carrier.stop(when + 2);modulator.stop(when + 2);}</script></html>
Open the page to listen to the sound https://mzxbox.ru/fmsynth/frequency.html
Phase modulation
Phase modulation shifts the phase of a signal, adding overtones and ultimately changing the timbre. For more information, see https://en.wikipedia.org/wiki/Phase_modulation
Connection mix:
Код примера
<html><button onclick='start();'>phase modulation</button><script>function start() {let audioContext = new AudioContext();let when = audioContext.currentTime + 0.1;let soundFrequency = 500;//frequencylet maxmodulation = 4;let carrier = audioContext.createOscillator();let modulator = audioContext.createOscillator();let level = audioContext.createGain();let phaseDelay = audioContext.createDelay();carrier.frequency.value = soundFrequency;modulator.frequency.value = soundFrequency;level.gain.setValueAtTime(0, when);level.gain.linearRampToValueAtTime(maxmodulation / (2 * Math.PI * soundFrequency), when + 2);phaseDelay.delayTime.value = 0.5 / soundFrequency;//shift by wave sizemodulator.connect(level);level.connect(phaseDelay.delayTime);carrier.connect(phaseDelay);phaseDelay.connect(audioContext.destination);modulator.start(when);carrier.start(when);modulator.stop(when + 2);carrier.stop(when + 2);}</script></html>
Open the page to listen to the sound https://mzxbox.ru/fmsynth/phase.html
AudioWorklet
The Web Audio API provides ready-made components for working with audio. Additionally, there’s an AudioWorkletProcessor component that allows you to write your own digital signal processing (DSP) code.
More — https://developer.mozilla.org/en-US/docs/Web/API/AudioWorkletProcessor
Here is the phase modulation code from the previous example, but using AudioWorkletProcessor.
<html><button onclick='start();'>phase worklet</button><script>let phaseWorkletSource = `class PhaseSineAudioWorkletProcessor extends AudioWorkletProcessor {phase = 0;cntr = 0;constructor() {super();}static get parameterDescriptors() {return [{ name: "carrierFrequency", automationRate: "a-rate" }, { name: "modulationLevel", automationRate: "a-rate" }];}readSample(inputs, xx) {let inputSumm = 0;for (let ii = 0; ii < inputs.length; ii++) {let singleInput = inputs[ii];let channelCount = singleInput.length;if (channelCount) {let channelSumm = 0;for (let ch = 0; ch < singleInput.length; ch++) {let singleChannel = singleInput[ch];channelSumm = channelSumm + singleChannel[xx];}inputSumm = inputSumm + channelSumm / channelCount;}}return inputSumm;}writeSample(outputs, xx, value) {for (let oo = 0; oo < outputs.length; oo++) {let singleOutput = outputs[oo];for (let ch = 0; ch < singleOutput.length; ch++) {let singleChannel = singleOutput[ch];singleChannel[xx] = value;}}}process(inputs, outputs, parameters) {let outSampleCount = outputs[0][0].length;let frequency = parameters["carrierFrequency"][0];let modulationLevel = parameters["modulationLevel"][0];let incrementBySample = Math.PI * 2 * frequency / sampleRate;for (let xx = 0; xx < outSampleCount; xx++) {let inputSumm = this.readSample(inputs, xx);let resultValue = Math.sin(this.phase + modulationLevel * inputSumm);this.writeSample(outputs, xx, resultValue);this.phase = this.phase + incrementBySample;if (this.phase >= Math.PI * 2) {this.phase = this.phase - Math.PI * 2;}}return true;}}registerProcessor("sinePhaseModuleID", PhaseSineAudioWorkletProcessor);`;function loadAudioWorkletCode(audioworkletcode, audioContext, onDone) {let blob = new Blob([audioworkletcode], { type: 'application/javascript' });let reader = new FileReader();reader.onloadend = function () {let blobURL = reader.result;audioContext.audioWorklet.addModule(blobURL).then((vv) => {onDone();});}reader.readAsDataURL(blob);}function start() {let audioContext = new AudioContext();loadAudioWorkletCode(phaseWorkletSource, audioContext, () => {playSound(audioContext);});}function playSound(audioContext) {let when = audioContext.currentTime + 0.1;let soundFrequency = 500;let maxmodulation = 4;let carrier = new AudioWorkletNode(audioContext, 'sinePhaseModuleID');let modulatorBeep = audioContext.createOscillator();let volume = audioContext.createGain();volume.gain.value = 0;let descriptors = carrier.parameters;let carrierFrequency = descriptors.get('carrierFrequency');let modulationLevel = descriptors.get('modulationLevel');carrierFrequency.value = soundFrequency;modulatorBeep.frequency.value = soundFrequency;modulationLevel.setValueAtTime(0, when);modulationLevel.linearRampToValueAtTime(maxmodulation, when + 2);volume.connect(audioContext.destination);carrier.connect(volume);modulatorBeep.connect(carrier);modulatorBeep.start(when);volume.gain.setValueAtTime(1, when);volume.gain.setValueAtTime(0, when + 2);}</script></html>
Open the page to listen to the sound https://mzxbox.ru/fmsynth/phaseworklet.html
As you can see, the result sounds the same as in the previous example, but the code has become much larger.
This approach only makes sense if you need to recompile your C++ VST plugin code to WebAssembly (https://wikipedia.org/wiki/WebAssembly) to run it in a browser.
Practice of use
After reading this, you may be wondering, “Why use FM synthesis in a browser?”
Answer: “Of course, to create music synthesizers!”
Desktop DAWs (Ableton Live, FL Studio, etc.) support plugin APIs (see https://wikipedia.org/wiki/Virtual_Studio_Technology). Anyone can write their own electronic instrument that will work in any DAW or sequencer.
In modern music, the importance of plugins is so great that beginners ask not “Which editor should I use for XXX music?”, but “Which plugin should I buy for XXX music?”
The situation is much worse online. Even Bandlab (30 million users) lacks an API for expanding its studio’s capabilities with third-party plugins.
I’m currently working on an online sequencer that implements its own plugin API. It’s still a long way off from release, but you can try it out now.
Emulation of Yamaha DX7
The Yamaha DX7 is a digital synthesizer released by Yamaha in 1983. It was very popular in the 1980s and, largely due to its low cost and compact size, became one of the best-selling models in synthesizer history — https://wikipedia.org/wiki/Yamaha_DX7
The Yamaha DX7 synthesizer uses phase modulation to synthesize instruments. Over the years, thousands of presets have been created for it, ranging from guitars to traditional and futuristic instruments.
You can see the plugin in action here:
https://rutube.ru/video/69cec43733da30418e9d56bd5225303c/
Due of using of the Web Audio API, the plugin code is much more compact in compare to existing VST implementations.
The text is too long, so code analysis and sound synthesis in DX7 will be discussed in the second part of the article.
Original — https://habr.com/ru/articles/1052640/
ссылка на оригинал статьи https://habr.com/ru/articles/1053484/