Probably a Bad Idea...

[John Ludlow]

I've had this idea for a while now that my lack of understanding, as to what actually happens that allows a sample to accurately represent a moment in sonic time, has prevented me from getting my arms around. I keep planning to educate myself on the subject - but... then I don't. I think I know that each sample represents the complex wave at the moment of the sample. But - how does it represent it? Is it just a very particular amount of pressure in that moment? Like, is 16 bit 65,536 different, step-wise, variations in air pressure between silence and the limit of the system's head room? And 24 bit, 16,777,216 variations?

This quandary is all in service to the idea I had. I was reading about how the particular magic of the new 50-series of NVidia graphics cards is about rendering frames that don't actually exist in the graphic recording by, essentially, looking at the value of two consecutive frames for each single dot on the screen and coming up with an average value between them that might well represent what that dot would be, had it been recorded at a higher frame rate. So, presuming grey scale, for ease in example, if a dot had a blackness of 4 in the first frame, and a blackness of 6 in the second - the dot for the generated frame between them would be 5 - half way in between 4 and 6. It's called 'interpolation'. It's a pretty slick idea, I think. And they are stuffing more than one frame in between every other actual frame - resulting in really high apparent frame rates without the burden of having to store them or send them over the wire.

And I wondered why that couldn't happen in sound recording as well. Like, in between each recorded sample at 44.1 sps, you would add, in real time, one that would represent a pretty good guess at what the wave might well have looked like at that moment in time (that was not recorded) if it had been recorded - resulting in double the resolution.

Mind you, I'm not asking whether it's a good idea or not - like, ethically or morally (emphatically intending to head off that train...). Just whether it is possible using however a sample is represented, and whether the resulting synthetic samples are likely to be sort of accurate in representing the complex wave at that (unrecorded) moment in time?

My reasoning for believing it might be is that a sound wave, by definition, is changes in relative air pressure, which is an analog phenomena, and continuous, and must be all possible values in pressure in between those pressures represented by the two samples at some point in time between them, as the pressure either grows or recedes. At the half-way point in time between them - wouldn't the pressure likely be half way between too? If not - would it necessarily be at least very close?

I said it would happen 'in real time'. Actually, I guess it wouldn't necessarily have to (it would be a nifty job for an RTX 5090 though). I suppose you could use less compute to convert a lower resolution file into a higher resolution file, save it - and then play it back too. That way it could take longer to convert than it would take to play. But, of course, then your 5090 is just sitting around chewing gum...

I envision a stereo with a button you could press to cause the music you're listening to to be interpolated to twice as many samples per second in real time, resulting in a change in 'feel' that some might like.

Another possible use would be in 'compressing' sound files for digital transport. The 'decompression' on the other side would just be interpolating samples in between the ones sent.

Anyway, what do you think? Anyone with mad digital recording knowledge care to comment? Is this feasible? I'm hoping Naturally Digital is still around and can weigh in. And Bob if he's not busy and it makes his propeller spin. But any others with knowledge are also encouraged. And - if it's a bad idea technically for some reason I haven't thought of - please educate my too-curious lazy ass as to why - and then I can stop thinking about it.

[Bob L]

Already been done for many years now... its how samplerate conversion is handled.

Bob L

[jmh]

DACs use what you are suggesting for oversampling. It is almost ubiquitous in certain types of DACs because it enables simpler analog filters that cause less audio artifacts to be used in reconstructing the analog signal - and it costs less.

[Macay]

The following video may be of interest to you: https://www.youtube.com/watch?v=cIQ9IXSUzuM
From its description:

This is a video about the digital vs analog audio quality debate. It explains, with examples, why analog audio within the accepted limits of human hearing (20 Hz to 20 kHz) can be reproduced with perfect fidelity using a 44.1 kHz 16 Bit digital signal.

That video was made to answer questions regarding this article, where the following was stated:

The most common misconception is that sampling is fundamentally rough and lossy. A sampled signal is often depicted as a jagged, hard-cornered stair-step facsimile of the original perfectly smooth waveform. If this is how you envision sampling working, you may believe that the faster the sampling rate (and more bits per sample), the finer the stair-step and the closer the approximation will be. The digital signal would sound closer and closer to the original analog signal as sampling rate approaches infinity.
...
All signals with content entirely below the Nyquist frequency (half the sampling rate) are captured perfectly and completely by sampling; an infinite sampling rate is not required. Sampling doesn't affect frequency response or phase. The analog signal can be reconstructed losslessly, smoothly, and with the exact timing of the original analog signal.

[John Ludlow]

Bob:
Thanks for clarifying that. I would suppose that the tough ones are the ones dealing with sample rates that aren't divisible into each other. Like... converting from 48 to 44.1. Since you can mathematically reproduce the complex wave for any point in time, you can do it, but you'd have to calculate the value of all of the samples except the very first one by finding each converted sample's position on your reconstituted complex wave by their location somewhere in between two existing samples. And it would never be the same location between two samples, time-wise. Very tricky business. Whereas going from 96 to 48 - you could literally just throw every other sample away.

John:
Yeah, I understand much better what oversampling is now. And, as I read somewhere recently, with enough generated samples you can reconstitute 24 bit accuracy with only 20 bits.

Macay:

The debate about whether something is missing from digital music that was available in analog music has come up over and over and over since the nineties here. And the explanation is always that at 44.1, the reconstituted wave form is 100% accurate between 20 Hz and 20K Hz (basic human hearing). And that explanation seems very defendable to me.

Yet, when I mix down to 44.1 from 96, if I A/B the resulting recordings at same high volume, there is something indefinable missing from the 44.1 version that remains in the 96 one. And I'm not alone. I've heard lots of explanations as to why that might be the case - including that perhaps part of the experience isn't being picked up by our ears. And I really haven't concluded anything. But it isn't nothing. I'm pretty sure of that.

But, I don't think that interpolation would necessarily work to bring that information back anyway - since it presumably must take place either below 20 Hz or above 20K Hz - a place that would not exist on a wave form at 44.1. Theoretically, it could be so fleeting as to completely exist in between two 44.1 samples. A sudden blip in the wave form before it returns. So, it could not be reconstituted by interpolation from the two samples - as any result from that blip would be missing from the available sample data. So, maybe my idea of improving the sound via interpolation is foolhardy anyway.

Thanks all!