The most amazing piece of software I have seen so far
by Volker Weber
[Thanks, Malte]
Comments
Amazing - this is just unbelievable !
As a hobby musician, I have seen the currently available professional studio software once during a recording session, but this outperforms everything. Let's hope the inventors have the success they desire with this piece of software.
I will believe it when there Plugin v.2 will be released. I really don't think it is possible to separate from one audio sample different notes and show them as subsamples. I think this is a kind of an very elaborate fake, or vaporware at best.
I do believe of the first part of the product - that they can transpose the sample to a different note without pitching, and that itself is very cool.
@Gregory
I doubt this is fake Melodyne is a very well used piece of software and you will have NOT heard it used on a lot of chart records (which is the point :-) ).
The manufacturers of Melodyne talk about Direct Note Access on their web site here:
http://www.celemony.com/cms/index.php?id=news&L=0
So apparently real and very cool :-)
Without having listened to the video (not possible right now), but why should features as Gregory describes not be possible? I have been using Ableton's Live and it recognizes different notes in a drum loop pretty well, allowing you to change the speed of the loop without pitching. Transposing a sample without pitching has been around for quite some time and it works well.
I'll have to see the video in an adequate environment to see what their software adds to existing stuff...
I don't know much about creating music but the video showed me now, what is possible...absolute fantastic
Damn it vowe, I can't go to Frankfurt this weekend and you show me this!
Had the chance to listen now...Oh, I heard Plug-in...this is quite lovely. Unfortunately, it didn't talk about limitations, but it seems that it deals with samples based on a single "instrument"...voice, guitar, whatever. What it probably won't be able to do is e.g. take a complete song and analyze it all apart, drum, bass, guitar and vocals. Still, it's a great plug-in and I'll have to watch out for it.
I agree with you on the single track. They only mention cords, but not instruments. I don't think that anybody can isolate instruments. That does not really matter actually, since you will always have each instrument on its own track -- unless of course you are recording an orchestra.
I can say, I am still shaking my head in disbelieve... this is just so close to magic.. :-)
Truly amazing. I forwarded this to a friend of mine, a professional composer producer and musician. I am keen on his comments ...
Here's more data than you probably wanted:
Interesting stuff. I have to do some similar kinds of things to make the Second Signal scanner monitor work -- nothing as fancy or complex, but the basis is similar.
You take a sample from the "Time" domain -- 1/5 of a second in my case, though I'm sure he's doing this at a way, way, more granular level. In any case, for me, at 44100hz sampling, in 1/5 (roughly) of a second you get 8192 samples. Take that, and run it across a Fourier Transform, and you end up with the same data but in a "Frequency" domain. In the frequency domain you can measure the "amount" (vague on purpose) of any frequency in terms of its activity over that time. So you could measure the level of activity of any "note" as it was active during that 1/5 of a second.
The problem is, you can't tell "when" during that 1/5 of a second on the frequency domain side, so unless you want very blocky sound, you're never going to be satisfied with blocks that long.
(it works for me because what I'm looking for is inherently different).
There are a bunch of variables that go into the accuracy and granularity you can measure:
More Total Samples = More granular frequency measurement, but exponentially more more calculation per block. Interestingly, you can pad the frequency sample data with 0's to increase frequency accuracy while reducing overall strength differentiation if you want.
Longer Sample Duration = More sample data, so you get better frequency measurement, but less granular frequency duration measurement. Sure, audio at 44khz sounds great. If you break it into 8192 sample blocks of frequency distribution, you're not going to be happy with it when you try to put it back together.
In my experiments, I found that a 700mhz Celeron can keep up with parsing blocks of data at about 5 per second into a frequency domain that has an accuracy of just about 1hz steps. If the code is pretty good, and assuming you're not stopping the operating system from doing other things entirely.
What he's doing isn't at all real-time though, and he's got WAY more power than a single core 700mhz Celeron to work with.
If you're not trying for real time, there are tricks you can use. You take that 44khz audio data and grab sample blocks of 8192 blocks to transform. That's going to give you a step rate of around 1hz which should be fine for what he's doing. The problem is that gives you the levels in 1hz gradients but only for a period of 1/5 of a second.
What I'd GUESS he's doing, is taking his samples with overlap, so even if he's grabbing 8192 data points, maybe all but 256 of them are overlap from the last sample. From there its tricky to keep track of but should be possible to overlap the frequency data in some way (I'm not sure if it would be additive, multiples, or more complex and I'm not going to think about it for this non-blog site)
Again, you could never do it in real time on low end hardware, but with a multi-core processor on a modern machine processing stored audio it should be just a question of granularity vs. time spent waiting for it.
Once you've isolated the frequencies and the durations at intervals that work for you, it should be easy enough to apply standard filters to them individually.
Putting them back together without screwing up the amplitudes in a huge way is probably some pretty serious math. Then taking into account the way the 2nd, 3rd, 5th, and 7th harmonics interact though the whole sample when you put it back together is enough to make my head hurt -- if he's even doing that at all.
I can see how he'd go about it -- and its brilliant work. The implementation must have been a killer.
** Of course, he may not be doing it this way at all. If the frequency response range he's looking at is much narrower and he's ONLY wanting data for certain actual notes he could just look at the raw frequency data.
The A below middle C is at 220 Hertz, the A above middle C is at 440 Hertz -- even if you were to look for variance by grabbing 24 notes in that range, you're still look at 24 frequencies instead of over 200. You could just set up a counter and see how many times a significant amplitide wave resonates with at each of the 24 frequencies you're watching for. That would be much faster, but you might loose other data. I'm not really sure. I guess you could almost make a purity mask filter out of that phenomenon -- only let the data for your 12 pure notes through and nothing else, then calculate the harmonics of each to re-create the overall tone. Something must be wrong with that idea though.
By the way, for more info on the frequencies of notes and how the math works in greater detail than I could have explained, see here....
http://www.musicmasterworks.com/WhereMathMeetsMusic.html
Nice - I'm guessing it will be used to fix the "clam" (i.e. wrong note) in a recording session alot and perhaps in other post-production type work.
Chris
Perhaps Cher will finally be in tune again :-D
Volker,
Thank you very much for posting this. I've got a brother who owns a recording studio and another friend who has one in his house.
As a part time musician (solely for my own fun), and an IT Professional it is really amazing to see that chords can be broken out like that. I've passed the information along.
Thanks again.
Perry
One of my favorite toys at the moment is my One Laptop Per Child XO laptop. One of the activities on it is called Measure, which is basically an oscilloscope connected to the microphone port. You can flip it from regular view to frequency view which means it does a real time Fourier transform on the incoming sound. Very cool and you can just about see the individual spikes of the different notes as you play different instruments. This is on a 430Mkz AMD Geode processor, as Andrew points out you don't need a huge amount of power to do a fast Fourier transform. The difficult part as I see it is isolating all of the sound of a real instrument. I can kind of figure out how to isolate three perfect sine waves playing a chord, but real instruments (apart from flutes) have much more complex waveform. I kind of understand what the guy means when he says it shouldn't work in theory but he managed to do it in practice. I am impressed at how clean the resulting sound is.
Das ist ja Wahnsinn. Mir war zwar klar, dass Dieter Bohlen nicht wirklich singen kann, aber wer damit gut umgehen kann und ein paar Tönde im Archiv hat, der zaubert kanze Virtuosen und Bands am PC herbei. Und das mit Drag & Drop. Ich bin sprachlos.
@Gregory:
"Any sufficiently advanced technology is indistinguishable from magic." - Clarke's 3rd Law
