0 my god yeah still does not feel natural yeah i i wish you know the the the the room would just detect when an when a meeting is started it starting voila cool okay so you better start i should start it was your idea so yeah no so i thought that we might s t speak a little bit about speech coding because nobody is doing here speech coding and i would like to have some maybe feedback because except me i have i have supposed to some speech coding stuff here with hynek but you know that little bit at least that based on some let us say hilbert transforms using a longer temporal context and deriving some parameters would be tranf transcs transmitted to a decoder and so it is a little bit different than the stuff which people are using now which is like a l p c based and but we are at the beginning with everything more or less so still we can not d encode a speech but of course there is there are many problems which we still did not solve and the biggest one is quite similar to l p c stuff where people still do not know how to exactly encode a carrier or the source signal we know what to do with the envelope or how to approximate the envelope let us say when you have got well maybe is is the white board working or or not 0 you mean that it would be nice to plot at least s y maybe show us the general sort of outline yeah at least on paper maybe i i can i can you can write on the paper yeah yeah yeah at least you can write on the p it is it is okay you keep w okay you can just write it on the paper so what we are doing now yeah you mean yeah okay i everybody knows l p c stuff how it works right yeah celp so i do not have to present anything mores but it is m more or less very simple we have let us say at the beginning there is some speech signal what we are we might see it in very simple way that we are using some hilbert transform h at the beginning to create analytic signal the analytic signal is the signal which is a complex which you can create from the real some sequence yeah right right and if you get if you apply hilbert transform on this on such a speech signal like a real trajectory then you get a complex signal and if you take just the amplitude then you get hilbert envelope code so i well i i do not know yeah let us say ups so is this different from the original p l p 2 stuff then is that yeah it is u it is not p l p at all d cause that is what you started with was not it this this stuff that marius no but marius is it is similar stuff no he is it is pretty much the same no marius is also using like hilbert transform and then okay you are lit i know what you mean with the pu p l p they are applying p l p with some iteration with this this one approach so they are doing some smoothing of a spectrogram in 2 ways okay yeah yeah h he is parameterizing it both temporally and across the spectrum yep one was right both right but p l p is using just the spectral direction let us say you are doing for every 20 millisecond processing 11 feature vector yeah here you are processing one spectral band frequency band right yeah yep okay so it is different direction which we are w using yep but anyway this is very simple so applying that absolute absolute operation you get a hilbert envelope and of course then there is the phase yeah so it is just the complex operation and that is more or less all what we are doing so now what we know what we know is how to approximate that hilbert envelope we are ap we are applying that hilbert him what is the marius stuff yeah to that so he knows how to encode the hilbert envelope using like l s f s or l p c yep yep yep is some kind of d c t or again yeah yeah ok you are applying again to some linear prediction model after that yeah yeah but it is a it is a it is a different linear prediction model is not it but it is right it is in a it is a little bit difference in not applied in the frequency domain yeah yeah yeah it is a complex linear prediction or whatever so instead of classical l p c domain where you have a spectrum let us say something like that how c and you are applying some linear prediction model which is trying to approximate the spectrum right so this is frequency yeah in our case we are we are trying to approximate s kind of instantaneous energy yeah so if this is a time and you get one and add one frequency band you can see something like this yeah and we are trying to approximate that envelope which is yeah that hilbert envelope yep right so again we know what to more or less we know what to do with the hilbert envelope okay we know that we can apply linear prediction model and it works pretty well okay but i if you want to really construct the speech signal or the you need also the phase signal you need the phase yeah ooh which is in some sense quite simple for voiced speech signal say because use linear phase or but s the phase no you ac it is a phase or you can encode it like a carrier well yeah 0 okay so the carrier is if you know amplitude modulation right you may see something like there is one cosine which is very would have quite a f right yeah yeah same frequency well it depends on how this i i mean you are talking b yeah just the carrier okay so it is not necessarily phase but if you are using some sort of sinusoidal coding it is phase yeah yeah it is called like the phase right right yeah exactly so the this is the signal which you also have to somehow encode or transmit from a code of the decoder in order to re synthesize it right okay okay w what so what sort of bit rate do you get for the encoding of the this the hilbert transform so the s the spectrogram what do you mean w what sort of coding do you get so far for the no no no no f f yeah again for this one for the the linear prediction model for l s f yeah yeah yep well it is just a linear prediction classical way i mean 0 okay w well if you want to see it in more details i mean wo well it is no no no but what sort of bit rate do you get is it is it id do you have any sort of m idea of what sort of band widths 0 you mean yeah bit bit rate you mean main bit rate what is yeah okay yeah good sure sure so whereas we are what we are now is let us say something like one2 kilo bits per 2nd for that hilbert envelope cause 0 okay and that is in that is with the fully optimized sort of compression scheme or that is just the raw data or is it is again a roughly estimated somehow okay yeah but we still know that we can go down and right yeah sure sure you just cut out m bands and just use interpolation or whatever exactly yeah yeah i don we are using like 15 bands t so that is one point okay that is with the full spectrum basically being used then right but they are 1st they are inter 0 well integrated into 15 bands using 0 classical way like in m f c c s or in p l p you just apply some those triangular windows or something like that in order to to get a li really like just 15 bands right yeah yep yep ah okay yep yep and each band is then processed independently yeah yep so for one band we more or less have w like w 100 bits per 2nd okay that is quite good okay so and hynek was showing me how scalable it is in terms of you can take half the bands away and still get decent performance for hilbert envelope right right right you take you take 3 quarters of them away it starts to degrade but it is still intelligible that sort of thing you yeah you can just right right right yeah yep okay s and that is without doing any sort of prediction of what the missing bands might be right you just leave them out right that is what we did i mean you you shou you could be able to do some sort of code missing data sort of bayes stuff yeah yeah sure yeah sure sure yeah there are many things we would would not at all but yeah exactly okay but now what i r really trying to do is somehow approximate that carrier signal to do something with the carrier okay there are many simple things which you can do like quantization quantization for example yeah it it would be it would be nice to have a scalable scheme though would not it like because you have got a nice scalable scheme for the for for the the coding of the the smooth spectrum right it would be nice to have some sort of scalable coding scheme for the for the voice signal as well right you mean something like f that celp coding system where you are or wh wh wh what is no what is exactly scalable system well i do not know i am just what you mean by that well at different bit rates so the actual excitation as well 0 yeah yeah sure this was yeah yeah yeah because i mean it would be nice l let us say if you had 1200 bits for the for the sp smooth spectrum and 1200 bits for the voice for dif yeah right right right yeah but i think this appr approach can do that because you really can split it into several bands and then you could scale them both yeah sure sure sure and you can decide how many bands you want more or less yeah so i have a question is the scale linear or is mel scale like in m f c c yeah this is a bar scale we are using is it bar scale like in like in sp yeah okay yeah or mel mel scale so it is it is being more wider for higher frequencies and of course the s the spaces right 0 yeah type the overlapping is different right yep yeah but what i w my sense is that it is not so much important still s i mean it does not matter if you are als use mel scale or bar scale but right yeah no no just long as it is some sort of non linear but that is what we are using yeah yep okay and actually we are using just gaussian windows here instead of those triangulars because they are more no s so 0 okay okay m smoothed or like apply more the place more emphasis on the it is better than for decoding it is better to play with them because they they are not 0 anywhere you know ah yeah yeah yeah they are they are approaching or in they are in 0 but actually they are not so d you do not have to care about some 0 values here and but more or less it does not matter yeah okay we are using just gaus gaussian windows yeah okay so after you are doing l s f s so or like i mean how you are finally hand coding okay so l those are that the hilbert envelope is is approximated by linear prediction ah so you have l p c coefficients which you can transcribe into l l l s f s which are right exactly 0 and then vector quantize or whatever yeah yeah so 1st you apply these triangle filters and then do l p c coding or like yeah right exactly so 1st the approach is quite sim simple yeah so at the beginning we apply d c t in order to get to frequency domain yeah okay yeah then you apply those triangular or gaussian windows filters yeah so this is done for 12nd 8000 samples for 8 kilo hertz sampling frequency right 0 okay so for such d c t trajectory you apply those gaussian windows and then i so you split it in 2 frequency sub bands and then each one f frequency sub band is more or less you can do the l p analysis on each sub band yeah right right exactly you just apply l l p so auto correlation let us say or you can do power spectrum whatever yeah m yeah ok yeah yeah yeah yeah so you for each one you get l s f s here yeah yeah so how many l s f s you are using and well i am using now like 20 l s f s per 12nd per one frequency band so times 15 0 okay so but you how many bands you are using twen that is t it is 20 times 15 it is not 15 if you are you are not skipping any bands or like 0 ok 15 into 20 then no no if i just keep all them so 20 l s f s each one is like is quantized by 4 or 5 bits okay so it is it is yeah it is 300 ah of what 300 bits it i mean it is it is total of 300 sort of parameters per 2nd 0 yeah 2 yeah yeah yeah but phase you are not encoding anything yeah exactly phase you are directly transmitting that you mean this phase the the hilbert carrier no th yeah yeah yeah or no of course we did the phase of hilbert envelope well y you are not you are not carrying transmitting that at all really at this point in time yeah i you are not transmitting at all like w well there are many things you can do we do not well the 1st thing what you can really do is just to replace it by gaussian noise yeah yeah is noise so you can really use t like in l p c system fourier and and you get whispering type synthesis yeah yeah right exactly or you can replace it by some cosine one cosine generated in a cen center frequency of that gaussian window yep or one of these glottal pulse things yeah f and just use basic l p c 10 style right yeah which is which is probably i mean yeah if you are going to aim for like a very low rate code to below 500 bits per 2nd i guess that is that is probably fine it mi it mi yeah so what is the celp standard now but like it is it is 2 kilo bits per 2nd the lowest standard is melp i think which is 2400 the standard or the 0 that 22 kilo bits per t it has been implemented at 1200 f bits per 2nd using more more complicated quantization schemes and stuff like that i think ah ok ah some 1200 yeah i think so yeah yeah it is possible some of it i s i saw that they used matrix quantisation on the l p on the l s f parameters or whatever yeah yeah yeah but but i mean that is quite it becomes quite computationally intensive r right but i mean i imagine the same sort of approach could be taken with yours as well well i am just comparing with l p c 10 standard which is like classical l p c yeah a little bit optimized no so there in the there is i think 24 kilo bits per 2nd the bit rate for that yeah that is the same 20 melp is a lot better quality though yeah definitely yeah yeah but this is codec centered no this is no i mean l p c 10 is just l p c you have code book so 0 it is not codecs code book no no no there is not code book no 0 ok just noise or just those impulses 0 yeah celp is celp no no yeah it is it is it is around one kilo byte like i think i think i think i think celp or celp or however you pronounce that yeah there yeah yeah it is about 4800 i think yeah yeah yeah yeah 48 or something yeah that is a i think even g s m and all this g 729 yeah or based around that yeah yeah yeah on based on celp okay right at least it gives it is not terribly interesting though really but still they use a code book no so that is really it is maybe better than like using just random noise or yeah so i am well you you have well i mean you have got a bunch of options have not you you can use like a codec sided or you could use a yeah sure yeah yeah well more or less what we did not do is analysis by synthesis right so that is what celp is doing right yeah sinusoidal based no so yeah so this is what we do and that might help a lot of course yeah yeah sure sure sure yeah yeah but what is more interesting but w yeah exactly huh i i mean cause c i i i suspect there somethings like celp that is that is the question you are going to get h lot higher band width are not you i mean it is a high band width standard it will be so yeah yeah yeah yeah sure yeah for example there what i also did was a simple peak peaking algorithm so if you take a look on that spectrum of that of that carrier signal which is more or less like the cosine if you really take a look on that but it is kind of frequency shifted cosine or frequency modulated cosine okay right so it is th the frequency still to be changing yeah yeah yeah or of course the amplitude is just not constant all time it should be theoretically but practically it is not ah okay so what you can see for a 12nd signal you m more or less you already might see some like one one impulse there or spectral line for okay but usually there are many others which are s quite small but they are quite important if you do not transmit them then the signal is quite robotic or something like that so if you are really transmitting just one spectral line but this is what you can do and you can understand pretty well okay the signal is audible or if you are adjust th find 11 line okay so i mean but how one speaker and dom dom dominant one in the spectrum of that c carrier hil hilbert carrier and you just transmit this parameter yep yeah cause it is a bit tricky is not it that is it is not like a standard i am just thinking i mean what sort of opt it is not like you can not just do standard you know inverse filtering for signal reconstruction or anything like that can you no n it is even easier because what you have to do is just divide the you just multiply that hilbert envelope by the table carrier that that is all 0 okay and then you just then you just do the inverse f f t or whatever yeah yeah inverse yeah yeah exactly inverse hilbert yeah yeah okay the oth i mean the other 0 so so you just multiply that one hilbert carrier which is transcribed by l s f s by that this one i f f t signal yeah so but that is going to be susceptible to having phase problems is that when you 0 f you mean the phase of that of that carrier or output signal like yeah yeah of of course i try to play also with that and i found that it does not matter so much which what is the phase because we are using just 12nd window right yeah okay so of course there there might be some disturb disturbed when you concatenate those segments together but then the phase might be important yeah but otherwise i mean i did not find any anything important with the phase like great keep the phase 0 and it works or and now i mean there is there are there is research pu that has been published showing you know speech reconstruction from the essentially the spectrogram without phase information right right right right yeah yeah so but it is again a little bit different because they are i they are doing this in a frequency domain yes but this is in time domain more or less yeah yes but i mean you can reconstruct the the f the th the entire right the spectrogram you mean spectrogram from that can not you yeah yeah yeah so you could do it that way yeah cause i am just saying there is a heaps of there heaps of options like if y you be exactly the same as sinusoidal coding or something like that right all you need is the reconstruction of the spectrogram in order to estimate the harmonic amplitudes and right exactly what we can also do very simple is just to get the magnitude spectrogram using this approximation without the carrier just hilbert envelope and then yeah yeah but you just get it in in this way in in the way of rob is right yeah but you can also get the s short time fourier transform and keep it right ye yeah yeah just by refitting just classical frame by frame algorithm 20 milliseconds yeah but you keep just the phases instead of magnitudes yeah yeah and you can put this together right okay and it works pretty well and then the signal is not just whispered then you really hear that there is the voiceness in the signal yeah okay 0 ok but of course the quality is is not so good still d it is th but you can use it for like 23 kilo bits per 2nd and it works yeah but that is that is the big thing at the moment is not it if if you are if i if the if the exci if the if the encoding rate is 2 kilo bits per 2nd for the the excitation component of the signal like yeah right it does not matter how efficient your encoding of the spectrogram is you are always going to have a high relatively high bit rate right yeah sure yeah anyway so m maybe it would be nice if you t if you tell us what you do with the speech coding cause i d i did not see any of p h d s of you i think so if you did something so maybe it would be interesting at least to i have never done anything nah yeah yeah yeah sure well one could argue it is it is a field that is quite a sort of a niche these days is not it yeah yeah sure but yeah but it is more towards industry no in standards and yeah like the yeah because i worked in a industry for like a small time it is true but they are they used to implement all these coders g s m and i t u standards so you were when when did you do that no just before my p h d 0 okay for like few months huh yeah but that is like industry it is really a big deal like so they they they work on a lot of these standards yeah i think so yeah well did you say who said yeah but mostly they are all based on celp like mainly yeah yeah so is it nokia is being sued in the u s for their tri band usage cause you know intellectual property how you mean using that c d m a no no g g s m because they do not have g s m in the u s or tri band is g s m or i did not have g s m in the u s yeah it is not g s m right it is kind of s no c d m a is another level i think it is f okay c d m a no no but c d m is there but now slowly they are introducing g s m also now in u s like no no n well that is tri band s yeah yeah g s m tri band yeah yeah yeah tri band g s m yeah like the frequencies are different like very yeah different frequency yeah exactly yeah yeah yep and apparently nokia is in a bit of trouble over that one but yeah ah well it is interesting yeah now it is really big competition i think there is a l lot of money in that like right yeah should have been a lawyer yeah like speech recognition of course it is very interesting but people do not use it so much as s just coding just telephoning right or just well that is but i mean coding is a much more sort of it is it is it well i mean it is it is it when it boils down to just mathematics to a large extent standard maybe like established right yeah does not it they s yeah yeah it is it is just sort of it is you know communications s sort of no yeah right right so where do you think your thing is useful in c d m a or g s m no i i i think maybe in c d m a it may be m c d m a is more like well what we were what we were thinking about is 0 c d m a i mean there is the b more time domain the they they are c doing only well the problem is this in with this application or it does not have to be problem but one constraint is that we are processing 12nd of the signal you can go down of course but then you are losing little bit of the bit rate let us say right yeah sure sure yeah so but there are many applications where you really can use it because you do not care about algorithmic delay 12nd algorithmic delay like for i do not know if instead of s m s you can really have some channel where you just right yeah yeah sure sure push to talk and stuff like that yep so probably and i mean as you say i mean that delay is i mean you can squash that delay to some extent right right right yeah yeah i n i think as far as the co you know c c delilah is coding goes i do not think you can probably do much more than what is already been achieved because to get higher compression you have to you know take advantage of redundancy over longer time spans yeah right so but you know really the these long line telephones they already have delay no yep at least half a 2nd or yeah yeah sure but i mean that is transmission delay not algorithmic delay if you add those together you end up with a the nightmare scenario i guess yeah right yeah yeah but mobile m okay what if s we might have someone else joining our meeting soon and this one is 0 yeah i do not know i mean you would be you tried sinusoidal coding based approaches did not you which one s 0 sinusoidal coding well but the sinusoidal coding is usually m when you s at least what i think or what i know is when you are doing sinusoidal coding it means you just take the speech signal at the beginning right and you are trying to find to yeah to find the harmonics in a temporal domain right for that harmonics and the yeah but sure yeah m yeah tr yeah it is a yeah yeah but i mean the th the the thing is you would need ti it part of the parameter set for that is the the f the harmonic amplitudes right and then one way of compressing the harmonic amplitudes is just storing the smooth spectrum and then you can and then you can interpolate yeah 0 yeah and then picking yeah you can just pick the the the harmonic amplitudes on that spectrum when you reconstruct on that smooth version of the spectrum when you reconstruct right more or less i did it maybe different way but kind of algorithm like okay y yeah yeah well i was just saying trying to find out the strongest or the the most dominant peaks in the spectrum which is more or less the same yeah yep but maybe the bit rate is the problem no like what david was doing yeah well i i th it is high like it is really yeah yeah yeah but th it is quite a high bit rate but i mean once we it maybe it is something that is scalable but you know the the f the interesting yeah maybe you can just see how many how many harmonics he was using 20 i think like yeah maybe you can try to reduce well it i mean you can use as many harmonics as you want i m h he was just using the fundamental frequency yeah yeah right and if you smooth it doing a smooth spectrum representation of the harmonic amplitudes then you only need to store the the the fundamental frequency and and the and the maximum voice frequency and the f right and that that is it and then all this amplitude and everything else can be yeah s yeah sure and then you do multiples of right ah yeah yeah yeah yeah so and then but then the tricky the only tricky thing that we had not figured out was the phase information yeah each again yeah but y but you can still get a like you get th basically if you do phase unwrapping and all of these other tricks you end up with some sort of phase representation which is some sort of monotonically decreasing function some kind of linear yeah like it if it is unwrap you mean unwrap phase yeah unwrapped yeah yeah yeah so i mean it is something that you could conceivably represent in some sort of low sort of r you know low bit rate representation i guess right some function or like a you l right yeah and then maybe you would need some sort of correction to to correct minor phase discontinuities at some point in time yeah yeah i also play with that and for example what i found i well in this case we are not trying to encode the speech signal we are trying to encode some carrier which is much easier i would say because you can really im you can see that even cau yeah yeah sure yeah yeah yeah okay so what i was trying to encode at the phases so i just found that 2 bits is enough for that and you really do not hear the difference okay so if you keep all the spectral lines as they were there but you were just quantizing with 2 bits or 3 bits for amplitude yeah yeah 2 bits for for phase so it is okay okay the problem is that you have to transmit many of them yeah 0 cause there is not just one dominant component but usually more others which are quite small right sure sure yeah yeah yep or what you have to do then you have to decrease the the the size of not 12nd true the size of the but you have to go down because then you can get just voiced part and unvoiced part you know 12nd it is usually mixed right somehow together yeah yeah but you do not do any voice un voice decision no not now no no and you you want to avoid hard choices on voicing decision yeah yeah the that is even advantage i think for this approach i wi yeah yeah we do not no want to do that yeah because it is quite tricky yeah sorta the modeling it is really another issue like well that is one of the big problems with l p c 10 for instance yeah right yeah yeah yeah yeah sure i it stinks because this hard yeah yeah exactly yeah and then you have to combine all this harmonic model and noise model right so again you may end up some but for example what we also found is that if you use just the hilbert envelope cause that that one which is trying to approximate that energy in some low band like ar around 200 t hertz and you take a look on the signal on the trajectory you can see that it works like kind of voice detector 0 okay yeah sure because for a low energies there is usually noise yeah for high energies there is kind of a harmonicity the pe peak yeah yep so it works pretty well for that so we might use this one information for that but but that that 200 this is what they use for like saint they they do use this for for pitch marking in speech but yeah i think so yeah so this is what alan black is code uses yeah yeah but it is very speaker dependent wh on what sort of band width you would look at this hilbert envelope yeah right huh yeah depending on the pitch depending on the person y you will get a different amount of smoothing and a certain sort of so i mean s yeah yeah you know if it is a female speaker for instance you yeah that is true yeah so it is sorta tricky so that is a you have to tune lot of parameters like i am yeah yeah yeah yeah yeah that is that is a problem but anyway you do not really need hard decisions no sure like voiced unvoiced n no f so m maybe it may help you but it yeah that would be nice n not to have anything yeah y yeah but from memory that that basic script that they had was not was not very effective yeah yeah yeah i mean maybe if you did some sort of initial 1st p guess and then and then chose your smoothing factor based on that yeah right cause it it you know selecting at any particular point of what the what the pitch is at the may be inaccurate yeah yeah my but if you estimated it over a 2nd you should be able to get a decent estimate i would i would think right exactly but do you really really need it or like voiced unvoiced well i do not know maybe if you we really want to improve the the quality then we will need it i do not know still yeah but yeah yeah but it is not really you need t you do not need to specify at each point it is voiced or yeah n not now yeah well i am going to t have a time next next month or in 3 weeks so i will try to play some examples there for you and yeah huh well it would be nice to it would be nice to really see what this what you are a actually trying to reconstruct here what we are trying to reconstr this carrier signal what what it really looks l yeah how does it look like or sure and you did say it looks like a frequency modulated cosine yeah more or less it is kind of not frequency let us say amplitude modulation when you have ju just the carrier and that modulation signal right yeah and then amplitude but you are telling even frequency is also shifting no yeah like your delay it would have to be cause pitch shifts yeah yeah we yeah yeah yeah exactly yeah yeah yeah yeah yeah so then yeah then it is not exactly amplitude modulations or it is it is in between maybe yeah yeah yeah it is in between exactly yeah so it is not this and that s yeah but still yeah but anyway what also the other approach which is possible to use so yeah the other is one to keep the phase like this one like 2 spectrograms one the phase and one the magnitude yeah yeah and well i can again play it and it works on 2 kilo bits per 2nd quite what no what he was doing he was using it for speech recognition that is what marius did no like but but he was keeping all the phase information what i what i know but for speech coding no no no it was i do not know no but even the his coded like he just he he did not do anything he just transferred it that way for speech coding you mean he was aiming for he is not really kind of doing speech coding but he want to do it on music so he does not really care about the band widths or he want to have the quality no like i do not know i do not know yeah but even especially the audio series and the that that is what he was telling like m maybe yeah at least even 10 k bit 20 kilo bits also is okay right so but he is he is not i think finished with all that thing yeah he is just transmitting all the phase and then but he is encoding the envelope yeah well for example this phase information right yeah yeah yeah i think so there are m more papers which are using this approach like trying to encode the magnitude spectrum or spectrogram yeah yeah and then leaving the phase yeah the phase or you just re can again the phase can be replaced by the noise which is pretty good yeah so that is the benefit variable approach so if you really do not have a band width you just can replace it by noise and it works yeah of course then it does not sound as the original because it is more whispered or whatever but you can understand well and for example if you mention this one approach with the that trajectory of the phase of unwrap phase recre i also use because you can do it here when you get into this domain where you apply or you are trying to compute s power spectrum in in order t let us say that this kind of trajectory yeah yeah yeah it does not in fit the same frequency over time domain but here you are trying to compute doing a to do linear prediction so we are just doing f f t right yeah then applying power in order to get to s to power domain and then i f f t right yeah or so you get the auto correlation coefficients auto correlation coefficient and so before you are applying power spectrum you can again get the you have got a complex f freq frequency or complex spectrum you can get the phase right yeah they again give the phase yeah again give the phase yeah and the phase look quite similar like this so you might see it is is more less like a line yeah it is like a yeah 0 and it it does start to get a little bit n non linear towards the end right but normally that is that is when the components are not actually probably going to be audible anyway in voiced speech right yeah so yeah right but this phase is different from your carrier no yeah it is it is very different yeah like yeah because you are just doing on your yeah then you are doing in different domain i think this is in frequency domain this in time domain and also like yeah it is quite it is quite sensible to any quantization what i found yeah and yeah really it looks like the lines so you just say okay let us p let us put the line there yeah but it does not work re well you can understand but you can here that there are many artifacts which right exactly yeah sure c c cause any sort of error in that in that line is is transmitted is realized in a net as a phase error right right right yeah and the another problem is that it is not band width variable so you cannot say that this part will be the just the noise which you can do in here in this this other approach so that is why i did not play with that so m more because i found there are more disadvantages than the advantages this approach okay so that is why yeah tricky yeah but even you can use as a can you use that phase directly you mean this one whereas the d yeah this one is direct yeah if you take just the original phase and the but this is phase of your d spectrum no like it is not in time domain right yeah it is it is it is not a phase so of your signal it is not so then it is bit tricky yeah yeah yeah it is the phase no you can again put the that envelope with this phase together because envelope is twe like how okay but still you get only the the magnitude of your hilbert envelope yeah yeah yeah y y y you have s 5 slices like n across that way i guess you will not get the no no you g no you get the f yeah you get the complex complex spectrum with this no but you started with hil hilbert envelope no like magnitude of hilbert envelope no no i started with the d c t magnitude of no before d c t your d 0 no after d c t you are trying this is so kind of again a real trajectory right and you are trying to 1st find that hilbert envelope yeah so let us well so this is a speech ah ok so input to the d c t is the speech or like your yeah okay yeah just you are starting with no but here what i want to say is really that y it is not part of hilbert envelope it is part of the d c t signal ah ah ok ok ok so it is not it is enough if you have just this phase and f and the hilbert envelope you can put it together yeah maybe the then yeah ah okay you do not need the carrier i mean this is kind of a carrier signal but in different domain represented yeah it is in frequency domain no right right so i what i think is that we already will need to apply s kind of analysis by synthesis approach in order to get something yeah that makes sense cause it is a time domain sort of approach right and i mean and and in in fact i mean it is what you are doing is so different to most approaches to speech coding that it does not matter that we do not know much about or have not had much i it is i mean it is quite different to a lot of the traditional sort so voi you know what i no you mean ye right source filter type approaches or or whatever yeah yeah definitely it is d right right yes that is true so but now i think people but i mean that is maybe even the advantage here because you it is not speech d based which means we are not it is just a matter of minimum distortion time yeah right so you can when i tried and comparing with l p c 10 on a twice bigger or higher bit rate this work works better it seems to me yeah just for unvoiced speech 0 i am not mentioning voiceless because that is unvoiced which means whispered sure unvoiced unvoiced speech is very difficult to b sort of judge on quality compared to voiced i think yeah but when you hear and try to use some examples with the music for example plosives yeah plosives are very important though i guess yeah right yeah so maybe it does a better job of plosives that is true yeah but anyway it seemed to me pretty good i mean that there might be okay it would be nice if we had some demos here that is true yeah well i suppose have to wait to your i guess so when is your turn it should be 13th december i think 0 0 okay 0 it is a little wa 0 bugger i mean just before i just you will not be here i will not be yeah 0 you no okay you will practice it before then will not you no i just swapped it with mike perrot i think cause h he is not going to be here this time yeah he is going to oz and i was supposed to have it in in january yep so whatever anyway so but anyway i i wanted to hear something maybe from you what you are doing f with speech coding because you were you mentioned that i did yeah but we are d we are doing a more traditional s s frequency slices rather than temporal based stuff by but it was based on like you were using kind of recognizer there right yeah well that is w that was one of the approaches sure yep low bit rate and the other approach was was this h m m based stuff which is which is basically just a parameterization of the you know the the the the the source filter type parameters and then and then learning the you know training like a h speech recognition system huh right right but that is not terribly that has got no relationship to this yeah it is i am i mean yeah it is different i it is yeah but it is speech coding at least yeah sure yes speech coding i i read some of the other techniques little standards but i mean i think i think you are on the right track with the analysis by synthesis but that does not tell you how what sort of bit rates you are going to achieve or and because you have to do it separately for each band i think so right yeah sure yeah band you mean right it would be the i imagine that is the tricky part because it would be varying quite rapidly would not it your your code book like w within each when you go along the the the b the 12nd sort of segment of the band you with different i do not know but maybe what we can already do is to decrease this this distance we can do a hilbert envelope approximation with 12nd but then we can split that signal into i do not know 1010 segments with the 100 milliseconds or even less yeah yeah and we can apply some other technique for that you know yeah well i mean yeah 1st you have got to choose your approach and not worry about the band width still right and then start thinking about how to given cause i mean i guess especially cause this is a this is a more of a commercial based project exactly exactly yeah you have g c you have you have got to i match the quality before you start worrying about other things right i mean d d do they i mean what what what sort of well th more less they do not care what is what will be the quality of the of the or the bit rate they just want to get something new you know huh okay so if you get the v very low bit rate with a reasonable quality or really high bit rate 0 okay well that makes things yeah with which is c again different but with high quality they would be happy with the both yeah they are happy yeah yeah okay so you know on one side it is pretty good but on the other you do not know which way you should go cause there are many of them cool yeah you know the the yeah sure sure and and at the moment you are at sort of a point where maybe you have to make that choice a little bit right right yeah yeah okay well i i will w do you want to mention your speech coding experiments yeah no no no yeah so you are saying that you are doing something i in india with yeah but it is mostly like this is s celp and these thing so you know the sub like more details no i was t doing some like writing some code for testing and these things where i am making some t standards for that yeah which is not so so th th there are like li the thing these standards how it works is like they give all the pseudo code kind of thing and then they put severe test conditions yeah so you have to pass those test conditions like then j you can say okay we implemented d 78 or d se 0 yeah okay yeah al although the standards depending on what standard you are dealing with sometimes it can be quite ambiguous right yeah i mean all the m peg standards are basically this is the container we do not care how you do it right yeah yeah yeah yeah yeah but it has got to fit inside this this sort of this sort of format right right yeah so that is why it is really even the t standards they give the test signals also you are t you can not really do it on whatever database you have okay so so you are to use the yeah and then you have the v and then you have to do m 0 s testing and it yeah it is more like they define so i did for some even also they are also they define all this the speech condition everything they define so you need to follow all those okay right i see yes this industry is again different like yeah yeah but it is quite interesting with the even with th s current speech coding technologies we are they are still trying to use k zap or those r p based approaches yeah i think there is not only se most yeah there is nothing new more or less well ho it and if i when i was listening the the talk of milan jelinek he is the czech guy but he lives in in canada i do not know the name of the university but they they have the pattern i think for celp even or ah okay celp is from a t and t no i do not know how it is exactly but they got very very famous for this for that celp like a t and yeah yeah it the all this l p c stuff is yeah what they did exactly i do not know if something different a little bit or not ah ok or they just keep those patterns or i do not know how it is exactly ah okay maybe hynek will know much more yeah yeah yeah but when i heard those that presentation they did not do so much new yeah well i th i mean it was last year and i think hynek no no no he was asking wh so what is new that you are so famous with that and they n they said okay we we know exactly how to do that i mean you know not to make any errors like in because everybody knows how to do that yeah the but it is not so easy to implement it right no no no so there is nothing so much new really like yep yeah that is the main thing even well the the what is what is new is in the transmission medium yeah like you know new like higher band width transmission or or like packet packetized based transmission yeah right yeah and then all th that sort of thing right the where th where they are they are not so much looking at the algorithm but so much as how to incorporate the algorithm within a new transi mission sort of s medium exactly yeah yeah like i basically s like i mean if you d if you see a lot of papers on like on voice over i p or whatever voice over i p or yeah the the publications are sort of like how do we deal with packet loss yeah yeah right or or how was it affected by packet loss and and and things like that right ri right exactly exactly but then they are s still using like error signal which is going to be approximated by code book right yeah sure and a linear prediction which is use which is approximating just the spectrum of for for and everything that that is that is all yeah yeah so still yeah and again lot of this stuff like whe even when i was working mostly that company was developing for voice over i p so they are to put all these things on again d s p s and also it is mostly like how to how many l p c s are yeah how many co yeah it is optimation and then it is even right yeah yeah and optimization of the code and all that sorta stuff yeah but even that is less of a big deal now right because all the little i s yeah yeah again now now now even they are also becoming yeah right even the memories are not really yeah not really you could trow matlab on a mobile phone these days yeah yeah yeah well maybe not but so even i think then you can have whatever band width is yeah it is short really well i hope that we still have a a lot of time for that because but no i think but still then that is what maybe people are going toward sign sort of modeling no like i was before when the bandwidth conditions were really stringent then they were using all the p l p c based source filter i think yeah it is pretty interesting yeah y yeah for example when i was you know harmonic voice modeling that h m m system which is using used for synthesis more less yeah which david did yeah yeah it is pretty interesting i mean it it works it works well yeah it is good yeah yeah if you want you can look at his code and yeah we have got all his code implemented but only th yeah yeah so it was not perfect right there was little bit of bugs with some of the matching but it was pretty hard to tell that it was m not not perfect unless you heard the samples sort of next to one another yeah yeah so that is not too bad but definitely it is better than co coded speech no right different 0 sure yeah it is really but how much you can get the compress these like what you are telling is you can do all these tricks like and i mean it yeah yeah it is well th yeah these yeah the only trick is the phase and we have got a student starting in january who is going to start working on on how to actually efficiently parameterize the the h m m yeah yeah you mean in sinusoidal modeling in yeah and i think i think we have got a number of ideas for the the spectrum that is easily enough handled just using standard techniques yeah but the and but anyway do you know exactly how it is done when you really have those m f c c coefficients which means magnitude spectrum and then you are trying to reconstruct the speech without phase without a phase i have got no idea you reviewed a paper on a because i i tried you know those th those papers well yeah i read i reviewed one paper like yeah the that is that is from no but s i am little confused about it i do not know how it works yeah maybe you can look at dates from you know east of east yeah east anglia but steven cox is not it yeah which one yeah this e east university of east anglia ben milner university of east anglia it is near c sort of it is it is near the sorta cambridge side of england i guess yeah i yeah it is yeah i can forward you that they are doing lot of tricks yeah that would be nice they are like they are again training gaussian mixtures for different frequencies and then but they can they are following the yeah 0 you know m but there are some algorithms which were quite no the much more mathematical i think yeah yeah yeah yeah yeah yeah just basically doing a search through or or stuff stuff like that do not they yeah or no no the those things are again based on l p c no like mainly like yeah it seems to me yeah something like that whereas like the pitch estimation in in the x waves or these things they are all dynamic programming based on again l p c analysis like try you know what you can do maybe yeah what you can do is take just the minimum phase of which you can get from a linear prediction right yeah you know what i mean minimum phase signal and then construct that and it sounds quite reasonably the 2 yeah but of course they are li it is not original at all so i do not know how they do that exactly huh no no i have no idea yeah again like no the problem is different for everybody like these guys what they are doing for distributor speech recognition mainly this so yeah yeah right true right sure where you have only got the m f c c s yeah m f c c s but th i think i think the i think it was to was it to do with aurora or something so no it is e t s a or something the okay european standard yeah hynek really n yeah even it is h kind of aurora framework for n for aurora we did not do anything with that i was so yeah okay yeah but we did not do any speech recon re reconstruction no but now it is bit yeah yeah now they are yeah yeah sure yeah but it w it was for say say you have got someone who has you know they want to dispute that this is what they actually said by the recognition right and the recognition network was wrong and you owe me money or whatever and then they can say well we have we have reconstructed your speech from the speech recogni input to the speech recognizer and then yeah m f c s hey voila it sounds like you i mean obviously there is a lot more to it than that there is a lot of sort of well does the quality stand up to legal sort of requirements or whatever right right right but but even at distant point they are planning to use for j standard d s m or d s it is not or 0 no that there is a separate set of standards i think for speech recognition yeah e e t s a i yeah y yeah yeah yeah european yeah commission yep so which i think requires you know that states the un the information they can send is this m yeah yeah so i do not know whether you tack on any extra phase information or whatever yeah you have to deal with what you are given yeah so yeah i yeah all these standards are really okay yeah i do not know i i it would be interesting you i mean maybe that is the best approach to look at to begin with is just pretend all you have got is you mean with this phase sure yeah because you can also interpolate the pitch information from the the spectrum as well yeah yeah but yeah they do some kind of reconstruction of speech they do that yeah yeah yeah so i mean maybe try to see how good you can go without transmitting any of that information right that is true yeah but the quality again problem no like the pitch was really good i thought i th did not think you could really tell no no the the eu again european standard what this give us they give the m f c c s and they give the pitch yeah then you are to construct the thing yeah but if have m f c c s then like again you have to get the high order m f c c s and then yeah then you are have to yeah but but the thing is if it is mel scaled if you have a mel scale you get quite fine reconstruction of the spectrum at the lower frequencies it which is what you need for reconstruction of the pitch anyway yeah right yeah yeah so i mean if you wanted to transmit with fewer bands you would maybe still keep the lower bands or or maybe you could still transmit the pitch so yeah the pitch is not that big a deal but it would be nice if you did not because then i mean that is well yep yep i mean it is worth giving it a go and then men maybe seeing wh what information you can add to the data stream to improve performance sure yeah i do not know yeah okay that that would be cool so how about you so get coding i have never done any coding in my life so i just have one curiosity so this is coding then s you code the speech then you are also supposed to transmit the speech right th let us say in g s m or something else yeah yeah i remember when i was doing some image processing there were techniques for compressing and then coding so adding channel information on the image at the same time in order to save sound bits you mean like channel and source coding together you see what i mean exactly or no i d was just wondering if there is some s someone was doing it with speech as well well i can imagine kind of like j peg operations which might be more less like it is source coding and then we also channel coding because we just smooth the or just i do not know well actually when you compress yeah all like 1st you compress the signal right i do not know and then you al you add com some complexity to the signal in order to recover the error right right right r right right so i was wondering if by any chance there was something like this in speech with the speech i do not know you do not know okay i think w i do not know so much about s channel coding like those huffman coding stuff and you know ah well i mean huffman is easy enough i mean it is just standard yeah yeah well it is not the most difficult right but but yeah sure they are just standard techniques but that that is where i stop run length encoding i am out of here but i think again it is different no like these standards again all these g s m standards yeah exactly maybe they have the compression standards and again coding stand but maybe i i read ah yeah yeah yeah sure sure and and this was the problem actually because you were i do not remember anything like yeah they are doing yeah combi yeah the even even g even normal g 79 normal speech coder no not not in g s m definitely definitely not so they have the speech coding standards again okay they are these channel coding then th they some you know source okay so yeah so it is so yeah yeah b okay just one i mean i i th i think they do try to i mean obviously the the the techniques they they choose are supposed to be less maybe you know susceptible to coding errors let us say yeah yeah you know i but i mean that is maybe the the limit to that definitely yeah again yeah i yeah so okay i think so is it good h yeah yep okay end th thank you then okay okay okay thank you okay 0 thank you guys if you find some good paper which you might there was that fam fam waveform pro prototyping cor
no 00 can not be able to go 0 good 0 my god still does not feel natural yeah i wish you know the the room just attacked when a meeting started starting okay so you could start it was your idea so i should start no so i thought that we might speak a little bit of speech coding because nobody is doing here speech coding and i would like to have some maybe feedback back because exactly me i am supposed to do some speech coding stuff here with hinek but you know that a little bit at least that it is based on some let us say hilbert transforms using longer temporal context and deriving some parameters which would be trans transmitted to the coder and so it is a little bit different than the stuff which people are using now which is like lpc based based and but we are at the beginning with everything more or less so still we can encode speech but of course there is there many problems which we still did not solve and the biggest one is quite similar to lpc stuff where people still do not know how to exactly encode the carrier or the source signal you know what to do with the envelope or how to approximate the envelope let us say when you have got maybe is the whiteboard working or no you mean that it would be nice to maybe show us yeah just on paper you can read it on the paper so what we are doing now yeah yeah you mean okay everybody knows our pc stuff how it works so i do not have to present anything more or else but it is more or less simple we have as i said at the beginning there is some speed signal what we are we might see it in a very simple way that we are using some hilbert transform at the beginning to create analytic signal the analytic signal is the signal which is a complex which you can create from the real some sequence and if you get if you apply a hilbert transform on such a speech signal like real trajectory then you get a complex signal and if you take just the amplitude then you get hilbert envelope code so i do not know yeah let us say so is this different from the original plp too stuff then is it yes it is not be a bit at all that is what you started with was not it the stuff that marius no but marius is it is only stuff no marius is all the same hilbert transform okay i know what you mean with the plp they were applying plp with some iteration with this one approach they were doing some smoothing of a spectrogram in 2 ways he is fromiturizing it both temporarily and across the spectrum but plp is using just the spectral direction let us say you are doing for every 20 milliseconds processing one feature vector here you are processing one spectral band frequency band so it is a different direction yeah anyway this is very simple so applying that absolute operation you get hilbert envelope and of course there is the face so it is just the complex operation and that is more or less all what we are doing so now what we know is how to approximate that hilbert envelope we are applying that hilbert what is the mario what is the mario is stuff so he knows how to encode the hilbert envelope some kind of dcd or like lss or lpc you are applying again some linear prediction model yeah but it is a linear prediction model but it is a different linear prediction model it is a complex linear prediction it is a little bit different it is not applied in a frequency domain so instead of classical lpc domain where you have a spectrum let us say something like that and you are applying some linear prediction model which is trying to approximate the spectrum right so this is frequency in our case we are trying to approximate kind of instantaneous energy so if this is a time and you get one frequency band you can see something like this and we are trying to approximate that envelope which is that hilbert envelope right so again we know what 2 know what to more or less we know what to do with herbert and envelope we know that we can apply their prediction model and it works pretty well but if you want to really well construct the speech signal or the you need the face you need also the face signal which is in some sense quite simple for voice speech signal say because you use linear face or the face no you it is a face or you can even call it like a carrier so the carrier is if you know modulation right you might see something like there is really one cosine which is very with quite a same frequency but depends on how this i mean you are talking yeah just the carrier okay yeah it is not necessarily phase but if you are using some it is called like the soyd or right right exactly so this is the signal which you also have to somehow encode or transmit from a code to decoder in order to resynthesize it so what sort of bit right do you get for the encoding of the the hilbert transform the spectrogram what do you mean what sort of coding do you get so far for this one no no no no the linear prediction yeah yeah well it is just the linear prediction classical way i mean 0 okay well if you want to see it in more details no no no but what sort of bit rate do you get is it do you have any sort of idea of what sort of bandwidth 0 you mean bittrate yeah bitrate yeah yeah sure sure so more as we are what we are using now is let us say something like one2 kilobits per 2nd for that hilbert envelope okay and that is with a fully optimized sort of compression scheme or that is just the raw data it is again roughly estimated so much but we still know that we can go down yeah sure sure you just cut out fewer bands and just use interpolation or whatever exactly yeah we are using like 15 bands okay that is with the full spectrum from basically being used right but they are 1st they are inter well integrated into 15 bands using a classical way like in mfcc or mpop you just apply some those triangular windows or something like that in order to to get really just 15 bands right and each band is then processed independently yeah yeah so for one band we more or less have like 100 bits for a 2nd that is quite good so here by and enoch was showing me how scalable it is in terms of you can take half the bands away and still get decent performance you take 3 quarters of them away it starts to decrade but it is still intelligible that sort of thing yeah okay and that is without doing any sort of prediction of what the missing bands might be right you just leave them out right that is what we did you could be able to do some sort of code book missing data sort of based up yeah there are many things we did not try at all but yeah exactly but now what i am really trying to do is somehow approximate that carrier signal to do something with the carrier there are many simple things which you can do like quantization scholar quantization for example it would be nice to have a scalable scheme though would not it because you have got a nice scalable scheme for the coding of the smooth spectrum it would be nice to have some sort of scalable coding scheme for the voice signal as well right you mean something like a kelp coding system where you are or well i do not know no what is exactly scalable system what you mean well different bit rates for the actual excitation as well yeah because i mean it would be nice let us say if you had 1200 bits for the smooth spectrum and 1200 bits for the voice but i think this approach can do that because you really can split into several bands and you can decide how many bands you want more i have a question is the scale linear or is melt scale like in m yeah this is a bark scale it is a bar scale like in so it is being more wider for high frequencies and of course the space the overlapping is different right right yeah but what i my sense is that it is not so much important still i mean it does not matter if you use malescale or biscale but no no just long as it is some sort of right yeah but that is what we are using actually we are using just gaussian windows here instead of those triangles because they are more smooth or no it is better than for decoding is better to play with them because they are not 0 anywhere you know they are approaching or in limit they are in 0 but actually they are not so you do not have to care about some 0 values here but more or less it does not matter we are using just gaussian windows okay so after you are doing lsf so like how you are finally encoding so that the hilbert envelope is approximated by linear prediction so you have lpc coefficients which you can transcribe into lss which are and then vector quantiles right exactly so 1st you apply these triangular filters and then do lpc coding or right exactly so 1st well the approach is quite simple so at the beginning we applied dct in order to get to frequency domain then you apply those triangle or gaussian windows so this is done for 12nd 8000 samples for 8 club hertz summary frequency right so for such dct trajectory you apply those gaussian windows and then so you split into frequency sub bends and then each one frequency submen is more or less you can you do lp analysis on each subband you right exactly you are just just apply llp so out of correlation let us say or you can do power spectrum whatever yeah so you for each one you get lsfs here so how many ls you are using well i am using now like 20 lsfs per 12nd per one frequency band so but you are 15 how many bands you are not skipping any bands or like no no if i just keep 15 into 20 then so to very ourselves each one is quantized by 4 or 5 bits okay so it is yeah it is 300 of what 300 bits it is total of 300 sort of parameters per 2nd true yeah yeah but face you are not encoding anything face you are directly transmitting you mean this fight the hilbert carrier or yeah yeah yeah no of course you are not transmitting that at all really at this point in time are yeah you are not transmitting at all well there are many things you can do we do not what 1st thing what you can really do is just to replace it by gaussian noise yeah yeah it is noise so you can really use like in a lpc system like whispering type synthesis exactly or you can replace it by some cosine one cosine generated in a center frequency of that gaussian window or one of these glottal pulse things and just use basic lpc 10 style right yeah which is probably i mean if you were going to aim for like a very low rate code below 500 bits per 2nd i guess that is probably fine so what is the standard knowledge kids it is 2 kilobits per 2nd the lowest standard is melp i think which is 2400 the standard it has been implemented at 1200 it is per 2nd using more complicated quantization schemes and stuff like that 1200 yeah i think so yeah yeah it is possible i saw it they used matrix quantization on the lp on the lsf parameters or whatever yeah but i mean that is quite it becomes quite computationally right but i mean i imagine the same sort of approach could be well i am just comparing in with lpc 10 standard which is like classical lpc a little bit optimized so then the there is i think 24 kilobits per 2nd to be trade for that yeah that is the same 20 mumps a lot better quality though yeah definitely but this is codexed you have codebook no i mean lpc then is just lpc 0 it is not codex no no i think just noise or just those impulses i think self or kelp or however you pronounce that it is about 4800 i think yeah it is one kilobite well the standard is 4800 or something that is i think even gsm and all this g 7 to 9 or based around that yeah on based on self at least it gives it is not terribly interesting though but still there is a code book no so that is a which may be better than using just random noise well i mean you have got a bunch of options have not you you can use like a code excited or you could use a well more or less what we did not do is analysis by synthesis right so that is what sal is doing right no yeah yeah sure sure yeah so this is what we really did not do and that might help a lot of course yeah but what is more interesting yeah exactly that is the question i suspect with something like help you are going to get a lot higher bandwidth are not you i mean it is a higher bandwidth standards it will be yeah yeah sure for example what i also did was a simple pick picking algorithm so if you take a look on the spectrum of that carrier signal which is more or less like the cosine if you really take a look on that but it is kind of frequency shifted cosine or frequency modulated cosine right so the frequency still will be changing yeah yeah yeah of course the amplitude is not constant all the time it should be theoretically but practically not so what you can see for 12nd signal you more or less you really might see some like 11 impulse there or spectral line for but usually there are many others which are quite small but they are quite important if you do not transmit them then the signal is quite robotic or something like that so if you really transmit just one spectral line but this is what you can do and you can understand pretty well this signal is audible or if you really just find one line one dominant one in the spectrum of the herbert carrier and you just transmit this parameter because it is a bit tricky is not it it is not like a standard i am just thinking i mean what sort of option it is not like you can not just do standard inverse filtering for single reconstruction or anything like that can you it is even easier because what you have to do is just divide you just multiply that hilbert anil by that hilbert carrier that is all 0 okay and then you just then you just do the inverse fft or whatever inverse yeah yeah exactly okay so you just multiply that one helbert carrier which is transcribed by lsfs yeah by that this one ifft signal so but that is going to be susceptible to having phase problems is that when you 0 you mean the face of that of that carrier output signal of course i tried to play also with that and i found that it does not matter so much which what is the face because we are using just 12nd window yeah right so of course there might be some disturbed when you can cut an end those segments together but then the face might be important yeah but otherwise i mean i did not find anything important with the face like quick keep the face 0 and it works or and i mean there is research that has been published showing you know speech reconstruction from the essentially the spectrogram without phase information right right right yeah but it is again a little different because they are doing this in a frequency domain yes but this is in time domain yes but i mean you can reconstruct the entire right the spectrogram spectrogram from that can you so you could do it that way because i am just thinking there is a heaps of options like if you would be exactly the same with soniosoidal coding or something like that all you need is the reconstruction of the spectrogram in order to estimate the harmonic amplitudes exactly what we can also do very simply is just to get the magnitude spectrogram using this approximation without a carrier just hilbert and evolve but you just get it in this way in the way of routes right right but you can also get this short time for your transform just by really and keep right just classical frame by frame algorithm 20 milliseconds but you keep just the faces yeah yeah instead of magnitudes and you can put this together okay right and it works pretty well then the signal is not just whispered and you really hear that there is the voiceless in the signal okay but of course the quiet is not so good still but you can use it for like 2 or 3 kibits per 2nd and it works yeah but that is a big thing at the moment is not it if you are like yeah if the if the encoding rates 2 kilobits per 2nd for the the excitation component of the signal it does not matter how efficient your encoding of the spectrogram is you are always going to have a high relatively high bit right yeah sure anyway so maybe it would be nice if you tell us what you do with a speech coding because i did not see any of phds of you i think so if you did something so maybe it would be interesting at least yeah yeah well one could argue it is it is a field that is quite a sort of a niche these days is not it yeah sure but yeah but it is more towards industry and standards yeah because i worked in the industry for like a small time but there they used to implement all these coders jism and itu standards so you were when did you do that no just before my phd i had for like a few months yeah but the like industry is really a big deal like so they work on a lot of these standards i think so yeah but mostly they are all based on self and like mainly yeah yeah so is being sued in the us for their tri band usage because you mean using that cdma no no g s because they do not have gsm in the us or they did not have gsm in the us yeah it is not gsm right it is cdm is another level i think no cdm is there but now slowly they are introducing gsm also well that is tri band yeah gsm tri bend yeah gsm frequencies are different yeah yeah and apparently not is in a bit of trouble over that one but yeah it is interesting yeah i know it is really big competition i think there is a lot of money in that yeah it should have been a lawyer like speech recognition of course it is very interesting but people do not use it so much as well that is just coding just telephoning right i mean coding is a much more sort of standard it is well i mean it is it is it boils down to just mathematics it is a right there does not it it is it is just a sort of communications right right so very do you think your thing is useful in cdma or jsm no i think maybe in cdme it maybe well what they were thinking about is 0 cdma i mean there more time domain they are doing newly well the problem is this in with this application or it does not have to be problem but one constraint is that we are processing 12nd of the signal you can go down of course but then you are losing a little bit of the bit right let us say right so but there are many applications where you really can use it because you do not care about algorithm delay 12nd algorithm delay like for i do not know if instead of sms you can really have some other channel yeah sure sure where you just talk and stuff right yeah so and i mean as you say i mean that delay is i mean you can squash that delay to some extent right right i think as far as the you know delayless coding goes i do not think you can probably do much more than what is already been achieved because to get higher compression you have to you know take advantage of redundancy over longer times spans so but even already this long line telephones they already have delay no at least half a 2nd yeah sure but i mean that is transmission delay not algorithmic delay if you add those together you end up with a yeah but mobile we might have someone else joining our i do not know i mean you be you tried sonysoidal coding based approaches did not you which one soniastal coding 0 well but the sinusoid coding is usually when you at least what i think or what i know is when you are doing sinusoid coding it means you just take the speech signal at the beginning right and you are trying to find to harmonics in yeah to find a harmonics in a temporal domain right for that yeah it is a term yeah but i mean the thing is you need part of the parameter set for that is are the are the harmonic amplitudes right now one way of compressing the harmonic amplitudes is just storing the smooth spectrum and then you can interpolate yeah you can just pick the the harmonic amplitudes on that spectrum when you reconstruct right on that smooth version of the spectrum more or less i did it maybe different way but kind of algorithm like okay trying to find out the strongest or the most dominant peaks in the spectrum which is more as the same but maybe the bitrate is the problem no like what david was doing like it is really it is quite high bit right but i mean once again it maybe it is something that is scalable but you know the yeah maybe you can just see how many harmonics he was using 20 i think like yeah maybe you can try to reduce well i mean you can use as many harmonics as you want i mean he was just using the fundamental frequency right and if you are smooth at doing a smooth spectrum representation of the harmonic amplitude you only need to store the fundamental frequency and the maximum voice frequency and everything else can be multiple so yeah sure yeah so and then but then the tricky thing that we had not figured out was the phase information you are taking yeah but but you can still get a like you get basically if you do phase unraffing and all of these other tricks you end up with some sort of phase representation which is some kind of monotonically decreasing yeah if you if it is unveraping i mean unveraping phase yeah so i mean it is something that you could conceivably represent in some sort of low sort of you know low bit rate representation i guess right and then maybe you would need some sort of correction to correct minor phase discontinuities at some point in time i also play with that and for example what i found well in this case we are not trying to encode the speech signal we are trying to encode some carrier which is much easier i would say yeah yeah you can see that even yeah okay so where i was trying to encode the phases so i just found just 2 bits is enough for that and you really do not hear the difference so if you keep all the spectral lines as they were there but you are just quantizing with 2 bits or 3 bits for amplitude 2 bits 4 of your face so it is okay okay the problem is that you have to transmit many of them because there is not just one domain component but usually more others which are quite small right or what you have to do then you have to decrease the size of not 12nd but you have to go down because then you can get just voice part and unvoiced part you know 12nd it is usually mixed right somehow together but you do not do any voice on voice decision not now no no it is even advantage i think you want to have choices we do not want to do that because in science model modeling it is really another issue like that is one of the big problems with lpc 10 for instance it stinks because this hard exactly yeah and then you have to combine all this right harmonic model and noise models again you may end up some joining but for example what we also found is that if you use just the herbert envelope that that one which is trying to approximate that in centaural energy in some low band like around 200 hertz and you take a look on the signal on the trajectory you can see that it works like kind of a voice detection because for low energies there is usually noise for high energies there is kind of harmonicity yeah so it works pretty well for that so we might use this one information that that 200 this is what they use like i am they do use this for i think so for pitchmarking in speech this is why alan black is code uses but it is very speaker dependent on what sort of bandwidth you look at this hilbert right depending on the pitch depending on the pitch depending on the person you will get a different amount of smoothing at a certain sort of yeah so i mean you know if it is a female speaker for instance that is true yeah yeah so you have to tune a lot of parameters yeah yeah that is a problem but anyway you do not really need hard decisions no like voiced and voice 0 so maybe it may help everybody that would be nice not to have anything later yeah but from memory that basic script that they had was not very effective i mean maybe if you did some sort of initial 1st guess and then then shows your smoothing factor based on that yeah yeah selecting any particular point of what the pitch is at the point is may be inaccurate but if you estimated it over a 2nd you should be able to get a decent estimate exactly but do you really really need it or like well i do not know maybe if we really want to improve the quality then we will need it i do not know but it is not really you do not need to specify at each point not now yeah well i am going to have a time next month or in 3 weeks so i will try to play some examples there yeah it would be nice to really see what this what you are actually trying to reconstruct here what we are trying to this carrier signal what it really how does it look like yeah and you did say it looks like a frequency modulated cosine yeah more or less it is kind of not frequency let us say amplitude modulation where you have just the carrier and that modulations now right but you are changing even frequency is also shifting not like you are telling you what have to make is pitch shifts yeah yeah exactly so then yeah then it is not exactly amplitude modulation so it is in between maybe it is in between so it is not this and that but still yeah but anyway what also the another approach which is possible to use the the other is one to keep the face like this 12 spectrograms one the face one the magnitude yeah and well i can again play it and it works on 2 kilobits per 2nd that is what marius did no but he was keeping all the face in front no what he was doing he was using for speech recognition what i know but i know that he is not he did not do anything he just transmitted it that is why he was aiming for speech coding i mean he is not really kind of doing speech coding but he wants to do it on music so he does not really care about the bandwidth so he want to have the quality i do not know i do not know but even especially the audio cds and that is what he was telling like yeah at least even 10 kb 20 kilobits also is okay right so yeah but he is he is not touching face at all i think he is just transmitting all the face and then well for example the envelope yeah right yeah yeah i think so there are more papers which are using this approach like trying to encode the magnitude and then leaving the face or you just really can again the face can be replaced by the noise which is pretty good so that is the bandwidth variable approach so if you really do not have a bandwidth you just can replace it by noise and it works of course then it does not sound as the original because it is more whispered or whatever but you can understand well and for example if you you mentioned this one approach with that trajectory of the face of unverp face trajectory are also used because you can do it here when you get into this domain where you apply or you are trying to compute power spectrum in order let us say that this kind of trajectory it does not fit this in frequency or time domain but here you are trying to compute doing to do linear prediction so we are just doing fftr right then applying power in order to get to power domain and then ifft right so you get the auto collection coefficient so before you are applying power spectrum you can again get the you have the complex frequency or complex spectrum you can get the face right and the face look quite similar like this so you might see it is yes more or less like a line yeah and it does start to get a little bit right non linear towards the end but normally that is yeah that is when the components are not actually probably going to be audible anyway in voice speech right right but this phase is different from your career yeah it is very different yeah because you are just doing on your you are doing in different domain yeah and this is in frequency domain this entire way it is quite sensible to any quantization what i found really it looks like the line so you just say okay let us put the line there but it does not work well you can understand but you can hear that there are many because any sort of error in that in that that line is transmitted is realized in the net as a right and the other problem is that it is not banner with variable so you cannot say that this part will be just the noise which you can do here in this this approach so that is why i did not play with that some more because i found there more these advantages than the advantages this approach so that is why but even you can use it can use that face directly you mean this one yeah just the original face and but this is face of your spectrum like it is not tip time yeah it is not a face of your signal it is not yeah yeah so then it is pretty tricky like how no you can again put that envelope with this face together because but still you get only the magnitude of your hilbert envelope you do not no you have face slices like slices like a w spectrum yeah you get a complex spectrum with this no but you started with hilbert envelope no like magnitude of no no i started with dct no before dct you are no after dct you are trying this is kind of again a real trajectory right and you are trying to 1st find that hilbert envelope so else so you input to the dct is the speech or like your this is a speech okay just you are starting no but here what i want to say is really that it is not part of hilbert enelope it is part of the dct skills you know so yeah maybe the then it is enough it is enough if you have just this face and and the hilbert envelope you can put it together you do not need the career i mean this is kind of carrier signal but in different domain represented yeah it is in frequency domain right right so i what i think is that we are really need to apply kind of analysis by synthesis approach in order to yeah that makes sense because it is a time domain sort of right approach and i mean and in fact i mean that is what you are doing is so different to most approaches to speech coding that it does not matter that we do not know much about no it is i mean it is quite different to a lot of the traditional sort of voice you know source filter type approaches or whatever right right that is true but i mean that is maybe even the advantage here because you it is not speech based which means it is just a matter of minimum distortion time right so you can when i try it and comparing with lpc 10 on a twice bigger or higher bitrate this work works better it seems to me just for unvoiced speech i am not mentioning voice ness because that is unvoiced unvoiced speech is very difficult to sort of judge on quality compared to the voice dice yeah but when you hear and try to hear some examples with the music for example yeah plosives are very important no i guess right yeah so maybe it does a better job of blazers that is true yeah anyway it seemed to me pretty good okay it would be nice if we had some demos here that is true i should have to wait to your term i guess so when is your time it should be 13 december 0 bugger i just i will not be here you will be here you will practice it before then right okay i just swapped it with mike perrow i think because he is not going to be here yeah it is going to us and i was supposed to have it in a in january so whatever anyway so but anyway i wanted to hear something maybe from you what you were doing with speech coding because you mentioned that i did yeah but we are doing a more traditional by frequency slices rather than temporal based but it was based on like you were using kind of recognize it there right yeah well that was one of the approaches sure yeah and the other approach was was this based stuff which is which is basically just a parameterization of the you know the the the the source filter type parameters and then and then learning the you know training right speech recognition system but that is not terribly that is got no relationship to this yeah it is different but speech coding at this yeah sure speech cutting i read some of the other techniques or standards but i mean i think i think you are on the right track with a nois or synthesis but that does not tell you how what sort of bit rates you are going to achieve or right yeah sure and because you have to do it separately for each band right would be there i imagine that is the tricky part because it would be varying quite rapidly would not it your your code book like you are within each with different when you go along the the one 2nd sort of segment of the band maybe what we can already do is to decrease this distance we can do hilbert envelope approximation with 12nd but then we can split that signal into i do not know 10 then segments yeah we did 100 milliseconds or even less and we can apply some other technique for that you know well i mean yeah 1st you have got to choose your approach and not worry about the bandwidth right exactly exactly start thinking about how to give it because i mean i guess especially because this is a more of a commercial based project you have got to match the quality before you start worrying about other things right i mean do they i mean what sort of more or less they do not care what will be the quality of the trade they just want to get something new so if you get the very low bitrate with reasonable quality or very high bitrate which is again different but with high quality they will be happy with the both okay so you know on one side it is pretty good but on the other you do not know which way you should go yeah sure sure and at the moment you are at sort of a point where maybe you have to make that choice a little right yeah okay i will do you want to mention your speech coding experiments no no so you are saying that you are doing something in india yeah but it is mostly like this is self and this so you know the self like no i was doing some more details like writing some code for testing and these things so i am making some standards yeah which is no so there are like the thing the standards how it works is like is like they give all the pseudo code kind of thing and then they put severe test conditions so you have to pass those test conditions like 0 yeah then you can say okay we implemented g 70 although the standards depending on what standard you are dealing with sometimes it can be quite ambiguous right i mean all the latest mpeg standards are basically this is the container we do not care how you do it right but it has got a fit inside this right this sort of this sort of format even the standards give the test signals also you can not really do it on whatever database you have okay and then you have to use and then you have to do mos testing yeah it is more like yeah they define so i did for some even a co cancel also there also they define what is the overlap speech condition everything they define so you need to follow all those yes this industry is again differently you know but it is quite interesting with even with current speech technology they are still trying to use salp or those lp days there is using only there is nothing new more than if i when i was listening the the talk of milan ealing he is the czech guy but he lives in in canada i do not know the name of the university but they they have the pattern i think for kelp even for okay kelp is from at t i do not know how it is exactly but they got very very famous for this for that cell what they did exactly i do not know if something different a little bit or not or they just keep those patents or i do not know how it is exactly maybe hineg will know much more but when i heard those the presentation they did not do so much new i mean it was last year and i think he nek he was asking so what is new that you are so famous with that and then they said okay we we know exactly how to do that i mean you know not to make any errors because everybody knows how to do that but it is not so easy to implement it right there is nothing so much new yeah there is the main thing what is new is in the transmission medium like you know new like higher bandwidth transmission or like packetized based transmission and all that sort of thing where they are not so much looking at the algorithm but so much as how to incorporate the algorithm within a new transmission sort of medium like i mean if you if you see a lot of papers on like voice over ip or whatever the publications are sort of like right how do we deal with packet loss or how is it affected by packet loss and but then they are still using like air signal which is going to be approximated by code book right yeah sure and the linear prediction which is you which is approximating just the spectrum 4 months and everything that that is that is all so and again lot of this stuff like where even when i was working mostly that company was developing for voice or ip so they have to put all these things on again dsps and also it is right how to how many lpcs are yeah how many yeah it is real optimization and then but even that is less of a big deal now right because all of it all you can know no no no even they are also becoming yeah even the memories are not really not really you can throw matlab on a mobile phone these so even i think then you can have whatever bandwidth is it is not really well i hope that we still have a lot of time for that because but still then that is what maybe people are going towards sensor modeling i always before when the bandwidth conditions were really stringent then they were using all the lpc base for example when i was you know harmonic noise modeling that h m system which is using used for synthesis more it is pretty interesting i mean it is good yeah if you want you can look at his code and not yeah but only it has got implemented so it was not perfect it was a little bit of bugs with some of the phase matching but it was pretty hard to tell that it was not not perfect unless you heard the samples sort of next to one another so that is not too bad but it is better than code speech no different 0 sure yeah and i mean how much you can how much you can get compressed is like what you are telling is you can do all these tricks the only trick is the phase and we have got a student starting in january who is going to start working on how to actually efficiently parameterize the in sinusoidal modeling yeah and i think we have got a number of ideas for the spectrum that is easily enough handled just using standard techniques but anyway do you know exactly how it is done when you really have those mfcc options which means magnitude spectrum and then you are trying to reconcile the speech without face without the face i have got no idea i you know those i reviewed one paper like yeah that is i am not too confused about it i do not know yeah maybe you can look at dates from university of east yeah which one university of east anglia university of east anglia it is near sort of it is near the sort of cambridge side of if yeah i can forward you they are doing a lot of tricks they are like there again training gaussian mixtures for different frequencies and then you know but there are some other things that were quite no much more mathematical yeah yeah just basically doing a search through or stuff like that do not they or no no those things are again based on lpc it seems to me yeah something like that was like the pitch estimation in the x waves or these things you know what you can do they are all dynamic programming based on again p c what you can do is take just the minimum phase of which you can get from a linear prediction right you know what i mean minimum phase signal and then construct and it sounds quite a reasonable that you can answer but of course it is not original at all so i do not know how they do that exactly i have no idea yeah again like no the problem is different for everybody like these guys what they are doing for distributors speech recognition mainly yeah right sure right the msccc the mfccc i think i think it was was it to do with aurora or something ctsa or something european standard for aura we did not do anything even it is kind of aurora framework yeah but we did not do any speech vacuum no but now it is a bit forward it was for say say you have got someone who has you know they want a dispute that this is what they actually said right by the recognition and the recognition network was wrong and you owe me money or whatever and then they can say well we have reconstructed your speech from the speech recognition input to the speech recogniser hey voila it sounds like you i mean obviously there is a lot more to it than that there is a lot of sort of well does the quality stand up to legal sort of requirements or whatever but even at some point they are planning to use for standard gsm or this it is not that i know there is a separate sort of standards i think for speech recognition ets yeah yeah european yeah which i think requires you know the state the information they can send is this so i do not know whether you can actually tack on any extra phase information or whatever you have to deal with what you are given yeah all these standards are really yeah i do not know it would be interesting i mean maybe that is the best approach to look at to begin with is just pretend all you have got is you mean with this face yeah sure because you can also interpolate the pitch information from the spectrum as well yeah they do some kind of reconstruction of pitch yeah so i mean maybe you try to see how good you can go without transmitting any of that information but the quality again the pitch was really good i thought i did not think you could really tell no no again european standards what this give is they give the mfccs and they give the pitch yeah then you are to construct the thing but you have mfcc then again you have to get the high road mfccs and then yeah but the thing is if it is mls scaled if you have a mls scale you get quite fine reconstruction of the spectrum at the lower frequencies which is what you need for reconstruction of the pitch anyway right yeah so i mean if you wanted to transmit with fewer bands you would maybe still keep the lower bands or maybe you could still transmit the pitch the pitch is not i think it would be nice if you did not because then i mean that is well i mean it is worth giving it a go and then maybe seeing what information you can add to the data stream to improve performance sure yeah i do not know okay that would be cool so how about you so how about you i have never done any coding in my life so i just have one curiosity so this is coding then you call the speech then you are supposed to transmit the speech right yeah let us say in gsm or something else yeah i remember when i was doing some image processing there were techniques for compressing and then coding so adding channel information on the image at the same time in order to save some bits you mean like channel and source coding together exactly i was just wondering if there is some someone was doing it with with speech as well well i can imagine kind of like jpec operations which might be most like its source coding and maybe also channel coding because you are just smoothed or just well actually when you compress yeah well like 1st you compress the signal right and then you add some complex to the signal in order to recover the error right right so i was wondering if by any chance there was something like this in speech with the speech i do not know you do not know i do not know so much about channel code like those hoofman coding stuff and well i mean halfman is easy enough yeah just standard technically it is not the most difficult right yeah sure that is where i stop one length encoding i am out of here but i think again it is different know like these standards again all these json standards yeah they have the compression standards and again coding standard but maybe i and this was the problem actually because you were i do not remember any anything like they are doing combined not in gsm definitely even even g 7 and normal speech coders so they have the speech coding standards again they have the channel code then they have some source so yeah yeah i mean i think they do try to i mean obviously the techniques they choose are supposed to be less susceptible to coding areas i mean that is maybe the limit to that again yeah it is okay i think so good yeah thank you okay thank you guys hey if you find some good paper which you yeah i totally know that that was that fame waveform prototyping code no waveform prototyping is different i was suggesting like this but that is again totally did you hear that bastion klein is you know wave he was he is really fond of the guy in speech coding like this where he proposed one prototype waveform like they take the signal and then they project into 2 dimensional and then they take the faces and it is like prototypes kind of way that you can see but it is not exactly valid so they take a slow moving waveforms slow moving signal and like fast moving and then they combine so it is like so you project signal into 2 dimensional space and then separate the slow moving parts and fast moving kind of capstone or something like that no this is a big product yeah yeah it is so now people mostly use it for like residual length coding so even if you have the lpc then you can use this for residual compression 0 that is quite interesting yeah yeah so it would be good for me exactly you can really good qualities with 4 kilo so that was that waveform coding which you put right for coding paper yeah but yeah so people can use at waveform level or at residual level so if you are using lpc you can use at residual you can see residual as some signals so it is really same residual you can clearly see whether it is fast moving yeah yeah with all these parts