so yeah we put this stuff on so that so that we have some record of the whole thing what are going to say and the whole idea here is says also put the lapel on yep the whole idea of this meeting is that it is not me who will be talking all the time so so that would be a boring that would be definitely boring meeting 0 what what is the topic here s so joel is going to talk no i am going to listen yeah yeah yeah so i so the p the actually the reason why we got together is that it is ab it is about the time to a make sure that when petr leaves us i am afraid it may happen right i mean it or did you did not change your mind yet well i guess so that is mine i could not or your girlfriend did not change your mind i would like to change but i cannot so so that whatever he is whatever he has been doing is not forgotten and then people can stay on his shoulders and i hope that he will not get out entirely out of out of this problem but still we we need to continue on this yeah so i think the joel will be is is my main hope that he is he is going to pick up the key word spotting and some of these ideas and also the reason why p i p petr motlicek is here is that joel is also getting interested in opal models of tem temporal envelope yeah yeah that is true yeah and of of course pet petr for signals so about it but petr motlicek too well i think that i am more now eh both of you you know much more than me well may from different point of view so but but but you will be here and from different point of view eh because but petr motlicek is working mainly on speech coding yeah 0 yeah okay speech coding yeah and so so this is yeah i guess this is it so m i i guess that really you should be talking more than anybody else because you should be asking questions no yeah yeah well that is yeah yeah right yeah i know so what is it that you need to know so and let me just tell us so what you know already about i do not know how far you are with the s features yeah well so basically i started reading about the features that is taking the long temporal and time frame that is traps or yeah and tandem approach so you know s something like that yeah s yeah yeah so then i also started with the there was a paper on fitting a ear model to the d c t of a signal started with that and slowly getting used to the stuff to the d c d s yeah but t what christos was doing 0 i cannot say it i think yeah yeah to the hilbert envelope you mean no it is the paper by dan ellis and co right 0 this were ho how it started basically because that was marios athineos yeah and dan ellis yeah yeah yeah yeah yeah yeah they had a paper i believe it was not in icassp a s r u or something right a s r u 2004 i guess yeah it was a a s r u but that was a time when they were not using very long d segments 2003 yeah yeah okay yeah it was like 50 millisecond or something what they were playing with at the time okay yeah yeah yeah yeah yeah yeah i know so then i the story was that then marios presented that at so some meeting and we realized 0 so for yeah that might be cute thing for estimating spectral energies because that is what we want and we will the the way we have we are doing it is kind of i think it works but it is little bit inelegant somehow stupid so so 1st do the frame by based estimate yeah and then you you get stick to 100 hertz sampling 10 millisecond sampling or something and then you just pick up the trajectory so so marios was here for half a year maybe or even more and started work with petr who also j just came why that i do not think so so they were working little bit together mainly m marios was programming in matlab and petr was putting it into fancy is yeah and but and but then we were hoping th to pick up on that little bit more yeah i just re wrote in in c +plus i did not do us so much on that and we we did right you did a little bit of work well we we tried many knobs to tune on th on that yeah because you can decide what compression factor and everything okay so i tried a little bit like pick maybe 3 knobs and try to go through the 3 d 3 d cage to see what happens in which which kind of settings so this we did but we we did not because there was still a question that i do not remember right now and we never published that anywhere right huh yeah because there was some essential question i i do not remember i would have to look at it f and because be before i wou before i we could u we could write anything i i would be have to be sure that something is right there yeah yeah i could not be sure about it at the time yeah so and anyway it it takes time this this computation yeah it is it is very inefficient it takes like 100 times more than computing some m f c c s computing these features because it is huge d c t huge f f t everywhere and yeah we were also doing a serious thing that we were stepping it by 10 millisecond right yeah at the time there is no optimisation right we just stick to the trap yeah we wanted to make as close to trap as is possible which is silly but that w you have to start somewhere right that was a reasonable start yeah yeah yeah so then i continued with another paper that you are mentioning l p trap so i just moving around with that ideas and also the main idea today why i told maybe is like if you can list down what all things are there to be taken care of yeah then one by one because finally face find i do have something else so maybe to list down the things other than the what i have already taken from you y okay and also to discuss generally so so what well last time we we were s talking about what about gaussians you you have the gaussians running somehow yeah like feature extraction yeah feature extraction and then the tandem training c also so the you are able to train the net now part tandem training so you may be the only one in the whole world now because because chi fong is gone and and barry is gone no no so they are going from icsi so morgan was joking no what he informed me they are gone from icsi chi fong went to t i and and b barry is at lawrence livermore lab huh he basically got a job he he graduated and he is he will be working on things which he cannot talk about yeah he can not even say it on a microphone here i heard eh it is i yeah it is classified work typically it is and so so jo morgan was h only half joking he was s somebody was asking him you were or so s or somebody was asking him something or there was there was matthew and he said well probably we will be coming to ask you because nobody really knows anymore at icsi so please stay kay but who who is the one who who is now developing the quicknet and these things the the m l p 4 and these things th nobody because it is that was barry and now it is cl close histo actually brno is doing a a lot of that now it was barry they picked up the software and they play with the and nets ah i see yeah so that might be that brno will be at the soonest as a ultimate expert on on the whole thing i see well still we have the experts who who made the software on here you know olivier bornet and n n where which software no they ne quicknet was done well quicknet itself was done actually long time ago at icsi quicknet training yeah yes but olivier make it run on on these machines and make the cross the 2 g barrier here at idiap is that so group because we could not we could not do something i do not know what but he is he is no good in quicknet right i s i guess what yeah yeah okay so we h have to treat him well yeah okay so yeah right now i have the feature extraction part where i can get the phoneme posteriors and through the neural net so so right so then i think i need yeah so you just miss the back end which is easy because it is h t k it is everyone knows here yeah i yeah but yeah it is t k through for some other task i have run it is t k so i have the general feel okay but maybe i can take the interceptor from you and run it once on my to have it yeah okay a and you you just need from us to hear w what are the other p front ends and other features to to choose or something yeah so yeah basically so then i can can share what i ca have read and well the whole idea then is that then you start playing with it right yeah and you yeah and yes okay so how do we do it shou shou should i list it so i will use this or maybe that might be the yeah the let us do the feature extraction that we have already taken then the back end for that yeah then well like i i d i distinguish between feature extraction and then the m l p yeah because see you feed something to the m l p which are not the waves yeah okay so so well and that is that is what i call it so yeah and then the back end yeah but okay well so what what i have here what i have been working on since i was here was m f c c p l p which is always needed for compressing purposes yeah then traps with w various lengths and also you know traps now yeah traps i know but though with various with var various lengths we were trying to to start them from from on the ultimate length of 11 frame up to 12nd or even more okay length ah okay on frame frame 0 but it was so this also w then we apply the d c t on the trap itself which is some something like the gaussians okay applying some filt filtering filtering yeah basically the trap so this but this is mainly franta who was doing this so this is related to traps i was also trying the warping okay means if you have a 2nd which is a 101 samples i was just keeping the central 1010 samples yeah and then i was throwing away some some of them you know like with sampling this this 12nd of the trap to something which i which i preserved all the points in the center and just picked every 10th point at the boundaries which means the trap 101 trap trajectory shrank to say 15 points with with still preserving the maximum resolution at the center and throwing away most of the points near the boundary okay so this i was trying so these i have some results on so these are traps then the gauss traps okay then tandem system okay the tandem you +minus half because that is that is just a p l p s okay and then there is a training with m l p that is taking some context of these p l p coefficients okay huh so it is like feeding the p l p l p s or m f c s w whatever to m l p and taking some context this is the tandem so this set up we also have here okay i did not experiment with this but maybe you some more results 0 it is hemant who has the results on tandem i guess no okay he used to have before he lost all the data yeah yeah we should not smile it is alright right yeah but i think the i think the most important results are already in the thesis he is writing but he will have it again yeah so so is there some somewhere so this is tandem then then what and then there is the f d l p then we can talk more about yeah yeah that is ri which is which i have more experiments on yeah also this warping and compressions and all these things right so compression basically you are looking at i guess yeah so yeah yeah but i am using that for different different reason right so it is speech coding so what is the task is usual just take the speech of course somehow to encode it into as as small number of beat as possible and with a reasonable quality of course coding yeah and again to e decode it okay and i am using f d l p for that still playing with more less 12nd long segments huh huh so just dividing the speech into one s 12nd frames doing d c t okay huh it is just the standard wor of work and then doing some compression and computation of yeah yeah yeah sh 0 yeah well it is bil still sub bands no but you are doing the same no well yeah yeah b yeah yeah you take the the d c t and you just divide it into several sub bands frequency bands with a different but that you just did not talk about any sub band so far okay and by the way you do not do the overlap on the sub bands no no that is for compression i d i do or you do yeah but you do not do the overlap on the time domain you mean you said somethi yeah no no no no i do not do okay so that is the difference i do not need for for compression you basically do not need the yeah just 12nd with without overlapping but in frequency domain there is a overlap because you can choose which filters you want more less or just trajectories of those bands yeah so there is overlapping i am using those gaussian filters or classical those those trapet trapezoidal like in m f c c s right and p l p and then processing each f yeah p l p s p l p s m f c c or triangles triangle yeah okay tr triangulars that one then i am doing s doing independent processing of each frequency band okay so i am computing that hilbert envelope which is just the envelope of some temporal trajectory okay the speech signal yeah and more or less you can divide this into a hilbert envelope hilb hilbert carrier we use something like l p a linear prediction envelope auto regressive model and excitation signal yeah so those are 2 parts or items you need somehow to encode if you want again to get original speech or reasonable quality okay and yeah there is some compression factor which is again when you are computing that auto regressive model from f d l p you can compress the spectrum somehow in in such a way that you rather preserve those even also those dips not only peaks in the spectrum okay and it sounds better for some kind of speech signals so you know about this compression how how does it why it is okay yeah yeah yeah knows more than than than i do i just do not know ah he is new he is he is doing you most of the things so it is kind of logarithm when you use logarithm for m f c c s right s so that is good yeah so it is again compression factor something like th and but then you have to uncompress it again of course to get back the speech and there other other things which i am doing so but it is not related to speech recognition so much or but okay but certainly when it comes to like the the opal modeling of temporal trajectories even pe petr fousek is not here you probably know a lot about it by now well what well no that ap approximating hilbert envelope by opal model right right i what i a v a v l p i i d yeah as as i say when you when you are not here that that petr motlicek knows enough about it too well the beginning of the well it is yeah sure sure i know i mean it is more or less the same approach right 0 yeah sure sure sure sure because same ap yeah different application different different software or something okay there is m just more right right right yeah okay yeah so just they in th in the speech re synthesis or coding what you are doing did you try to to play with the knobs somewhere yeah with what like many v widths i m no no the compression number of bands and lengths of the so what you what you were playing with and what other result yeah yeah sure number of bands and like what to use so that it sounds the best you know that is the problem for speech coding because i do not have results like in speech recognition yeah 0 of course you you listen to it yourself so of course i have to listen it and and s well so now what i am using are 15 bands i d i am fine with that because which seems to me fit 15 frequency band which seems to me quite reasonable we can go down till like 12 and still okay i would say yeah but then you start hearing difference or also that is another problem okay once you can go down with frequency bands then you will have much more c complicated data but let us say excitation signal or that carrier which yeah because broad bands right if you go down yeah if if you make a b broader bands because because when the band is narrow the c excitation is close the carrier is close to s sinusoid basically co sinusoid yeah maybe like that no so there is more and more s frequency components let us say or something like that in one broad band i see so so you will not say that the l p c is not able to fit it so that the residual was easy so no l p c will maybe but then you have still another part sometime something like a source and a filter so when you are doing speech coding or you may almost want to dis i b yeah b use the white board yeah use maybe you can use the board for that yeah yeah i mean and that is why we have it here right so that will be hero why can we remove this when you are going to this place i do not think so i do not know now i do not think so it is quite long no no no no 0 i think no we are wired are not supposed well i mean it is it is the same like when you are doing l p l p synthesis right so you have got source signal i do not know source i think you have to push it push it as much as possible because it reacts to the except yeah and then there is some filter which is that one divided by a z and that is and that is it and the source can be a noise signal or it can be some sequence of what is that pulses periodic pulse yeah pulses right this is l p yeah this is a standard time domain l p right this is yeah yeah so and it is more less or less the same what i am doing so this is some envelope which is in our case related to hilbert envelope and this is a source signal which is hilbert carrier but since you are in a different domain this is he here we are in time domain right but this is but but you are you are really in a frequency domain right so it the line is turning to into sines right right and you p and since this this i mean because this thi this this spectrum is al has also line spectrum right 0 which lines exactly yeah so this is now going to be frequency yeah yeah okay okay okay okay okay i mean all components so if you are then if we are in a frequency th your source looks like this yeah so more or less if we can we can plot it and so what we can see is some hilbert envelope like this i see what you mean and then there is that hilbert carrier or this not hilbert carrier but this is the signal which we get from one di one frequency band so you compute d c t right from signal this is the sub band time signal yeah yeah exactly it is yeah right it is and what we get from that is this hilbert envelope right so you know that and the 2nd which should be that carrier should be something like this yeah i see it is just really close to cosine we m we might see if we do it do it properly right okay so and this is what you also have to somehow transmit or at least try to transmit in order to get i see and this is where i was thinking the residual in l p c fit of the hilbert envelope not this i was not thinking of this at all before you know right right cause there are 3 things 1st you have to encode this then the l p c coefficients and the residual after l p c fit if you like right no i do not need l p c no no this is like a residual for him 0 you do not this is a residual for me yeah in the for frequency it is this is the residual yeah i am just wan i just want to transmit this signal right it is and what you need is this envelope and this one carrier yeah well i and you again get the same signal this this this shape doing i d c t okay yeah yeah and the biggest problem currently is that the carrier is actually not cosine but something little bit more interesting putting everything together yeah right right but if you take a look for example for a s if you have a very g good music signal which is very periodic you can really see something like there is one peak and and if the frequency does not change over the wa over over the lengths of the and frequency more or less does not change it is related to that frequency of the broad band or how it is called yeah yeah that that is nice if you add those filters which are wider and wider this is that center frequency which is somehow related to this one yeah okay so then you can transmit just fe and so so so the so the broader bands do what on this picture l left left it is worse and worse do not then you do not see any anything like cosine but but f f fortunately because of he because of properties of hearing more or less but for the higher bands you if you put in just s some garbage some noise it does not really m w matter all that much it w sounds little bit worse but it is not a big tragedy where you really w if you want to g do a good voice speech you will want to get the lower bands correct right right yeah that that is and if you put in noise altogether it sounds like a whispered speech but you understand perfect everywhere it sounds like l p c when you excited by noise by noise it is better i would say yeah better quality definitely better quality it lower bit rate but that is a problem so now we have to kill them both so so because i mean gave all the secrets away yeah sure i mean qual computers is going to for any object you measure you use for a reconstructed signal and objective measure well it was some those distances spectral distances well edinburgh saw probably yes so well i mean if motorola or nokia gets hold of this tape then we are in trouble huh hopefully it by then it will be protected and and everything right although herve just said that we should not s talk about confidential things here it is it is it is but listen this is a science that is that is the problem i mean it is confidential to the extent that we are not i publishing it yet yeah but we w we of of course we will i h and and but we have to make sure that that if qualcomm has a interest of who is funding the work the they have to have a chance to to to patent it or something yeah but that is about all this is not a public public presentation alright so this i is not even a problem yeah sure the only problem would be if but you know i see nobody can beat p petr petr motlicek anymore in i mean if even if if nokia knows everything if you if you sell it tomorrow who did you work for th the text no okay t h p h p h p they will take half a year before they would even realize what it is yeah and by then so it is not no problem okay so that is where that is that is 0 okay so so there is a compromise between the the lower bands the the worse stuff you just said and and the more bands the more coefficients to transmit right so 15 is about okay yeah yeah yeah also there are papers about it that f if you have like 12 frequency bands it is somehow optimal okay so this and then there is a compression what about that compression is only to really to somehow feed that hilbert envelope to yeah to get the so that you get so that you can not model it actually yeah yeah you can model it better there is no echo or something like reve reverberance in the signal if you really go down with that compression factor to like 0 one something that yeah of what 000 compression 0 one kay so what is the compression factor we are getting here so when you comp w what you com when you compress the effectively they will want the signal which we are modeling huh right and so then you get the fitting to compress signal the sub band signal okay essentially if you were in spectral domain you would compress the power spectrum yes here you compress the the th this th this i the what ful yeah of d c t of of the signal right sub bands power spectrum it is it is s right and it is still yeah and so the w then you have predictor is fitting actually this compress signal but you know how much c it was compressed so you can always expand it back so that you get the c that you get a fit to the to the original thing to fit the the except that if it is different because it ha th basically this opal modeling has a different properties and it is yeah you just preserve something different little bit yeah so we can tell you a little bit more about it let us say in r s s this is an sort of old technique which it yeah i think hynek has got ma many papers or some papers over that and it was my thesis even so 0 p h d thesis actually in i think i read it a small sorry l p c l l p yeah it was s t l p was called spectral transform linear prediction it was my 1st icassp paper ever it was it 1983 boston you guys were still in kindergarten right yeah right more or less 0 this is terrible yeah so but no nobody nobody was using it for anything except it was cute but that was about it and and but now it seems like that is since since what we are doing here we are normal l p c is fitting peaks right coming back though yes but when you are modeling hilbert envelope you do not necessarily want to fit only peaks yes you sometimes want to get a basically good estimate of the of the envelope envelope yeah so that helps you because in this thi thi this spectral transform allows you to to find the right balance between how well it fits peaks how how well it fits dips and and yeah right exactly yeah and sort of the so this is g so i think that basically i do not know what else i mean you should be asking questions right s yeah so one thing i wanted to know is any a good literature on these lines well yeah yeah yeah go on well i think you can really take those papers from marius about f d l p where you started with yeah yeah yeah they are yeah that is one yeah 0 th 0 about 3 papers they are that is really enough on there was the s s s sapa or how it is called the confere then then i have 2 with the sapa and then yeah this is a this they are 2 papers in one is in sapa one is in u interspeech interspeech paper i and one is in in the america yes in a c cheju and one was in the a s r u it was from virgin islands yeah what is sapa sapa was a s i i tell you what it stands for statistic and ad and something ad and perceptual approaches i i do not know i just n you were i was on f i was on a f w s organizing committee i should know just something you do not know but statistic and and something approaches evaluation that is that is bad pattern right pe perceptua and statistics and perceptual a approaches approaches okay okay kay that was yeah but there are also reports from idiap i think at least 2 reports from marius seem to me but they are the same right as as and then also is there is some more work actually can be traced down yeah they are more less they are same like huh okay i think there wa the reading les atlas is papers are very good yeah because les atlas is doing a lot of work on coding in this domain okay he does not use opal model but he uses d c t and then he p so the so so and it and les is excellent guy huh yeah yeah he is very good mathemat h very good theoretician huh basically he is a he is at stanford the guy and really he likes to write things well right right yeah yeah so he is very rigorous as opposed to s to me for instance yeah okay yeah definitely so if you read that then it then you know that there are no errors and sometimes unreadable i would say okay yeah but so whatever you g whatever you get hands on from les atlas for over past 5 years regardi this 2 years ago it is quite difficult but yeah last year on one of the icassps he had like 5 papers right or various aspects of this because he is eager to yeah i think this year a lot interspeech there was one paper but it was regarded to speech recognition somehow okay is that so i did not even see it y yeah yeah yeah or it was not something like speech coding or that d c t transforms huh n nothing like that i do not so also i can before marius left he he has he had actually left himself but all the files are here you know so all the m matlab implementations of these are here okay and there is also one big table that he used he used to yeah to fill in yeah with all kinds of parameters he was also playing with all these compression factors and lengths of the of of that huh and there is a big table in in excel which you can look at huh he has all kinds of speech recognition results right yeah yeah yeah it was the 1st time you start using the cluster i remember or some some parallel processing this wa yeah because it it it took a huge amount of time that was a big thing here and he basically he he was the 1st one to realize that these these guys are pretty slow to to do these things everybody hated him for t temporarily so he he he just he just put here the p make environment and he did not stick to any cluster or anything right i remember huh he just put p make and run it on on all the machines here huh so at some point he was using all the available memory by any mistake you know so at th in the morning all the machines were were just stuck d no one could ever m move the mouse i yeah i remember frank came to me and says is everybody was computing like marius we would be all dead right but i say frank but i want everybody to compute like marius and so these these resul yeah yeah so because he was doing everything in matlab so it was again 10 times more demanding than c +and then yeah maybe yes maybe no i do not it is hard to say now it is not so bad with that if you really write it f well but he did he wrote it well and he did but anyway anyway i think so because i w i in the code that i wrote i was like playing also with if if the mat matrix is parse i really do not use it all sure i just multiply from here to here from here to here r right not to matrix p my matrix yeah yeah and then i i just optimized it a bit and also blast libraries and all these sta things yeah yeah but okay 0 well whatever whatever that is not important well it is important you know but the t whereas i mean this is something which we never talk about right i hear it how much work does put in here 0 yeah sure it is important for this case because it is true it is true this that that things sped up tremendously since i came yeah but still i mean of course partially because they started to buy some more more hardware yeah but also be p some at least some people basically know how to write faster programs yeah but there is a the back end recognizer which i will give you which is turn around of 18 minutes or well from 11 to 15 minutes this is fantastic normally it would take hour and a half and used to be 2 days yeah yeah no no today it was over one hour it is just like 10 times less maybe or 5 to 10 times less but still still you can you can 5 times more compute so do you already know where what is the situation with you you go back to prague for a while and then did you get anything in f in france great i got some ideas by the way i yeah i have to talk to you about these yeah i i do not know yeah i i did not try to i did not even mail to to chernotsky i must say huh i just talked to one girl there which told me that he would be possibly able to find me a job there any any job like related maybe to field but job not not anything any research 0 it is that is the that is the would be a pity right so if if nothing then then i could be able to get the job somewhere you you ca you t so you do not die of hunger yeah probably and so yeah yeah but t s but yeah yeah that is not the idea no we will we will we will we will try something let us see what we can do kay so joel your main basically thing is as i told you to t when we were walking that that what whatever petr did that it does not die but it i it carries yes and obviously v we very much hope that you get into that and you get excited and you do s more and or maybe even different things we hope that you find out that everything was wrong and that the potential is much bigger yeah but that it actually works yeah it is not that it does not work yeah it actually work that too right no but after you find out the back then it will be that will be the yeah yeah then so we will send you the papers related yeah so and i will point you to all the code which i have yes yeah and yes well regarding this f d l p and i and this and this is all on the record yeah yeah yeah yeah so if we if if we you can hold him to his word he does not send yeah i can play it back yeah yes so i can just now tell the the locations of all the files and you can rewind j ju you rewind browse through the meeting okay yeah yeah ferret yes so so how we doing time wise yeah i think half hour it is fine right yeah yeah and definite any any more things but you will be here until almost until end of yeah until 24 november it is the friday november so november yeah yeah november so breaks my heart okay it is just like i was just start teaching and then i n i will need an assistant yeah maybe yeah d hopefully it is make m makes pay pay some extra so we will see what are you going to t talk or to teach it is just that perceptual stuff and either speech but it is co actually compilation of really compilation of 2 classes i was giving at 0 g i which is like speech processing and and perception and so i tried to put it together so that it makes some sense do you already know how many students you have no no no i have it is i it i should have at least 4 because this is a p like graduates but i have one alright yes 0 it is graduate for graduates guaranteed i have one guaranteed that i do not know if anybody else is well i think guillermo was thinking about yeah bu 3 i mean 4 credits and or a total of 4 sorry i s i do not know ah i do not know if it needs to be 4 credit or not i do not know how much i like people without n not taking it for credit because then they just sit there and having good time yeah but it is okay no no it is okay because s s in i s it is 4 credits and guillermo is taking i think he is g yeah yeah guillermo is taking very good and so i will i s who already have at least 2 if yeah we pro i probably get 4 well i would like to go there but not just for credits just maybe to hear something new and it is possible that is for sure i mean that is okay it was yeah the last time it was for me it was good the course from my from one special reason because in the f when there th were the flats in i lost all my papers from my school all the all the exercise books how to say this yeah the notes notebooks notebooks note note yeah all the notepads and everything all the notes from any courses huh so i was remembering what i was learned or what i was taught at the time but that is good you know it is so so i have it now back at least partially cause all the all the algorithms you were speaking about yeah i did not get into too many things actually i would add we know the reason was that at the time we thought i was teaching with herve and we thought i we have much more time yeah and suddenly we ran out of time basically because what happened was that it was supposed to be 3 hour lecture now but it was only 2 hour lecture and herve assumed that it is going to run longer or i do not know what he assumed 0 but suddenly we sa s he said well so so where are you some i said well i am just slowly getting into l p c he said why but i we have only 3 more lectures yeah i have to teach h m m and everything so i the l p c part was like i had to do it in one hour and then everybody was c totally confused i guess yeah and yeah and also in the in the l lab labs the d people did not know anything about it and they just follow the papers yeah they were also confused yeah it was where was this it was at d b f l yeah in okay but yeah 0 we will see okay guys i think that this unless somebody has still anything incredibly important not to this topic yeah but i can come to yeah right it is ju it is a little bit up to you later you have to p you have to back this ha h specially petr because just to make sure that and yeah yeah but anyway after i am back i will be still on skype and all these things yeah so no problem to contact me until i do not know may next year okay so it is fine huh do you think that you will be done by may is it plan well i would like to yeah i think so yeah we made a mistake i tell you i should i think if i had to do it again and we made a mistake we should have register you here at e p f l you would have had a degree by now from e p f l yeah that was a mistake but i would have to study here for that that long time no i th sorry yours you were or you were here t you were you were here how long you were here 2 years 22 years yeah yeah and i would have to stay one more where are the well it would be good reason for petra that i have to stay here yeah right right so is she getting another position there or what is the re reason of what a boyfriend huh yeah yeah because she has a good position because she has a friend which is starting who is starting a lab there and everything is it is a great opportunity for her at least 0 i see so that is good yeah but the problem is and then she will go back and there is f family issues and all these things you know for a girl it is always the pain yeah i know because she has to break for some time at least a little yeah but has to break yeah and in in the science it is not that easy to to break i know yeah okay okay yeah so in case i need something on this f d l p i will come to you yeah 0 sure otherwise i am also much more available than i was a any i mean when you came it was just like cra one of the craziest times of this year okay okay it is not it is not that i was very productive that i was doing anything useful yeah yeah but it just like it is a little bit like writing thesis all the time re writing and change the do make a little changes and and but takes your huh yeah but i think definitely you are the best person here to with related to availability of you because if if if i want to go to herve i probably call secretary he is there or and when i can and i there was no op no ev every time i came to you you were available yeah he is ver no he has many more sp many more people so to take care of yeah well i am typically i try to be because because you know i mean for s this is also my livelihood right students at least it that is great yeah so i have to make sure that that that they need to know something that so feel free basically to stop by any time and yeah and it is not that i know everything yeah but if i know i i think i sort of know a little bit like what you guys were doi doing right i think so but not not obvi every detail obviously yeah you know the core which we do not remember anymore programming what yeah you do not know why why you are doing it anymore we know the details and you know what what is th what is the global stuff why you are you doing this yeah good question sometime asking ask hynek 0 boy yeah ask my boss oops okay i think my mic is falling falling yeah so i think we go huh right yeah yes thank you very much yeah thanks thanks a lot thank you 0
set up huh yeah all right let us set up so yeah we put this stuff on so that we have some record of the whole thing what this is that we are going to say the whole idea here is also put the lapel on yeah so the whole idea of this meeting is that it is not me who will be talking about so what is that it would be a boring that would be definitely boring meeting so joel is going to talk i am going to listen so actually the reason why we got together is that it is about a time to hey make sure that when petter leaves us i am afraid it may happen right i mean it did not change your mind yet goodness 0 your girlfriend did not change your mind i would like to change but i cannot so so that whatever he has been doing is not forgotten and then people can stay on his shoulders and i hope that he will not get out entirely out of this problem but still we need to continue on this so i think joel is my main hope that he is going to pick up the keyword spotting and some of these ideas and also the reason why petja mottlitschek is here is that joel is also getting interested in all pol models of the temple envelope yeah that is true and of course petterfosagnos about it but i think that you are more than you know from different point of view but you will be here and from different point of view is working mainly on speech coding and so so this is i guess this is it so i guess that really you should be talking more than anybody else because you should be asking questions yeah right yeah and so what is it that you need to know and let me just tell us so what you know already about i do not know how far you are with this yeah well features or so basically i started reading about the features that is taking the long temporal and time frame it is traps and tandem approach so you know something like that yeah so then also started with there was a paper on fitting air model to the dct of a signal started with that and slowly getting used to the stuff to the dcg yeah that is what chris was really it is a little bit of the paper by dan ellis and co 0 that is how it started basically because that was mario satineos and dan alice they had a paper i believe it was not even like asru or something right it was a but that was the time when they were not using very long segments yeah okay it was like 50 milliseconds or something when they were playing with that time time so then the story was that then marios presented that at some meeting and we realized 0 that might be cute think for estimating spectral energies because that is what we want and the way we are doing it is i think it works but it is a little bit inelegant somehow stupid so 1st do the frame based estimate and then you get stick to 100 hertz sampling and a millisecond sampling or something and then you just pick up the trajectory so marius was here for half a year or maybe or even more and started work with better just came so they were working a little bit together mainly marius was programming in matlap and petter was putting it in yeah i just rewrote it in the c pos i did not do too much on that and then we were hoping to pick up on that a little bit more and we did right you did a little bit of work well we we tried there are many knobs to tune on on that because you can decide what compression factor and everything so i tried a little bit like picked maybe 3 knobs and tried to go through the 3 d 3 d cage to see what happens in which kind of settings so this we did but we never published that anywhere right we did not because there was still a question that i do not remember right now because there was some essential question i do not remember i would have to look at it and because before i before i we could we could write anything i would have to be sure that something is right there i could not be sure about it at the time so and anyway it takes time this this computation it is very inefficient it takes like a 100 times more than computing some mfcs computing these features because it is a huge dct huge fft everywhere we were also doing a silly thing that we were stepping it by 10 milliseconds right yeah there was no optimization we just stick to the trap we wanted to make as close to trap as possible which is silly but that you have to start somewhere yeah so then i continued with the another papers that you are mentioning at ptrap so just moving around with that ideas and also the main idea today why i told maybe is like if you can list down what all things are there to be taken care of then one by one because finally if i find that you have something else maybe to list down the things other than the what i have already taken from you okay and also to discuss generally so so well last time we we were talking about what about goshen you you have the gosions running somehow like feature extraction yeah feature extraction and then the tandem training also so the you are able to train the net now you may be the only one in the whole world now because chifeng is gone and barry is gone so morgan was joking they are gone from ixie no they are gone from ixie chifeng went to t i and barry is at lawrence rivermoor lab he basically got a job he graduated and he will be working on things which he cannot talk about he can not even say it on a microphone here yes it is classified work quickly and so morgan was only half joking he was somebody was asking him you were or somebody was asking him something 0 that was matthew and he said well probably we will be coming to ask you because nobody really knows anymore at ixie so please take who is the one who is now developing the quick net in these things the mlp 4 and these nobody did it was barry and now it is a close it was very actually berno is doing a lot of that now they picked up the software and they play with this 4 layer nets so there might be that bernouff will be an ultimate expert on the host well still we have the experts who made the software run here you know olivier for that which software no quicknet training quicknet was done clicknet itself was done actually long time ago yes but olivier make it run on these machines and make the cross the 2 g barrier here at the dda because we could not we could not we could not do something i do not know what but he is he is now good in quickness okay so we have to treat him well so yeah right so i have the feature extraction part where i can get the phoneme posterior yeah generally through the neural net right so then i think you just miss the back end which is easy because it is hdk it is everyone knows yeah but yeah hdk through for some other task i have run hdk so okay i have the general feel but maybe i can take the entire setup from you and run it once online this thing to have yeah okay and you you just need from us to hear what are the other front hands and other features to choose something yeah basically so then i think i can share what i can have read and yeah well the whole idea then is that you start playing with it right yeah and you yes yeah and okay so how do we do it the list it or this yeah that might be the 1st is the feature extraction that we have already taken then the back end for that then well yeah like i i distinguish between feature extraction then the mlp because you feed something to the mlp which are not the waves so so well that is that is what i call it and then the backend but okay also what what i have here what i have been working on since i was here was mfc cplp which is always needed for comparison purposes and traps with warriors lens and also you know traps no yeah traps i know but with various with various lands okay length we were trying to start from the ultimate lens of 11 frame up to 12nd or even more but it was so this also then we applied the dct on the trap itself which is some something like the gosians applying some filtering basically the trap so this but this is mainly franta who was doing this so this is related to traps i was also trying the warping means if you have a 2nd which is a 101 i was just keeping the central 10 samples and then i was throwing away some of them you know like resampling this 12nd of the trap to something which i preserved all the points in the center and just picked every 10 points at the boundaries which means the 1st trap 101 trap trajectory shrank to say 15 points with still preserving the maximum resolution at the center and throwing away most of the points near the boundaries so this i was trying so these i have some results on so these are trubs then the gauss traps okay then tandem system okay the tandem you +minus have because that is just the plps okay and then there is a training with mlp which is taking some context of these plp coefficients so it is like feeding plps or mfcs or whatever to mlp and taking some context this is the tandemn so this setup we also have here i did not experiment with this but maybe some more results it is hemant who has the results on tandem i guess he used to have before he lost all that right yeah but i think the most important results are already in the tcc is writing so it is somewhere so this is tandem then then what and then there is the fglp then we can talk more about which is which i have more experiments on also this warping and compressions and all these things so compression basically you are looking at yeah yeah but i am using that for different different reason right so it is a speech conning so what is the task is you just take the speech of course somehow to encode it into as small number of bits as possible and with a reasonable quality of course and again to decode it and i am using fdlp for that still playing with more or less 12nd long segment so just dividing the speech into 12nd frames doing dct it is just the standard word of work and then doing some compression and computation so sub bends no yeah yeah yeah 0 yeah but you are doing the same no well yeah but you just did not talk about any sub bent so far yeah you take the dct and you just divide it into several sub bands frequency bands with a different and by the way you do not do the overlap on the sub bands no no you do i do yeah but you do not do the overlap on the time domain no no no i do not do okay so that is the difference for compression you basically do just 12nd without overlapping but in frequency domain there is overlap because you can choose which filters you want or less or just trajectories of those bands so there is overlapping i am using those gaussian filters or classical those trapezoidal like in mscc is right and then processing each mnccc are trying okay triangles that one then i am doing independent processing of each frequency band so i am computing that hilbert envelope which is just the envelope of some temporal trajectory and more or less you can divide this into a hilbert envelope hilbert carver which is something like lp linear prediction envelope autrogysm model and excitation signal so those are 2 parts parts or items you need somehow to encode if you want again to get original speech or reasonable quality and there is some compression factor which is again when you are computing that after aggressive model from fdlp you can compress the spectrum somehow in such a way that you rather preserve those even also those deeps not only peaks in the spectrum and it sounds better for some kind of you know about this compression how yeah why it is okay i just do not know more than than i do well he is new he is doing most of the thing so it is kind of logarithm when you use logarithm for mscc is right so it is again compression factor something like that and but then you have to uncompress it again of course to get back the speech and there are other other other things which i am doing but it is not related to speech recognition so much but certainly when it comes to like the op on modeling of temporal trajectories if that petit fosek is not here you probably know a lot about it by now well what no that approximating hilbert developed by all pol model what fdlp yeah well if you are not here that petter montu chic knows about it is more or less the same approach right there is different applications different software or something so just in the speech resynthesis or coding what you are doing did you try to play with the knobs somewhere with what with i know the compression number of bands and yeah sure number of bands and what you were playing with and what other and what are the results like what to use you know that is the problem for speech scoring because i do not have results like in speech recognition yeah of course i have to listen to it yourself i am fine with that because well so now what i am using are 15 bands which seems to me 15 frequency band which seems to me quite reasonable we can go down yeah till like 12 and still okay i would say but then you start hearing difference or also that is another problem once you can go down with frequency bands then you will have much more complicated that excitation signal or that carrier if you go down yeah because you might get broader bands because when the band is narrow the excitation is close the carrier is close to sinus sinus so there is more and more frequency components let us say or something like that in one broad bands so you want to say that the lpc is not able to fit it so that the residual was no lpc will maybe but then you have still another part something like source and filter so when you are doing speech cooling or you may almost want to this yeah whiteboard i mean yeah that is why you are here right so can we remove this when you are going to this place i do not know i do not think so no i am not supposed no i mean it is the same like when you are doing lp lp synthesis right so you have got source signal i do not know source i think you have to push it as much as possible because it reacts to the and then there is some filter which is that one divided by a z and that is it is and the source can be a noise signal or it can be some sequence of what is that falses right this is a standard time domain right so it is more or less the same what i am doing so this is some envelope which is in our case related to hilbert envelope and this is a source signal which is hilbert courier but since you are in a different domain this is here you are in time domain right but where you are you are really in a frequency domain so the lines turn into signs right right and you are exactly yeah so this is now going to be frequency yeah okay this spectrum is also a line spectrum right i mean all components so if you are then if you are in a frequency your source looks like this yeah so more or less if we can we can plot it so what we can see is some hilbert envelope like this and then there is that hilbert carrier or this is not hilbert career but this is the signal which we get from one frequency band so you compute dct right this is the subband subband time signal and what we get from that is this hilbert envelope right so you know that and the 2nd which should be that carrier should be something like this it is just really close to cosine we might see okay so do it properly right and this is what you also have to somehow transmit or at this try to transmit in order to get i was thinking the residual in lpc fit of the hilbert envelope not this not this i was not thinking of this at all before right right because there are 3 things 1st you have to encode this then the lpc coefficients and the residue after lpc fit no i do not need lpc no no this is like a residue for him this is a residual for me yeah i just want to transmit this signal right yeah and what you need is this envelope and this one carrier and you again get the same signal this this shape okay do you idct and the biggest problem currently is that the carrier is actually not cosine but something little bit more for example for us if you have a very good music single which is very periodic you can really see something like there is one peak and if the frequency does not change over the over the length of the range it is related to that frequency of the broadband or what is called if you have those filters which are wider and wider this is that center frequency which is somehow related to this one okay so then you can transmit so the broader bands the what on this picture it is worse and worse then you do not see anything like cosine fortunately because of because of properties of hearing more or less for the higher bands if you put in just some garbage some noise it does not really matter all that much it sounds a little bit worse but it is not a big tragedy right if you want to do a good voice speech you want to get the lower bands and if you put in noise all together everywhere it sounds like a whisper speech it sounds like lpc when you excited by noise it is better i would say better quality definitely better quality it will lover or betrayed but that is a problem so now we have to kill them most because i mean we gave all the secrets away yeah so any objective measure you use objective measure you use for objective measure well some those distances spectral distances well it did not work so operable so well i mean if motorola or nokia gets hold of this tape then we are in trouble hopefully by then it will be protected and everything right how the hell i just said that we should not talk about confidential thing it is it is it is a it is a problem listen this is a science i mean yeah confidential to the extent that we are not publishing it yet but we will and and but we have to make sure that if corcom has an interest who is funding their work they have to have a chance to patent it or something this is not a public public presentation right so this is not even a problem yeah the problem would be if but you know nobody can beat petra motley check anymore i mean if if nokia knows everything if you sell it tomorrow who did you work for hp hp they will take half a year before they will even realize what it is and by then so it is no problem okay so that is where that is okay so there is a compromise between the lower bands the worst stuff you just said and the more bands the more coefficients to transmit right so 15 is about okay yeah yeah also there are papers about that that if you have like 12 frequency bands is somehow optimal okay so this and then there is a compression what about that compression is only to really to somehow fit that hilbert developed to get that so that you get so that you can model it actually yeah yeah you can model it better there is no echo or something like reverberance in the signal if you really go down with that compression factor to like 0 one what 0 compression okay so what is the compression factor we are getting here when you compress the effectively the what the signal which we are modeling right and so then you get a fit into a compressed signal essentially if you were in a spectral domain you would compress the power spectrum yes here you compress the this this what subband power spectrum yeah the city of the signal right and so then your predictor is fitting actually this compressed signal but you know how much it was compressed so you can always expand it back so then you get a that you get a fit to the to the original thing except that the fit is different because it basically this opal modeling has a different properties yeah you just preserve something different a little bit you can tell you a little bit more about this is an old technique which yeah i think he has good many papers or some papers all that so it was my thesis even phd thesis actually there is no piece i think i read it was called spectral transform linear prediction it was my 1st icas paper ever it was in 1983 boston you guys were still in kindergarten right right more less but nobody nobody was using it for anything except it was cute but it was about it and but now it seems like that this since what we are doing here we are normal lpc is fitting peaks right yes but when you are modeling hilbert envelope you do not necessarily want to fit only peaks you sometimes want to get a basically good estimate of the of the envelope so that helps you because in this spectral transform allows you to to find the right balance between how well it fits peaks how well it fits dips and right so i think that basically i do not know what else i mean you should be asking questions right yeah one thing i wanted to know is a good literature on these where you started with i think you can really take those papers from marios yeah they about fdlp about 3 papers that is really enough i have to this is sapa or how it is called sapa and then yeah this is this is then then there are 2 papers in one is in sapa one is in the u interspeech and one is in the american and one was in the asru yes it was from virgin islands what is sapa i do not know i tell you what it stands for statistic and and something and conceptual approach i was on organizing committee i should know but you know and everything approaches perceptual and statistics and perceptual approaches approachs okay okay but there are also reports from idiab i think at least 2 reports from mario seem to me but they are the same right yeah they are more you are same and there is there is some more work actually can be traced down i think they are reading les atlases papers are very good because les atlas is doing a lot of work on the coding in this domain he does not use opl model but he uses dct and then he so unless he is excellent guy he is a very good mathematician very good theoretician basically he is a stanford guy and really he likes to write things well so he is very rigorous as opposed to me for instance so if you read this then you know that there are no errors sometimes it is unreadable i would say yeah but quite difficult so whatever you get hands on from les atlas for over past 5 years we got this 2 years ago last year or one of the ikes he had like 5 papers right all various aspects of this because yeah i think this year i do not interspread there was one paper but it was regarded to speech recognition okay somehow is that so i did not even see yeah yeah it was not something like speech coding or that dct transforms nothing like that also i can before mario is left he he has he had actually left himself but all the files are here and also all the matlab implementations of these are here and there is also one big table that he used to use to use to yeah with all kinds of parameters yeah he was also playing with all these compression factors and lens of that and there is a big table in excel which you can look at all kinds of speech recognition results right yeah yeah it was the 1st time you started using the cluster i remember or some parallel processes it was a big thing here it took a huge amount of time and basically he was the 1st one to realize that these guys are pretty slow to do these things everybody hated him temporarily i remember he just puts here the p make environment and he did not stick to any cluster or anything he just put p make and run it on all the machines here so at some point he was using all the available memory by any mistake you know so in the morning all the machines were just stuck no one could ever move to the house i remember frank came to me and said everybody was computing like marios we would be all dead i said frank but i want everybody to come in like various right yeah so these these results yeah because he was doing everything in matlop so it was again 10 times more demanding than t +and then maybe yes maybe you know i do not it is hard to say no it is not so bad with that if you rewrite it well but he did he wrote it well but anyway anyway because i in the code that i wrote i was like playing also with if the matrix is sparse i really do not use it all i just multiply from here to here from here to here not the matrix my matrix and then i just optimized it a bit and also blast libraries and all these things but okay whatever whatever important you know what i mean this is something which we never talk about right how much work 0 yeah sure it is important for this case but it is true that things sped up tremendously since i came i mean of course partially because they started to buy some more hardware but also some at least some people basically know how to yeah but there is a the back end recognizer which i will give you which is turn around of 18 minutes or well from 11 to 15 minutes normally it would take an hour and a half yeah no no today it was over one hour it is just like 10 times less maybe you are 5 to 10 times less but still still you can 5 times more so do you already know what is the situation with you you go back to prague for a while and then did you get anything in france i got some ideas by the way great i have to talk to you about these i do not know i did not try to anything i did not even mail to hondeljavlotsky i must say i just talked to one girl there which told me that he would be possibly able to find me a job there any any job like related maybe to field but job not not anything any research so if if nothing then then i could be able to get the job so you do not die of hunger yeah probably yeah but yeah that is not the idea yeah we will try something okay so jorge your main basically thing is as i told you to when we were walking that that what whatever patterns that it does not die but it carries and obviously we very much hope that you get into that and you get excited and you do more and or maybe even different things we hope that you find out that everything was wrong and that the potential is much bigger yeah it actually works that too yeah it is not that it does not work no but after you find out the bug then it will be that will be there so we will send you the papers related and i will point you to all the code which i have yeah regarding this fdlp yes and this is all on record so if we yeah he does not hold him into his word actually i can just now tell the the locations of all the files and you can rewind but i was through the meeting yeah yeah that yes so are we doing time wise yeah i think half hour it is fine right yeah any more things well you will be here almost until end of yeah till 24 is the friday november it breaks my heart it is just like i will just start teaching and then i will need an there is no petra svairnski and frater it hopefully makes a pay some extra what are you going to talk or to teach it is just the perceptual stuff it is actually a combination of really combination of 2 classes i was giving at ogi which is like speech processing and perception and so i try to put it together so that it makes some sense do you already know how many students you have students you have no no it is it i should have at least 4 because this is a like graduate school i have one right 0 it is a graduate i have one guarantee that i do not know if anybody else is well i think guillermo was thinking about 3 i mean 4 credits or a total i do not know i do not know if it needs to be for credit or not i do not know how much i like people without not taking it for credit because then they just sit there and having with time but it is okay no no it is okay because in ice it is for credits and germo is taking i think germostaking very good and so i already have at least 2 if i would probably get 4 i would like to go there but not just for credits just maybe to hear something new it is possible that is for sure i mean that is okay yeah the last time it was for me it was good the course from my from one special reason because in the when there were the flats in czech i lost all my papers from my school all the exercise books how to see this yeah the notes in your books yeah all the note pads and everything all notes from any courses so i was remembering what i was learned learned what i was taught at the time so i have it now back at least partially because all the all the algorithms you were speaking about i did not get into too many things actually you know the reason was that at the time we thought i was teaching with erwe and we thought we have much more time and suddenly we ran out of time basically because what happened was that it was supposed to be 3 hour lecture it was only 2 hour lecture and everybody assumed that it is going to run longer or i do not know what he assumed but suddenly we said well so where are you i saw well i am slowly getting into lpc so why we have only 3 more lectures i have to teach hm and everything so i the lpc part was like i had to do it in one hour and then everybody was totally confused i guess also in the in the lab labs the 2 people did not know anything about it and they just followed the papers it was there was this there was a d yeah okay yeah yeah let us see okay guys i think that unless somebody has still anything very important not to this topic yeah but i can come to right it is a little bit up to you you have to you have to back this especially better because just to make sure that but anyway after i am back i will be still on skype and all these things so no problem to contact me until i do not know may next year so it is fine you have seen that you will be done by may is it planned well i would like to yeah i think so yeah we made a mistake i tell you i think if i had to do it again we made a mistake you should have registered you here at epfl you would have had a degree by enough for epfl i was a mistake but i would have to study here for that long time no 3 years you were here how long you were here 2 years yeah i would have to stay one more well it would be good reason for petra that i have to stay here so is she getting another position there or yeah yeah because she has a good position because she has a good position because she has a friend who is starting a lab there and everything is it is a great opportunity for her at least but the problem is then she will go back and there is a family issues and all these things for girls it is always the pain because she has to break for some time at least a little but has to break and in in the science it is not that easy to know to break so in case i need something on this dlp i will come to you 0 sure otherwise i am also much more available than i was i mean when you came it was just like one of the craziest times of this year yeah it is not that i was very productive if i was doing anything useful but it is just like it is a little bit like writing pieces all the time rewriting and make little changes and yeah but i think definitely you are the best person here to with related to availability of you because if i want to go to herva i probably call secretary he is there and when i can many more people to take care of you there was no no every time i came to you we were available typically i try to be because because you know this is also my livelihood right students i have to be sure that they need to know something so feel free basically to stop by any time and it is not that i know everything but if i know i know i think i sort of know a little bit like what you guys were doing i think so but not every detail you know the core which we do not remember remember any more while programming yeah you do not know why you have to do we know the details and you know what is the global stuff why are you doing this ask my boss ask my boss okay my mic is falling so i think we go huh thank you very much yeah come thank you i am going to all if it is a little bit of the other i am going to not i am going to i must you have to see there is a starry it is just one the k n tlisks and they are going to they are going to you will just you will just you will yeah