0 yeah you do not put your headset microphone so we are here to just make the point well make the point yeah make the status about the shot detection okay to so you have applied the shot detector to some of the cinetis video or at least one and so we can basically maybe just comment on on this yeah maybe quickly remind what the shot detector is how the shot detector is working basically we might be able anyway to give you some reference i mean your report it is it should be described there or yeah yeah yeah yeah i have actually yeah the we can see the report here and the demo you have the report okay and the goal of this meeting is to present the video shot detector to olivier okay to olivier yeah yes okay so yeah it was my project during my internship at idiap under the supervision of jean marc and as you saw as you saw in the email i sent okay i did not send this video is the new one i just processed from your images ah okay yes the kayak wi i was not even with the other one okay yeah but let us let us start by the video from friday cinetis demo one by the way i was going to ask so i had seen the video i you did a downsampling of the video right i mean for browsing i mean yeah okay yeah so this was okay so this is hierarchical normally so you have scenes and shots but yeah here on the left you have different things but as applied kay because for instance we could probably you know okay this is like group all the initial ones into one scene and maybe after when there is some things inside it s should be another scene so okay so this detector detected the 30 shots and if you click on plus button you can expand it and see the key frames that are in fact 5 extracted frames from the shot okay which are extracted i think but i think th as it is it is t extracted without using sub shots it is simply 5 down sam yeah down sampling downsampling okay so i we can see the video actually the entire video can open with real player yeah i hope it will work so we do not have the the sound and it will not be grabbed the video will not i do not know if okay it is not good quality yeah the quality is that is why i was asking that 0 yeah yeah yeah it it is not this that you processed right no no no no no no okay okay it is in real media so it is very compressed so here we see that we the shot at 18th and after 22s and after again okay and if we go down i mean maybe in terms of shot here we can see that kay right i mean so now we see shot 7 i suppose yeah okay yeah okay shot 9 currently shot 10 probably okay 11 or 12 no 11 10 yeah or no this is 10 indeed okay and then there is yeah yeah this one it is 1112 okay or 14 seconds okay yeah okay okay yeah it is inside and yeah yes and this one yeah it is a dissolve it is a dissolve so it does not detect yeah it is really a dissolve or is it really like is you have the feeling that it is both okay when i saw the m with m player it is a dissolve yeah okay but it is it is it is not standard man made movies because i would say most of the time people couldn did not do any dissolve yeah i could see here i mean that it was like horse it was only one single one yeah okay yeah i do not know if this one if we see was correctly you see n yeah yeah yeah this is the one i i had seen yeah yeah but as i said i mean with even the motion detector when it is black it is there is not s no structure so you do not esti yeah but you do not really estimate motion but there is a moon but okay because there is no motion in fact between the 2 shots well there is no structure there is no texture if you want to say so and this is this is not so good for the motion yeah okay okay but for instance of course between the horse and and the c the c yeah the cows yeah maybe we could have detected things the cow yeah but it is not even sure because the background is quite similar and it is it is it is not that sure yeah maybe to explain this method was using a simple distance between consecutive histograms okay and we did not use motion features at all for that that is why i we so it is yeah the histogram so we have experience with it yeah yeah yeah okay so it is it is scrambled so you can scramble the image and you get the same histogram okay yeah so there is no structure indeed so of course when you look the horse and then you can see as a cow the cows okay of course you have a black in the top and you have also maybe 80% of like green which is even not green there but grass you have stones and so there of course are differences because on the one hand you have horse but it is black so you have more sky which is black on the right so you do not care about this okay so it is a glo it is global it is a global histogram so yeah kay but it is when for this i would for this application i could imagine that it would were quite high enough and it is a threshold but still it is adaptive right if i remember well yeah yeah yeah with a sliding window yeah okay so because of course if you just compute the yeah i can show you if you have the report because if you compute the distance between the 2 histograms i think this was bhattacharya yeah right so now we know okay and so if you take these 2 with the with the bhattacharya then of course you should just do a threshold when there is motion you will see a lot of things changing so by using an l r t t threshold okay if you see that there is a continuous change of the d distance you do not do a threshold it is only if you have an identified peak which is m 50% or maybe 2 times higher than the the next 2nd value in the window okay then you say it is it is a shot yeah right i do not know i do not know if we c we want to make use the whiteboard or here it is it was to dis it is infra only the this part about shot boundary detection is in french unfortunately yeah yeah but the this image shows the results using motion when you s use the motion features and here on the right is when using the histogram distance yeah and the the motion features can better show what is going on when there is such a difficult transition while with the histogram okay yeah it just shows that there is a shot change at the end but even for dissolves i do not have examples here but there is not a strict transition it is continuous for dissolves it is continuous for dissolve yeah okay yeah okay but an anyway it is difficult i mean in th in in in general dissolves is yeah yeah i understand okay because sometimes you have some dissolve you could imagine they are part of of of a scene and so yeah but of course when you have cuts you can see they are easy to detect right i mean like something yeah yeah okay yeah so yeah so with the motion basically the motion tried to estimate well the motion between 2 frames okay right if my image is there but after it is there and it looks at how many points c are well matched okay okay so you can see that like somebody speaking and so and so before the sh before the shot you can see that usually there are like 90% of the points at least right who have some motion so but then after suddenly if you go from one frame and you try to match all the points here with like a translation plus the zoom plus maybe some rotation you c you can find anything well right so basically you would see that the number of points maybe only 10% of the points you can match well so it is easy to detect okay and after you can see that indeed because it is a long shot and it is almost like a flat plane you can do a very good registration through the number of points yeah okay is very high i mean the number of points that are matching we will see okay and here you basically here with a histogram you can see it very flat but of course you should go from this histogram to this which is essentially green you have this big measure which is here the chi 2 i do not know whether this is what you implement is used in the no it is see the cut no no now it is well for the video you saw it wa it is bhattacharya it is bhattacharya okay okay yeah so yeah and you have maybe a one image with a adaptive threshold i think you had something like this right yeah just to explain how okay set the beginning yeah yeah you see like here yeah so there is the idea is having a sliding window we have already all the data that is on the graph okay and then to detect the the the the changes we we do a calculation with points under this sliding window within this sliding window and we calculate so what do we here in fa we want to detect that this points in the middle is higher than alpha times the maximum of the other ones th okay exactly yeah of the other ones alpha alpha times the maximum of the other ones okay so the of course if it is you are not a maximum you are not going i inside the window you are not the max anyway you are not going to be to be above alpha times the max of the others right because alpha is low lower than one the c okay but if you are in the maximum already and at this point we check simply if this value is alpha times the the max of the other points like for instance yeah let us say these points here like for instance here i i see 6 but i do not know whether this was for bhattacharya or for chi s chi 2 okay okay i maybe for chi it is it 6 but yeah but for bhattacharya it is for the tests i did it 24 yeah yeah yeah yeah 24 yeah so it is a bit less because the amplitude is much smaller yeah yeah yeah yeah okay so you can it was not so just i mean because the motion i mean it was in the shot we have seen no f because f especially with chi 2 square it r the amplitude it is much bigger than with bhattacharya right yeah okay you can see that it could go to 700 and when it is there is nothing it is can be 0 with bhattacharya it is between 0 and one so you have not so much amplitude okay but still in some of the v these videos for instance even if we gon go now if we want to but now for the kayak case i do not know at some point targeting 2 kayaks and it is moving quite fast towards to other people i do not know what happened there if it was detected as a shot transition or i in the the 1st shot yeah yeah yeah i saw that it was detected as a shot yeah it was it as a shot okay i was wondering yeah yeah maybe you want to see now okay but he did not okay but here for instance you understand so we can show this tomorrow to to the people at the ci cinetis meeting i mean to yeah okay okay i will look to have a connection right yeah okay yeah we need a connection that is true okay yeah and we need a laptop or you have yours yeah that is true yeah i can bring my laptop if necessary yeah yeah okay i will i will bring it okay so it is it is done okay it is okay this is idiap laptop so you can u okay yeah so here in fact after 12 i do not know if it is i think it is 12 frames well yeah but normally there is a a burning if you c maybe you can play the shot or okay yeah it is strange i mean maybe you play yeah if you play the the video well insi at the beginning i do not know what is happening 0 yeah maybe you are burning but 0 there is no ha ha i did not ha i did not have time to the the real media ah you did not generate the okay the real media okay but yeah it is complicated well no i mean here i can see that it was it was detected as a single shot yeah here we can see that yeah okay i was because here you can see that the motion here is you know he is doing a zoom on the 2 kayaks here at 24 yeah and then there is translation on the right yeah yeah and then he is doing quite a fast transition from these to another one kay and i was wondering whether so if you well though of course it is true that it is mainly water so maybe the color is indeed quite but you know as a kayak is entering quite fast i mean you can have quite maybe good distance i can see there is another program that can see but i see that it seems to be how many shots do you have indeed de de detected the n it is i think it is f there there are 5 normally 6 yeah the one is maybe so you did it with with what by the way if we put expand all okay it is with the images i gave you or okay yeah the other one i had problems converting the video into d v x yeah and then so i used the program from software from sebastien p p a p m to a v i ah to regenerate an a v i file okay yeah yeah because yeah one thing maybe we will talk about that is the library it is it does not compile anymore on debian machines and yeah some maybe some work has to be done on this okay i do not know why this image does not appear this is the the frame that is extracted to be the shot key frame here this one is this one the middle one so i do not know what why it does not appear but it just a simple problem in the get frame okay yeah yeah but what i can show you now is a little program if we want to be very precise about so it is ma get frames okay c g i here and we were interested in around what di do you want around 67 seconds yeah i mean initially i do not know why there is a a shot detected okay if we do next 0 yeah there is something wrong yeah yeah 0 yeah so it detects ah okay so the okay so the here is an image number there was some burning yeah in fact yeah that is why i said it is it is a frame number in your program ah okay okay yeah w i do not know why yeah yeah yeah it is a frame number okay okay so then this this is a frame number this is in 2nd but i i kept the frame number so but it is in seconds or it is it is 12 frames 0 seconds okay okay yeah okay okay seconds plus number okay and the next is 11 seconds and 12 frames yeah okay so basically what has been yeah extracted is yeah this point rather than okay so this is normal okay yeah yeah so if we do every 2nd i d the middle frame is this 1000 okay do you thi it is another kind of browsing page but to be more precise about to to check maybe the this yeah exactly yeah yeah okay okay okay like transition something that is okay anyway you go back to one frame around f 160 for instance i do not know and and there is yeah that is it you know okay ah okay understand prev okay you see that there is okay it is not a shot change it is a motion actually here yeah it is motion yeah it is not a cut yeah yeah yeah yeah yeah yeah but i was thinking that maybe it could have been yeah i s yeah but it is true that it is mainly water when we see things so it is yeah it is not such a big but yeah sometimes i mean you can have various things hap i mean just think about more action movie and thing like this kay well sometimes it is very difficult so focus usually there is no problem people say this is a cut this is not a cut but like for dissolve even sometimes you know people may it is people may have some trouble i mean this is a dissolve this is not this should this should be labeled as a transition or not yeah some people will say yeah but it is part of the thing or for instance in like in also in come on advertising usually yeah because the shots are very short in advertising usually it is very fast and so sometimes okay yeah yeah when you have an incrustation like last time what what should you say it is it is yeah it is a change of when you have yeah the the goal results then yeah in fact it is not a shot change but yeah yeah i mean it is a so sometimes very difficult yeah yeah a at least what i can see that currently like for cinetis for instance if it is to do color correction or for instance if it is too black too a bit whiter i think this type of things should be fine because basically it is even if they are not real shots they are not so interested if it is not real shot yeah maybe after they have this and they may say okay just by looking at least the summary so maybe for the summary we could do better key frame extraction for instance but by just looking at the key frames and so on they may say okay from this shot to this shot we apply like specific color enhancement algorithm or restoration algorithm or to improve the brightness and something like this right yeah and so i think for them this is fine after i yeah so this would ma be made manually actually when you say from this shot to this shot is using a graphical user interface someone who would do manually for each video well maybe after i mean they would sh okay they would go directly and say select the shots from here to here and they would say you apply this algorithm yeah method yeah so of course we could try to make scene clustering and so on but i am not even sure that this is necessary for them at this point because if they have a 6 minute movie if they have 30 shots yeah for them it might be easier to look at from here to here i apply this and they can do it quite quickly yeah and they have to do it an to do it anyway because what is going to take time might be to select the type of restoration algorithm or the type of things ah okay so this might take more time than having to click 2 times on 2 shots rather tha tha than having directly a scene right yeah yeah yeah okay so so i think for them this type of things is going to be useful at least for this task okay if they want to do stabilization it may not be sure of that this is the right segmentation for some of these but so but this is another point which for motion stabilization as i said several time we are not we did not commit to provide anything but of course if we can provide things we will do it but yeah so do you have more question or do you have a no i have look at the demo film and it is about 40 minutes and about 173 different shots so it is yeah this is the one in the d v d maybe okay we can see a little bit if you want to play the yes and i do not know if this is a real sample d v d because this will make about 5 seconds for each shot so i think it is very short well no it is okay well i just i i i it is i think this is no i if this is to r to to to make a change in the color light and they have a lot of shots yeah but after i mean it may be more like a guy i int i in an interface a guy interface i mean where i mean if 2nd select from shot one to 7 yeah okay ah okay yeah okay this is a same seg they may do the grouping by themselves quite by looking at this it may take w w less 11 minutes okay understand and i think it is anyway if it is too summarized after they may want to go inside to see whether there is not special things i mean so yeah because after it depends i mean if you give only one key frame per shot after i mean it is 170 images okay it is quite fast right and so i think and the hierarchical thing is good because still you can check if you have some ambiguities you are not sure you could okay in fact i did not understand that okay but now i understand better that cinetis wants this kind of method to help them to restore and to apply the algori some algorithms to to to make a better movie yeah yeah yeah yes yeah at least yeah yeah yeah yeah yeah yeah okay yeah it is not for the client for the user who will maybe okay well ultim ultimately i think this is the type of things but at this point we yeah 0 maybe they can okay also but okay i i hope that if it is a big success they will ask idiap to to provide something like this of course with some money or at least to pay a student or somebody for doing it okay but yeah of course this was ultimately i think one way of if you could provide like chapters or something like this indeed you could provide like the a v i to the clients and he may ask for special services or and even to just to watch the movie to have direct access to different parts and so on but it is i mean this labeling might be more easy to do for the client by the client yeah okay because he knows what they are i mean he we wants to have this and this and right so okay ah okay yeah yeah yeah because he did the movie so he wants t okay to archive he wants to okay yeah yeah he may want to organi yeah yeah otherwise i mean this but of but of course it might be easy to do a very simple software this is yeah another which would allow the user to select just the shots and give names and after it would generate automatically some index and as a i suppose the client might be interested in such a such a things if he could generate automatically on the d v d yeah yeah yeah okay but for and you could imagine that before sending in the d v d you could allow him to l to watch this type of interface and he would be able to do the selection say this is this this is this this is this this is this and when he would receive the d v d yeah yeah he would indeed get already the chapters that he decided to put there and this could be done by through internet yeah yeah yeah and i think this may it it is good that this is a demo like this because i am sure that they might be interested indeed because i imagine that some people yes from the client point of view and from the preparation of the movie d v d yeah yeah yeah yeah yeah so i think this is this is a very good and a yeah but okay i will play yeah so this is a kind of service i do not know either they give it for free and but we had an arrow here or what it is sorry but you see that sometimes even here it is a bit like it is for instance if people would like to have like some motion compensation th you i it could be proposed yeah but for instance here it i wou it would be problematic to have motion so i here it is fine to do motion compensation but stabilization rather than yeah but for instance here it is it is it is it is yeah i think i think they made some special effect in this d v d effects okay there is 2 or 3 time you see you see the same same kind of scene just 2 or 3 seconds if i go to menu there is something ah okay so maybe like here for instance it is a just reproduced 2 times yeah do you know what this is no old menu there is no menu in the d v d no there is no menu okay okay there is just 11 chapter and no menu okay so this is the same video a the the a v i number one yeah yes yeah yes and on the on the d v d i saw that there are some other stuff i dunno this one i think in the video there is 2 there is 2 video one is a real video and the other is a just introduction when you have the menu the same okay oops so this is vi okay this video is about 4 minutes 5 minutes right no it is 13 minutes ah okay so this is not the one that has been no but yes this is the same but there was a problem when do the grab from the d v d to the files ah okay okay so this is why they have make 22 files there are 2 files okay i understand but now i have one file also in the so you can process again one time okay run the okay okay it is it is it is real time right i mean the processing yeah almost yeah i would maybe a less a little bit less than a bit more than real time maybe a bit more than real time well anyway yeah i will have to check but if it is 5 minutes videos it is like done in yeah quickly yeah yeah normally it could it could go faster but i probably it is probable that even for the project you would do some of the reprogramming yeah i mean to especially if we need to interface new yeah or for just reading yes okay components open c v i mean the the d v x and thing like this so it is so tomor anyway for this we do not know exactly yet yeah yeah yeah because it is not sure what they want to what the user is tools for reading d v d and so on so okay yeah yes and s because i do not think this is so difficult now you have already looked at the histograms and open c v computing distances and so on open c yeah so if we want to do a special thing that probably runs fast it may be easier to do it directly and yeah yeah but but yeah the interface is very so the interface uses the output of your program essentially okay yeah so that we will use the whiteboard okay so in fact yeah the interface is quite is quite simple to use for instance you see on the on the u r l i put okay i i disco there is m m m dot idiap dot c h slash v c g i bin slash video browser okay slash video browser 3 frames 3 frames dot h t m l and then this c g i program ask the parameter c video and m equals cinetis okay demo one in fact i just explain how to put the data on m m m but it is quite simple when you log into m m m as with a c c s s h for instance so you just put it is working like this m m m data video browser okay slash and then this is the name of the new directory you will create demo one and there stands there is couple of things well it is this name so i will say this is video name so video name dot a v i you just because we need the a v i so that we can you can do the get frame yeah we can do the the get frame but otherwise it is not necessary 0 yeah no no no otherwise it is not necessary ye okay 0 f and also for the 0 no no you need you need to generate these images right okay yeah and this is no no no this is done on the fly with get frame okay okay yeah so you need that you need the x m l v so it is video name okay and i could data dot x m l but in fact it is just a simple convention i w could have put nothing but i just put data the it is a convention okay data and then video name okay it is r m 9 dot r m for other stuff okay but in fact yeah when i think now we could have okay yeah only with extension you can you can see yeah yeah okay but the the name of the video file is video name is it is the same as a directory the name is no of in fact yeah is is yeah as a directory yeah yeah yeah is the same as the directory it could have been video naming also but it is the same as a directory okay okay yeah and that is it actually when you you u you do that i did the same for kayak you say kayak in fact now it is kayak and then you can see yeah yeah okay yeah and this output yeah as your this output this file is an output from the the software the shot boundary detection software x m l yeah on the shot okay okay and it is a simple format i suppose like like you have scenes and shots and key s some time stamps it is a very simple format yeah yeah actually we yeah okay okay yeah i do not yeah we cannot see yeah okay yes okay so but you are yeah it is yeah it is tha video scene shot yeah i mean it it might be good to yeah okay it is hierarchical in x m l and key frame time stamp and the and from the time stamp and the video location he knows how to get the frame yeah okay each of these frames okay and if we do in fact i do not know yeah it is mozilla firefox view page source okay view frame source okay okay so every every frame on the f page is online and every frame is stored into a b a backup in fact every frame it is m m m who manage this automatically is is a okay so that when you ask for this frame in fact we can okay i just view frame source when i view image when i say view image when i say view image on this frame yeah yes up we go there and this is in fact it is a perl program that outputs in fa instead of outputting h t m l or text it outputs j peg okay okay okay okay understand from the video with a time stamp it outputs a j peg yes okay okay but on the on the on the server the original is a f video film this is quite nice and yeah okay and this image is i this image is output outputted to the client so for example you can change you can change a number to take which one okay yeah exactly yeah yeah that is what i did okay understand here in the other yeah yeah in the other application yeah okay perfect yeah i dunno and who has a right to put who has a right to put data on the m m m yeah here in this one okay i f there are yeah the m m m developers so i am part of that but we could we could yeah it is not a problem okay yeah i think it is not well done yeah okay yeah yeah i wanted to saw to see to do s say something else 0 yeah and the images are sent to your browser the client to this browser okay but yeah but also kept in a directory in on m m m okay okay there is li local cache exactly a cache okay okay i say backup but it is a cache okay okay that is that is how it works okay so if we for example now we have the c the the case that this video is not correct because i have make event grabbing so we need to remove the cache for this video or this is automatic no no no it is okay for the cache we have plenty of space on m m m so the this is not a problem ah yeah for a new window yeah yeah but what i mean is i i i i i if i if i put the same video with the same name instead of this one yeah exactly yeah and it detects okay they should different yeah so you go inside the get frame function you see where it i it saves it ah okay we need to remove them and then there is a special naming convention and you can remove the but i it is c yeah it is true yeah okay okay understand okay yeah it is not a real usi usage no okay because there it is in cache so that if i put another video with the same name they th they will happen again okay yes yes understand okay ah okay yeah yeah they will appear this frames will appear yes bec because this one i was yeah if yo if you have the same decomposition right th this one was yeah yeah ex if you have the same decomposition but yeah yeah but it may happen that for 90 fra 90% of the frame it is different yeah so you see it fine but suddenly you have one other frame they do not know what it is and where it comes from right okay okay but it is okay but yeah on m m m f now we do not have such many videos so it is okay okay yeah yeah yeah yeah but if you put yeah in another directory then it would work or or a new naming convention with special yeah code yeah yeah 0 you mean bac because it is it is it is the m m m cache yeah m m m cache okay so this means that when it is done if i go on my computer and i do the same it is not going to enter in my cache but it is going to be already there in m m m okay yeah on the m m m yeah but it it can go in the browser cache also so it is th it is so it is not done on it is so it is not done online the 1st time is done online the 1st time yeah but after it is it take okay yeah you can see here yeah okay okay the 1st time is done 0 i thought it was everything was generated on the fly th so you know you see that well here it is it was cache because i already pressed preview but now it is on the fly yeah yeah okay okay so it is quite quick 0 it is greek but okay this okay so i if you come back now you and then if you come yeah yeah yeah because the images are already generated okay but now you have s you have both cache yeah okay and maybe they are on the cache of this browser this computer yes computer too yes to but okay so okay but it is fast that is why it is yeah it was used with yeah sebastien did the c function and yeah okay and the the is calling this c executable and it was used with m m m server with pierre a and the beginning when yeah yeah understand okay yeah okay okay so okay i ju just a question on the the organization of the m m m yeah yeah so you do not have a part for specific project every for example if we make a lot of film for cinetis all are on the top yeah i mean all the film are y you you do not have a separation for the film for cinetis or for the project and s 0 the movies or yeah yeah yeah in fact we we we could you know okay you know all these graphical user interface that you see okay well you say video name equal all are on the same level so could you say video name cinetis slash something in fact okay yes or not yes we could do that okay okay we could put cinetis slash here and then a cinetis slash is the sa yeah it works okay you th okay you think it would work okay okay but the th the point is this interface is i think 6 or 7 files only yeah okay x m l well it is per c g i that output x m l and x s l so it is it is not a big deal to to to make a new like a cinetis browser or you know video browser ah okay to make okay if we need to make some special thing on the browser it is t yeah we w we copy all the files and there is just 11 parameter one string okay that we need to know where this data is and we could pr create a m m m data video browser cinetis or something else yeah okay it is okay y it is ex extensible and is okay great even if you would r change a c g i for instance let us assume that we say we do not do any scene thing then we could remove like for instance the scenes right yeah okay yeah exactly yeah yeah okay i was thinking that here it says that you have online video structure correction which means that yeah but i it is not it was you know thi this is working the expand all but this is okay yeah work in progress this is wish wishable functions yeah to do list or something okay to do this on a video annotation on a video structure co correction but i it is not easy on the web with h h t m l pages to to do such corrections like video structure corrections for instance okay yeah input okay no for that you need it is possible maybe on interfaces like flash or java but still i i i there is it takes bas more work and but i yeah yeah i mean it is good good point yeah okay yeah mainly the t webpage or for output okay no i was thinking no because i was thinking if it is there i mean we could use it we could you see but yeah yeah okay and all these file are only internal this mean if i go t on c cintetis cinetis no no fro from outside we can view them from outside of idiap we can access this yeah from home i was able to access them with a password or no well you we can make for for instance for t s r t s r videos alessandro asked me to do the same thing and we i put we had like a password convention yeah yeah okay yes okay okay so this is possible to make for these videos okay nobody actually it has been downloaded several times and people there are some lawyers that are putting no so it is not yeah well if yeah it is there is no possible productions for these videos but yeah maybe what i say i am very tired y yeah okay but if po yeah it is demo and one can say you have to guess the name to but yeah yeah okay yeah yes yeah okay yeah yeah yeah yeah no no yeah okay no but yeah it is good to bu y yeah a good point about password protection issues or the privacy issues yeah okay yeah if we i yeah it might be good indeed if we start d building some experiments like this still to have a password okay right yeah yeah ha yeah all the framework is ready with a t r s t s r videos so yeah yeah yeah yeah i understand yeah yeah okay usually what we do sometimes okay just to say something about password protections sometimes instead of having a purely secure directory we p we we put a co code number like so that it is quite impossible to guess instead of video m name equals kayak then we would have 0675 or yeah yeah okay yeah yeah yeah yeah you know 2 and then this directory from outside we have to take care that okay not only the video name this u r l is not accessible but also we should take care that in fact m m m it is in m m m data video browser and this is accessible from outside we should take care that this directory or in fact this directory is not listable you know you cannot list from outside with f t p or h t t p right okay yeah yeah what is yeah yeah yeah okay yeah simple things with privacy issues but i there are some methods and everything is possible to make things private okay i yeah i do not know if you want to add anything but i would say that for the c code yeah i i i just want would like to stress something is that yeah if you want want to use the new versions etcetera i think we need to go a little bit through the code and i would be happy to to to help to to for instance the classes about video structure yeah yes etcetera to put them yeah okay yeah and yeah it was done indeed f to be more general than just to do the shot detection so that is why the code is a bit more complex i think currently yes yeah okay but and the code is based on open c v yeah in fact it is not there is no tosh i it was called vision lib to read to to extract one frame each frame of a video yeah you know 0 okay so it wa from sebastien it was in the tosh library but it is nothing w to do with tosh and it is using the open c v and some classes from jean marc for yeah yeah okay just okay yeah yeah bhattacharya distance etcetera bhattacharya okay and so on okay ah okay yeah yeah okay image processing yeah it was the same the same ones as the one you saw in my code yeah okay okay image processing and stuff so we can forget tosh and use input se the input reader for the the m peg we can re lo use open c v yeah yeah we can forget tosh yeah yeah yeah yeah yeah yeah yeah yeah definitely yeah okay okay yeah okay so it is 430 so we need one hour it is it is good just if you want yeah yeah i think it is it is good yeah it is good this is the parameters of the of the c program but usage yeah yeah video in okay yes the output file x m l output output file dist it is the distance between each histogram yeah so that if the x m l you feel there is not enough shots you can instead of reprocessing the video you you reprocess this file okay that is good yeah and also some other stuff s for instance alpha threshold we talked about is set to 24 and yeah okay okay okay yeah so the the the step we always use one and it should be one because otherwise it is stuck between consecutive frames yeah yeah okay ah okay okay we can skip some frames yeah okay yeah but it does not w yeah it it is better to to keep it one yeah okay yeah but why is after i mean if there is a shot you do not know between because you dropped one frame you do not know where is the sh the d the shot yeah yeah yeah yeah so detection latency is about the it is half of the windows tha yeah sliding window sliding window option for key frame extraction okay sub threshold yeah yeah right okay leave it to one yeah okay it is perfect yeah maybe i i will i will give you this file in as a rhy it is a kind of rhythmia yeah i just for preparing this nice discussion yes yeah m i am quite happy to to to do that i it is good when you do something that there is someone using it i do not know i am i am sure that they will be well i hope they will be a of something like this yeah yes but but yes yeah at least yeah with the interface it is quite funny to to see d that the q quickly you can do something in in yeah the output in 30 t if the video i 10 is 10 minutes in 30 minutes cause it is no not a lot of things to change yeah yeah the the browser or even it can do it it can we can do it quite automatically yes yes it can be done yeah so just last question maybe so this makes a x m l file yeah okay this is the the program to to generating the x m l file and the the shot this generates yeah yeah yeah and with the kayak you have one problems you do not have the real media so this is another thing 0 yeah it ho how do you generate the real media yeah so the real media is yeah a parallel thing that from the a v i you convert it into real media and it is done on windows okay okay okay okay so you have some special tools from from real media yeah okay there is a a the free one is okay you can use the free we have a license here at idiap but the if you want to pay you pay for options such as resize or cropping but in fact maybe we do not need that and it is called elix producer okay okay okay okay i heard it yeah i forgot yeah yeah elix producer and and that is it okay you can you can i it it i but anyway you need you need to have windows to make these yes so i will ask you or someone else okay yeah yeah okay okay understand and yeah with real media okay but this is this is made by a program done outside idiap okay well this is you can buy or you can order for free on internet this is yes right okay this not idiap program no no no no for the real media okay okay and you did not see but the re the real media is called via a real server so it is r t s p the connection and i it is streaming video streaming so it is not h t t p yeah yeah okay so that it can play smoothly and but we are still investigating for the video formats yeah because i it is not supported for instance in java in java applications etcetera 0000 yeah you mean the java framework what mike presented real media yeah we cannot play real media okay real media so there are many issues about that but yeah okay okay it is another problem formats formats yeah yeah okay so i think this was good to see the the tool and tomorrow we are going to re say most of the things okay thank you but shorter i f i believe right i mean it is bad to explain okay yes and i think maybe later w yeah we we can start doing simple i think for test i can take the real demo and make the the full processing yeah yeah yeah yeah the full yeah mi this might be interesting to see tomorrow so i will of course maybe there was the ah 170 shots or it is a lot yeah but after i mean at this point we do we have nothing we can do unless they know what they want to do with it right yeah yeah okay yeah that was a single video structuring but it is other level it can it is quite it is researched yeah yeah yeah yeah but i think i think is it is too it is going very be very to be very good for them to visualize how you can organize things and maybe what they can do yeah how they and so this is important yeah last thing this is a kind of x m l that is outputted so so it is the structure is such as vi video segmentation with videos in begin and then for each shot you have an i d well this is the terms from m peg 7 convention okay okay i have some informations but anyway shot begin and end a sh the shot key frame that is the middle here okay sub shot and in this in this the begin and end the number is the for example the end is 0 dot 1212 is this frame too yeah yeah you are right 12 frames okay and in fact that is what i output in on the yeah i think y on the interface okay okay yeah okay and there is only one sub shot because w yeah okay yeah initially yeah there was there was problems with detecting the sub shots etcetera what what well we did not because you tried to do something but yeah well yeah i tr well not with this one but yeah i tried to code some stuff it but i am not so confident in the way it is coded okay yeah like for instance if we need to find sub shots usually it it for instance is better to have the whole shot to decide on how to build the sub shots yeah but it is true that maybe i if you store all the distance at that the distances we could do some basic stuff yeah but we yeah even we should for instance for the videos for the spectral clustering for the clustering okay here i store the consecutive the distance between consecutive histograms but we could store the matrix that is yeah okay yeah yeah yeah yeah yeah yeah yeah yeah that is the thing yeah and then yeah okay and i think i there are many possible things with spectral clustering just what what but but the with there is one parameter that is very c wha i can not remember what is the parameter ah th the gap you mean or no yeah 0 no you mean in the bhattacharya distance no no no no the in the the gap to find the the good number of clusters is so sensitive yeah yeah yeah yeah yeah yeah yeah anyway it is it is a bit later in the in the report okay so anytimes yeah i would be happy to to to help you with that yeah okay okay so it is okay it is okay yeah yeah you can we can go
0 you have got your laptop yeah okay i do not know i have got it i have got it i have already the laptop because i think no no no but it is it is good to if you want to use it after 0 no no no no because we are going to be in the same 0 yes we are supposed to speak in english 0 yeah okay you do not put your headset microphone no no so it is in the country everything so we are here to just make the point well make the point yeah yeah it is a status about the shot detection okay so you have applied the shot detector to some of the syneties video on this one and so we can basically maybe just comment on this maybe quickly remind what the shot detector is how the shot detector is working basically we might be able anyway to give you some very very very good reference i mean your report yes it should be described here yeah i have actually yeah we can see the report here and the demo and the goal of this meeting is to present the video shot detector to olivier yeah yeah okay so yeah it was my project during my internship at ediap under the supervision of your mark and as you saw in the email i sent okay i did not send this video is the new one i just processed from your images okay yeah it is okay but let us start by the video from friday cynitis demo one by the way i was going to ask i had seen the video you did a downsampling of the video right i mean for browsing i mean yeah okay yeah so this was okay so this is a yeah ki called normally so you have scenes and shots yeah here on the left you have different scenes but because for instance we could probably group or the initial ones into one scene and maybe after when there is some things inside it should be another scene so this detector detected 30 shots and if you click on a plus button you can expand it and see the keyframes that are in fact 5 extracted frames from the shot which are extracted i think but i think as it is it is extracted without using sub shots it is simply 5 down some yeah not something so i we can see the video actually the entire video and open with real player i hope it will work i do not have the sound and it will not be grabbed the video will not i do not know if okay it is not good quality yeah the quality is that is why i was asking that 0 yeah yeah it is not this that you process no no no no no okay so it is in real media so it is very compressed so here we see that we have the shop at 18th and after 22 then after again and if we go down i mean maybe i mean maybe in terms of shot here we can see that right i mean this is shot 7 i suppose yeah okay i shot 9 currently 10 probably 11 or 12 no 11 10 and then there is yeah this one it is 11 12 for 14 seconds it is ah we have missed this is this one and this one it is a dissolve so it does not detect is it really a when i saw the m player it is a dissolve okay but it is not standard man made movies because i would say most of the time people did not do any this yeah i could see here and that it was like here of course it was only one single one i do not know if this one if we see was currently you see yeah yeah this is the one i had seen here but as i said i mean even the motion detector when it is black there is not no structure but there is a moon yeah but you do not really estimate motion okay because there is no motion in fact between the 2 shots there is no structure there is no texture yeah okay okay this is not so good for the motion but for instance of course between the horse and the yeah the yeah i dare the q more the co the coes yeah maybe we could have detected things but it is not even sure because you see the background is quite similar and it is not that sure maybe to explain this method was using a simple distance between consecutive histograms and we did not use motion features at all for that that is why you so it is yeah and the histograms so you had experience with it so it scrambles so you can scramble the image and you get the same instagram so there is no structure in the greek so of course when you look the horse and then you can see the cows of course you have black in the top and you have also maybe 80% of like green which is even not green there but grass you have stones and so there are of course some differences because on one hand you have horse but it is black so you have more sky which is black on the right so you do not care about this so it is a global it is a global instagram so yeah okay but it is when for this i would for this application i could imagine that it would be not quite fine and it is a threshold but still it is adaptive yeah yeah yeah with a sliding window yeah because of course if you just compute the i can not show you yeah if you have the report because if you compute the distance between the 2 histograms i think this was batacharya yeah right and so if you take these 2 with the batasharia then of course if you just do a threshold when there is motion you will see a lot of things changing so by using an adaptive threshold if you see that there is a continuous change of the distance you do not do a threshold it is only if you have an identified peak which is 50% or maybe 2 times higher than the next 2nd value in the window then you see it is a shot yeah i do not know if you want to use the whiteboard or here it was to do it to this part about shot boundary detection is in french unfortunately but the the this image shows the results using motion when you use the motion features and here on the right is when using the histogram distance and the the motion features can better show what is going on when there is such difficult transition while with the histogram it just shows that there is a shot change at the end but even for dissolves i do not have examples here but there is not a strict transition it is continuous for dissolve but these are anyway it is difficult i mean in general because sometimes there are some you could imagine they are part of a scene and so but of course when you have cuts you can see they are easy to detect right i mean like so yeah so with the motion basically the motion tried to estimate what the motion between 2 frames right if my image is there but after it is there and it looks at how many points are well matched so you can see that like somebody is speaking and so on so before the before the shot you can see that usually there are like 90% of the points at least right there are some motion so but then after suddenly if you go from one frame and you try to match all the points here with like a translation plus a zoom plus maybe some rotation you can not find anything well right so basically if you see that the number of points maybe there are only 10% of the points you can match well so it is easy to detect and after you can see that indeed because it is a long shot long shot and it is almost like a flat plane you can do a very good registration so the number of points is very high i mean the number of points that are matching and here you basically here with histogram you can see it is very flat but of course you should go from this histogram to this which is essentially green you have this big measure which is here the kite 2 i do not know whether this is what you is implement is used no now it is well for the video you saw it was it was it is batasharhyan okay okay so yeah and you have maybe one image with an adaptive threshold i think you had something like this right just to explain okay it is at the beginning yeah yeah it is like here so there is the idea is having a sliding window we have already all the data that is on the graph and then to detect the changes we do a calculation with points under this sliding window within this sliding window and we calculate so what do we do here in fact we want to detect that this this point in the middle is higher than alpha times the maximum of the other ones exactly alpha times the maximum of the other ones so the of course if you are not a maximum you are not going inside the window you are not the max anyway you are not going to be to be above alpha times the max of the others right because alpha is lower than one but if you are in the maximum already at this point we check simply if this value is alpha times the max of the other points let us say this point here like for instance here i see 6 but i do not know whether this was for batasharia or for k maybe for kai it is 6 but yeah but for batasharia it is for the test i did it is 244 yeah so it is a bit less because the amplitude is much smaller yeah so you can it was not so obvious i mean because the motion i mean it was in the shot we have seen because especially with kai 2 square it is the amplitude is much bigger than with batasharia right you can see that it could go to 700 and when it is there is nothing it can be 0 but batash area is between 0 and one so you have not so much amplitude but still in some of these videos for instance even if we are going to go now if we want to go now for the kayak case i do not know at some point he is the targeting 2 kayaks and is moving quite fast towards 2 other people i do not know what happened there if it was detected as a in the 1st shot yeah yeah i saw that it was detected as a shot as a shot okay i was wondering you want to see now yeah maybe but he did not okay but here for instance you understand so we can show this tomorrow to the people at the cinetist meeting i mean so okay we look to have a connection yeah okay yeah we need a connection that is true and i need a laptop or you have you yeah i can bring my laptop if necessary okay i will bring it okay okay okay this is ediap laptop so you could okay okay so here in fact after 12 i do not know if it is i think it is 12 frames 0 yeah but normally there is a burning maybe you can play the shot or it is strange 0 yeah maybe a burning but maybe you play yeah if you play the video well at the beginning i do not know what is happening 0 there is no i did not i do not have tried to the real media but yeah it is complicated well no i mean here i can see that it was detected as a single shot yeah here we can see that okay i was because here you can see that the motion here is there you know he is doing a zoom on the 2 yeah and then there is translation yeah and then he is doing quite a fast transition from this to another one and i was wondering whether so if you well vote of course it is true that it is mainly water so maybe the color is indeed quite but you know as kayak is entering quite fast i mean you can have quite i can see there is another program that can see it seems to be how many shots do you have indeed the end it is i think it is there are 5 normally yeah the one is if we put it so you did it with what by the way it is with the images i gave you yeah okay the other one had problems converting the video into divix and then so i used the program from a software from sebastian ppm to avi ah an avi file okay yeah yeah because yeah one thing maybe we will talk about that is the library it is it does not compile anymore and dbn machine and dbn machines maybe some work has to be done on this i do not know why this image does not appear this is the frame that is extracted to be the shot keyframe here this one is this one the middle one so i do not know why it does not appear but it is just a simple problem in the gate frame but what i can show you now is a little program if we want to be a little program very precise about so it is gate frames okay fijiyae and we were interested in around what do you want around 6 7 seconds yeah i mean initially i do not know why there is a shot detected okay if we do right next 0 here there is something wrong so the 000 here is an image number yeah in fact yeah that is what i said it is a frame number i do not know why yeah yeah it is a firm number this is the frame number this is in 2nd but i kept the frame number so but 002 it is in seconds or it is 12 frames 0 seconds okay yeah okay okay seconds plus the next is 11 seconds okay so basically what has been yeah extracted is this pointer other then okay so this is normal yeah so if we do every 2nd the middle frame is this 1000 okay just it is another kind of browsing page but to be more precise about to check maybe the this okay like transitions yeah exactly yeah then you go back to one frame around 160 for instance you do not know and there is okay yeah that is it prev okay that there is a okay it is not a shot change it is a motion actually here yeah it is motion yeah it is not a cut yeah but i was thinking that maybe it could have been yeah but it is true that it is many water when we are doing so it is yeah such a big but yeah sometimes i mean you can have very things i mean just think about more action movie and things like this sometimes very difficult so for cuts usually there is no problem people say this is a cut this is not a cut but like for dissolve even sometimes you know people may have some trouble i mean saying this is a default this is not we should this should be labeled as a transition or not some people who say yeah but it is part of the thing or for instance in like in also in come advertising because the shots are very short yeah i think usually it is very fast so sometimes yeah when you have an incrustation like last time when you what should you say it is a change of when you have the goal results then yeah in fact it is not a short change yeah i mean so sometimes very difficult at least what i can see that currently like for sinaitis for instance if it is to do color correction or for instance if it is too black to have it whiter i think this type of thing should be fine because basically even if they are not real shots they are not so interested if it is not real shots maybe after they have this and they may say okay just by looking at least the summary so maybe for the summary we could do better keyframe extraction for instance but by just looking at the keyframes and so on they may say okay from this shot to this shot we apply like specific color enhancement algorithm or restoration algorithm or to improve the brightness and something like this right and so i think for them this is fine so this would be made manually actually when you say from this shot to this shot is using a graphical user interface someone would do manually for each video well maybe after i mean they would go directly and select the shots from here to here and they would say you apply this algorithm yeah so of course we could try to make scene clustering and so on but i am not even sure that this is necessary for them at this point because if they have a 6 minute movie if they have 30 shots for them it might be easier to look at from here to here and they apply this and they can do it quite quickly and they have to do it anyway because what is going to take time might be to select the type of restoration algorithm or the type of thing so this might take more time than having to click 2 times on 2 shots rather than having directly a scene right so i think for them this type of things is going to be useful at least for this task if they want to do stabilization it might not be sure that this is the right segmentation for some of these but this is another point which for motion stabilization as i said several times we did not commit to provide anything provide anything of course if we can provide things we do it so do you have more questions or do you have a i have looked at the demorphine and it is about 40 minutes and about 173 different shots this is the one in the dvd yeah maybe okay we can see a little bit if you want to play yes and i do not know if this is a real sample dvd because this will make about 5 seconds for each shot so i think it is very short well no it is okay well i just i think this is if this is to make a change in the color like color light and they have a lot of shots yeah but after i mean it might be more like a guy an interface a guy interface where i mean if they can select from shot one to 7 this is the same thing they may do the grouping by themselves but by looking at this it may take less 11 minute and i think it is anyway if it is too summarized after they may want to go inside to see whether there has no special things i mean so because after it depends i mean if you give only one keyframe per shot after i mean it is 170 images it is quite fast right so i think anzeriarchical thing is good because still you can check if you have some ambiguities you are not sure okay in fact i did not understand that but now i understand better that cinnitus wants this kind of method to help them to rest and to apply some algorithms to make a better movie it is not for the client for the user who will well ultimately i think this is a type of thing but at this point i hope that if it is a big success they will ask you to provide something like this of course some money at least to be a student or somebody for doing it but yeah of course this was ultimately i think one way of if you could provide like chapters or something like this indeed you could provide like the avi to the clients and he may ask for special services or and even to just to watch the movie to have direct access to different parts and so on but i mean this labeling might be more easy to do for the client by the client because he knows what they are okay he wants to have this and this 0 okay yeah yeah so because he did the movie so he wants to okay yeah to archive yeah yeah otherwise i mean this but this is yeah but of course it might be easy to do a very simple software which would allow the user to select just the shots and give names and after it would generate automatically some index yeah and i suppose the client might be interested in such a thing so he could generate automatically on the dvd but for and you could imagine that before sending him the dvd you could allow him to watch this type of interface and he would be able to do the selection say this is this this is this is it and when he will receive the dvd he would indeed get already the chapters that he decided to put there and this could be done by through internet yeah yeah yeah and i think this may it is good that this is a demo like this because i am sure that they might be interested in it because i imagine that from the client point of view and from the preparation of the movie and dvd yeah so i think this is a very good but okay i will play in so this is a kind of service i do not know if they give it for free and you had a narrow here but sorry but you see that sometimes even here it is a bit like for instance if people would like to have like some motion compensation it could be proposed but for instance here it would be problematic to have motion so here it is fine to the motion compensation but stabilization rather than yeah but for instance here it is here it is you returge you 2 or 3 times no no that is more it is really how it is i think they made some special effects there is 2 or 3 times if i go to menu do something you see the same kind of scene just 2 or 3 seconds okay so maybe like here for instance it is just yeah we will reproduce 2 times do you know what this is no what is this no what menu there is no menu in the dvd no there is no menu okay okay so it is just one chapter no menu okay so this is the same video the avi novi no one yes yes and on the dvd i saw that there are some other stuff i do not know this one i think in the video there is 2 there is 2 videos one is the real video and the other is just an introduction when you have the menu okay oops so this is okay this video is about 4 minutes 5 minutes right no it is 13 minutes okay so this is not the one that has been no but yes this is the same but there was a problem when i have to do the grab from the dvd to the files ah okay so this is why they like to files okay i understand now i have one file also on the so you can process again okay and it is real time right the processing yeah i would maybe a bit less a bit less than real time a bit more than real time i will have to check but if it is 5 minutes videos it is like done quickly yeah normally it could go faster but it is probably that even for the project you would do some of the reprogramming yeah okay especially if we need to interface new components opencv yeah or for just reading the dvx and things like this so tomorrow anyway for this we do not know exactly yet because what they want to what they use as tools for reading dvds and so on and because i do not think this is so difficult now you have you also already looked at the instagrams and opencd computing distances and so on so if we want to do special things that probably run fast it may be easier to do it directly and yeah but yeah the interface is very so the interface uses the output of your program yeah so i will use the whiteboard so in fact yeah the interface is quite simple to use for instance you see on the url i put okay i just go there is dotidia c ch slash cgi bin slash video browser okay slash video browser 3 frames 3 frames dot html and then this cgai program has the parameter video name equals cinetis okay demo one in fact i just explain how to put the data on but it is quite simple when you log into as with a css for instance so you just put it is working like this working like this data video browser and then this is the name of the new directory you will create demo one and there stands there is a couple of things well it is this name so i would say this is video name so video name avi you just because we need the avi so that we can we can do the get frame yeah we can do the get frame but otherwise it is not necessary otherwise it is not necessary okay and also for the these images right okay yeah and this is no no no this is done on the fly with get frame okay yeah so you need that you need the xml so it is video name and i could it data xml but in fact it is just a simple convention i could have put nothing but i just put data it is a convention data and then video name okay it is rm 9 rm for the stuff but in fact yeah when i think now we could have only with the extension you can see the name of the video file is video name the name is no in fact it is the same as a directory yeah it is the same as a directory it could have been video naming shows also but it is the same as the directory and that is it actually when you do that i did the same for kayak you say kayak in fact now it is kayak and then you can see and this output yeah this file is an output from the software the shot boundary detection software and it is a simple format i suppose like it is a very simple format like you have seen shots yeah actually we some time stamps yeah i do not yeah we cannot it is yeah it is video scene shot it is hierarchical so in xml and keyframe timestamp and from the timestamp and the video location he knows how to get the frame each of these frames and if we do in fact i do not know it is modilla firefox vue page tools so every every frame on the face is online and every frame is stored into a backup in fact every frame it is who manage this automatically so that when you ask for this frame in fact we can okay i just viewframe for those when i view image when i say view image when i say view image on this frame up we go there and this is a in fact it is a pearl program that outputs instead of outputing html or text it outputs jpeg okay i understand from the video with a timestamp he outputs a jpeg this is quite nice on the server the original is a original is a video film yeah okay and this image is output i was put it to the client you can change a number yeah exactly yeah that is what i did okay i understand here in the other application yeah yeah okay perfect and who was the right to put data on who was the right to put data on there are yeah the developers so i am part of that i am part of that okay but we could we could yeah it is not a problem yeah it is not a problem i wanted to see to do something else yeah and the images are sent to your browser the client to this brother but also kept in a directory on okay there is a local cache exactly a cache i say backup but it is a cache that is how it works so if we for example now we have the case that this video is not correct because i have made a wrong grabbing so we need to remove the cache for this video or this is no no it is okay for the cache we have plenty of space on so this is not a problem what i mean is ah yeah for a new video if i put the same video with the same name instead of this one and it detects again the shoot are different so you go inside the get frame function you see where it saves it and then there is a special naming convention and you can remove the okay but it is true no it is not a real usage no okay because it is in cache so that if i put another video with the same name ah okay yeah they will happen again they will appear yeah because this one was if you have the same decomposition yeah if you have the same decomposition yeah but it may happen that for 90% of the frame it is different so you see it is fine but suddenly you have one of a frame they do not know what it is and the way it comes from but yeah on now we do not have such many videos but if you put in another directory then it would be or a new naming convention with a special code 0 you mean because it is an cache yeah cache so this means that one it is done if i go on my computer and i do the same it is not going to enter in my computer going to enter in my cache but it is going to be already there yeah in an okay but it can go in the browser cache or so it is not done online the 1st time is done online yeah but after it is yeah you can see here yeah the 1st time i thought it was everything was generated on the fly you know you see that well here it is it was cash because i already pressed few but now it is on the fly yeah okay so it is quite quick quick okay okay so if you come back now you and then if you come yeah because the images are already generated and maybe they are on the cache of this browser this computer so okay that is why it is it was used with sebastian did the c function and the bird is calling this c executable and it was used with the m m server with pierre at the beginning okay okay so just a question on the organization of the mnm so you do not have a part for a specific project every for example if we make a lot of film for synethys all are on the top i mean all the film are the film you do not have a separation for the film for synethys or for other project and in fact where you say video meme equal we could you know all are on the same level you know all these graphical user interface that you see so could you say video name equal cdeme slash something okay yes yes we could do that okay we could put cinity slash here and then cinity slash it works but the point is this interface is i think 6 or 7 files only xml well it is per cgi that output xml and xsl so it is not a big deal to make a new like cynetis browser or you know video browser if you need to make some special things on the browser we copy all the files and there is just one parameter one string that we need to know where this data is and we could create data video browser synethys or something else it is quite extensible even if you would change the cgi for instance let us assume that we say we do not do any scene thing okay yeah exactly yeah i was thinking that here i see that you have online video structure correction which means that yeah but it is not it was you know this is working expand all but this is a working progress this is wishable yeah to do list or something okay on a video annotation on a video structure correction but it is not easy on the web on the web with html pages to do such corrections like video structure corrections for instance for that you need it is possible maybe on interfaces like flash or java but still it takes place more work and but yeah yeah it is a good point no because i was thinking if it is there i would visit it and all these files are only internal this means if i got on the city no no from outside from outside of egypt we can access this yeah from home i was able to access them with a password or well we can make for instance for tsr videos alessandro asked me to do the same thing and we i put a we had a password convention okay okay so this is possible to make for these videos i am sure it has been downloaded that several times and there is some lawyers that are there is no password productions for these videos maybe what i say i am very tired but if yeah it is demo and one can say you have to guess the name the name yeah yeah yeah yeah okay okay no but yeah it is good to yeah a good point what password protection issues yeah privacy issues yeah it might be good indeed if we start building some experiments like this still to have a password okay yeah yeah yeah yeah all the framework is ready with trs videos yeah yeah okay usually what we do sometimes okay just to say something about password protections sometimes instead of having a purely secure directory we put a code number we put a code number like a so that it is quite impossible to guess instead of video name equals kayak then we would have 6 7 5 or 2 and then this directory from outside we have to take care that okay not only the video name this url is not accessible but also we should take care that in fact it is in data video browser browser and this is accessible from outside we should take care that this directory or in fact this directory is not listable you know you cannot list from outside with the ftp or http okay simple things about privacy issues but there are some methods and everything it is possible to make things private i do not know if you want to add anything but i would say that for the c code i just want to stress something is that yeah if we want to use the new versions etc i think we need to go a little bit through the code and i would be happy to help to for instance the classes about video structure etc to put them yeah yeah it was done indeed be more general than just to do the shot detection so that is why the code is a bit more complex i think currently the code is based on open cv and torch yeah in fact it is not there is no torch it was called vision lib to read to to extract one frame each frame of a video you know so it was from sebastian it was in the torch lavery but it is nothing to do with torch and he is using open cv and some classes from jean marc for batasharia distance etc okay yeah yeah image processing the same ones as the one you saw image processing so we can forget torch and use yeah yeah we can forget in torch for the impact we can use opencv yeah yeah yeah definitely okay it is the farrar of the vale that is no it is the garage of the batheer so it is 430 so we did one hour it is good just if you want yeah this is the parameters of the c program but usage video in okay the output file xml output file dist is the distance between each histogram so that if the xml you feel there is not enough shots you can instead of reprocessing the video you you reprocess this file okay that is good and also some other stuff for instance the alpha threshold we talked about is set to 24 and yeah so the sig 1st that is the step we always use one and it should be one because otherwise it is step between consecutive frames you can skip some frames yeah but it does not yeah it is better to keep it one otherwise after i mean if there is a shot you do not know between because you dropped one frame you do not know where the shot so detection latency is about the it is half of the window yeah sliding window option for keyframe accession again sub threshold yeah leave it to one yeah okay it is perfect yeah maybe i will give you this file in as a it is a kind of read me just for preparing this nice discussion to do that because it is good when you do something that there is someone using it yeah i am sure that they will be well i hope they will be at there with something at least here with the interface it is quite funny to see that quickly you can do something in if the video is 10 minutes in 30 minutes because it is not a lot of lot of things to change or even it can do it it can we can do it quite automatically it can can be done so just last question yeah so this makes the xml file okay this is the program to generating the xml file as the shoot yeah and with the kayak you have one problems you do not have the real media so this is another thing 0 yeah how do you generate the real media yeah so the real media so the real media media is a parallel thing that from the avi you convert it into real media and it is done on windows okay so you have some special tools from real media yeah there is a the free one is okay you can use the free we have a license here at idap but the the if you want to pay if you want to pay you pay for options such as resize or cropping but in fact maybe we do not need that and it is called elix producer okay the elix yeah i forgot elix producer and that is it you can but anyway you need to have windows to make this yes so i will ask you yeah okay i understand and yeah with real media this is made by a program done outside ejap well this is you can buy or you can all for free on internet it is not a ejap program no no no no no for the real media and you did not see but the real media the real media is called via a real server so it is rtsp the connection and it is streaming video streaming so it is not http so that it can play smoothly but we are still investigating for other video formats because it is not supported for instance in java in java applications etc 0 yeah you mean the java framework what might presented you cannot play real media so there are many issues about that but yeah it is another problem formats formats okay so i think this was good to see the tool thank you tomorrow we are going to receive most of the things but shorter i believe right i mean that to explain and i think maybe later we can start doing this i think for tests i can take the real demo and make the full processing yeah it is interesting to see tomorrow of course maybe they would say ah 170 t shirts it is a lot but after i mean at this point we do not we have nothing we can do unless they know what they want to do with it with it right yeah there are other things such as video structuring but it is other level yeah it is quite research i think is it is going very be very good for them to visualize how you can organize things and maybe what they can do how they and so this is important last thing this is the kind of xml that is outputed so so it is the structure is such as video segmentation with a video scene begin and then for each shot you have an id well this is the terms from mpec 7 convention but anyway shot begin and end the shot keyframe that is the middle here in this the begin and end the number is the for example the end is 0512 frames this frame 2 here and in fact that is what i output on the interface and there is only one sub shot because yeah there was problems with detecting the subshots etc but i because you tried to do it yeah i tried well not with this one but i tried to code some stuff but i am not so confident in the way it is okay good yeah like for instance if we need to find sub shots usually it for instance is better to have the whole shot to decide on how to build the sub shots but it is true that maybe if you store all the distances we could do some basic stuff but we yeah even we should for instance for the videos for the spectral clustering for the clustering here i store the consecutive distance between consecutive histograms but we could store the matrix yeah yeah yeah and then yeah and i think there are many possible things with spectral clustering just what there is one parameter that is very i can not remember what is the parameter the eigengap you mean no 0 no you mean in the battery distance no no no no the eigen gap to find the good number of clusters is so sensitive yeah anyway it is a bit later in the report so anytime i would be happy to help you with that okay it is okay it is okay so we can do it so we can do it yeah we can do it this is not your laughter this we need it in fact we do not name that okay everything is stored here i suppose i suppose