which data you know i i want everything and then i will decide what i want yeah so this was that face to face meeting okay but it it does not there is not th so there are things in the eye tracker record that we definitely do not want in the g d f format like the frame rate eye movement you know we we i we are if we are expecting the g d f format to be sort of a parsed version of events and you know there is just too much raw frame f frame rate stuff coming out of this you would not prob you would not want it in this kind of format for up translation to the other things we can get back to it we can get we can always get back to it yeah we yeah you can get back to it we are not th actually throwing anything away yeah we are not throwing anything away and if we decide to re jig this at some finer this is just the stuff yeah mecha method we will still have the original and a method of of filtering yeah but the th but the way to think of this is this is the data that we want to be able to analyze against the other tracks of data so so the data that we want to compare with a language or with whatever and so okay as long as the format in which we pick it up is a format which could be generalized to a finer take because the worst thing that can ever happen to you is you discover that you have done huge amounts of analysis and th you actually had the data but you threw it away because you treated it so grossly yeah okay so whatever we should be able to b you know backtrack and say okay instead of every 2nd every 10th of a 2nd or some such thing s so in essence what you would then do is but you know th well we are not doing frame rate at any frame for well maybe we are i do not know well the eye tracker does i i had thought we were yeah the eye tracker does yeah 500 hertz though but yeah but right w what w the kinds of events that we had r before talked about putting into the record for then use in elan and n x t is not based on saying every x seconds something you know give me give me what is happening every x seconds well you need it is more like you know give me the fixations and the the blinks well one needs to know the duration of these things right yeah 0 yeah so one needs to be able to say durations are in there but it is not there is a difference between saying that something is in a particular state every frame whatever the frame rate is you know 10 seconds a minute you know that is that is one kind of w way of looking at data and a parsed version of the data which is not at any particular length it relies on this really close frame rate that is underneath well so there is no real time there is only ev there is only events what i am we are trying to figure out is whether we have thrown time away no no the time the time is still in there but you will get what you want is to figure out what are the the concepts behind that data that you want represented so rather than saying the eye was at this particular place you know here and then a 10th of a 2nd later here and then a 10th of a 2nd later here here you say there was a fixation from this time to that time and there was blink from this time to that time okay and so one of the formats in which eye tracker data is analyzed is percentage of time spent on some target as opposed to some competitor over the 1st 2nd after some event okay so you actually have to be shu have to show at some time slice rate whether the eye was on the same target as the other guy okay a similar target what we would both been deali you know so we have to b we have all the parts the interesting parts of the screen identified and be able to show distribution of gaze over time alright now we could be really unlucky and somebody would expect us to do that at the real frame rate yeah but i think that is really unlucky but but my point is that we must not throw away in or lose the capacity of being able to deliver that kind of data right so there is 2 issues here we are not throwing anything away but the question is what tool would you use to get that information out of the data would you be so the more you can plan s so typically the eye will move right so over any 12nd they eye is actually f fixing on a bunch of different things and you and so when you say the eye was on this from here to here what is going to happen is that is that if you use the real frame rate it is going to jiggle all over the place it is going to be on this landmark yeah it is on this object that object this fixed thing that fixed thing okay you are going to get lots and lots of stuff and it is percentage distribution that you are going to want it is not i went here i s you know it is not like person walking i went here i stayed here it is it is more like a fly hovering so you need to know the percentage of time it was on during that fixation yeah because if you if you take them as separate events you get 1000s of separate events m m i i yeah but the p sort of start and end times will give you back to that output a as a and then you just measure it against you know the c the c yeah the cumulative total time record or something against bigger frames okay so the percentage is i i is a cut it is not something you are using for measurement so it is a cut off for whether or not it counts as a fixation well that is the that is the problem is that there are 2 ways of doing it one is that there is always a cut off right but the other is that there will be a time span a kind of reaction to some event span b m yeah so i will i will lag in some of it during the i suppose there is the the saccadic movement itself for example right right and so i mean there are there are really 2 ways of looking at this one would like to know for example the percentage of time overall in which 2 people are looking at the same thing okay and one would like to know sequences of they did not look at the same thing and then they broke it or they did look at the same thing and then they did or did not brea you know 11 could imagine those 2 categories that is that is a kind of testing of our hypotheses but there is a certain amount of dues you pay to the way they do things in the literature to get your papers published and one of the things they will want to know is an event series after some critical event right what percentage of time is given to looking where the other guy is looking and what percentage of time over some you know reasonable time span of a couple of seconds alright so they will be they they will want to see essentially the gaze settling on particular incidents on particular places so we have to be able to deliver those 2 things and they are rather different demands th th th yeah right the it should still come out that the little sketch i asked this p for for last week about n x t and whether it would have that sort of tiered effect so you could see the overlap of where one person is looking compared to the other and their mouse movements and things yeah that that should ra address all that cause all you want to know is whether they were looking at the same object in the period after but but because it is also in the i based on the the time course of the procedure you will be able to basically get the scan path of the pattern of events from that as well yeah okay can we draw it to make sure that that that this is satisfying ellen is concerns yeah yeah so i did have it somewhere i did not have it r have not brought it right well there is a white board right trying to remember it we love this thing see if the pens work so this is a is eyes right and b is eyes and i think what you are saying you should probably draw this right is that they you know if from this time to this time they are looking at triangle one w what you are saying is that you want to know in the critical you want to know after they are they are looking at triangle one what is happening in this period with this guy right or how l alright let us let us imagine a typical construction event okay there is some part there are there are separate movable parts on the screen ready for use right there may be or not a construction already begun right somebody makes the 1st move there is some kind of communication either gestural imaginary or verbal that plans how we are going to do it right before that happens the person who speaks is going to do some kind of visual scanning while they are speaking or communicating in some way the other person may or may not be looking where they are looking okay they may be overlapping gaze at particular objects which are of interest let us call it the c the construct existing construct which could be 0 alright or the the addendum the thing which is next going to be added next piece okay now in neither case when you draw this he is looking at triangle one which you are actually drawing you know for a the period of time in which he is steadily looking at triangle one is going to be very very short right because staring fixedly without interruption blink or or saccade is is an extremely short event the it m it m but br at that point they are all of it lumped all the individual fixations as long as it does not move off that triangle together as a total sort of gaze gaze time on that that is fine so it is in a region it is in a region yeah so yep yeah that that is a target within the target region okay but even so you are likely to get bouncing in and out of the region right m so even so you want to look at if we are interested in how long before the construction move takes place how much of the time they spend looking at the same thing let us call that our measure of alignment so they are going to be yeah that is right a is going to be on triangle one and various other places alright huh that is right and and b okay m yes somewhere and we are going to look at the percentage of of that time where they are both in the same region right so what i want to make sure is that we do not simplify do you know what i mean so temporarily simplify too much so that yeah it it sh yeah it i mean as long as we are dif r yeah it there should not be anything missing cause then you lose percentage as an as a as a d v right and so b has some periods of looking at one and all it will which will also be intermittent that should be alright yeah because yeah yep so you know he might get triangle one there and no maybe this one is going to overlook lap quite a bit m m yeah yeah yeah yeah right so what you want to do is be able to define bigger periods which is the period when they are sort of interested in triangle one overall right yes this is sort of some meta level analysis when they should be so we yeah so we if we have an an event yeah that that yeah the 2nd line then for the 2nd before it right and then we can just take that chunk out and do something with it yeah but that is that is not a something that you can do in the n x t query language but it you can not do that in any you know that should i i okay this is such special purpose s but it is easy enough given the data format for any of these things to do that so you could do it on cause i it is just a matter of i the hard part is deciding how close together these have to be before you decide that this is a event that you want to pull out right because you know algorithmically you are you are already putting together i mean this is essentially already you know these fixations with stuff in between yeah yeah yeah and you know we we have got an algorithm for deciding when that is a look i guess you already have that or no well if it is just based on like a a stable fixation for so long or or something and crossing into that region well it 0 yeah yeah yeah so it is all fixations they are these are all fixations and saccades but within the yep yep right so that is easy enough what so the hard thing is they could be moving their eye arbitrarily here right and you know maybe even having fixations on other things yes yeah yes because for example for part of that time both b and a are probably looking at the construct okay yep if if your intention is to move the red triangle to sit on top of the thing already constructed you would tend to look back and forth yep back and forward yep right so this is not yeah and we should be able to to figure that out but what you need to do is we we can already build into this g d f format these regions if you can give a definition of what you think that is but i think the right way to do this is to be able to inspect the data in some tool and play around with the definitions because you will not get it right the 1st time yeah yeah sure so so the question would be under what tool should we be looking at these events yeah and elan is a reasonable choice for that right so what you need is something well no because you need to to y you need to be able to see the effects of this do not know so i g i guess this is a case of craig writing some scripts so robin having some ideas about what the relationship is and saying you know add this data a automatically it i mean it is not that hard right it is just there will be a bunch of these yeah you will have some rules about how maybe how long you spend on other objects and w whether the other object is the existing construction or not right right so so essentially there is there is a e is divided into t r one and c and other stuff okay where c is the construct the existing thing and so ah right so if they if they they had a fixation on c you would not be worried but if they had a fixation on well no i mean we would take a separate we would look at the percentage time i mean i have no idea because i do not know how people look when they are building things together okay so there is a there is the addendum and the and the co the existing construct and i do not know whether they are going to spend more time looking at one or the other but if they are whatever it is that one is looking at if the other is looking the same place they are in good shape yeah they are yep but yeah they are going to look at the clock and they are going to look at they may look at completely irrelevant yeah sure sure and that is what the other stuff that is what the that is what the diagonal stripes are for that is true but we c we can one of the things we will be doing is categorizing people i take it or interactions by the amount of time people spend looking at the clock we would expect that if we put people under time pressure they will look at the clock a lot more yeah more but that is like a separate analysis right sure it is a separate analysis so but you do not want to throw it away right it no no so so you can define if all of these things are actually categorized by the eye tracker as to where the eye is alright the so every separate yeah you should be able to pull out any interesting category and say alright for this phase it is t r one or c for this phase it is s q one or c okay yeah but what we find e what we can get out of the data easily at the moment is at this kind of level right and then it is a case of defining algorithmically all these other transformations that you want so this is one we had not thought of adding before but we should right which is well how would you decide whether something was the current addendum the cu you have to actually transcribe the thing or watch the film further on so that is a human decision yeah yeah absolutely yeah yeah yeah so so m so i assume that well once you have that sort of information you well you got the g d f format but the ex yeah you then filter it to say right between time x and time y they are they are constructing that having triangle one to the construct and then you just say in that time period what percentage are they looking at triangle one what percentage are they looking at the construct right does that sound does that sound reasonable right yeah that is the addendum in the construct should be you know kind of they they change their true identity yep but they are i their categories of stuff now i w my kind of assumption was that that this was j p land yeah that that is what he was re what he was really interested in was the kind of coding which would define the building sequence okay well he is really interested in right so and the and maybe he wants to think about that more and say well there were several candidates for the addendum alright so which one does and and it was the negotiation of which candidate was going to be the right candidate that is actually the interesting stuff so i should talk to him about this he is he is actually emailed me with a l with a list of things he wants to discuss at length but we believe so i w i will bring this one back to him but we believe j p to be defining that maybe well this is my question for him is he going to define this kind of building sequence in a way that we can get out or is he so b i mean you know they have been working on construction there for a while so it may be that they have a coding system that is ready to go and we should just apply it yeah okay they do a lot of sub assembly in the tasks they have been doing so they may well have a yeah i could believe them yeah no taking an interest in that let us find out i mean the general division was language here and other stuff there right right right that is what i have understood it to be so wha right but well i do not know i will just i will just minute that what well we but we have the now actions i mean actions might mostly be theirs right but w but since we have the the eye tracker i th we are going to have to answer their questions about gaze right 0 yeah eye tracker here yes huh yeah right yeah i mean we know that yeah i right but so they will they will they will tell us something about that but presumably what you piously hope is happens here is what happened with transactions right that the verbal analysis and the and the visual analysis give you the same breaking points the same chunking points well yeah but i mean with transaction coding on something like this you would use i y you would not consider it verbal analysis exactly it would be verbal plus action because it is a task breakdown well i mean what you know it is just a segmentation it is it is it is a task breakdown but for the verbal version there are ways of announcing that now we are doing a new one right yeah but you would not have you you would not do 2 codings for task level one you would not do transaction coding in a action segmentation or that we are finished with the old one no you would not but i mean one of the things one of the things you would like to establish you know as an as an outcome of this is that you could analyze either end and you would get the same chunking of the material the thing which you guys thought to make in a paper i i do not think that is realistic because i think what you would actually do is use the full record of what you have and do a segmentation i think it would be hard to understand what was going on if you used like just the language without watching the video to decide whether we are breaking down the task it is a hypothesis 0 you do not want it to be coding th based on that though and then that seems a strange way to go about things well i mean this the language events are multi modal right you know they are they are doing all these things together they are doing these things together but one of the things that you one could imagine doing is senator is you know having the transcription there and pl just play it back to a bunch of captive undergraduates and say when do they stop and start wha why do you care whether they can do it just based on the the language because because essentially you are looking at cycling sequences in discourse is and if the discourse tells you what is going on that is information i mean i g one of the questions is how the information is going to be shared across these media and if you can get it all out of the speech you know if the if all of the chunking is available when there is speech okay then it is carrying a lot of the burden well it seems to me much less important than the other things one could get out of this data it is saying we are done with that and but i you know i it is a possibility but this is but we have not looked at data like this with this with our old fashioned analyzes you know it is we will and i it is it would be really nice to know that that much information that chunking of the task information is being carried by the language because j p is question the overriding question is so what is language for you know if people are busy interacting all the time and all of our colleagues do not even bother to control for whether people are talking to one another when they are doing these joint tasks because it seems to them to be irrelevant cause language is irrelevant it would actually be nice to demonstrate that in a place where we are controlling whether you have language or not you could get the entire chunking of the task out of a language yeah well i c i can see maybe wanting to know whether you can do the chunking just based on the actions without the language in cases where they use both it is more g i can not quite see why it is important to know that they can do it just based on the language when w you know that they had both because you know it is a self contained system as opposed to a system which is which can not be interpreted without the other system huh well i think that one is just bound to fail okay usually the claim almost all studies on language claim that language is a self contained system which will give you everything you need to know but yeah really that is the claim okay alright anyway i did not say it was true i said that is the claim and so it is worth testing so back back to this main problem which is yeah w the record that we are getting in the 1st instance is about the m about these you know they are looking at some region of the screen defined dynamically and then we need some way of knowing what the you can you can see adding these other analyzes about you know they are they are jointly focused on this region and trying to figure out what s percentage of time they are looking at it as adding new tiers of information to either the elan track or n x t they are they are both sort of track based in this way but the hard part is knowing how you want to do that and i i think in the 1st instance what we are trying to get into the g d f format is just this and then we have got no option but to figure out a way for like to look at this just explore this data suggest ways about doing it and be able to play them back and see when we think we have got these things right i do not think we even know what the set of these things are that we want much less how to get em yet no no and we will not and establishing the full set will you know take us much of the project so th so you know yeah that is right so this is not the you know this is something we are aiming for but this is not something that affects what craig programs for the initial g d f translation you can we can announce now fine my only concern w you a as i said when i barged in was just to make sure that we did not lose the things that we might need to pick up later yeah that is all so let us just return to this question of frame rate because this is the thing we were not planning to transfer into this format so again you know what a frame rate kind kind of coding is ag it can again be seen as a track but it says it is a coding like this right where you say it is in state a state b state c state a every single frame well okay but since the l the definitions of look a thing we are looking at are d are the ones which get rid of the lower level like jiggling around in the area of a particular object right so if we are moving the the green triangle we have defined a region which is the dynamically the green triangle wherever it is we will jiggle around in there but it does not matter where we are in there that is beneath our level of analysis right so you right so i am just you are happy that it is not going to have frame rate like this s well it is going to have an interpretation like this well the in that interpretation is definable to frame rate by our by the only way we are going to use frame rate what does that mean sorry i c well i mean i because i can i can say for how many frames this fixation went is officially defined so that as long as i as long as it is not a untimed event okay yeah all all the all events have a start time and an end time a and an end time right so so it is you know it is fine i can say it is a long one or it is a short one w right or it is you know it is 20 milliseconds more than that one yeah but it seems to me it seems to me that ellen is concerns might mean we need to add more information to these tags because that is you know this thing in itself is th bunch of fixations right with saccades but y y y you mean to the tags of the data w within it yeah so so things like fixation the number of fixations a and fixations and the percentage of time those fixations cover you might want bi yeah with some some measure of the so we could work out the number of fixations that made and the average fixation duration as well because having saint long steady fixations can be informative okay and okay i am i am asking you as an expert so number of fixations percentage of this is this this is not a leading question this is a question question why would that be informative particularly if if they are holding their if there is less dancing around the the cognitive focus tends to be in one sort of part huh so there is a difference between if they even though they are still looking at the same part if they are looking around it rather than just holding their gaze steady or and longer on it huh on it so you think that if for example if they are looking at one or another apex of a triangle it would just that would make a difference yeah they were exploring the thing as opposed to they were simply alright there is it there it is it yeah it is it is certainly it is it is worth okay so th so that really means that we are not throwing away absolute fixations no i would not want to throw out the screen location of absolute fixations put the actual location no no this is this is still summary data they are he is we are talking about just adding attributes to these things that say the number of fixations that counted as that looking at triangle one right alright huh so y it is possible as opposed to out of it i mean presumably so you so i thought i understood you to mean exploring the figure because you know your fixation is a point and the figures are bigger than points so you could be exploring the figure or you could be just somewhere in the region defined as the figure but not on it if well yes but that probably is not going to be easy to get out because you would then have to break up the parts somehow okay alright fine the so you just want to know the variability of the of the fixation yep and would not that differ from person to person it it yes it can do but it is also looking at it within their their own behavior to see yeah okay so it is the number of different spots within that region where the eye has fixed the number of different fix 0 ok okay th w i i would say just the number of fixations i mean it could just be yeah the number of different yeah yeah the spots will be different anyway so it is a jiggle rate okay cause i you are not going to g have the same pixel yeah yep exactly right so so we have the option that we can put in an a fixation thing here right as well if you want the smaller if you want the smaller coding in in here i mean to so that it is not just parsed into this idea of which object but you also have the raw fixation data we can put that in as another track if you think that that is something that you might want to look at in one of these tools that shows you the tracks against each other well all i th understood you to ask for was the duration of each fixation inside yeah i d that is all he asked for it yeah yeah yeah if if we have got everything then i suppose it is you can always yes it is you are not throwing anything away then well i think you can come back yeah and in n x t there is no cost right because the y the you just do not choose to load those or right all we are doing is dumping it as output that you can load if you choose to i g s you know so if they are things that you just know are crazy and you are not going to want then we do not do it okay but otherwise we go ahead and dump em right cause it is easy to dump out the fixations yeah yeah so it is the number of different fixations okay so it is the number of fixations the did you want the you well actually do you want the average duration or do you want the number of different ones but number well the duration time as well somehow whether it is the average one or duration of each fixation in the region of the fixation m not e well if it is each you have to d you have to break it down into another tier right well yeah you can not you can say there were 3 fixations and they averaged a certain time the the average should be alright to work with but you do not want to d well what about the the overall sum of the durations the the is that i mean which way do you want to well you have already got that well if you have got the number of fixations in the average time you can then just generate it yeah then that is that is the way you want it that way around yeah yeah okay so that is a measure of jiggle in the region roughly and y did you want the percentage of time fixated as opposed to which is sort of another view of this number moving you can have as many of these as again that that should be you should be able to derive that presumably from yeah well it is not the the extra cost of having these as attributes is not high so you know it saves you i having to do write scripts to do arithmetic on that later if you know that these are numbers that are going to be useful to you okay right what is our error of measurement on location on the screen so to what degree do we actually know whether the 2 little eyes little circles are exactly in the same place or just somewhere in the same region that it can vary a bit depending on the calibration but if we are building in a sort of error margin it it will be based on pixels around right okay right but there will also be a minimum one cause i am trying to think that it is probably the small the actual mouse icon is so small it will need a slightly larger just because if what you have i is a sort of larger error or of b error margin around it it is yeah okay right s okay so my question was going to be alright the the mouse is a dynamic object right yeah the eye track is a dynamic object or is it only the the piece that the eye tra the other person is eye track is on that is that is an object in this definition no the 0 the other person is gaze position as well is a is an object yep okay trea treated the same as as the mouse say with some with plus or minus something or other yeah around it yep some circular area yes yep okay and so you can be on the object but not on the other person is gaze so there can be a triangle which has a is gaze in the center and your gaze actually happens to be here yes so if there if yes you are in the triangle region does this but not looking at their individual and pr fact they probably will avoid looking directly okay because if they are looking directly at it it is obscuring the the part but it is directly added when when these 2 in this situation when these 2 overlap right like this yeah okay yeah cause i think that that is going to co that is cause b the diagram we got there suggesting that they are only looking thing at time so whether it is triangle or whatever else but so what you are suggesting there is they can be looking at the triangle and maybe a mouse pointer and maybe an eye at the same time yeah the okay okay well it would come up it would say yes right i mean because they would be in the same place i do not know so if you y you are going to have dynamic regions which overlap one another by definition if the mouse is on the construct then right right the m the mouse the gaze and the part are all layered yeah yep yeah yeah well well but y those are different tracks of information cause you get c cause you so what you want is one track of information about which object it is 11 about whether or not it is on the other guy is gaze right and one about whether it is on one of the mouse positions we treat those as as other objects but the difference between them and the the parts of the c thing to be constructed is that they are allowed to overlap with other things so nothing breaks okay yeah but i in in the analysis you should treat those as completely separate tracks of information if we because they can be looking at they could look sim simultaneously at somebody is mouse somebody is gaze and some object right ye yes as opposed to simply looking at somebody is gaze when the mouse is not also there right so yeah and you want to know those things yep so the these are independent right they are i they are independent dynamic objects so so what we want is whether they are looking at s at the other person is gaze right and say they are in this period well i mean they are probably not given what i have trying to okay so a definition of we are all looking at and touching the same thing so imagine that we have just added a piece to the construct alright so now that is the construct and both mouse symbols are on it right yeah and both gazes are on it right yep so at that point you should have a line up of a is on the construct a is on the mouse a is on a is on a is mouse a is on b is mouse a is on b is gaze and b likewise yeah okay so g so there is there is a lot of tracks here right right because well we just added this fixation track ooh i love the eraser and then there is a is mouse not their eyes is on an object right and the mouse might be on triangle one yeah but the mouse is also a dynamic object but it does not matter i mean i the mouse we are just saying where is the mouse that is fine okay you know it is on this object and then a is eye is on well a is mouse i guess right a is eye might be looking at a is mouse here a is eye might be looking at b is mouse here right i mean these are all the different ways of taking cross products at the things that could be co located right so you just treat em as independent and i am guessing from what you say yeah and then you then you look for combinations of them yeah i guessing from what you are saying you want all these tracks i think we need to because tha some measure of alignment is looking where the other guy is directing attention yeah which is fine there is no problem with that at all and there are 2 measures of where the other guy is directing attention that you get from the visual track one is where the mouse is and the other is where the gaze is yeah so s so you kinda got the idea here yep yep okay yeah yeah so so yeah so so it is just a matter of adding these the 2 mouse pointers and the 2 gazes as extra objects so yeah as long as as long as they have got an i d in there we can just say right at this p between these times they are looking at that object or they are looking at the mouse right right they are looking at whatever but the important thing is you do not treat that entire set as mutually exclusive and exhaustive so it is just so tha the objects is like a separate level of analysis yeah that actually true be cause if whether they are looking at the objects than whether they are looking at the gaze right cause they can happen at the same time but that it is also going to be the case that whatever region you define as the region of the dynamic object of the red triangle may get to the point where it overlaps the region of the dynamic object of the green square yep okay what do we do about that does that happen yeah because if you have them close together there is always some fuzz factor around them if when they get moved together 0 right and so if they are left lying close together okay or in the in fact in the model in the not in the model it huh in the in the supplies set they are actually neatly packed into a little space okay so what you are saying is like a new problem for us which is i am going to change 2 objects close together your eye gaze can actually be looking at the left edge of one and the right edge of the other that is the bad one and they are sor therefore looking at 2 objects at once yeah yeah so you are saying squeak squeak saying they need some new pens okay yes can you write on my pen pad new pens just say yeah pens pens white board pens white board pens so you are saying you know they could be looking at square one at this point i confuse em cause they because the 2 are so close together that their regions overlap close together okay that is a problem for the data models and either of the things that we were using yeah unle but it unless we just define it as there is only they are they are exclusive that would and which ever area m has the greatest overlap is what they are actually looking at what do you mean by greatest overlap the greatest overlap with what well it w if if you have got your sort of eye position ho ho and it is it is unlikely to be exactly 50% in one object and 50% in the other draw my g my spacial reasoning is no good yeah well i put these things on i think i think you are throwing information away see i i am the one that uses the white board cause i actually put the the microphones on yeah what i am i am thinking is if you have got say one object there and right you know so this is this is the better pen thank you part one and part 2 and you have got they are sort of they are looking there huh then that overlap means they are looking at the 2 pieces simultaneously yeah but but you had a way of choosing which one right yeah yeah just if this is there is more of it yeah i yes but it may be a bad idea it may be a bad idea you have just thrown data away yeah that is w that is that is a decision we will i am saying is if they are looking if most of what their overlap is here then we stick with that one yeah well thing is that with j so you just mean which i is it closer to the is the circle i mean what is this circle well they are looking at a pixel or something right that is th we no that because the gaze position will be a sort of fuzzy area it will not just be a pixel it will it will be an actual like circle and area the gaze will wobble right yeah the gaze will wobble so i i it it that is that is going to be w it is it is not just a single point that is natural s right 0 so those vary in size it is it is a it is a sort of lump target yeah yeah and so there will be a bit of a a sort some sort of overlap of 2 objects yeah or indeed they might be looking from one to another or b yeah they could just be oscillating yeah d or deciding which one to choose or thinking so you do not want to decide it is only one and throw the other one out they could be yeah thing thing is w g d s formats is wor is working at the moment it is actually got a separate track for each part okay well that is fine so that i was going to say that is the obvious way to do it 0 but what pain that is going to be analytically yeah because it is going to how many parts are there well each one is yes or no well the like well the the the it depen depends on the depends on what you build in couple of dozen there is different parts there is different parts whether you are doing the tamgram or whether you are doing an airpa airplane or whatever right 6 to a dozen usually okay but th but sa but thing is if if we if we got if we got tracks for each for each part we can we can easily analyze it down later on i know ways of getting around this analytically cause you c yeah i mean i think it is very likely that if people are considering f i mean if if they are oscillating though then that is presumably going to saint change the weight so you are actually going to see them flipping between when you decide they are looking there if they are really close yeah well not if the regions overlap okay if they are really close to each other it is going to get foxed if the regions i yeah right yeah and if you define the regions so neatly that there is that you know that we are losing gaze because of jiggle outside this closely defined region when it is in the middle of space then h yes i mean i i actually like the solution however ugly it it looks in terms of a data format of having a track for every possible object of gaze yeah well the it is not the data format i am worried about it is the the way you do the analysis because you do not want to have to say you know did they jointly look at triangle one okay did they jointly look at square one okay did they jointly look at you know you need some way of going over the whole thing s a thing but i c we can find ways around that well okay i think for now we we do it this way and then we we think about what we need out of it in the end so it is just you know in elan it that has the side effect that if you go for the naturalistic way of of up translating to elan there will be a zillion tracks and it will probably ruin their viewer cause you will get very sparse data on each track huh it will be like the old referring expression generalis visualization the map task you know where they talk about one object one landmark and then another landmark i always thought that was brilliant well it is big right yeah it is big and you do have to scroll through it to see what is going on but it gives you a very clear picture of what is going on well we will see if elan likes it or not yeah well okay but so we would we would better do a test one and and listen to them scream the yeah yeah and we b we better also like have in the representational list of par like even in the n x t up translation a list of parts with parts it it affects the way we do the n x t u up translation and make things easier for people yeah right so i mean so yeah well the the the the does not really list part in the in the g d f so yeah that that that is not a problem there so that bas basically what i have got is well there there there is places where you got the i d though so you have the an i d for a part you got an i d for a location on the screen so it shows you the target config the clock whatever else and then we can easy add i d s for the 2 mouse points and the 2 gazes and from that and ba basically all all the all the tracks are doing is saying at this between this time there is a look at this 0 this object yeah th this i d so so it is effectively a track for each i d okay so what about the point where a part becomes the construct the th yeah the but but do you still identify it as a part i mean i think one should lose that i take it well wha wha what happen what happens there is that the the 2 the the existing parts basically cease to exist and a new part with a new i d starts existing at the point where they are joined cause right so you cha so every time a part is added to the construct it becomes a new part so the construct actually has say it has 6 things that are added to some initial thing okay so if there is just the initial thing you have put it in the middle of the screen that is just the initial object right yep soon as you put a part there is a construct but it is construct one because when you add another part construct one ceases to exist and you get construct 2 right yeah so actually defining all of those as the construct is going to be the trickiest thing i it is also going to be a problem because you are going to have one part or a construct forming in the middle of a fixation well the right so it is not going to be quite as clear if you can not can not chop it off right so you are you are looking at triangle one right and all of a sudden it ceases to exist huh and then it yes and it suddenly becomes and there could be more than one construct at a time right and it becomes construct or assembly whatever 2 yeah 0 it is there is no the construct right cause you can build sub assemblies no and then you can start again you can build sub sub assemblies and then link the the part really yeah also when you when you screw it up you g start again yeah what why does it why does construct one become construct 2 when you add something to it because the definition yeah that that that that is what that is what that is the way he is working the moment because it th it is it is generating a new i d when you c when you create a new part i so each yeah 0 and this is because we do not want it to stay triangle one when you start with a construct it is not a construct at all it is just a one thing and we do not want triangle one to suddenly have bigger stuff well the problem is that all of these things are popping out of existence yeah so do we know the difference between we broke it and it went away and it is now part of the construct yeah but there there is there is joint events so if the if a joint event if if if it is there is joint event linked to linked to the item then you know it is become part of a construct of can we ta can we tag the constructs with what is in them yeah the the the yeah yeah yeah there are wai w i th the con the con the constructs are already tagged with that yeah there are so that i am act there is actually a s a separate file of the data format which says for each construct which which 2 parts made it made that came together to make that construct which constituent parts okay so we could we could essentially well then we could cumulati or it is a new construct cause it has a new list of a new cumulative list of parts right yeah but you can still ta you can still track what parts went into it but we will right so so you can still find triangle one whatever so some fancy programming is going to have to be done to say i was looking at triangle one and now i am looking at triangle one in construct one cause the 1st construct was triangle one and square one okay yeah yeah but but that is easy n it is easy to do because the the co each construct has the i d s for the pa the things that construct it alright so now we have to ask whether so l let us imagine this we have made a construct of 2 parts and we are c i a triangle and a square and we are considering now adding something on to the side of the square that is not attached to the triangle okay are we looking at the square or we looking at the construct yeah construct that was the next question i had yeah yeah you will be looking at the construct cause that is the way it is dis defined in this definit should be looking at the construct yeah there is not a way to figure out that you are looking at the square rather than the triangle yeah well we will d yeah well with the yeah d well thing is that we we still have the x wa the x wave traces from the ascii formats looking at areas within it so if if you want if you want to be able to say i am looking at a square instead of i am looking at the construct that is it is possible to get that data out it is it is yeah it is 0 yeah it is it is not doing it is not doing it in the gest analyzer at the moment but i can easily add it to give that information if you want well okay because i mean that is that is the example m f presumably that is the best example of overlapping overlapping areas cause if if the thing the 2 s the 2 things are now abutted of course there is an area which is common to both of them okay that will cover most of em so what i am trying to run through in my head is that we can always tell the difference between something going out of existence cause it is joined to construct something going out of existence cause we threw it away right suppose we pick up an we we we screwed up the 1st time we put the triangle with the square and we threw it away and decided we did not like it and we took another one is it triangle one is it triangle one prime or something like that it is the replacement for triangle one i yeah yeah if if you take it if you t if you crea if you create a new part it it gets a new i d it is triangle 2 so if you get if you drag a new part out of the parts bin well how do i know what it replaces you do not because that is you would have to have human coding to do that yeah there might be multiple triangles that you have broke and you will not know which one they are planning on using it for yeah i mean the we e w n n this is true but if i have triangles 123 right and i throw them away any triangles i take out are to replace 123 well you know which mold they come from and so you know which shape they match okay but you can not possibly know which one of the ones they are meant to replace yeah i mean you know whi you know which way they match except that ex no you do not know which one they are meant to replace so you can but you know they are not meant to replace any that are all that are that have not been thrown away they can not replace the new parts only appear when the previous one is broken okay right so the new parts are replacing those which have disappeared from the screen yes and if there is no more t r one track the t r one track is gone now they ca they can appear without them you c 0 so there is no 2 parts with the same shape yeah the 0 there are many with same shape but not the same shape and color then same identity same eye 0 same shape and color yeah right okay yep 0 is that true you have decided not to make all the triangles red and squares green and well that is well that is that is what they are at the moment 0 okay 0 okay so you can tell what it is meant to replace then right because it is the same it is just a case of the mold right 12 and 3 they yes so that is the way it works at the moment i just assume that is how ni nijmegen had set it all up okay th that is that is back to tim is day so so there yep so shape plus color they are individual you have different colors for the yep okay they are unique unique yeah yep all the parts are unique so wai hang on that makes the task a little bit easier right yes the tam tamgrams are much easier when you can see the all the lines yep so it is just a construction task rather than a puzzle indeed which kind of leaves you with an obvious way of making the same task harder and easier does not it yep yeah but the this also but but that is hard to code i mean that is really a thing we might do okay so now let us make sure that the coding would survive that it is just it is a beautifully controlled situation the construction task is essentially the same it is the figuring out stuff that gets harder if everything is purple for a just to choose a color at random here yeah yeah so yeah yeah yeah so i think that anything is loading there is there is then 2 triangle is not there there is 2 identical triangles in the tamgram yes it at the moment there are d there are 2 i think 2 small ones and 2 large triangles they are actually the same physical size and shape at the moment they are in different colors that also means in the p new parts list they are there for you know 2 v large triangles and 2 small one small ones say whereas in in the revised edition if they were all just black there would only be one mold in the new parts yeah we in when you specify these things do you specify the molds or does it figure out what the molds are based on so is this hard wired into tim is program or is it just an artifact of the way we s as far as i know it is hard wired it just appears so well yeah yeah when you when you specify the molds in the in the in the p the file that c that generates the parts like a list of parts the molds are bas just taken out of the the file that generate that specifies the parts right okay so so ho however many part however many mo however many parts there are in the file that is how many parts appear in the in the parts box the polygon one polygon 2 whatever it might be so it is hard wired so if you have 5 black triangles there going to be 5 black triangles in the parts box or one if you have 5 black triangles going to be 55 parts in the fi 5 black black triangles in the parts box huh so minor modification needed for but what what what but what you can do well if in the in the initial configuration you can re use a triangle so if you so if you only want one black triangle you just create one black triangle in the parts box and then re use that pa that triangle 5 times in the configuration okay as long as it this is possible so c that that just is a kind of thing which is so simple yep so much like a single 10 variable change that it if it that does not already exist yep same same pieces but what were the req wha cause i am tr i am trying to kinda map this onto baufix and baufix has has multiple you know all the the has multiples yeah the the v the nuts are are red or and all the flat things are wood color something no so they they can have different nuts there w well nuts that are different colors the same part the same part is different colors but same part in different colors yeah 0 yeah yeah but they do not have enough to make unique no what i mean there is more than they are multiple pieces that look the same so what so when you say a red nut or a green bolt or a long green bolt then there will be lots of them lying around and i think that is how the robot is set up to there are 55 green bolts yeah so i th here is one you know i think we have gotten to the point where you know to try to summarize we we think tim is program can do both of these conditions that you want okay okay but i am still worried about what the effects are for analysis so because you were aiming at something in the g d f format and i was not quite sure what okay and w so w all i am i am doing is kind of worrying out loud about all the things that will happen that we will miss by th a simple view of for example regions and looking at at triangle one yeah when is triangle one not triangle one anymore so so okay does it s keep its identity so i will summarize what we said about that part that every time you cast off a new part from a mold or you know every black triangle has a different i d identity yes and you know it is a black triangle cause you know which mold it came from but you do not know which part it was intended to replace on the screen because you can not mind read yep that is true so how can we define the shapes that people build what do you mean define well if assuming i was for a moment i was j p and i wanted to know how they actually went about building the thing okay and i wanted to to look at the strategy and i wanted to see if the strategy was different when we could talk about it and when we just picked up whatever i mean suppose i can not talk to you and i am doing this task with you and i can not talk to you the thing which is going to be hardest for me is making elaborate plans with you i can reach for the next thing and you can go where i reach but if i have some you know sub goal some long term sub goal of doing something clever with putting these together cause it is hard and then putting these together cause it is hard there it is going to be almost impossible to convey that to you and the difference in the history of construction right is an important thing and i wonder how we can get that information back i think that this is a human coding that is part of the action coding do you think because i do not see how anybody but a person watching this can guess why they cast off this thing at this time no it why is not that is that is not the question i am asking the question i am asking is what is the history of the construct so you have the n the names of the things in there and you have 2 constructs suppose we have 2 sub constructs and whichever one had 2 pieces put together 1st is the earlier numbered of those constructs right yep how do you tell the difference between adding a piece to this 1st construct and creating a 2nd construct well when the when the f when the final construct the the way the the way the the constructs are defined in the file you have got like your p your parent construct and the parts that it is constructed out of okay and yeah as i say the it is it is it is parent child relationship yeah yeah so you got parent child relationship and i the child can be a part or it can be another construct okay so construct 2 could b construct one is a red triangle and a green square construct 2 is 2 green squares okay construct 3 right is a red triangle a green square and another red triangle construct 4 okay yeah but that is okay cause they all have different underlying part numbers and when you say what a construct okay and you can so how are we going to tell when for example you think we are going to have to do human coding on when 2 sub constructs are put together no that is a definition wherein a construct has a c has constructs as children no no no yes yeah yeah yes cause there is a joint action okay the result the t what you need for a join action you need to yeah a construct and a part 2 parts whate whatever you are taking together okay and then the result of that is a construct you have got time when that happens you can see that construct appeared at that time right and you can still track the children of that construct so we could zip we could zip through this and look at all the all the interactions in which people build sub construct contstructs 1st which we expect them to do bu as a wild hypothesis more when they have verbal communication than when they do not okay and we can do that by simply searching for any constructs that have constructs as children yep yeah so we can easily look at things like that automatically okay it is just the why they g why they get new parts that is the problem for us we can not if they suddenly if they decide to get a part of a mold we do not know we can not know why until they do something with it if they do yeah we have not actually made a rule that you can not collect extra parts right and the extra parts do not appear so you can not collect them 0 yeah you said that yeah there there are a couple of them that it is like grayed out they really very faint yeah but you can so you see the mold is there but they do not come filled until no no i mean you have to b you have to break one to get one okay but you can break you you can but you can break one when you break one you do not cast off the mold yourself it happens automatically yes yes 0 okay never mind then i i it appears in the new p like suddenly the th th the new part lights up sort of thing and it is there so they just have to use it yeah huh 0 but they have to drag it up past the mold line yes yeah they have to bring it into play and you can not just bring extras in case you screw up no you have to screw up 1st and then you can bring an extra nope that yeah that that is one think unlike the say the baufix or anything we i assume the the baufix yeah where there is extra parts the there are not any extra parts and at the moment the software does not a allow for that i but it does have replacement yes yeah replacement but no superfluous so you can not have a a standard stock with extra parts that it will never use yeah okay there there is a so you so so you appreciate you appreciate the cost of so this why question does not even arise because cause you know what they are going to do with it screwing up when they decide to move it past the mold line they are p doing something with it probably and you will know what they are going to do with it because yeah yeah well by by the by the time by the time they are doing that they must have already broken something they do it that is right so you will so you will know what they are replacing yeah yeah so right okay so they rejected something or they broke something inadvertantly okay is there a way of telling the difference between intentional and unintentional breakages intentional in the sense that they break deliberately when they bring them into the new parts how would you do that bec yeah but they c you know it is like the trash can idea that they bring a no you p they both put they both put their hands on a construct because they do not like it yeah but there is there would be no way of coding that automatically right how could you possibly do that you have to hope that there is language and one of them says something but that is a human coding right you c yep so that just has to be coded there is no there is there is no way you could tell that just from the the raw movements or or something not generically so w yep because they will if they are going to do it by gesture they will build up a convention right so that is going to particularly be a problem for j p right what else do we need to know remember m all the things we are looking at are f with language with and without language and it is going to be very difficult to tell whether they broke something intentionally without the language which means it is probably not unless they develop a convention suppose they start off with language and then they they say hey look you know let us do it like this yeah okay yeah but th this is properly part of the action coding it i sure so we should well we should make a wish list right as a side effect absolutely i just i just want to make sure that a wish list yes although yes what i am trying to remember is if the people deliberately break things by moving 2 pieces together they they do not both break they do they both they both if if if what if one if one person is trying to get a part to a part that is not that that so the part that is just sitting on the screen that is not being touched they both break so the i intentional thing is when they actually both click on the same object and it breaks for sure you know that is and l or well n is not necessarily intentional no they could just make a mistake or screw up yeah yeah they could just screw up yep yeah they could discuss deciding to break something and and then one guy does it yeah y yeah or i do not think that is very good no we will never get away with that we will get a bad score but that is bringing in the the language thing again yeah yeah let us just throw that away and start again yeah yeah okay which reminds we have fixed the score problem have we well yeah cause yeah c cause it was through the rotational part 0 yes at the the percentage score right yeah yeah so our yes rather than the number of penalties or or something no the the score w is fine was the percentage overlap and that made the score ye that is that is the way it is worked out yeah that is when i left but the problem was to do with the m the p they had a relative and absolute piece and it is all to s to the symmetry of that 0 yeah well do you know which which which which one was the top right yeah but that is all been fixed so as long as it is defined fine that is that is that is back yes yep it was not it was not a bug it it it was a do not tell me it was a feature well no no yes it was something that had to be defined and the dutch lot did not it was them just doctor forgetting to define an attribute they had to alright yeah 0 okay so that was it was actually their their fault well whi which is probably a documentation fault right yeah i mean i am 0 so but it is not clear enough in the documentation i have th i have yeah it is it is in the documentation yeah yeah but it is like 2 thirds of the way down a quite a long page but i it it is quite fundamental and causes quite immediate problems if you do not know about it but the right yeah so not prominently flagged yep exactly okay yep yeah a communication flaw but okay so we can now that is now just lovely and there will not be any crazy scores anymore yeah is there a y yeah well while we are on past things is there anything else you want to tell us about the prior part of this which is the you know what else is he holding you up from or 0 we are well we yeah we should should go back to the beginning the camtasia was tested last week and initially we were a bit worried because in the old version it seemed to interfere with it 0 i have to use a new page huh but when you when craig re compiled it and did his new version it started wor wor it seems to work fine yeah right yeah so it does not slow down the eye tracker it does not do anything nasty so we we p so we pay for it and nothing we can tell no no the the the there is there is very small graphical bugs no but it is it is not it is not big thing it is like couple of white pixels in the course of a in the course of a the course of a task so it is there is distracting nothing important and noth nothing big enough they are going to look at it right no no alright so the last i think i am sorry you know i had as assorted emails from you and i was probably did not catch everything the problem was that there are 2 camtasia records yes there will now have to be 2 camtasia records because e each person actually sees a different screen because of the gaze feedback they do not see their own gaze they do not see their own gaze to the other person they see the 0 only the other guy is gaze right yep yeah so there will need to be 2 separate camtasia videos generated yeah yeah i know 0 do you need 2 copies of camtasia to do that hopefully we will not huh huh you mean officially or yeah but then the que the que the question is there a license for camtasia where you just need one editing c thing and 2 recording things cause yeah the r the recording yeah the recording thi pro program is just quite a it is quite a small program they just general it just generates the video and so i do not know do not know if it is worth having a look at the licensing for that just to see okay i mean yeah i had not thought about the i was assuming we would just get one and one license both of us so you are going to look into this right check out suppose technically we should have 2 because it will be 2 machines well y c look at the license conditions it might not i suspect that it will it will be per machine i would think i c i never pre judge licenses until i have read em okay i i will have a yeah i will have a l a look at the actual camtasia license so we the other thing we are we so well and hold on before we get go any further so this has implications for data storage right because w it means we have got well no this is 3 times right cause we have got 0 no it cause it is now twice twice the size of a we are going to build we are going to use javascript to build the videos on the fly right cause it is fast so the the permanent storage is just the camtasia stuff yeah so is that a problem where are we putting things is there is there such a thing as you know dual track video because you can certainly mix things what ideally one wants one does not really want to l to have these things because we are using them for back up and for coding we do not really want to play them independently 0 we can we can put em in the same thing right we do almost always want to use them in exactly in parallel time aligned you can dump em into one video frame if you do not mind losing resolution but that is maybe a problem right because well the other problem is camtasia has to be manually launched on both machines separately so the synchronization is going to be a problem yeah what are we doing for synchronization otherwise during the task there is a flash and a bleep 0 i mean not during the task before for each task yeah before the task yes okay so that we would have to re align them we could not dump them from their start points yes well maybe you could the thing that you are dumping them onto just starts running with the flash and the bleep you turn on the 2 copies whenever at different times but they are still no is the what what i th what are they i do not really understand the the parts that are operating here camtasia y yeah 2 videos are turning on there is no time stamp that comes from some common source but no they are they are on 2 s the 2 separate machines and the you you start and end them separately no the they they are not talking to each other they are just recording you start it recording and it starts recording what is on one screen right and you start the other recording on the other machine right so and they are they are totally separate okay they do not know about the other one these are ba these are back ups right so the flash get allows you to hand synchronize them later if you need to by stripping extra video off the front right is there a div as long as you make sure you start em before the flash then you are fine yep yep but it will there is a a cost of having to go to back up which is synchronizing them so they will include things like the the eye tracker calibration right which is not a bad thing actually yes because sometimes you want to go back and find out if this was just a particularly duff subject yeah see how how long it took them actually that is that is true how long did it take them to calibrate right yeah or yeah and hand every time we re calibrated the damn thing slipped yeah how many t how many times before the validation worked or something yeah that is true 0 okay that actually helps so you so you should start the camtasia really early then i i mean that ni right well it at the moment the that is what i was assuming on on on doing that it is done 1st yep right okay right no they that actually saves you hours trying to make somebody is data smooth out when it will not because yeah you go back and actually yes they were crap is not that the answer that is a technical term that is a a technical categorization in psychological research yeah yes the other thing we were getting sorted out was the the microphones cause only one of the camtasia videos will actually have the soundtrack and that is because we are just getting we have got one cable and to the 2 mics i suppose we can do it is it would it be worth splitting it well we need sound we we need real sound recording right that is that is 2 track that is stereo yeah so the at at at the moment it is ins well what we have finally just at 120 clock today looks like we have got it sussed is to get one channel one microphone being the left channel and one being the right channel to merge into a stereo single file alright huh huh unfortunately the on one machine on w on what sorry on one of the the display p c s con yeah the camtasia v video 00 it is or m m multi media thing on camtasia that is not where you are putting it i mean u that is our only sound record yeah yes well okay you have got the proper sound record but you are not dumping it through camtasia to record it right it it is you are yep it is camtasia that is recording it what before we were going to use camtasia at all what well what we were going to do what were we going to do with the sound huh yeah so ha wou is there any sound degradation that comes about from putting this through camtasia rather than running it out straight how good is it yeah have to really question no you got you yeah you got exactly the same access to sound quality as you got from anythi anything else on the windows so you can you decide whatever quality you want so got a bit of okay so what you are planning on doing it is bunging it on one of the camtasia tracks and then splicing it off the camtasia track it is an easy bit of yeah and then you are going to bung it on the other camtasia track too if you need it right and then then they can do sound analysis on that but we will just we are going to store all 3 separately then and the sound is going to start at the same time as one of that arbitrary one of the camtasia videos but you will know when the real experiment you will know the relationship between that and the eye tracker timings yeah because the the synchronization the audio and vi the audio visual synchronization comes at a specific point in the eye track file yeah no human intervention required it is time stamped no it is yeah th it is i i sta stamped into it so it is just after just after it prints the line starting experiment yeah i will i am pretty sure it is just yeah yeah yep but ho hold on do we need that i mean it i i thought the way that tim had this set up originally the audio w the time stamps used there would be joint time stamps between the audio and the eye tracker so that you know the audio record 10 seconds in was the same as 10 seconds into the eye tracker record so that we did not have to do any extra hand work you know any chopping the starts of audio signals to get there was nothing okay so we are relying on your matlab script to get us the chopped version of the audio right well the well the matlab script bas just sa just says yeah gi gi gives you the times of the audio beep in the s in the signal right so we can either adjust the eye trai track data or we can actually chop the audio signal to take off the 1st whatever however many seconds so i just 0 okay yeah right so we there is going to be a way to sooner or later to align the eye track camtasia one camtasia 2 with 2 sound channels yeah yeah but but d the lining up the camtasia 2 is going to take we w he is working on a matlab script for the sound but not for so the one that has got the sound on it right that will also tell you where to chop the video to get it to line up the one that does not have the sound on it it will not ah the bleep we will find 0 so it is just got the bleeps 0 yeah 0 okay so w no hand synchronization required at all as long as the scripts work right yeah kay how confident are you about the finding these things pretty confident but you you have you have seen them it it was it was so we just have to w hope we do not have an any subjects with the odd vocalizations right yeah yeah i do not see him with 10 he kilohertz right right they they also tend to have a funny funny shape beeps so yeah yeah okay yeah cause this this is the drop out test yep okay yeah is there an end signal when you decide it is all over does it ping you back and yeah the the the there is a signal at the start and the end of every trial yeah yeah right okay so so when you have when you have pressed we are finished it pings back at you yep yep okay it is just that all kinds of crazy things happen when you are running things and so up saving a percentage of a trial is sometimes a good i you know being able to do that is sometimes a good idea okay 0 yeah yes it is a stereo jack it is a stereo jack i know but so you g you got a solution for that then but it is a mono but it is a mono microphone mono mono recording on microphones line in sp going going through a line in 0 right okay so it is a so it is a lit so it is quieter because it is not as though it is not powered yeah but it is still think yeah we we still pick up a signal can w you are sure there is not a better solution that involves yeah can we do can we do something better than that like having cause you know people might want to use this for speech recognition or something yeah you never know if it is the data is there then yeah i mean it is it is crazy we are in an anechoic room which we specially yeah built an anechoic you know but you know sound room which we specially built for this yeah and you are going to want forced alignment we are d in good circumstances we should not screw it up more than we have to is there anything we can do to improve this yeah 0 no yeah no well no no no no better to collect and sort do you have a powered microphone or something well the the when they are they are the the they have batteries in them well it is the mono but i there is not does not but does not seem to be powered because the because the line in input is so low wait wait w i i thought the sol that a better solution was fixing this problem with the mono microphone socket right sound card so you said the problem was the m it the m m i is expecting a mono microphone input the so the sound the sound the s that the line is stereo the input is mono right is that what you said well the yeah the the yeah the the the the jack that is that it is plugged into is stereo right is ah but when you plug in when it is recording from microphone it records in mono so is that fixable like is there any way yeah that is somewhere in the sound card and yeah a new sound card is fixable i think a new sound card sounds a better solution than monkeying about with trying to filter white noise out yeah als think we we have a well i suppo i suppose it well one th one thing you do is re is record the sou speech signal separately for each for each person but then you would have to pull the 2 pull the 2 sounds together which is not hard yep but hi but it is only got one input right so if so if i i if each machine recorded its own with a recorded the recorded like the the p th the the user on that one its own track its own video you have to align the videos anyway right yeah i i believe it is easy to put 2 monos into a stereo signal yeah 0 they yeah and then the so the yeah so th yeah it is easy yeah easy well i am not the technical person who would do it so if if nothing if nothing else you h if nothing else you can do it in a matlab script but i know who to ask you just write read read file one read file 2 yeah yeah and then you just yeah so is there any reason why we wou do not do that yeah but then the the the matlab the matlab script will tell you how l it will tell you how far apart they are but it is the same as cause you can do you can just measure the drift between the 2 of them cause you got a l you got a list of where the where the beeps are and then you say right this stream is like i do not know nought 25 seconds behind the other one says robin thinking about cutting and splicing yeah yeah yeah okay well it is in the old days we probably would have want a little if they had been in the same room we would have had a little stereo thing in the background as backup as well just to make sure cause it is slightly dicier you know if his scripts do not work then you do not get the overlap yeah i want to see several of them i want to see i want to see one of them look really nice before we start running subjects can we do that it does not have to be a real trial yeah it just has to be 2 guys talking in the 2 bits yes that is right and then i just want to know it works the way you expect it to work this is not a a gesture of mistrust this is just experience that if anything can screw up it will yeah we have both been so badly bitten in the past i i am trying to choose my language carefully because we are being recorded but you will hear more choice language at the point when we have all done a lot of work and and then we discover we can not use the session because of some thing we did not think about yeah right yeah but this this sounds reasonable to me so then i so i think yeah right and then then we combine do do they both have the same sound card problem i would not count on it they have both got the same sound card okay so yeah yeah yeah yeah cause cause cause it is probably worth mentioning this to the to the guys in the garage in toronto right to say yes yeah so the the external version has got plugs that size and that that is stereo microphones huh right but yeah the other one is plug is like the smaller size and that is not stereo so the color the color can do it 0 it is just that the connectors on the the connectors at the back of the computer can not well yeah i think tell the guys i think that is a if the documentation said it w could record in stereo then i think we should talk to them about how they have done this is you are not sure from the documentation what it says it can do to yeah yeah yeah but as as far far as we can and find the the web site of the yeah the the creative labs web site and the review of it and they were not entirely clear but they both suggested that it is there is a mono that there there is a mono microphone yeah yeah yeah it is just the way they wired it so it is getting late and i have got a 20 clock and i have got a nice lunch right okay so i want to eat my lunch today what did you want to get through today more okay i mean i think i think it was useful going through your expectations about this cause that is quite a bit clearer in my head at least and but i think mostly we leave you to go away and re design the g d f or add new bits to the g d f along these lines yeah i am yeah so and basically out the output the g d f is just going to be a whole series of events you know with start and end time and then w yeah right and the event is going to have whatever all whatever the object is that is being looked at and things right okay and then yeah and then it is a matter of we are going to put whatever filters you need after that so you can say right right i just want so you can say from the time where they picked up triangle one to the part time it became part of the assembly find out how it is how m percentages of time they were looking at it or whatever right right okay so when you i mean given that there are these events recorded we can use any of them as the beginning of any of them as a start point for example right there are also motions being recorded of objects yep so yeah right so you got mo yeah yeah yeah yeah there there is move events there is look events there is fixations and blinks now at the moment as well right and yeah and 0 yeah and see looks at looks at objects and looks whatever else so s okay but i think the thing to do is for you to go away and think about it this way with the different tracks for the different objects and the the kinds of things we have added about whether they are looking at the other person is gaze and what have you and and try to change the spec so it it reflects this and maybe 0 maybe by that point there will be some sample data or something and the right thing to do is for us to look at the sp the spec the way you understand it now and some data yep yep and then ellen will have new ideas about what is needed or about these post analyzes you know the and then and we can add em in right yeah yeah yeah i will so i am right but it is going to emerge over time i mean that is clear that that that is great i i would before we break up for lunch i just want to make sure that tha that i know how long it is going to be before we are in run mode okay so the sound thing the sound thing is stands in our way we need to be recording sound online 0 so the spec to the other conversation yeah okay so that is a thing that has to be solved the shape thing is solved yeah that is that is solved now we think yeah test we believe we have to test right so that is that that that should be done this week yep in fact like tomorrow for example the shape all the shape problems are solved generating shapes no problem scoring shape overlap no problem nothing is problem there there are no visual display problems well you do not have your models or do you yeah they had their they had their inspection last week j p staggered away from it and ah yeah yeah well anything you actually need fixed you know yep okay s joe does not live here anymore right so well i but he was not going to be booking any time to us after the 31st of october so that that means that any bugs that need fixed are now craig is responsibility right you do not know have any change on that right okay fair enough i have not i have not had any discussions with him since cause he that is what he said to us originally so i would assume you know no more contact with joe it is not fair to get work out of somebody for free so you know ask craig instead okay well he is ph he is physically here yeah if we want to pay him for some more time we can do that would that be a just better use of our time in get i do not think yeah well w he was not paid in advance he is being paid in arrears so it does not matter right i think it is but m he he he is around and i believe and if if the if i if it is a i think it is better to have all the software i mean it is not big things it is just maintenance at this program at this and right but if if everybody has a lot to do and you know joe could give it a couple of hours which i is what it might take just to fix it then let us do it because starting run time is now getting to be fairly urgent yeah i if he can give it you know like because the problem is you got to have an immediately when you discover a bug in that and we can make craig shift all his other priorities to do this cause we bought him right right right but we can not we can not make joe do this ec alright yeah so it is a kinder way so there is some trade off between availability and speed from start to finish given that this is somebody else is code it yeah yeah so you know but it sounds like you do not think there is anything that actually needs fixed at the moment well yeah yeah because w we only need this to work well enough that you can use it for something and you got work rats right so we are we are b we are building models so right we are we and how many models do you need to completion and how long does that take we have the design somewhere and you got well and you know about complexity you are waiting on marloes okay you can cold call her today this afternoon yeah i think so cause i have to check back with with j p fairly soon okay he wanted to kn the 18 the next 18 month plan great deliverables great progress with the experiment well that is what we are doing now okay so what do we think if you have a fairly hectic week are we looking for subjects next week okay ads go up yeah i do i do but i think ad should probably go up this week we want people for n for next week is that okay is anybody now terrified at the thought that we are live next week there are people coming in here one after another or p 22 after 2 more likely that is her job yeah yeah okay is the is this is this generalized is this generalized anxiety robin come on or do you have a specific thing that okay yeah so aside from the usual angst which we all suffer when we go live so we w we want to do some piloting at the end of the week yep right yeah so we will we will get we will get the get the audio stuff tested you know single microphones yeah get that tested next couple days and i assu i assume that is not something you need me for t in terms of that okay right s right 0 yeah she is using but she is actually relaxed that somewhat okay yeah ostensibly booked time which is not so you can dash in there for half an hour at a time does not sound very useful for you huh okay well okay we have but we have we have a solid booking for all the mornings anyway you guys sort it out right and l except by permission of us alright we own it so we can change that i mean they everybody acknowledged the the other week that we that we had priority right but it would be kind to tell her ahead of time which of your morning sessions you are bumped 0 yeah yeah yeah yeah 0 yeah yeah yeah 0 yeah yeah do not worry there is there is a booking system right so it is public what is booked and what is not i thought that was the problem that your mornings were not sliced off on the booking system okay no alright maybe w should get on there and book mornings as far as the eye can see alright huh cause you know i if you are sa told use the booking system yeah just a well we think we spoke to all the people who who used those facilities the other day but we could be wrong so huh yeah right and what are the instr what are the people upstairs downstairs have i you know as rules about who you hand the keys to well the 2 key holders are here cause you know we like people to know to ground the 0 i thought the the lending for the booking system caroline 0 i see from caroline or the general office the general office see cause you are supposed to have training in i see but no they need a they need an account on this machine to actually to use it they need account on these machines so that they could book 0 okay so that is the safety they could not actually log in and you control those accounts 0 okay that is fair enough okay okay we are also working on a way of making sure that the matin that the machines return to 0 state when they are when you come off them huh yeah that is safer because the it is the people have been leaving var well people have been leaving various bits of who knows what around on them cause then they learn the 1st time right which which we think is what is critical yeah yeah okay i really have to get out of here okay so what else is important before you run nothing right piloting piloting piloting you have we have to build the things you got willing heads we have to try the machi th things we have to check them with marloes and j p that they like the representation the variables right how about you have any time on friday 0 i do not think i brought my diary up but maybe i get a email back why maybe you and i should be the pilot subjects 1st pilot subjects the ones who know which questions we want to ask and then we should get another pair who are naive to the whole thing okay so okay yeah this is generalized floating anxiety no no it is it is it is wisdom but yeah well because actually we can i mean you know if we make comments they feed back in a lo into the development so do you know what time of day you might want cause i should c i should check that okay you do not you do not need me involved in the pilot do you so well unle unless unless it is a bug report yeah you all have to b you will have to be there to when it falls over i will w yeah w i will i will i will b i will be around but yeah i will make sure but ho hopefully it will run smoothly and yeah i can relax yeah so okay so that all sounds f good fun well i can just leave you guys here you know why not
data man you know yeah i want everything and then i will decide what i want so this was that face to face meeting okay but it it does not there is not so there are things in the i tracker record that we definitely do not want in the gdf format like the frame rate eye movement you know we are we are expecting the gdf format to be sort of a parsed version of events and you know there is just too much raw frame rate stuff coming out of this you would not probably you would not want it in this kind of format for translation to the other things yeah we can get back to it we can always get back to it we are not actually throwing anything away we are not throwing anything away this is just the stuff jig this at some finer yeah mech method we will still have the original and a method of yeah but the way to think of this is this is the data that we want to be able to analyze against the other of data so the data that we want to compare with the language or with whatever and so as long as the format in which we pick it up is a format which could be generalized to find or take because the worst thing that can ever happen to you is you discover that you have done huge amounts of analysis and you actually had the data but you threw it away because you treated it so grossly yeah okay so in essence what you would then do is whatever we should be able to backtrack and say okay instead of every 2nd 2nd every 10th of a 2nd or some such thing but well we are not doing frame rate at any frame for well maybe we are i do not know i had thought we were yeah the i tracker does but right what the kinds of events that we had before talked about putting into the record for then use in alana and nxt is not based on saying every x seconds something you know give me what is happening every x seconds it is more like you know give me the fixations and the what needs to know the duration of these things right yeah 0 yeah durations are there but it is not there is a difference between saying that something is in a particular state every frame whatever the frame rate is you know 10 seconds a minute you know that is one kind of way of looking at data and a parsed version of the data which is not at any particular length it relies on this really close frame rate so there is no real time there is only events what i am trying to figure out is whether we have thrown time away no no the time is still in there but you will get what you want is to figure out what are the concepts behind that data that you want represented so rather than saying the eye was at this particular place you know here and then a 10th of a 2nd later here and then a 10th of a 2nd later here you say there was a fixation from this time to that time and there was blink from this time to that time okay so one of the formats in which eye tracker data is data is analyzed is percentage of time spent on some target as opposed to some competitor over the 1st 2nd after some event so you actually have to show at some time slice rate where the eye was on the same target as the other guy a similar target what we would both been dealing you know so we have to we have all the parts the interesting parts of the screen identified and be able to show distribution of gaze over time all right now we could be really unlucky and somebody would expect us to do that at the real frame rate but i think that is really unlucky but my point is that we must not throw away or lose the capacity of being able to deliver that kind of data so there is 2 issues here we are not throwing anything away but the question is what tool would you use to get that information out of the data would you be so the more you can plan so typically the eye will move right so over any 12nd the eye is actually fixed on a bunch of different things and so when you say the eye was on this from here to here what is going to happen is that if you use the real frame rate it is going to jiggle all over the place it is going to be on this landmark it is on this object that object this fixed thing that fixed thing okay you are going to get lots and lots of stuff and it is percentage distribution that you are going to want it is not i went here i you know it is not like a person walking i went here i stayed here it is more like a fly hovering so you need to know the percentage of time it was on during that fixation because if you take them as separate events you get 1000s of separate events yeah but the sort of start and end times will give you that output and then you just measure against you know against bigger frames yeah the cumulative total timer okay so the percentage is a cut it is not something you are using for measurement it is a cutoff for whether or not it counts as a fixation well that is the problem there are 2 ways of doing it one is that there is always a cutoff there is always a cutoff right but the other is that there will be a time span a kind of reaction to some event span yes a lag in some of it i suppose there is the saccadic movement itself and so i mean there are really 2 ways of looking at this one would like to know for example the percentage of time overall in which 2 people are looking at the same thing and one would like to know sequences of they did not look at the same thing and then they broke it or they did look at the same thing and then they did or did not you know one could imagine those 2 categories that is a kind of testing of our hypotheses but there is a certain amount of dues you pay to the way they do things in the literature to get your papers published and one of the things they will want to know is an event series after some critical event what percentage of time is given to looking where the other guy is looking and what percentage of time over some reasonable time span of a couple of seconds so they will want to see the gaze settling on particular incidents on particular places so we have to be able to deliver those 2 things and there are rather different demands right it should still all come out the little sketch i asked this for last week about nxt and whether it would have that sort of tiered effect so you could see the overlap of where one person is looking compared to the other in their mouse movements and things yeah that should because all you want to know is whether they were looking at the same object in the period after because it is also based on the time course of the procedure you will be able to basically get the scan path of the pattern of events from that as well can we draw it to make sure that this is satisfying ellen is concerns i did not have it well there is a whiteboard right we love this thing see if the pens work so this is a is eyes right and b is eyes and i think what you are saying you should until we draw this right is that they you know if from this time to this time they are looking at triangle one what you are saying is that you want to know in the critical you want to know after they are looking at triangle one what is happening in this period with the sky right or how let us imagine a typical construction event okay okay there is some part there are separate movable parts on the screen ready for use right there may be or not a construction already begun somebody makes the 1st move there is some kind of communication either gestural imaginary or verbal that plans how we are going to do it before that happens the person who speaks is going to do some kind of visual scanning they are speaking or communicating in some way the other person may or may not be looking where they are looking okay they may be overlapping gaze at particular objects which are of interest let us call it the the construct existing construct which could be 0 all right or the the addendum the thing which is next going to be added next piece okay now in neither case when you draw this he is looking at triangle one what you are actually drawing you know for the period of time in which he is steadily looking at triangle one is going to be very very short right because staring fixedly without interruption blink or saccad is an extremely short of end at that point it will have lumped all the individual fixations as long as it does not move off that triangle together that is fine gaze time so it is in a region it is in a region that is a target in the target region but even so you are likely to get bouncing in and out of the region right so even so you want to look at if we are interested in how long before the construction move takes place how much of the time they spend looking at the same thing let us call that our measure of alignment so they are going to be yeah that is right a is going to be on triangle one and various other places that is right and b okay and we are going to look at the percentage of that time where they are both in the same region right so what i want to make sure is that we do not simplify do you know what i mean temporarily simplify too much yeah i mean as long as we are because then you lose percentage is not it yeah there should not be anything right and so b has some periods of looking at one which will also be intermittent yeah yeah so you know he might get triangle one there and no maybe this one is going to overlook quite a bit yeah so what you want to do is be able to define bigger periods which is the period when they are sort of interested in triangle one overall right this is sort of some metal level analysis if we have an event at that yeah the 2nd line then for the 2nd before it and then we can just take that chunk out and do something with it yeah but that is not something that you can do in the nxt query language but you can not do that in any you know this is such special purpose but it is easy enough given the data format for any of these things to do that so you can do it on because it is just a matter of the hard part is deciding how close together these have to be before you decide that this is an event that you want to pull out because you know algorithmically you are already putting together i mean this is essentially already you know these fixations with stuff in between in between and you know we have got an algorithm for deciding when that is a look i guess you already have that or no well it is just based on a stable fixation for so long or something crossing into that region 0 yeah yeah so it is all fixations these are all fixations and saccades but within the yeah so that is easy enough so the hard thing is they could be moving their eye arbitrarily here right and you know maybe even having fixations on other things yes yes because for example for part of that time both b and a are probably looking at the construct if your intention is to move the red triangle to sit on top of the thing already constructed you would tend to look back and forward so this is and we should be able to yeah but what you need to do is we can already build into this gdf format these regions if you can give a definition of what you think that is but i think the right way to do this is to be able to inspect the data in some tool and play around with the definitions because you will not get it right the 1st time yeah sure so the question would be under what tool should we be looking at these events and ilan is a reasonable choice for that right what you need is something well no because you need to i do not know you need to be able to see the effects of this so i guess this is a case of craig writing some scripts so robin having some ideas about what the relationship is and saying you know add this data automatically i mean it is not that hard right it is just there will be a bunch of these you will have some rules about maybe how long you spend on other objects and whether the other object is the existing construction or not right right so essentially there is this is ae is divided into tr one and c and other stuff where c is the construct the existing thing right so if they had a fixation on c you would not be worried but if they had fixation on well no i mean we would take a separate we would look at the percentage time i mean i have no idea because i do not know how people look when they are building things together okay so there is the addendum and the existing construct and i do not know whether they are going to spend more time looking at one or the other but if they are whatever it is that one is looking at it if the other is looking at the same place they are in good shape but you know they are going to look at the clock yeah sure sure and they are going to look at they may look at completely irrelevant that is what the diagonal stripes are for that is true but we can one of the things we will be doing is categorizing people i take it or interactions by the amount of time people spend looking at the clock we would expect that if we put people under time pressure they will look at the clock a lot more but that is like a separate analysis right sure it is a separate analysis but you do not want to throw it away right so you can define if all of these things are actually categorized by the eye tracker as to where the eye is you should be able to pull out any interesting category and say all right for this phase it is tr one or c for this phase it is sq one or c but what we find what we can get out of the data easily at the moment is at this kind of level right and then it is a case of defining algorithmically all these are the transformations that you want so this is one we had not thought of adding before but we should right which is well how would you decide whether something was the current addendum you have to actually transcribe the thing or watch the film for the line so that is a human decision yeah absolutely yeah so once you have got that sort of information well you have got the gdf format you then filter it to say right between time x in time way they are constructing they are adding triangle one to the construct and then you just say in that time period what potentially they look at triangle one what percentage are they looking at the construct does that sound reasonable the addendum and the construct should be you know kind of they change their true identity but they are categories of stuff now my kind of assumption was that this was j p land that is what he was really interested in was the kind of coding which would define the building sequence okay so and the and maybe he wants to think about that more and say well there were several candidates for the addendum all right and and it was the negotiation of which candidate was going to be the right candidate that is actually the interesting stuff so i should talk to him about this he is he is actually but we emailed me with a with a list of things he wants to discuss at length so but we believe jp to be defining that maybe well this is my question for him is he going to define this kind of building sequence in a way that we can get out or see so i mean you know they have been working on construction there for a while so maybe that they have a coding system that is ready to go and we should just apply it yeah okay they do a lot of sub assembly the tasks they have been doing so they may well have a i could believe them you know taking an interest in that let us find out i mean the general division was language here and other stuff there right right so i have understood it right i do not know i will just admit it back i mean actions might mostly be theirs right but since we have the eye tracker we are going to have to answer their questions 0 yeah eye tracker here yeah i mean we know that but so they will tell us something about that but presumably what you piously hope is happens here is what happened with transactions right the verbal analysis and the visual analysis give you the same breaking points the same chunking points well yeah but i mean with transaction coding on something like this you would use you would not consider it verbal analysis exactly it would be verbal plus action well i mean but it is a task breakdown it is just a secondation it is a task breakdown but for the verbal version there are ways of announcing that now we are doing a new one right yeah but you would not have you would not do 2 codings for task level one no you would not do transaction coding one of the things one of the things you would like to establish you know as an as an outcome of this is that you could analyze either end and you would get the same chunking of the material i i do not think that is realistic because i think what you would actually do is use the full record of what you have and do a segmentation i think it would be hard to understand what was going on if you use like just the language without which in the video to decide whether we are breaking down the task it is a hypothesis 0 you do not want to be coding based on that though and then that is a seems a strange way to go about things i mean the language events are multimodal right you know they are doing all these things together they are doing these things together but one of the things that one could imagine doing is having the transcription there and just play it back to a bunch of captive undergraduates why do you care whether they stop and start whether they can do it just based on the language because essentially you are looking at cycling sequences and discourse and if the discourse tells you what is going on that is information i mean one of the questions is how the information is going to be shared across these media and if you can get it all out of the speech you know if all of the chunking is available when there is speech okay then it is carrying a lot of the burden well it seems to me much less important than the other things one could get out of this data but i you know but this is but it is a possibility we have not looked at data like this with our old fashioned analyzes you know and it would be really nice to know that that much information that chunking of the task information is being carried by the language because jp is question the overriding question is so what is language for people are busy interacting all the time and all of our colleagues do not even bother to control for whether people are talking to one another when they are doing these joint tasks because it seems to them to be irrelevant because language is irrelevant it would actually be nice to demonstrate that in a place where we are controlling whether you have language or not you can get the entire chunking of the task out of the language yeah well i can see maybe wanting to know whether you can do the chunking just based on the actions without the language in cases where they use both it is more i can not quite see why it is important to know that they can do it just based on the language when you know that they have both because you know it is a self contained system as opposed to a system which can not be interpreted without the other system huh well i think that one is just bound to fail almost all studies on language claim that language is a self contained system which will give you everything you need to know really that is the claim okay all right anyway i did not say it was true i said that is the claim and so it is worth testing so back to this main problem which is you know the record that we are getting in the 1st instance is about these you know they are looking at some region of the screen defined dynamically and then we need some way of knowing what the you know the you know they are looking at some region of the screen defined dynamically what the you can you can see adding these other analyzes about you know they are they are jointly focused on this region and trying to figure out what percentage of time they are looking at it as adding new tiers of information to either the elan track or nxte they are both sort of track based in this way but the hard part is knowing how you want to do that and i think in the 1st instance what we are trying to get into the gdf format is just this and then we have got no option but to figure out a way for like to look at this just explore this data suggest ways about doing it and be able to play them back and see when we think we have got these things right i do not think we even know what the set of these things are that we want much less how to get them yet and we will not establishing the full set will take as much of the project yeah that is right so this is not you know this is something we are aiming for but this is not something that affects what craig programs for the initial gdf translation fine my only concern as i said when i barged in was just to make sure that we did not lose the things that we might need to pick up later yeah so let us just return to this question a frame rate because this is the thing we were not planning to transfer into this format so again you know a frame rate kind of coding can again be seen as a track but it says it is coding like this right where you say it is state a state b state c state a but since the definitions of the thing we are looking at are the ones which get rid of the lower level like jiggling around in the area of a particular object right so if we are moving the green triangle we have defined a region which is dynamically the green triangle wherever it is we will jiggle around in there but it does not matter where we are in there that is beneath our level of analysis right so you are happy that it is not going to have framework this it is going to have an interpretation like this well that interpretation is definable to frame rate by the only way we are going to use frame rate what does that mean sorry well i mean because i can say for how many frames this fixation went is officially defined so as long as it is not an untimed event yeah all events have a start time and the end time so it is you know it is fine i can say it is a long one or it is a short one it is 20 milliseconds more than that one it seems to me that alan is concerns might mean we need to add more information to these tags because this thing in itself is a bunch of fixations right with saccades so things like fixation the number of fixations and the percentage of time those fixations cover you might want yeah with some measure of the so we could work out the number of fixations that made and the average fixation duration as well because having long steady fixations can be informative and so number of fixations percentage of this is not a leading question this is a question question why would that be informative particularly if they are holding their if there is less dancing around the cognitive focus tends to be in one sort of part so there is a difference between if they even though they are still looking the same part if they are looking around it rather than just holding it their geese steady on it so you think that for example if they are looking at one or another apex of a triangle it would just make a difference that we are exploring the thing as opposed to they were simply all right there is there it is okay so that really means that we are not throwing away absolute fixations no i would not want to throw out screen location of absolute fixations but the actual location no no this is still summary data we are talking about just adding attributes to these things that say the number of fixations that counted as that looking at triangle one right so it is possible out of it so i thought i understood you to mean exploring the figure because your fixation is a point and the figures are bigger than points so you could be exploring the figure or you could be just somewhere in the region defined as the figure but not on it well yes but that probably is not going to be easy to get out because they have to break up the parts so you just want to know the variability of the fixation and would not that differ from person to person yes it can do but it is also looking at it within their own behavior okay so it is the number of different different things of different spots within that region where the eye has fixed i would say just the number of fixations the number of different yeah the spots will be different anyway you are not going to have the same pixel so we have the option that we can put in an a fixation thing here right as well if you want the smaller if you want the smaller coding in here so that it is not just parsed into this idea of which object but you also have the raw fixation data we can put that in as a little another track if you think that that is something that you might want to look at in one of these tools that shows you the tracks against each other well all i understood you to ask for was the duration yeah that is all he asked for yeah well i think we have got everything then i suppose it is you can come back you can always yes it is you are not throwing anything away then yeah and in nxt there is no cost right because the you just do not choose to load those all we are doing is dumping it is output that you can load if you choose to you know so if the things that you just know are crazy and you are not going to want then we do not do it but otherwise we go ahead and dump them because it is easy to dump up the fixations yes so it is the number of different fixations okay so it is the number of fixations well actually do you want the average duration or do you want the number of different ones the number but duration things will somehow whether it is the average one or duration of each fixation in the region not each well if it is each you have to you have to break it down into another tier right you can not you can say there were 3 fixations and they averaged a certain time but you do not want to the average should be all right well what about the the overall sum of the durations is that i mean which way do you want a number of fixations in the average thing yeah then that is that is the way you want it that way around yeah so that is a measure of jiggle in the region roughly and did you want percentage of time fixated as opposed to which is sort of another movie you have this number you can have as many of these should be you should be able to derive that presumably yeah well it is not the the extra cost of having these as attributes is not high so you know it saves you having to do write scripts to do arithmetic on it later if you know that these are numbers that are going to be useful to you what is our error of measurement on location on the screen so to what degree do we actually know whether the 2 little eyes little circles are exactly in the same place or just somewhere in the same region it can vary a bit depending on the calibration but if we are building in a sort of error margin it will be some pixels around but there will also be a minimum one because i am trying to think it is probably the small the actual mouse icon is so small small it will need a slightly larger just because it is yeah okay right sort of a larger area or margin around it okay so my question was going to be all right the mouse is a dynamic object right the eye track is a dynamic object or is it only the piece that the other person is eye track is on that is an object in this definition no the other person is gaze position is a is an object okay with some with plus or minus something or other yeah around it some circular area okay and so you can be on the object but not on the other person is gaze so there can be a triangle which has a is gaze in the center and your gaze actually happens to be here yes you are in the triangle region but not looking at their end of the center individual but they probably will avoid looking directly if they are looking directly at it it is obscuring the part but it is directly at it when these 2 in this situation when these 2 overlap right like this yeah so that is kind of that is because the diagram we have got out there suggesting that they are only looking at one thing at time so whether it is triangle or whatever else so what you suggest in there is they can be looking at the triangle and maybe a mouse pointer and maybe an eye at the same time okay well it would come up it would say yes right i mean because they would be in the same place so if you are going to have dynamic regions which overlap one another by definition if the mouse is on the construct the most of the parts right well those are different tracks of information so what you want is one track of information about which object it is on what about whether or not it is on the other guy is gaze and what about whether it is on one of the mouse positions we treat those as other objects but the difference between them and the parts of the thing to be constructed is that they are allowed to overlap with other things and nothing breaks okay yeah but in the analysis you should treat those as completely separate tracks of information because they can be looking at they could look simultaneously at somebody is mouse somebody is gaze and some object right yes as opposed to simply looking at somebody is gaze when the mouse is not also there and you want to know those things so these are independent right they are independent dynamic objects so what we want is whether they are looking at the other person is gaze right and say they are in this period well i mean okay so a definition of we are all looking at it and touching the same thing so imagine that we have just added a piece to the construct all right so now that is the construct and both mouse symbols are on it right and both gazes are on it right so at that point you should have a lineup of a is on the construct a is on the mouse a is on a is mouse a is on b is mouse a is on b is gaze and b likewise yeah okay so there is a lot of tracks here right because we just ended this fixation track and then there is a is mouse not there on is on an object right and the mouse might be on yeah but the mouse is also a triangle one dynamic object but it does not matter that is fine the mouse which is saying where is the mouse you know it is on this object and then a is eye is on well a is mouse i guess right a is i might be looking at a is mouse here a is eye might be looking at b is mouse here here right i mean these are all the different ways of taking cross products the things that could be collocated so you just treat them as independent and i am guessing from what you say and then you look for combinations of them yeah i am guessing from what you are saying you want all these tracks i think we need to yeah which is fine there is no problem with that at all looking where the other guy is directing attention and there are 2 measures of where the other guy is directing attention that you get from the visual track one is where the mouse is yeah the other is where the gaze is so you kind of got the idea here yeah yeah okay yeah so yeah so it is just a matter of adding the 2 mice pointers and the 2 gazes as extra objects so yeah as long as they have got an id in there we can just say right at this between these times they are looking at that object or they are looking at the mouse they are looking at whatever but the important thing is you do not treat the entire set as mutually exclusive and exhaustive it is just so that the objects is like a separate level of analysis whether they are looking at the objects then whether they are looking at the gates but it is also going to be the case that whatever region you define as the region of the dynamic object of the red triangle may get to the point where it overlaps the region of the dynamic object of the green square okay what do we do about that does that happen yeah because if you have them close together there is always some fuzz factor around them and so if they are left lying close together okay or in the in fact in the model not in the model in the supplies set they are actually neatly put packed into a little space okay so what you are saying is like a new problem for us which is i am going to change 2 objects close together your eye gaze can actually be looking at the left edge of one and the right edge of the other and therefore looking at 2 objects at once yeah so you are saying squeak squeak okay hey can you write on my pen pad just say yeah pens pets whiteboard pens so you are saying you know they could be looking at square one at this point because they because the 2 are so close together that their regions overlap okay that is a problem for the data models and either of the things that we are using unless we just define it as there is only they are exclusive and whichever area has the greatest overlap is what they are actually looking at what do you mean by greater distance overlap with one well but if you have got your sort of eye position and it is unlikely to be exactly 50% in one object and 50% in the other draw it my special reasoning is no good i think you are throwing information i am the one that uses the whiteboard because i actually put the microphones on what i am the thing is if you have got say one object there this is the better pen thank you part one and part 2 and you have got their sort of they are looking there huh then that overlap means they are looking at the 2 pieces simultaneously yeah but you had a way of choosing which one right yeah but it may be a bad idea there is more of it it may be a bad idea you have just thrown data away yes yeah that is a decision i am saying is if they are looking if most of what their overlap is here then we stick with that one so you just mean is it closer to the circle i mean what is the circle they are looking at a pixel or something right no that because the gaze position will be a sort of fuzzy area it will not just be a pixel it will be an actual like gaze will wobble the gaze will wobble so that is natural that is going to be it is not just a single point it is a sort of lump target so there will be a bit of a some sort of overlap or indeed they might be looking from one to another yeah you know deciding which one to choose or thinking so you do not want to decide it is only one and throw the other one out the wally gds performance is working at the moment it is actually got a separate track for each part okay well that is fine i was going to say that is the obvious way to do it 0 but that is going to be analytically because it is going to how many parts are there well it depends on what you are building there is different parts there is different parts whether you are doing the tangram or way you are doing an airplane or whatever 6 to a dozen okay the thing is if we have got a series of tracks for each part we can easily analyze it down later on i mean i think it is very likely that if people are considering if they are oscillating to us then that is presumably going to change the weight so you are actually going to see them flipping between well not if the regions overlap if they are really close to each other it is going to get foxed yeah yeah right and if you define the regions so neatly that there is you know that we are losing gaze because of jiggle outside this closely defined region when it is in the middle of space then i mean i actually like the solution however ugly it looks in terms of the data format of having a track for every possible object it is not the data format i am worried about it is the way you do the analysis because you do not want to have to say you know did they jointly look at triangle one okay did they jointly look at square one okay did they jointly look at you know you need some way of going over the whole thing but we can find ways around that i think for now we do it this way and then we think about what we need out of it in the end it is just you know in a lawn that has the side effect that if you go for the naturalistic way of up translating to alon there will be a zillion tracks and it will probably ruin their viewer because you will get very sparse data on each track it will be like the old referring expression general visualization in the map task where they talk about one object one landmark and then another line mark well it is big right yeah it is big and you do have to scroll through it to see what is going on but it gives you very clear picture of what is going on well we will see if alon likes it or not yeah well okay so we better do a test one yeah and we better also like have in the representation a list of part like even in the nxt of translation a list of parts it affects the way we do the nxt yeah well there is only list parts in the gdf so yeah that is not a problem so basically what i have got is well there is 3 places where you have got the id day so you have got an id for a part you can id for a location on the screen so that is just the target configured the clock whatever else and then we can easily add ids for the 2 mouse points and the 2 gazes and from that basically all the tracks are doing is saying between this time there is a look at this object this id so it is effectively a track for each id okay so what about the point where a part becomes the construct you still identify it as a part they do not think one should lose that well what happens there is that the 2 the existing parts basically cease to exist and a new part with a new id starts existing at the point where they are joined so every time a part is added to the construct it becomes a new part so the construct actually has say it has 6 things that are added to some initial thing so if there is just the initial thing that you put in the middle of the screen that is just the initial object as soon as you put a part there is a construct but it is construct one because when you add another part construct one ceases to exist and you get construct 2 yeah right so actually defining all of the constructs those as the construct is going to be the trickiest thing it is also going to be a problem because you are going to have one part or construct forming in the middle of a fixation right so it is not going to be quite as clear right so you are looking at triangle one right and then all of a sudden it ceases to exist and it becomes construct or assembly whatever too yeah and there could be more than one construct at a time right there is no the construct and then you can start again you can build sub sumably and then link the point start again why does it why does construct one become construct 2 when you add something to it yeah that is the way it is working at the moment because it is generating a new id when you create a new part 0 and this is because we do not want it to stay triangle one when you start with a construct it is not a construct at all it is just a one thing and we do not want triangle one to suddenly have well the problem is that all of these things are popping out of existence so do we know the difference between we broke it and it went away and it is not part of the construct yeah there is joint events so if a joint event if there is a joint event linked an item then you know it is become part of a construct can we tag the constructs with what is in them yeah the contracts are already tagged with it so there is actually a separate part of the data format which says for each construct which 2 parts made that came together to make that construct we could essentially well then we could cumulative or it is a new construct because it has a new list of a new cumulative list of parts yeah but you can still track what parts went into it right so you can still find triangle one whatever so some fancy programming is going to have to be done to say i was looking at triangle one and now i am looking at triangle one in construct one because the 1st construct was triangle one and square one yeah but that is easy it is easy to be because each construct has the ids for the things that construct it so now we have to ask whether so let us imagine this we have made a construct of 2 parts a triangle and a square and we are considering now adding something on to the side of the square that is not attached to the triangle okay are we looking at the square or are we looking at the construct yeah that was the next question i had yeah you will be looking at the construct in this step it should be looking at the construct there is not a way to figure out that you are looking at the square rather than the triangle well well yeah well the thing is we still have the xy traces from the ascii format so if you want to be able to say i am looking at the square instead of i am looking at the construct it is possible to get that data out it is it is not doing it in the jasanlad of the moment but i can easily add it to it to give that information if you want okay because i mean that is that is the example presumably that is the best example of overlapping overlapping areas because if the thing the 2 things are now about it of course there is an area which is kind of both of them okay so what i am trying to run through in my head is that we can always tell the difference between something going out of existence because it is joined a construct something going out of existence because we threw it away right suppose we pick up and we we screwed up the 1st time we put the triangle with the square and we threw it away decided we did not like it and we took another one is it triangle one is a triangle one prime or something like that it is the replacement for triangle yeah if you take if you create a new part it gets a new id so if you drag a new part out of how do i know what it replaces you do not because that is you would have to have human coding to do that yeah there might be multiple triangles that you broke and you will not know which one they are planning i am using it for this is true but if i have triangles 123 right and i throw them away any triangles i take out are to replace 123 well you know which mold they come from and so you know which shape they match but you can not possibly know which one of the ones they are meant to replace yeah i mean you know you know which one they are meant to replace but you know they are not meant to replace any that are all that are they have not been thrown away they can not replace the new parts only appear when the the previous one is broken right so the new parts are replacing those which have disappeared from the screen yes and there is no more tr one track the tr one track is gone now 0 so there is no 2 parts with the same shape 0 there are many with same sheet but not the same sheep in color 0 same shape in color right okay 0 is that true you have decided not to make all the triangles right and squares green well that is what they are at the moment 0 okay okay so you can tell what it is meant to replace them because it is the same it is just in case of the mold right yes that is the way it works at the moment i assume that is how nymegan had set it all up okay so that is back to tim is so shape plus color they are individual they are unique all the parts are unique so hang on that makes the task a little bit easier yes right tangrams are much easier when you can see all the lines yeah so it is just a construction task rather than a puzzle indeed which kind of leaves you with an obvious way of making the same task harder and easier does not it yeah yeah but that is hard to code i mean that is really a thing we might do okay so now let us make sure that the coding would survive that it is just it is a beautifully controlled situation the construction task is essentially the same it is the figuring out stuff that gets harder if everything is purple just to choose a color at random here yeah yeah so everything is purple there is then 2 triangles is not there there is 2 identical triangles in the tangram yes at the moment there are 2 i think 2 small ones and 2 large triangles they are actually the same physical size and sheep at the moment they are in different colors that also means in the new parts list they are there for 2 large triangles and 2 small ones whereas in the advised edition if they were all just plaque there would only be one mold in the new parts when you specify these things do you specify the molds or does it figure out what the molds are based on so is this hardwired into tim is program or is it just an artifact the way we yeah yeah when you specify the molds in the in the file that generates the parts we have got a list of parts the molds are based just taken out of the file that the general that best 1st place the parts right okay so how many parts that are in the file that is how many parts appear in the parts box so if you have 5 black triangles there are going to be 5 black triangles in the parts box or one if you have 5 black triangles in the parts box so what you can do in the initial configuration you can reuse a triangle so if you only want one black triangle you just create one black triangle in the parts box and then reuse that triangle 5 times in the configuration  okay as long as this is possible so i mean that just is a kind of thing which is so simple so much like a single yeah same 10 variable change that it does not are existing because i am trying to kind of map this onto bowfix and bowfix has multiple yeah has multiple you know all the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the all the flat things or wood colors they can have different nuts that are well the same part the same part as different colors yeah but they do not have enough to make unique no i mean there is more than there are multiple pieces of look the same so when you say a red nut or a green bolt or a long green bolt then there will be lots of them lying around and i think that is how the robot is set up to 5 green bolts here is one i think we have gotten the point where you know to try to summarize we think tim is program can do both of these conditions that you want but i am still worried about what the effects are for analysis because you were aiming at something in the gdf format and i was not quite sure what okay so all i am doing is kind of worrying out loud about all the things that will happen that will miss by a simple view of for example regions and looking at triangle one when is triangle one not triangle one anymore so okay so i will summarize what we said about that part that every time you cast off a new part from a mold or every black triangle has a different idea identity yes and you know it is a black triangle because you know which mold it came from but you do not know which part it was intended to replace on the screen because you can not mind read that is true so how can we define the shapes that people build what do you mean define well if assuming i was for a moment i was jp and i wanted to know how they actually went about building the thing okay and i wanted to look at the strategy and i wanted to see if the strategy was different when we could talk about it and when we just picked up whatever i mean suppose i can not talk to you and i am doing this task with you and i can not talk to you the thing which is going to be hardest for me is making elaborate plans with you i can reach for the next thing and you can go where i reach but if i have some sub goal some long term sub goal of doing something clever with putting these together because it is hard and then putting these together because it is hard it is going to be almost impossible to convey that to you and the difference in the history of construction is an important thing and i wonder how we can get that information back i think that this is a human coding that is part of the action coding because i do not see how anybody but a person watching this can guess why they cast off this thing at this time no why is not that is not the question i am asking the question i am asking is what is the history of the construct so you have the names of the things in there and we have 2 constructs suppose we have 2 sub constructs and whichever one had 2 pieces put together 1st is the earlier numbered of those constructs right how do you tell the difference between adding a piece to this 1st construct and creating a 2nd construct well when the file construct the way the constructs are defined in the file you have got like your parent construct and the parts that was constructed out of okay and yeah as you say they are it is a parent child relationship it is a parent child relationship yeah yeah so you have got a parent child relationship and the child can be a part or it can be another construct okay so construct 2 construct one is a red triangle and green square construct 2 is 2 green squares okay construct 3 right is a red triangle a green square and another red triangle construct 4 yeah but that is okay because they all have different underlying part numbers and when you say what a construct so how are we going to tell when for example you think we are going to have to do human coding on when 2 subconstructs are put together no that is a definition where a construct has constructs as children yes yeah because there is a joint action the result what you need for a joint action you need to you know a construct on a part 2 parts whatever you are taking together and then the result of that is a construct you have got a time when that happens you can see that construct appeared at that time and you can still track the children of that construct so we can zip through this and look at all the all the interactions in which people built subcontract 1st which we expect them to do as a wild hypothesis more when they have verbal communication than when they do not okay and we can do that by simply searching for any constructs that have constructs as children yeah so we can easily look at things like that automatically it is just the why they get new parts that is the problem for us we can not if they suddenly if they decide to get a part off a mold we do not know we can not know why until they do something with it yeah we have not actually made a rule that you can not collect extra parts right extra parts do not appear so you can not collect them 0 yeah you said that yeah there are they are like grayed out yeah but you can so you see the mold is there but they do not come filled no no i mean you have to break one to get one yes you can but you can break one when you break one you do not cast off the mold yourself it happens automatically yes 0 okay never mind that it appears in the new like so they just have to use it the new part lights up sort of thing huh 0 but they have to drag it up past the mold line yes you have to bring it into play and you can not just bring extras in case you can screw up you have to screw up 1st and then you can bring it that is one thing unlike the say the bow fix or anything yeah well there is extra parts there are not any extra parts and at the moment the software does not but it does have replacement yes replicement but no superfluous so you can not have a standard stock with extra parts that they will never use okay so this y question does not even arise so you appreciate the cost because you know what they are going to do with it when they decide to move it past the mold line they are doing something with it probably and you will know what they are going to do with it because yeah but they are doing that they must have already broken something so you are saying what they were replacing so they rejected something okay is there a way of telling the difference between intentional and unintentional breakages how would you do that intentional in the sense that they break deliberately when they bring them into the new parts it is like the trash can idea they both put their hands on a construct because they do not like it yeah but there would be no way of cutting that automatically right how could you possibly do that you have to hope that there is language and one of them say something but that is a human coding right so that just has to be coded there is no way you could tell that just from the role movements or something not generically no because if they are going to do it by gesture they will build up a convention so that is going to be a problem for jp right remember all the things we are looking at are with language and without language and it is going to be very difficult to tell whether they broke something intentionally without the language which means it is probably not a convention suppose they start off with language and then they say yeah okay but this is properly part of the action coding sure so we should i just want to make sure we should make a wish list right wish list as a side effect yes what i am trying to remember is if the people deliberately break things by moving 2 pieces together they do not both break they do if if one if one person is dragging a part to a part that is not that is just sitting on the screen not being touched they both break so the intentional thing is when they actually both click on the same object and it breaks for sure you know that is or well it is not necessarily intentional they could just screw up yeah but they could discuss deciding to break something yeah i do not think that is very good no we will never get away with that we will get a bad score let us just throw that away and start again yeah fix the score problem have we well yeah because it was through the rotation 0 yes percentage score yes rather than the number of penalties or something no the score there is the percentage overlap and that made the score that is the way it is worked but the problem was to do with the which they had a relative and absolute peace and it is all too to the symmetry of that so as long as it is defined fine it was not a bug do not tell me it was a future no it was them just forgetting to define an adjective it was something that had to be defined and the dutch lot did not yeah which is probably a documentation fault right it is not clear enough in the documentation but it is like 2 thirds the way down a quite long page it causes quite media problems not prominently flagged yeah exactly a communication floor so that is now just lovely and there will not be any crazy scores anymore well while we are on past things or anything else you want to tell us about the prior part of this which is the you know what else is holding you up from well yeah we should go back to the beginning the camtasia was tested last week and initially we were a bit worried because in the old version it seemed to interfere with it but when craig recompiled it it did his new version it seems to work fine right so it does not slow it on the eye tracker it does not do anything nasty so we have been told there is very small graphical bugs but it is not a big thing it is like a couple of white pixels in the course of a task so it is nothing to distractive nothing important enough nothing big enough they are going to look at it right no so the last i am sorry i had assorted emails from you and i probably did not catch everything the problem was that there are 2 kemptasia yes there will now have to be 2 camtasia records because each person actually sees a different screen because of the gaze feedback they do not see their own gaze they see only the other guy is gaze so there will be 2 separate camtasia videos yeah i know 0 do you need 2 copies of camtasia to do that hopefully it will not you mean officially or yeah well the question is it the license for camtasia where you just need one editing thing and 2 recording things because yeah the recording program is just quite a small program that just generates the video so that is worth having a look at the licensing for that just to see so you are going to look into this right yeah i had not thought about i would assume we would just get one and one license code we would unlock both i suppose technically should have 2 well look at the license conditions it might not i suspect that it will be per machine i think i never prejudge licenses and blubred them okay i will have a look at the actual kantasia license the other thing we well and hold on before we go any further so this has implications for data storage right because it means we have got well no this is 3 times right because we have got 0 no we are going to use joescript to build the videos on the fly right because it is fast so the permanent storage is just the camtasia stuff so is that a problem where are we putting things is there such a thing as a dual track video because you can certainly mix things what ideally one wants one does not really want to have these things because we are using them for backup and for coding we do not really want to play them independently we would almost always want to use them in exactly parallel time aligned you can dump them into one video frame if you do not mind losing resolution but that is maybe a problem right because well the other problem is camtizzi has to be manually launched in both machines separately so the synchronization is going to be a problem yeah what are we doing for synchronization otherwise during the task there is a flash and a bleep 0 i mean not during the task before before each task okay so that we would have to realign them we could yes dump them from their start points well maybe you could the thing that you are dumping them onto just starts running with flash and a bleep you turn on the 2 copies whenever at different times but they are still no what are they i do not really understand the parts that are operating here 2 videos are turning on there is no time stamp that comes from some common source no they are on the 2 separate machines and you start you start and end them separately they are not talking to each other they are just recording you start it recording and it starts recording what is in one screen and you start the other recording and the other machine and they are totally separate these are backups right so the flash allows you to hand synchronize them later if you need to buy stripping extra video off the front as long as you make sure you start them before the flash then you are fine but there is a cost of having to go to back up which is synchronizing them so they will include things like the eye tracker calibration yes which is not a bad thing actually because sometimes you want to go back and find out if this was just a particularly tough subject yeah see how long it took them actually that is true how long did it take them to calibrate or yeah every time we were calibrated the damn thing slipped the validation worked or something 0 okay so you should start the camtasia really early right well yeah i was assuming in doing that it is done 1st that actually saves you hours trying to make somebody is data smooth out when it will not because you go back and actually yes they were crap is not that the answer it is a technical term it is a technical categorization in psychological research yes the other thing we were getting sorted out was the microphones because only one of the camtasia videos will actually have the soundtrack and that is because we are just getting we have got one cable and to the 2 mics it is supposed to well we need sound we need real sound recording right that is 2 track yes at the moment it is in what we have finally just at 120 clock today looks like we have got it sussed is to get one channel one microphone being the left channel being the right channel to merge into a stereo single file unfortunately the on what sorry on one machine on one of the the discreetecesia video it is or multimedia that is not where you are putting it i mean that is our only sound record yes well okay you have got the proper sound record but you are not dumping it through camtasia to record it right yeah it is camtasia recording before we were going to use camtasia at all what were we going to do with the sand huh yeah because it is because it is a particular going to be able to be in the area so is there any sound is there any sound degradation that comes about from putting this through camtasia rather than running it out straight no you got you yeah you got exactly the same access to sound quality as you got from anything else on the windows so you can decide whatever quality you want so okay so what you are planning on doing it is bunging it on one of the camtasia tracks and then splicing it off the camtasia track it is an easy bit of yeah and then you are going to bung it on the other camtasia track too if you need it right but we will just we are going to store all 3 separately and the sound is going to start at the same time as one of an arbitrary one of the camtasia videos but you will know when the real experiment you will know the relationship between that and the eye tracker timings yeah because the synchronization the audio and the audio and visual synchronization comes at a specific point in the i track file yeah no human intervention required it is time stamped no it is stamped into it so it is just after it prints the line starting experiment yeah i am pretty sure yeah yeah but hold on do we need that i mean i thought the way that tim had the setup originally the audio the audio the timestamps used there would be joint time steps between the audio and the eye tracker so that you know the audio record 10 seconds in was the same as 10 seconds into the eye tracker record so that we did not have to do any extra handwork you know any chopping the starts of audio signals to get there was nothing okay so we are relying on your metlab script to get us the chopped version of the audio right well the matlab script base just says yeah it gives you the time of the audio beep in the signal right so we can either adjust the i track track data or we can actually chop the audio signal yeah take off the 1st whatever 70 seconds there is going to be a way to sooner or later to align the i track camtasia one camtasia 2 with 2 sound channels yeah yeah but lining up the camtasia 2 is going to take we he is working on a metlab script for the sound but not for so the one that has got the sound on it that will also tell you where to chop the video to get it to line up the one that does not have the sound on it it will not the bleak will find out 0 so it is just the bleeps okay so no hand synchronization required at all as long as the scripts work right yeah yeah i okay how confident are you about the pretty confident are you about finding these things pretty confident you have seen them it was it was yeah so we just have to hope we do not have any subjects with odd vocalizations yeah i do not see it on 10 hicklehertz but you know they also tend to have a funny funny shape so yeah okay there is one also in each file so we need one to start to the dropout test yep okay is there an end signal when you decide it is all over does it be there there is a signal at the start and the end of every trial right okay so when you have pressed we are finished it pings back at you yep okay it is just all kinds of crazy things happen when you are right things so saving a percentage of a trial is sometimes a good idea you know being able to do that is sometimes a good idea okay yeah pretty sorry because it is just a good thing and they are just a patient in something like that they are 0 yeah yes it is stojack but it is a mono microphone so you got a solution for that then lining it is going through a line in so it is quieter because it is not powered but it is still yeah we still pick up the signal you sure there is not a better solution that involves you sure there is not a better solution that involves can we do something better than that because you know people might want to use this for speech recognition or something you never know if the data is there yeah i mean it is crazy we are in an anicolored room which we specially built an anacolitan room which we specially built for this and you are going to want for certain circumstances we should not screw it up more than we have to is there anything we can do to improve this 0 no no no better to collect a sort of powered microphone or something but when they are there there they have batteries in them but it is not there is it does not seem to be pad because the lining input is so low wait wait i thought a better solution was fixing this problem with the mono microphone socket sound card so you said the problem was the the the sound the sound the sound the sound the sound the sound the sound the microphone is stereo the input is mono right yeah the the the jack that is plugged into is stereo but when you plug it when it is recording from microphone it records in mono so is that fixable like is there any way somewhere in the sound code yeah i think a new sound card sounds a better solution than munking about with trying to filter white knives out 0 i think the other one thing you do is is record the speech signals separately for each person but then you would have to pull the 2 sounds together which is not hard but if each machine recorded it is own they recorded the they recorded like the yeah it is a lot of video you have to align the videos anyway right yeah yeah i believe it is easy to put 2 lanas into a stereo signal yeah it is easy i am not the technical person if nothing if nothing else you do in a matlab script you just read file one readfeltt and then you so is there any reason why we do not do that then the back that script will tell you how will tell you how far apart there because you can just measure the drift between the 2 of them so you get a list of where the beeps are and then you say right this stream is like i do not know 025 seconds behind the other one says robin thinking about cutting and splicing yeah yeah well it is in the old days we probably would have run a little if they had been in the same room we would have read a little stereo thing in the background as backup as well just to make sure because it is slightly diceier you know if his scripts do not work then you do not get the oven up i want to see one of them look really nice before we start running subjects can we do that it does not have to be a real trial it just has to be 2 guys talking in the 2 bits and then i just want to know it works the way you expect it to work this is not a gesture of mistrust it is just experience that if anything can screw up it will we have both been so badly bitten in the past i am trying to choose my language carefully because we are being recorded but you will hear more choice language at the point when we have all done a lot of work and then we discover we can not use the session because of something we did not think about yeah but this sounds reasonable to me so i think yeah do they both have the same sound kind problem i would not count on it they have both got the same sound card okay so yeah yeah yeah because it is probably worth mentioning this to the guys in the garage in toronto right to say yeah yeah so the external version is that the game of the circuit system the front circuits in the version of the version of the version of the version of the yeah so the external version has got the version has got plugs that size and that is stereo microphones right but yeah the other one plugs the smaller size and that is not stereo so the car can do it it is just that they connectors on the connectors that are the back of the computer can not well yeah i think tell the guys i think that is a if the documentation said it could record in stereo then i think we should talk to them about how they have done this is you not sure from the documentation what it says it can do do yeah but the website and they are yeah yeah but as far as we can tell when found the the website of the the creative labs website and a review of it and they were not entirely clear but they both suggested that it was a mono there was a monom microphone yeah it is just the way they wired it so it is getting late and i have got a 20 clock and i have got a nice lunch so i want to eat my lunch today what did you want to get through today more i mean i think it was useful going through your expectations about this because that is quite a bit clearer in my head at least but i think mostly we leave you to go away and redesign the gdf or add new bits to the gdf along these lines yeah so basically the output the gdf is just going to be a whole series of events yeah we are starting in time and the event is going to have whatever the object is that is being looked at and things and then yeah and then it is matter of we can put whatever filters you need after that so you can say right i just want so you can say from the time where they picked up triangle one to the part time it became part of the assembly find out how much how percentage of time they were looking at it or whatever okay so given that there are these events recorded we can use the beginning of any of them as a start point for example right there are also motions being recorded of objects yeah so yeah so you have got move events there is look events there is fixations of blinks in there at the moment as well and yeah and 0 yeah and looks at objects and looks wherever else but i think the thing to do is for you to go away and think about it this way with the different tracks for the different objects and the the kinds of things we have added about whether they are looking at the other person is gaze and what have you and try to change the spec so it reflects this and maybe maybe by that point there will be some sample data or something and the right thing to do is 1st to look at this the spec the way you understand it now and some data and then alan will have new ideas about what is needed or about these post analyzes you know and then we can add them in but it is going to emerge over time that is clear that is great before we break up for lunch i just want to make sure that i know how long it is going to be before we are in run mode okay so the sound thing stands in our way we need to be recording sound online so that is the thing that has to be solved the shape thing is solved yeah that is solved now we think yeah test we believe we have to test so that should be done this week in fact like tomorrow for example the shape all the shape problems are solved generating shapes no problem scoring shape overlap no problem nothing is a problem there are no visual display you do not have your models so do you they had a proper meeting off of the mortgage of the time yeah they had their inspection last week jp staggered away from it so they are that is right there is the model was still odd me but this day knew that it was both down and so when they were yeah well anything you actually need fixed you know i mean you know joe does not live here anymore right so well but he was not going to be booking any time to us after the 31st of october so that means that any bugs that need fixed or not craig is responsibility right you do not know how to change on that right that is what he said to us originally i have not had any discussions with him so i would assume you know no more contact with joe it is not fair to get work out of somebody for free so you know ask craig instead well he is physically here if we want to pay him for some more time we can do that would that be just better use of our time and get yeah well he was not paid in advance he is being paid in arrears so it does not matter right i think it is he is around and i believe and if i think it is better to have all the software i mean it is not big things it is just maintenance of this program at this point but if everybody has a lot to do and joe could give it a couple of hours joe could give it a couple of hours which is what it might take just to fix it then let us do it because starting runtime is not getting to be yeah if you can give it you know like because the problem is you have got to have it immediately when you discover a bug and that and we can make craig shift all his other priorities to do this because we have bought him right but we can not we can not make joe do this yeah so there is some trade off between availability and speed from start to finish given that this is somebody else is code but it sounds like you do not think there is anything that actually needs fixed at the moment because we only need this to work well enough that you can use it for something and you got workouts we are building models right and how many models do you need to completion and how long does that take we have the design and you got well and you know about complexity well that is what i am you are waiting on me loose you can call her today this afternoon yeah i think so because i have to check back with it with jp fairly soon okay he wanted to the next 18 month plan great deliverables great progress with the experiment well that is what we are doing now okay so what do we think if you have a fairly hectic week are we looking for subjects next week okay ads go up yeah i do i do but i think ads should probably go up this week we want people for next week is that okay is anybody now terrified at the thought that we are live next week there are people coming in here one after another or 2 after 2 more likely okay that is her job come on is this generalized anxiety robin or do you you have a specific thing that okay so aside from the usual angst which we all suffer when we go live so we want to do some piloting at the end of the week yep we will get the audio stuff tested yeah single microphones yeah get that tested next couple days okay i assume that is everything you need me for in terms of that right right 0 yeah but she is actually relaxed that somewhat okay yeah ostensibly book time which is ostensibly book time which is not book time which is not so you can dash in there for half an hour at a time does not sound very useful for you okay well okay we have we have a solid booking for all the mornings anyway and except by permission of us all right we own it so we can change that everybody acknowledged the other week that we that we but it would be kind to tell her ahead of time which of your morning sessions you are both yeah yeah yeah yeah yeah yeah yeah yeah yeah do not worry there is a booking system right so it is public what is booked and what is not i thought that was the problem that your mornings were not sliced off on the booking system all right maybe you should get on there and book mornings as far as the eye can see because you know if you are said told use the booking system well we think we spoke to all the people who used those facilities the other day but we could be wrong so yeah and what are the people upstairs downstairs have you know as rules about who you hand the keys to well because you know we like people to know to ground the 0 i thought the lending for the booking system caroline from caroline or the general office the general office see because you are supposed to have training in but no they need an account on this machine to actually use it they need to count on these machines so that is the safety so that they could book they could not actually and you control those accounts okay that is fair enough okay okay we are also working on a way of making sure that the machines return to 0 state when you come off that is safer because then they learn the 1st time well people have been leaving various bits of who knows what around on them right which we think is what is critical yeah yeah okay i really have to get out of here so what else is important before you run nothing right piloting piloting you have to build the things we have to try the machine things we have to check them with marlis and jp that they like the representation of the variables right how about do you have any time on friday 0 i do not think i brought my diary up but maybe why you can email back maybe you and i should be the pilot subjects 1st pilot subjects the ones who know which questions we want to ask and then we should get another pair who are naive to the whole thing okay so yeah this is generalized floating anxiety no no it is wisdom but yeah well because actually we can you know we make comments they feedback and a vote into the development so do you know what time of day you might want because i should i should check that okay you do not need me involved in the party though you all have to unless the bug report yeah you will have to be there to yeah i will be around but yeah i will be sure hopefully it all runs smooth in yeah i can relax okay so that all sounds good fun well i can just leave you guys here you know why not thank you i guess the word just the week that is pretty good yeah they are about pollution and we do not know what they are on you know we are not going to have to be in a place where they are in a coming out and how to get a lot of that they are like to get out of the next to a fall yeah well what took is we can not have you have i have i mean this at once you know so much of coming that is well i assume i assume that is the same to the point the fact of the other people yeah yeah and that is the 0 i think the hotel today right yeah yeah well i will have so many why there i do know so hey look i am doctor elmonds and john is in the mystery i think that is you know thanks for me doctor p but mister doctor doctor doctor there is no doctor no no there is a professor he was not orranged he has not had his inaugural he has not had his baptism by fire but i am looking forward to it because it would be a good media and bob lowe is and inaugural is now a new 0 yeah that is right is that no i am 5 0 5 yeah yeah it is a bit of use thing that okay something going to have some traditional open the world i need to talk today okay okay okay and i am going to close we talk today is the talk today if they are like wine talk the