5 years just because of the floods bringing the ceilings down alright is it it is ready okay great so on its great joy are not they suing him i think they probably ought to but they well they have only just started realizing that treating this as a legal issue is probably the right way to look at it so they are not really the sort of people that are up for understanding the intricacies of the committee i guess i mean i you would have thought they could but it is r it is actually quite hard work the law is not very clear can not th can not they get legal aid that is a good question they p they they might be able to it is just it is so strapped now you know like the citizens advice bureau is so hard to actually get access to because they have they have so many cases and so few people so but yeah that is probably the next thing they will have to do yeah everything is okay are we supposed to count to 10 or something we do not have to do all that yeah okay m okay okay okay yeah okay yeah okay thanks thank you thank you right well i thought the right thing to do was to review what wei xun has been doing in the area of dialog structure for work package one over the last period so basically for him to talk about what he what progress he has been making and then for us to for hi for him to describe what work he intends over the next period of the project and for us to figure out what help he needs with that m maybe give him some help figuring out what publications he is going to get out of this and for us to make sure that it fits into the overall project so this is something that you know we are not going to do everything today but i wanted to make sure that we sort of all aligned on what he was doing at the moment and and then sort of figured out how to give him the best advice coming up so that is that is why i called this okay so okay okay thanks yeah you are going to give a presentation for us then yeah yeah a very or a quick slides great so i have been here for 14 month i think yeah so i have a accustomed to the environment here and enjoy staying here so from last year i was s supervising the annotation of segmentation summarisation with jean of the i ic icsi corpus and on the other hand i w n did s some very preliminary work on chunking the switchboard corpus but that was not well picked up in the ami corpus which is supposed to be so and currently i am more interested in see yeah analyzing the discourse structure in multi party meetings this is what i am going to present here so in this presentation i am going to see how we could analyze utterance to utterance relations in multi party meetings cause meetings is a lit different from normal text or a n a normal dialog or monolog so interactivity is a concept is a concept that i am using to distinguish the 2 extremes of of interac of discourse in meetings and for dialogs and monologs i think we might have to use different concepts to m m model the discourse structure then i w i will give a very rough plan for the next couple of month and finally i i will intr i introduce some related work yeah so the big question that i am concern here is what is the relationship between utterances in multi party meetings or how do we analyze discourse structure right above utterances instead of those high level intentional structure of discourse so the concern here is about the very local low level utterance to utterance relations in multi party meetings this is the local discourse structure so the building block of that kind of structure would be the utterance and possibly one of its properties the dialog act which we are currently working on or and for meetings yeah so i have mentioned before that meetings is a bit different from monolog and dialogs so in meetings i from my reading of the one of the ami meetings i s 108 the the 4 meetings i noticed that the interaction degrees of interaction in the meetings varies from parts to parts so for some part the interaction is very strong so it is obviously a dialog but for some part part where the interaction is very weak just like the chairman or project manager is coordinating the meeting he is pushing the meeting forward then this kind of s organ organizational structure or part and also other pr parts like the back channels without any semantic content from other participants so this kind of interaction might be better analyzed as monolog which in which there is interaction i suppose so i am a little bit confused huh so when you say interactivity do you mean well how would you characterize a highly interactive meeting yeah what are the measurements you would apply so you i think what you are doing is segmenting meetings into highly interactive parts and not so interactive parts where you think some of them are like a bit like dialog and some are a bit like monolog yeah yeah yeah so what measurements would you apply to find out which is which so yeah so this is going to be based on the utterance to utterance so for this utterance the next utterance if it is from another pa speaker then if it is not a back channel or something similar then it is a dialog so if speak changes so yeah speaker change is so you mean relatively short turns and speaker change yeah yeah yeah b but sh short turns like or something very similar might be the seg 2nd type of interactivity the weak interactivity that might not be analyzed as a monolog but as a dial no no no wrong not as dialog but as a monolog yeah so for that part for that kind of back channels we can just skip that and to not treat them as interruptions or something like s then yeah we could just so so am i right in thinking you think that you can segment meetings maybe into 2 t types of areas interactive and non interactive yeah so what are th what are the characteristics of the interactive areas yeah speak change speak change would be yeah so w so okay so but you just said i think that you would characterize short turns as being the non interactive parts if if it is just some kind of back channel or it is just a signal signal the s the speaker is listening or is understanding what is going on yeah 0 okay well we we can discount back channels i mean c are you is it okay if we talk about this for a minute cause i am confused yeah yeah or or do you get a c i was just wondering if you were going to define this more on the slides to come or if this is as much as you are going to tell us alright huh alright i am just wondering if he is prepared cause if this is all we have got i think we should have interactive and non interactive on the board and then figure out what properties going to each just so we are clear what the 2 mean sure so so what did you have more on later slides about this yeah so yeah so for interactivity this is the only slide so yeah activity is i would define interactivity to be as a degrees of interaction in meetings okay well okay so which might speak change whi would be a very good indicator yeah but are you you need to be more precise what you mean by speaker change so y do you mean all of the speakers are involved in it yeah yeah yeah no no is it sufficient for 2 speakers right and then yeah no yeah so actually so this is a very rough idea of interactivity if we go further we might distinguish the interactivity between 2 f 3 or all the groups so that might be some final definition for the interactivity yeah but we have also established that it is not just speaker change it because if you have okay then yeah so that might be the 2nd class yeah okay but that is n you can that is definitional right weak this is a kind of weak interactivity okay y yeah yeah yeah yeah so so i mean you would not necessarily consider that to change the turns those are just back channels so one speaker will have the floor channel throughout that and so i guess what you are saying is that the interactive sections would have shorter speaker turns when you are not considering the back channels as interruptions right yeah yeah yeah yeah yeah yeah yeah yeah definitely yeah so well it might be the case that to be interactive there have to be dialog acts of some specific type in a turn rather than yeah yeah yeah that is also true yeah s yeah yeah actually cause i have looked well do y like statement statement reject accept you know something like that yeah yeah sufficient but not necessary maybe it depend if you had so a bunch of people all just giving information and opinion around the table and they never actually ask questions of each other okay but the turns were actually quite short which one would it be so for that case yeah yeah so this might be a weak definition of interactivity so if we could figure out if they are really interacti interacting with each other or just incremental presentation of some information so that is d different we might yeah need a more strict or finer definition of interactivity yeah ju for this use of the for the use of this term for the indicator of the degrees of interaction i think for for the current proposal i think that it is just for distinguish between dialogs and monologs cause for these 2 types of discourse the the the cons the the 2 that we are going to analyze the discourse structure might be d different right okay i remember you saying this before yeah yeah so basically you want to do the segmentation because you are going to apply completely different analyzes to the 2 sections yeah yeah and also cause i have read some of the m 0 t a corpus so for some part of th that kind of meetings several utterances or dialog acts are just tagged as statement so that i do not think that would be very informative for high level str analysis so that is all what that is why why i am distinguishing the 2 kinds of discourse here right so wei xun quite takes quite a strong position on this difference between these 2 kinds of group activities because he says that they admit different kinds of analyzes right the with the non interactive more monolo mono logic sections of the meeting you have to analyze them using something akin to rhetorical structure or something and with the other sections you you you use some other theory to look at them and i actually think that that is a slightly extreme position with regard to the literature that a lot of people might say it does not really matter who the different speakers are in an interactive section you can still build maybe the the same argument structure over the top of the basic acts and propositional content that is there and what you get is a messier version of the argument structure because not everybody has the same argument in their brain you know for this but but that the same theory or the the same analysis can be applied to both parts so i think it is important to keep that in mind and if you if you do need to argue that these 2 kinds of group activities can can only be analyzed if you use different analytical mechanisms then you have to explain why that is okay right so it it puts an extra burden of proof on the work that you do well actually i do not quite agree i mean i for example you can not use r s t across speakers because the definitions of the relations depend on the intentions of the speaker and there is different speakers have different intentions so you can not use the theory it is just fundamentally wrong it is it was meant as a theory for a one person right so that does not mean there are not other theo so for yeah the like yeah right but to the grou degree that a group is building an argument together i mean it i think wh what you are doing in a group discussion is not necessarily one coherent argument f for the person but an argument that is the group view yeah and so it is maybe it is not r s t but it is more akin to i do not know is it toolman but you know not that i but you know the kinds of things where people to try to represent argument structure right but those are very those are really different theories that are at are trying to get at different things and it and they admit of m you know some of them admit of multiple parties and some of them assume you are in one speaker is head right and the are are the important aspects of it for this analysis really that substantively different though yeah the entire theory is based on intention and the intention of one individual so r s t would be a bad theory to use but i if somebody if there has been an argument so far in a group meeting w and somebody makes a statement they have an intention about how to modify that group argument so modeling modeling a single contribution by a single speaker you can within that you can use r s t if you want to right but across speakers you can not right but but you are they have no relations for things like reject or agree they the theory does not have them okay so the r the relation names are not are not right but so the the so it is not the right theory cause the theory is the relations plus the notion of nuclearity so we need a different theory now i think alex s d r s allows both kinds right yesterday yeah might be a so you know within a speaker turn you can have things like explanation and cause and all that but then across speaker turns you can have things like correction or i did not see any examples of agree or disagree in s t r s but i there are not okay yeah no i do not think so i mean there are 30 relations i d so we should look at at what they are yeah but i d i do not well the relation names are less important i mean people have argued for ages about exactly what relations were in any of these theories yeah yes well her the relations have a s and that is that is less important than what the theory is trying to achieve and it just seems to me that if you have got to have 2 dif separate apparatuses for the speaker internal argument structure and the group argument structure that that is more appara apparatus than you should actually need for from doing this kind of analysis cause in both cases you are building up an argument publicly in this group meeting and then you know adding something or changing the existing argument in some way by relating a contribution to what is already on the table so d yes but so i mean the relation names it is not the names that are important it is their semantics that are important right and they are the semantics of the relations are based on the semantics of the utterances that they are relating as alex s explained yesterday yeah yeah and so it is important that these relations have a semantics that is tied to the semantics of the utterances right yeah c i just think well i mean it all of these things when they encounter a new genre of of actual data then they end up modifying building new kinds of relations with new kinds of semantics and adding them p poss possibly but few people have tried to you know that is the entire history of these do the kind of truth conditional semantics that you know asher and lascarides have done right yeah so the theory can handle the kinds of things that happen in meetings as she explained the actual computational mechanisms they have currently for building up the trees can not now i do not remember was which speaker you in the c in the example she showed yesterday was it the case that the representation of the utterances and the relations included which speaker was no just a turn exchange yeah i mean it had the at the l at the leaves they had the well she did not show it but they would have the semantics of the utterances and then she had relation names and handles yeah actually sh she used some a a a bunch of features to do the automatic parsing right i just i just do not did not remember seeing in the representation which speaker contributed what cause she did represent discourse structures cause these were the verbmobil scheduling dialogs yeah so the utterances in the in the representation were uttered by 2 different parties that was not the you know in the in the examples she showed us there were there was some that was just you know one party yeah it was a monolog but then there was a snippet of dialog as well with right so there you have got somebody who is doing theoretical development that would claim that you do not need 2 separate an analytical frameworks for these 2 types of activities that wei xun has identified for this specific set of relations that that they have defined yeah there might be some higher level structure that you want to pull out of that i mean i cause you were saying you are just going to you you are you want to look at this you know very local stuff yeah utterance to utterance yeah very low level yeah but from that you c one would hope you would pull out something high you know you cou yeah yeah i to just we need to see how well we could work out of the utterance to utterance no i think working from the bottom up sounds great yeah but i am just y i mean what you can imagine someone saying well how did they decide to make it blue and maybe there was a long blow by bl and you do not you kind of want to abstract away from he said this she said that he sai you know to anyway i you know i mean i i do not think especially yeah so yeah you know if you are not steeped in these theories that it makes a huge difference to you whether we use r s t or something like s d r s but it might make much more you know i mean yeah i just prefer it to be one thing rather than 2 completely separate things and then the added explanation of why it is not possible to have the same thing applied to both halves well i think we should ass i think we should try to find a theory that allows multiple speakers and relations to span over more than one spea you know yeah because it is true that i mean if you look at things like damsel and other attempts to come up with annotation schemes for for dialog acts there was this notion of forward looking and backward looking anyway they are relational i mean in some way you know they are all relational yeah but you needed but there is a different set of tags in there from the kinds of relations you see within r s t for example so if you look at amanda stent is thesis and the analysis that she did sort of within a turn she used something like r s t and then across turns she used damsel right yeah which would not probably would not give us the kind of information that we need about relations across turns just because we really are critically interested in the argument that the group is building up so that we can find the decisions that they made and any p counter proposals to proposals that have been made and things i mean it really is more in the spirit of the r s t ish relations than than in the damsel kinds of relationships what we want out of the c the relationships between speakers here they are jointly building up an argument structure and that is the critical thing that we are missing in order to be able to understand what is happening in these meetings so we would not we would not want to do the same thing as what she did in this case well i bec just because damsel does not give you the right sort of representation between speakers well it gives you things like accept and reject and 0 does it okay i have not seen it on that this kind of data 0 yeah so maybe in fact r s t does not give you anything like that no right okay but are what i mean is you kno want to make sure you get the argumentative relations however you do it okay whichever relations you have to build you know right not i am not saying i am not saying that there is a particular set of tags or relations that have been defined out there that are the right ones right i am just saying we need the correct ones for th the structure that we are trying to get out of this cause we need certain information out of this analysis right okay d so i think it is much more impor i think it would make more sense to go about this problem figuring out what kinds of relationships we want and then finding and that yeah you know like setting down what the constraints are on the analysis game well wei xun has been looking at the data so that is that is a step for this he has been looking at the data okay s to good and then and then there are lots of different theories of argument i mean you can also think about the sort of joint problem solving kinds of approaches yeah yeah yeah yeah so you know clark and lockbaum and you know that that kind of thing yeah that were designed for dialog right i mean and can be expanded to more than 2 persons actually the lockbaum stuff is for groups yeah yeah yeah yeah so yeah for interactivity this is a very weak definition yeah if we need more then we might use some more finer f finer definition for interactivity well what you will need in the end is something operational yeah so it is something that you can use to actually segment meetings into the the areas that you think are important for this analysis right yeah so it is not quite clear to me what the different kinds of things are okay so it would be i think handy if you could take a meeting at some and say you know this is a interactive area and this is a mono logic area because and we could come up with what kinds of measurements you need i mean we have w we have not done past work here on segmenting meetings in that way but we have done work on characterizing different kinds of meetings in terms of how highly interactive they are but it is a different definition right it is in terms of how much all of the people involved in the meeting take some take equal part in it as opposed to like the highest state is people in the meeting just driving it and and the other people following whatever the high state is people want so but those kinds of measures you know yeah we could use to to do this kind of segmentation if it is important to the analysis right yeah okay w but we need clearer definitions of what the different things are how many different kinds of of activities are there you know is it just interactive and non interactive or is there are there there more grades of shading than that and you know what are the things that you expect in each what is the back channeling behavior what is the speaker turn you know what kind of dialog acts do you expect i think that would help you pin down what the definitions are yeah okay so so for dialog yeah there would be some kind of interaction structure so it for in previous discourse studies especially in discourse analysis exchange is used to describe this kind of local structure and in conversation the analysis so adjacency pairs are used to describe this kind of structure but in these 2 strands or work they just come up with a list maybe not exhaustive so i am been think about if we could come up with some kind of more principal scheme taking some of the information in the in the interaction into account like is the small part of interaction is is exchanging some information here information is a little vague it could be some kind of knowledge what the speaker knows or or some kind of thought what the speaker is thinking so these 2 kind of inform 2 kinds of the information might be some kind of a difference so this is a little bit like the bell is exchange of information for the knowledge if exchange of opinion some kind of so it is very related to thought of thinking yeah do you think it matters whether something is knowledge or opinion in the way it contributes to the group process in some case it s differs cause when we exchange knowledge we might not challenge the speaker but when the speaker is pre is talk about what he feels what he thinks is proposing something or s then other speakers might might chan challenge him or her yeah so that kind of structu i i sort of have always taken the view that that well people are not very good at at this kind of rationality you know yeah yeah and the difference between information and opinion is not that prominent in the way people do their reasoning so yeah but that might b be some difference inter interaction par patterns might be cause for information if we somebody is seeking some information then he ask some question and the one knows will gives gives the answer but for the opinion or thought stuff then the speaker might just present some idea and others will comment or challenge the idea yeah well you you can not define em that way you can not define the difference between information and opinion that way because that is basically defining as the what happened about it okay and i think what tends to happen is that if people challenge the the facts you often discover that they were not actually solid facts anyway you know that they are i think the boundary between these is kind of fluid yeah yeah it is vague yeah yeah yeah yeah so i guess the question is is it important to what what is it important to so do you need the difference between information and opinion to do this analysis see cause so far i am s still looking for a so called principal scheme for the interaction for the interaction so so what have you been looking at yeah i was just going to say it would be useful to know what m yeah so yeah so you are you have got books that high on your desk yeah which ones are they yeah so yeah for p the discourse analysis they are using exchange but they have no no classification just to come up with okay so you mean but who is they that is the question who sinclair and coulthard that is right also at geneva school they i do not think they list some kind of exchange but the exchange structure and yeah exchange is 1st developed in systemic functional grammar along the halliday and and martin is work yeah so actually they they have some something in in a textbook about the exchange they call it negotiation so they have some flow charts of the detailed analysis of exchange structure yeah okay so y i am sort of getting that you think there is a lack in the theory to do with exchange structure at the moment yeah but i am but i am not quite clear yet what you think the lack is so so i do not think there is any kind of principle classification of exchanges this is i yeah w the well are you thinking it has been a while since i have read this literature but i mean my understanding of exchange is is that the initiating move introduces a goal to the discourse and then the remaining moves will contribute to either the the meeting of that discourse goal or the decision not to bother with that discourse goal right and so i would be surprised if there was not work describing okay things in those terms i mean i know sinclair and coulthard is is basically about teaching yeah yeah i r f kind of structure yeah yeah yeah so you would not expect that to necessarily apply in in wider contacts in a way like you might expect to have to extend the theory to do that but i th i think there are other other people have looked at exchange structure and described it in these terms in a way that should really suit your needs okay cause from my previous experience we see ananys analysis of information seeking dialogs over telephones so it is about the tourism information so from my analysis of that kind of dialogs i could say that there is only there only 4 basic exchanges depend on the information direction of the information flow yeah and also the speaker role cause in the information seeking dialogs the t roles of the 2 participants are different one is mainly the from the agency a tourism agency that is main information provider while the customer is the information seeker mainly yeah that is so theref and also the direction of information flows for some kind of information like the price of a specific route the spots they are going to visit and also on the kinds of ve vehicle or or transportation yeah yeah so this is a l a lot like george horton is work on saying what are the different parameters for these exchanges huh yeah yeah yeah yeah yeah yeah yeah yeah you know what are the things that characterize the set of exchanges that can happen yeah yeah s yeah yeah so but there also some variations which i call complex exchanges so that is that is just a list not like the basic exchange exchanges yeah so so is your complaint less about less about knowing what makes an exchange and more about knowing how you parameterise exchanges for this genre so what is the range yeah yeah okay so you are worried about what is the range of exchange types for this genre and that is what you think has not been done yeah yeah parametri yeah i i i am looking for parametric yeah scheme huh okay that is clearer yeah yeah yeah that is y yeah a parame parameter or principle yeah some something like yeah yeah and also yeah a and also s in cause in the meetings the speakers might might report something they have done in the past and talk about th what they are currently doing and also planning some future actions so this kind of information might also be helpful yeah so so have have you tried i mean i know of one previous attempt to establish what exchanges are needed for f some kinds of industry meetings and our experience in that project was that you can not really identify exchange structures in these meetings because they they do not really follow that pattern that exchanges are clear in well i there are clearer in when you have got fewer people so there is fewer people trying to pull the discourse goals in different directions yeah and they are clearer when your goals are quite limited so when you have just got information exchange and you know the or there is a clear task that the people are trying to do and they are focussed on that task as is normal in dialog elicitation like map task or like your like your informa which like your information seeking dialogs which was not dialog elicitation but it was very constrained in genre yeah yeah so the the answer to the question might be nobody knows what the catalog of exchanges is for real meetings because or for these decision making meetings because you can not identify exchanges right so have you gone through the meetings and looked for initiations and responses with without a prior catalog just knowing that init initiation in introduces a new discourse goal and a response meets or negates that goal in some way yeah so have you tried marking these exchanges not marking just for my reading of the i i s 1084 meeting series yeah just to reading this part yeah well i mean i think that w it i do not know w it would probably get to re hear you know more of your presentation because i think there are lots of different candidates and you certainly can not go through and mark up every meeting i mean mark up meetings with every scheme we think of so i think we have to kind of get an overview of what the candidates are m if there is like 2 or 3 that we think are good contenders yeah then we try to see what works better right yeah but but a reading of this you know th could tell you that there is no case where you are going to be looking for initiation and responses that is going to work you know cause you could go through and see that actually the pattern of of addressing different discourse goals is too dispersed to to be able to do that kind of mark up but i g yeah i mean i obviously you want to have a front runner scheme i in mind before you there are a lot of option there are a lot more options so so let us see what what else you have got okay so this is for the dialog part for monologs this is yeah so that is it 0 so that was it on dialog okay yeah okay so no further information is here okay so it is very unclear to me why you think exchange of information is more principal t did you i mean what did you have specific schemes in mind there or spec specific people is work that you cause so for the information and action the the action here is not dialog act but some kind of physical action that the participant is i i is going to or has done in the past he is going to do the future do you mean an action in terms of what the group is requiring him to do or do you mean an action in like picking something up so not so action in the technical meeting sense or just action in the physical sense physical sense yeah right okay physical sense yeah yeah so this kind of might be the some kind of semantics of the utterance or or exchange so this is just one thing i one parameter for but can you give us an example of a scheme that is based on exchange of information i mean here the scheme is for exchange scheme so yeah for for for exchange of information okay so there might be the s speaker and he a hearer so who speaks to whom and s so this is the so direction of the information flow and also so speaker role might be 1st parameters for the ex information exchanges so but for action that might be some kind of propose or accept or reject kind of exchange structure i mean 0 so that is not a physical action right if i pick this up which is a physical action you can not reject that huh no no what do you mean no that is that is the so we are going to do something and so could you do something or this kind of action related utterance i mean right yeah that is right that is a action in the sort of technical sense of meetings okay technical sense so you are actioned to do something which is part of the permanent record that yeah okay yeah yeah yeah yeah yeah yeah okay yeah yeah okay yeah that is n okay so i mean i think that there are a lot of things that we have to look at before i mean before come up with the scheme you mean yeah yeah i mean and well there are very few schemes for marking up exchanges on actual data right because people have not considered it very important exch well i always thought they get tired after marking up the dialog moves so they nev never bother with any higher level dialog structure so we did it in the map task but i do not actually know of other cases where people bothered marking anyth anything higher level than the map than the move structure right so yeah well we are marking up a lot of higher level things in our you do not know of yeah s stent yeah stent might does yeah on yeah i have s sorry stent yeah but it c it is not been very popular car probably cause it makes no okay so have you looked at nate blaylock is work so so cause i am i might not familiar with these na so he has things like you know adopt so basically it is to you know collaborative problem solving you know more than one person involved and trying to characterize things as you know if i i can not remember the i can not remember the names of all the actions so is it the work described in the c in in in the coconut project or something similar no so so the coconut project is a project that i did with rich thomasson and barbara diginio yeah yeah yeah yeah have you looked at that y yeah look not very much detail yeah not very clear no though there is a lot of detail in the paper yeah but you have not looked at it no in very much detail no not okay so i think you should okay because it was all these things that are done for joint problem solving and joint action i think are very relevant to look at because these people are solving a problem together yeah right so while these schemes may not have they have some es of the essential components that you need to characterize joint problem solving so while th you may think that they are missing something then we we could certainly augment them but i think that they have to be looked at and so so nate blaylock tried to okay so you know from gross and krauss and gross and sidener came all this stuff about shared plans and how you really can not understand you know dialog u unless you understand shared plans but the actual theoretical or formalization of it was incredibly complex yes yes and so lockbaum simplified it quite and you know sidener and rich and collagen have simplified it quite a bit but nate blaylock tried to you know define you know what happens after somebody suggests something well you can adopt it you can well i can not remember you know okay but he has he has a taxonomy yeah right yeah i know okay 0 i know this guy yeah yeah jim is student yeah i know yeah yeah no no in saa saarbruecken yeah yeah yeah exactly right okay yeah yeah yeah okay yeah i know her which we tried to apply to tutoring but it actually does not work very well for tutoring cause tutoring is just a totally different ball game different okay but it you know we spent a fair amount of time looking at it and it did strike me that for joint problem solving there w there were some good ideas and there is you know a set of tags and what they mean and you know that sort of thing and then you can also look at you know cl you know clark on joint action now he is not going to give you a set of tags cause that is not the kind of guy he is no but but the overarching idea you know i mean i think accounts for a lot of the ways in which you know these other people are thi are thinking but i mean i think that we we basically have to merge some of this stuff that has been done you know on meetings with some of the stuff that has been done on collaborative problem solving cause there is elements of each that seem really important yeah and i think th well the 1st step is find out in the what how much of the material does the collaborative problem solving cover so how much of a genre clash is there here because it really seems to me that you know if you look at and some of this stuff on exchange is like ancient and you know they did not have the ability to mark up lots of data and count things you know so i mean all the conversational analysis stuff i mean those guys never count anything you know no i mean they are ps so they see a pattern and then they think it is kinda cute and then they scour the world for 3rd turn repairs you know they do not actually say how often do those happen well it is i mean they worked i mean i know manny schegloff really well and they are not counters they are purposely not counters they define themselves as non counters yeah b th but there there are people in the field that are moving that direction like martin and also what is important here is the theoretical clarity about what an initiation and a response is so what is it that ties different acts together at a higher level you know yeah th i mean it it does seem like you know there is something that we want from that and then there is something that we want from collaborative problem solving and if you l if you take the exchange stuff it is not going to d quite do everything you want and if you look at the collaborative problem solving stuff maybe it will not do every that you want and so but i think more important than looking at whether parts of the meeting are highly interactive or not so interactive is looking at what sort of task the people are trying to do at the moment so you know are they doing i are are they i as in the kick off meeting just exchanging information to get started with a project or are they actually addressing some specific design criteria at the moment should try to decide which shape the thing is because that is going to affect the kinds of relations that you get between the utterances and so that is much more like the the literature from mcgrath about the different kinds of activities groups get into yeah so i think that is another thing that has to be pulled into this because you you do require different different kinds of relations or you expect different kinds of relations from some over arching set in those different kinds of activities yeah so main actions i guess might be some some high level problem solving or decision making concepts not like the exchange which which i i would like to be very low level utterance to utterance relations so what is the high level version again p s i mean i mean the m main actions might be some it might be yeah built built meeting action right okay i mean i the meeting actions is just going to be a segmentation it is not going to present any s any micro structure okay so well w so it is just going to say from here to here they are doing something that is all that that scheme is going to do okay like somebody is writing on the white board or i mean the w meeting actions they presented at the review seemed really low level to me like somebody stood up okay w it is n that is not no those well that is individual actions okay physical some kind of physical actions yeah and the meeting actions are meant to b become more semantic and more theoretical than the kind idiap have traditionally used okay okay so so we need to still preserve some sense of the activity okay but rather than saying it is somebody is at the white board it will be you know that somebody is explaining something that the others are focussed on okay okay i mean i i realize it is important to recognize that you know what wei xun got up and went to the white board i realize that is important you know yeah but i d was kind of surprised that those meeting actions yeah different scheme yeah so yeah okay it is it is a different scheme yeah so yeah th the the main actions that you are talk about i think might be yeah built up from the exchange yeah ex at least exchange might be one kind of important information for that kind of well you might need the meeting action segmentation in order to know where what what area you are circumventing for so it is just exchange structure in this area that you are looking at and then that is a solid boundary between the next bit or something i do not know yeah but w well time is getting on yeah okay so you know what practical things can we do to help this work move forward so y show what else next you are you are thinking about yeah yeah so yeah for the monolog i have i have not looked into the details about the this kind of relations for the monolog parts just a list of these relations and so next i need to review the literature and come up with s schemes for the real data annotation and after that we w cause this is not in the hub corpus so we are not going to get l all the meetings annotated with that kind of so so we we might ju 1st try a few meetings and have a reliability study if it is reliable enough then we must just just annotate a small part of the meetings and using some learning techniques to boot boot strap the automatic analyser well m marking up exchange structure if if it is there and if if you have got a working scheme for it is relatively cheap it is a lot cheaper than doing dialog act coding in the 1st place yeah if if it is based on the dialog acts yeah right well one presumes it is otherwise it is going to be ra radically expensive to do any kind of mark up so the you know it is something that we can look at but you need to what we need it is more important than knowing you know that we need exchange structure marked up on x amount of data at this point is knowing where you are headed with this i think so what is the good that that the project gets out of this in the end so you know what is the w what is the headline results of the work on the project i think is is what we are trying to get a sense of here yes so you get your exchange structure marked up and say we just put in the money get exchange structure marked up on that entire hub or you know figure out how to boot strap it so we we get this how do you how do you sell that as a as a project goal you know cause people are going to say exchange structure so what what do wh what do you what how do you plug that into the rest the stuff right or fits the meeting browser yeah yeah okay so this might not be the final goal that that could be fit into the browser but it might be some kind of information which might benefit meeting actions or some more high level decision making or problem solving cause this is the building blocks i guess yeah so yeah see cause i think that just to make this m more efficient you should be thinking about the high level goal in order to focus the work that you are doing rather than thinking well yeah you know there is these theories you know how can we apply these theories to it and then seeing what comes out of that that might be useful for the meeting browser actually trying to say yeah okay we have a goal of being able to do this with the meeting browser what kinds of dialog structure would we need to be able to do that i mean i do not know maybe you would want to disagree with that but i this more goal directed might actually make it easier to bring these things to publication in the long run right well i think we should approach it both ways i mean you know i i d you know the meeting browser people have a lot of constraints and so what they end up with saying is most critical for them right now might not be the most interesting thing in the longer term right and so it is not necessarily what they say but it is something at least identify something that we would hope to get out of it you know right i mean people have said over and over again that they would like to understand how a decision has evolved right and i think that well i do not know if all of this low level stuff would allow you to track that kind of thing but it might at least be the building blocks of allowing you to track that kind of thing and yeah yeah so after this if if we might do some pe from here we might be some yeah some higher level yeah yeah c problem solving decision making process to track that kind of information yeah yeah so i mean i think it is clear that something at this level needs to be there i mean and to ask questions like who disagreed or were there any counter proposals or whatever even those kind of questions you need some kind of mark up like this i it just helps you know which theory you should be concentrating on so you know that immediately discounts exchange structures that are mostly for information seeking alone because the notion exchange is to restrict it there and in favor of these these more argumentative things so i mean i do think you should keep in mind what the final goal is so that you can plug this into something at the end and and show some benefit of having the mark up yeah well that brings up something interesting because i think probably what we should do is try to get you know some of these tasks sort of put on you know sort of as part of the official agenda so being able to answer questions like were there any counter proposals to the bana you know banana shape or whatever or to some specific you know recommendation and things like that i mean you know those you know strike me as cl clear things that you know people who go to decision making meetings want to know if they have missed one or are arriving late or anything like that yeah so it wou it seems like we should probably get some of these things on the official agenda and say look you know would not it be nice if you could have answers to these questions or something yeah i mean operationally it is not going to much difference whether it is hub or not right because we have got the bank of people 0 we have to get the hub stuff through but you know he is n not ready with an annotation scheme that can be used at the moment anyway so and it it is not that expensive no m so i mean it is something that we can afford off budget cause and i think we would not want to necessarily do this on icsi data i think the icsi data is too too complex and there are not enough real decisions made you know that you could clearly identify messy yeah it and it is too different a genre yeah yeah yeah i mean there are some but and not that i have read more than about 2 meetings ye yeah but i think w it you know w well so one thing that is going to be interesting is the i think to me yeah is the what can you do because presumably the d f k i folk or somebody is going to mark up a lot of the data with sort of tags about ontology right so we know you know somebody will have gone and by hand and sort of marked up som something l like shallow semanticy kinds of things right or maybe deeper semanticy things but that but you know yeah that is not help either but yeah they are sure to do that not really in a in a formal language but mostly just you know tags right and then the question is well how does you know just the way people typically do this is you know let us take these semantic features into or these features into account and see how well we do with the the learning and then let us throw these features out and see how well we do because we know we are going to have them because one or 2 sides is really intr are really interested in this ontology stuff right i mean t n 0 is and d f k i is right so there will be some kind of domain semantics and there will also be presumably some kind of task hierarchy modeling going on in i mean cause the whole browser working group final sort of presentation on that 2nd day was about in the m you know knowing the task right and very you know details about which task is this you know which sub task is this little chunk of the meeting addressing and furthering in various ways right y which seems like it would be really hard in an open domain well i do not remember that about the w i mean the you remember when steve came in steve whittaker came in at the end and said you know they would spent the morning talking and they came up with this idea 0 but that was about what task the person doing the browsing was going to do it w they did not say anything about what sort of information they would want from the meeting itself it w it was about i they just mention that they want decision for issues action items i think they are more talking about user requirement for the meeting than soft text they want to know no they said people are talking about tasks you know what happened happening in a meeting i mean they really they made it clear that they were that that was going to be heavily that was the that we the browser task you know they have identified these 7 different tasks people might want to do with browsers you know like join a meeting late and ch catch up on what is happened in the meeting so far be at home waiting for the washing repair guy no those are the s those are the scenarios and they say that those are the user scenarios the task what what he came back and said that at the end yeah i mean it actually kind kind of scared me because it was all based around tasks okay well c maybe and and the domain tasks okay they are now they now have to decide how many buttons to put on this thing i mean it was all about the domain task okay and it is really the it and given that it it is really scary to think about what you could possibly do in an open domain where you do not where you do not know what the tasks are and i do not think you are going to be able to figure them out very reliably but i thought you wanted to avoid doing stuff that was s stuck in this particular domain if possible but you know th they were not concentrating i do but the whole project is you know focussed on ami hub data as their 1st priority well yeah but that does not mean you have to be using the techniques that you can be using open domain techn n no no no no no but i mean if the if the if they had th they had this sort of epiphany at the end of this meeting where they decided well why are people browsing this well it is all to see how you know how the participants were furthering their tasks and you do not think that can be done in open domain at all cause we do not want to get into working with ontologies i mean one of the strengths of this grouping was that yeah they all work with the closed domain ontologies and we will do the open domain side of things so i do not know well right and then yes we will see and then there was this confusion about the last spider phone conference that the messages telling people to call came out after the meeting happened at least to some of us 0 right yeah so we do not know what happened at that one yeah so i mean we will we will get that back on track but i think that we do not you know there will be some meeting browser demo that you know i i think it is an interesting question of okay well if you have knowledge here is what you can do if you do not have that knowledge you have to do different things and you know you can not necessarily yeah so you are advising wei xun that he should be using like any ontologies and propositional content that d f k i and t n 0 bank in the c v s repository or or not yeah ma ma s well i think when he does re when he does lear these learning experiments he should do em 2 ways that is what i am not clear on because he will have this opportunity with or without with our without and then we can see like well without you know these tags so you know yeah that is kind of okay alex was showing yesterday well without knowing the head word of the utterance at the leaf you know you get these kinds of results and when you add that in you get these other kinds of results yeah yeah yeah and you know i think yeah different feature combination yeah so i think if we have those features it is you know it is a rare orp or rare opportunity that you can you know you can try one head to head against the other can actually try yeah and then of course to make sure that you really i do not know when they will bank these by the way cause that is it is not hub so i it is one of these things where everybody is supposed to bank as many things as possible right so you know w well we will not i mean we will not use em till they are ready obviously so we might do the sort of open domain i mean a although it will be a closed domain i mean we we can not use any domain information necessarily yeah but open i mean there probably are a bunch of issues like if you have got a 100 hours of meetings and they all talk about a lot of the same things yeah it is not really an open domain task yeah then it is ea it it is not really open domain and somehow that sneaks into all the statistics anyway cause you do not have such a wide vocabulary and you know well eventually the hub is supposed to be 40% non scenario right but that data is definitely going to be kind of coming in the last so sure so we might find that when we move from what we thought was open domain but on the hub you know our results go down when we truly are getting meeting yeah yeah that is quite reasonable so i mean that will be interesting w we have got 5 isca meetings actually which are still an elicitation task but they are not our domain so that should help some okay well that is good and they have been declared hub so so they will accrue all of the normal an annotations of course yeah you will you will never get the t n 0 and d f k i stuff on it cause they are only billing stuff for the one to make the so okay so we want to decide what wei xun should do next and maybe think about meeting in i do not know a month or something yeah just figure out how we are going to help him to to try to make this more efficient yeah so before we do that can i just ask you a question about chunking yeah yes in the beginning you said something about you were doing this chunking and you are doing it on the icsi data but this was did you say it th was not picked up in ami or you i i i was confused about w yeah yeah i mean i was supposed to chunk some part of the ami meetings you know to see if the chunker channel switchboard works well enough or not if not then we have to use some kind of semi superv supervised learning or some boot strapping to to to do the chunking and you and you have not done it yet no no cause i was a bit worried about this kind of discourse structure yeah tha that is fine so that is still on the agenda to do at some point yeah yeah but we should make sure we finish that off because otherwise that work is just wasted right yeah exactly yeah we are going to use chunks for something useful that is okay yeah chunks are going to be named entities if nothing else well so i mean right or building blocks i mean so i mean we have to think about how s chunking interacts with the automatic segmentation you know because barbara was showing the low level automatic segmentation at which level dialog segmentaton huh dialog act yeah yeah okay and i mean it seems to me that chunking should have some kind of so if if your dialog act segmenter says there is there is a segment here yeah but your chunker said no this is a coherent chunk then y y you know w i mean maybe they could work together to do a better job than yeah and that is a problem yeah certainly if if you have got chunks it ought to help you know where your dialog act boundaries better not be right yeah yeah and it may be even more mechanical than that based on the examples we are seeing from d f k i cause they are they are hand marking stuff and the dialog act boundaries are supposed to be intentional but they seem to always be marking them that after verbs you know with clause m it is very it looks quite mechanical periods 0 after right so we are querying them about this because that is not the way they are supposed to be doing it okay right okay and good so it sounds like well w well hold on but when do we need chunking by and it you know so y i do not like the idea that you know you start something and then 18 months later you pick up to do the last 10% of the work to finish it off or whatever so it i like things kinda tidy right so how much work is there left and when do we actually need it by would be useful to discuss well how much ami well you wanted to do the chunking on the ami data right yeah because m the ami data is is there there is i mean there is hub set a is transcribed i think in final form you we can check the wikis but you know the data is rolling in so so we got stuck at some point on evaluation of chunking right the problem was you did not have icsi data that was hand labeled for chunks or ami data n no and we decided labeling the icsi data for chunks would be probably difficult and maybe not very useful and so it it mi and but barbara still says somebody is done this for chunks okay i will have to look up you know so so okay if there if there was if there was hand labeled data that you could evaluate again so train on some and evaluate on some would not that cause you have trained on switchboard right where there there are chunks so if you had some icsi data that you could maybe use i do not know yeah as a what is the n there is t there is a training set and then there is the development set and then there is the test set maybe you would use some chunked icsi stuff as development or something and do some last tweaks and then evaluate but you know w h i mean i can see why wei xun did not want to spend his time chunking icsi d which means we have to get some portion of the data hand chunked hand chunked but that should not be that hard to do right how much data do you need how much well i do not read lips what was that was it hard how much in switchboard switchboard is all chunked yeah yeah right they the data how much data you need was not a problem with the switchboard cause he had lots so he did not have to think about how much so we need to know how much data you need hand chunked right well and then how do you hand chunk stuff right so you were were working on instructions for doing hand chunking no you had penn tree bank parses and you made chunks from that for switchboard yeah but then he there were 2 possible options for for doing this on the ami data yeah yeah for switchboard yeah you could either essentially go the same route as they did with the switchboard you know do the penn tree bank and then create the chunks off that or you could chunk directly and it was not clear to us which one of those 2 methods was the right way to do this direct chunking cause we are not going to in a tree out of the ami data right well so you would have to produce as part of this exercise is anybody going to parse is any going to try to anybody try to parse the ami data you would have thought so but i dunno so th we d f k i cause they are working on some h p s g parsing 0 yes yeah i would forgotten about that yeah might be might be yeah if the but that is not very similar well i do not know very much about grammar is th is that similar enough that that this same method could apply that we derive chunks off the h p s g output if i am i guess that if they are different then we might just write another script to do the translation i think you might lose a lot in you know this step so basically your your chunker is trained on something that was derived off penn tree bank and it might be different enough that that actually causes it not to match you know if you are suddenly evaluating against chunks that were done using h p s g i do not know about grammar but it you know it okay yeah yeah i need to look at some of to see well they are using the e r g right the which probably has a l maybe finer g which i believe has finer grained distinctions than and different labels than the penn tree bank yeah and the mappings are can be cr pretty difficult for this right or is is easy to map between finer grained distinctions and coarser ones it is i am t it is probably easy to dumb down the h p s g parse too think so cause the penn tree bank parsers are pretty minimal because they did not want to commit to a specific grammatical theory yeah yeah yeah so i mean it probably is worth looking at okay cause they are doing this with dan flickinger right which is the same parser as the redwoods tree bank was made with that alex talked about yesterday so you can look at redwoods parses and see how different they are from tree bank parses and you might want to talk to alex to find out if somebody is already done this turn redwoods stuff into tree bank stuff so that they can compare you know against results that are reported on tree bank huh the names would suggest that this was the idea right well that was the idea i believe but yeah it was supposed to be another tree bank right so y that sort of assumes that we think it is better to do this by deriving chunks from some parsed format than just hand chunking the stuff well the good thing about that is that then you know that when you go through and parse you will be getting the same chunks as when you do this process automatically cause the problem w when you go through and hand do it is then suddenly you introduce your automatic method and your accuracy goes can go way down i mean sometimes it does not if you know sort of for part of speech tagging you know whether you do it by hand or machine it is pretty much the same accuracy right yeah but so let us try it this way and then that that puts off the question of when do we have to do this to when we get h p s g stuff right so can you ask d f k i whether they can give us a couple ami meetings run through their h p s g process i do not even know what time scale they had in mind i mean it is this something they w they were doing it at the moment but i do not know whether this was very preliminary or do you remember they talked about it at the meeting that we went to in 20 minute yeah twente twente yeah i was thinking enschede why can not i remember the n the f the right name of this place yes twente contact which you were there jean yeah yeah okay yeah y you are expecting me to remember stuff are not you no no no no no no no i i i know you missed a meeting and s and so there there may have been you missed the one in t n 0 0 yeah 0 i do not even know which one i have missed brussels amsterdam t n 0 okay t n 0 yeah see i was not there so i do not know where it was okay yeah cause yeah they did some work bio project with the h p s g parsing yeah at least bio on on which project sorry i did not get that verbmobil right yeah well maybe yes yes they did yeah okay yeah so ask em right it well that is that is what alex talked about yesterday right okay the parse i mean that is was not something she did but that is the data she is using right so we have all that stuff here you know the parsed verbmobil material so you can look at that and see what the parse trees look like you do not need to wait for d f k i no and if it turns out that you think 0 there is no way i can map these into penn tree bank style bracketing then then we have to think of something else right okay okay but i think they had that in mind so it is it is worth also talking to her about that okay good so it is talk to alex see if it looks like it maps by looking at the verbmobil if it does ask d f k i either for data or a time scale and you are going to need to know how much data you need in order to do that so do you have a technique for estimating how much data you would need yeah not sure c i do not know how you do that so who knows how you do that everybody always says there is no data like more data okay so just ask d f k i how much data and when right okay if you know i mean the typical thing is to do leave one out cross validation with you know 89 dialogs yeah yeah i know yeah okay so i trained on 88 and then or i train on 1999 and okay but we should speak to steve okay we do not have any particular time in mind when we need a chunker that works on this stuff we might have some so well the question the question is what what is chunking going to help i mean i think chunking could help automatic dialog act segmentation yeah i think chunking could help as you mentioned named entity recognition yeah well i gue if we did a chunked version of the ami data and stuck it into c v s under the automatic contributions then people would end up using it cause they do that right so the question is at what stage should we put that in well we would want to make sure we are pretty happy that it is reliable because yeah so post evaluation so we can put in some idea of what the evaluation is like yeah i mean otherwise you get these very bizarre results because the data is not really very good so but i and nobody has expressed like massive interest in t in chunking so i think you know nobody is waiting for it no but he is relying it on it th this summer but but we could we c i think we should talk to barbara about how to do some joint experiments or or if she is going to be sending other people maybe liz or whoever comes over next time and it would actually be great if it was liz because i think she is the one who did the dialog act segmentation automatic so when when do you think that would be when when is she sending somebody she just said that hopefully by the next meeting she would be bringing someone else or someone else would be coming instead of her remember 0 for the next one but that is a pretty tight time scale then right cause it would be for the fall we would want to know the evaluation is that the time scale you are thinking of or the th not to have not to have it done but just to talk about how it could interact with automatic dialog act segmentation how it could be done yeah and we i guess we should talk to yoshi about about it in relationship to named entities who is who is doing named entities besides yoshi that is all i think okay so we could talk to vasilis about that as well i think you should still ask barbara again you know c can you have another think about who might have the chunking stuff cause she has said several times there must be somebody and she can help with that and i think she is just forgotten so and then so and then we can evaluate on the icsi you know pretty easily so one thing that would be very helpful for me because i have not read i think we are in a situation where jean you know has read a lot of literature and i have read a lot of literature and we there is some overlap but there is also quite a big space where we do not know the literature from the other side and since you are going to be reading it i mean it would be probably be useful if you made some kind of you know report that described you know okay this what these people you know this is you know a synthesis okay here is the basic idea behind exchange structure and you know here is a couple of you know here is the 2 most relevant works that i have read or something like that you know to just yeah even if it is just a you know on a wiki the the references and then why they are relevant right i mean you do not want to make this really formal or do you or even just like a paragraph of you know the basic idea yeah a paragraph each just because you know obviously i am not going to go rea off and read all this kind of stuff because i do not have time yeah and you probably do not have time to read all this collaborative prol problem solving yeah but that wou that would help us be a lot more specific ab than than just you know so listing bullet points on slides cause then we would know sort of which literature you are thinking of putting in and we could read the most important argumentative stuff okay yeah yeah yeah well in any way every time you write a paper you have got to go back and do this related work thing anyway i can have these things anyway yeah so it just it just makes it a lot easier to get that yeah common ground i think you know kay yeah yeah i think it is a real advantage that we have read different things and come at it from a different side because i think this ha they the the previous approaches have not really you know taken you know the good features from you know both sides and and tried to put them together they have not addressed this problem yeah yeah i am going to have to go very soon because i have got this this hearing to go to yeah me too but so the practical actions you know what are the things we are going to go away and do i can send you a l yeah well i think we both want to send w well i do not know maybe you have already done this but i you know i will send you or we can you know chat off line for a minute a about a list of things that i think you really need to take a look at yeah yeah and jean probably maybe you should do the same okay s so then later yeah okay and if sabrina has any ideas we would be happy to hear from sabrina as well yeah right now d i am doing i am trying to do the meeting action segmentation j because after talking to jane just we have different ideas whether meeting action will be about the same size as topic segments okay so it would be good to t right now i am doing this yeah looking at the data and trying to see if i can do the meeting action segmentation 1st and the scheme m and then do topic segmentation again to see if the boundary actually coincide with each other yeah if if that that do then maybe we can talk again whether i it it make sense yeah yeah so you can do this exercise and then come and talk to me some time next week i am not in monday but the other days i will be around alright yeah i have just started and so far i s think it is it i think it it is a still about the s about the same level of segmentation we are talking here yeah but maybe the boundary will not be exactly the same yeah s yeah so this this is interesting but i w i do not think we can afford to l look at this right now but that that the segment right sure so why do not you bring it by next week yeah and i what i really want to do concentrate just now is is finding out the how we are going to help wei xun with with making progress on this sure yes so we have already got you know he is going to construct a list of the references he thinks is important and johanna and i and and you as well are going to contribute to that and we can put that on a wiki or we can do it some other way i mean you know i am wiki mad okay so if we share it it should go on a wiki cause i can not find anything in my email but you know you do not have to do that yeah it is yeah and then how do we you are actually so what you are doing is taking all of the the the most important references for finding argument structure that might i eventually be of some use to a meeting browser and trying to look at actual data and apply and and see like which one helps you figure out what you are actually going to do with this particular data right so it seems to me that we should take like the front runner scheme and the data and you should look at not front runner scheme but front runner theory and i see if you can speak to the data you know using that kind of theory at some point and might might be an interesting exercise right well i think we need to like you mentioned earlier factor in what this might be used for in the end and so even if it we will not it will not be used maybe if it would need some higher level analysis before it could be directly used for that purpose we should still keep that in mind the update and so i think going through the presentations that were given at the work package 6 meeting where and i think it was it was not wha was it anita or was it 0 i have forgotten her name the other woman from twente you know they talked about they did they had done a lot of questionnaires and they got some user requirements out of that and then steve presented the results of following people around and looking at how they used their notes from meetings and and so that would give us some ideas about you know what people want you know out of meeting records but then we should also add things that we think are interesting yeah but that is a good starting point yeah but just to have to keep some of these goals in mind cause they are t 01 scheme might let you answer this set of questions but would not help you at all with another set of questions that will be the exactly yeah so you want to go through basically the user requirements and you know maybe think about some user requirements of your own and make a case for you know these are the kinds of things that we would want and look you are going to need this kind of dialog or argument structure for it so should we expect that all the slides from like the work package 6 meeting or the joint work package 5 work pa will be u up on one of the work package web sites and if not how do we make that happen i will think about that okay some of this was presented in at least the one spider phone or the 2nd spider phone telecon i went to for work package 6 so if they are not on the web we can try to look at that you know at the materials that were presented in that telecon without necessarily listening to the whole telecon yeah yeah so we are just trying to get records wh i mean i am sure they are not on the web yet and steve m and his i have forgotten the name is it was it simon his yeah so yeah they they have i think they may have written this up in a report yeah but i wou would hope that is on the work package 6 web site as well yeah okay so so we are going to go through the requirements like that and did did you think that is that was reasonable to after that to try and actually you know take some theory that seems fairly reasonable and think about the particular data and come and talk i do as i once he is read a couple more i mean he is just completely missed out the collaborative problem solving world yeah i there is a few references missing yeah which i d i really think we need to take into account i mean you may take it into account and say no you are wrong and here is why but i i i am going to need some convincing yep so then try front runner scheme on well a front runner theory or t or or 2 and see this one gives you this and this g and can not you know are they fundamentally incompatible or can we yeah yeah but try it informally yep when do we get together again cause i really do have to go okay i promised these guys i would be there i think this is going to take about 3 weeks cause he has got plenty of reading to do huh but w so maybe we get together again in the yeah i mean i will send them the references immediately reading lists before that though so right so we will get together in about 3 weeks okay is that enough time for you do you think sorry you mean is that enough time for you to to read and think about i will try my best yeah if y if if it is too tight you know i think m we can say 4 0 yeah i m i might yeah cause julie and the boy is here or are you going to be on holiday so it is a little tiring cause my parents have left yeah yeah huh so i might take 2 weeks off 2 weeks off in may yeah sometimes yeah maybe 4 weeks okay okay so we should get together probably in about 4 weeks yes off yeah 4 weeks yeah okay thanks so when are you going to bring this beautiful baby in to see everybody yeah hope i hope so very soon i want to see yeah yeah good good something that is great sure right i am going to head off yep well good luck solving your problem okay thanks c cause floods are more important than dialog structure i am afraid i have to tick and tick it does something weird when you do that
5 years just because of the floods bringing the ceiling down okay great and it is great joy are not they suing him i think they probably ought to but they well they have only just started realizing that treating this as a legal issue is probably the right way to look at it so they are not really the sort of people that are up for understanding the intricacies of the committee i guess i guess i mean you would have felt they could but it is actually quite hard work the law is not very clear can not they get legal aid that is a good question they might be able to it is just it is just strapped now you know like the citizens advice bureau is so hard to actually get access to you because they have so many cases and so few people but yeah that is probably the next thing they will have to do yeah there is a thing that thing are we supposed to count to 10 or something we do not have to do all that okay okay yeah so just like the last one okay okay okay okay okay okay okay okay right well i thought the right thing to do was to review what weeching has been doing in the area of dialog structure for work package one over the last period so basically for him to talk about what progress he has been making and then for us to for him to describe what work he intends over the next period of the project and for us to figure out what help he needs with that maybe give him some help figuring out what publications he is going to get out of this and for us to make sure that it fits into the overall project so this is something that you know we are not going to do everything today but i wanted to make sure that we sort of all aligned on what he was doing at the moment and then sort of figured out how to give them the best advice coming up so that is why i called this okay thanks you are going to give a presentation for us then yeah yeah great very quick slides so i have been here for 14 months i think yeah so i have accustomed to the environment here and enjoyed staying here so from last year i was supervising the annotation of segmentation and some relation with gene ex acus and on the other hand i did some very preliminary work on chunking the switchboard corpus but that was not well picked up in the ma corpus which is supposed to be so and currently i am more interested in the analyzing the discostructing multi party meetings this is what i am going to present here so in this presentation i am going to see how we could analyze utterance to utterance relations in multi party meetings because meetings is a little different from normal text or a normal dialog or mind log so interactivity is a concept is a concept that i am using to distinguish the 2 extremes of discussing meetings and for dialogs and monologs i think we might have to use different concepts to model the discourse structure then i will give a very rough plan for the next couple of months and finally i will introduce some related work so the big question that i am going to be concern here is what is the relationship between utterances in multi party meetings or how do we analyze discourse structure right above utterances instead of those high level intentional structure of discourse so the concern here is about the very local low level utterance to attrance relations in multi party meetings this is the local discourse structure so this the building block of such kind of structure of basic utterance and possibly one of its properties the dal act which we are currently working on and for meetings yeah so i have mentioned before that meetings is a bit different from monolog and dialogs so in meetings from my reading of the one of the m meetings is 108 the 4 meetings i noticed that the interaction degrees of interaction in the meetings varies from part to parts parts so for some part the interaction is very strong so it is obviously a dialog but for some part where the interaction is very weak just like the chairman or project manager is coordinating the meeting is pushing the meeting forward then this kind of organizational structure or agenda falling part and also other parts like the back channels without any semantic content from other participants so this kind of interaction might be better analyzed as monolog which in which there is neo interaction i suppose so i am a little bit confused so when you say interactivity do you mean well how would you characterize a highly interactive meeting what are the measurements you would apply so i think what you are doing is segmenting meetings into highly interactive parts and not so interactive parts where you think some of them are like a bit like dialog and some are a bit like monolog so what measurements would you apply to find out which is which yeah is going to be based on the utterance to utterance so for this utterance the next utterance if it is from another speaker then if it is not a back channel or something similar then it is dialog so it is speak change so you mean relatively short turns and speaker change yeah yeah but short turns like or something very similar might be the 2nd type of interactivity the weak interactivity that might not be analyzed as a monolog but as a dialog not as dialog not as dialog but as a monolog so for that part for that kind of back channels we can just skip that and do not treat them as interruptions or something like then yeah we could just so am i right in thinking you think that you can segment meetings maybe into 2 types of areas interactive and non interactive so what are the characteristics of the interactive areas speak change speak change would be okay so but you just said i think that you would characterize short turns as being the non interactive parts if it is just some kind of back channel or just signal the speaker a is listening or a is understanding what is going on we can discount back channels i mean are you is it okay if we talk about this for a minute because i am confused i was just wondering if you were going to define this more on the slides to come or if this is as much as you are going to tell us i am just wondering if he is prepared because if this is all we have got i think we should have interactive and non interactive on the board and then figure out what properties go in each just so we will clear what the 2 mean sure so what did you have more on later slides about this for interactivity this is the only slides so yeah actually i define interactivity to be the degrees of interaction in the meetings well okay speak change would be a very good indicator yeah but you need to be more precise what you mean by speaker change so yeah do you mean all of the speakers are involved in it no is it sufficient for 2 speakers right no yeah so actually this is a very rough idea of interactivity if we go further we might distinguish the interactivity between 23 all the groups so that might be some final definition for the interactivity but we have also established that it is not just speaker change because if you have okay yeah so that might be the 2nd class okay but that is you can okay that is definitional right yeah yeah so i mean you would not necessarily consider that to change the turns those are just back channels so one speaker will have the forward channel throughout that so i guess what you are saying is that the interactive sections would have shorter speaker turns when you are not considered the back channels as interruptions yeah yeah yeah so well it might be the case that to be interactive there have to be dialog acts of some specific type yeah yeah yeah that is what is true yeah yeah actually because i have looked to well like statement statement reject except you know something like that yeah sufficient but not necessary maybe if you had a bunch of people all just giving information and opinion around the table and they never actually ask questions of each other okay but the turns were actually quite sure which one would be this might be a weak definition of interactivity so if we could figure out if they are really interacting with each other or just an incremental presentation of some information so that is different we might need a more strict or finer definition of interactivity yeah for this use of the for the use of this term for the indicator of the degrees of interaction i think for the current proposal i think that it is just for distinguishing between dialogs and monarchs because for these 2 types of discourse the 2 that we are going to analyze the discourse structure might be different right okay i remember you saying this before so basically you want to do the segmentation because you are going to apply completely different analyzes to the 2 sections yeah yeah and also because i have read some of the mfta corpus so for some part of that kind of meetings several utterances of dialects are just to tag us statement so i do not think that would be very informative for high level analysis so that is why i am distinguishing the 2 kinds of discurses right so waging takes quite a strong position on this difference between these 2 kinds of group activities because he says that they admit different kinds of analyzes with the non interactive more monologic sections of the meeting you have to analyze them using something akin to rhetorical structure or something and with the other sections you use some other theory to look at them and i actually think that that is a slightly extreme position with regard to the literature that a lot of people might say it does not really matter who the different speakers are in an interactive section you can still build maybe the same argument structure over the top of the basic acts and propositional content that is there and what you get is a messier version of the argument structure because not everybody has the same argument in their brain for this but that the same theory or the same analysis can be applied to both parts so i think it is important to keep that in mind and if you do need to argue that these 2 kinds kinds of group activities can only be analyzed if you use different analytical mechanisms then you have to explain why that is so it puts an extra burden of proof on the work that you do well actually i do not quite agree i mean for example you can not use rst across speakers because the definitions of the relations depend on the intentions and the different speakers have different intentions so you can not use the theory it is just fundamentally wrong was meant as a theory for one person right so that does not mean there are not other theories but to the degree that a group is building an argument together i mean i think what you are doing in a group discussion is not necessarily one coherent argument for the person but an argument that is the group view yeah and so maybe it is not rst but it is more akin to i do not know is it toolman but you know not that but you know the kinds of things where people to try to represent argument structure those are really different theories that are trying to get at different things and they admit you know some of them admit of multiple parties and some of them assume you are in one speaker is head right are the important aspects of it for this analysis really that is substantively different though the entire theory is based on intention and the intention of one individual so rst would be a bad theory to use if somebody if there has been an argument so far in a group meeting and somebody makes a statement they have an intention about how to modify that so modeling a single contribution by a single speaker within that you can use rst if you want to right but across speakers you can not right they have no relations for things like reject or agree the theory does not have them okay so the relation names are not right so it is not the right theory because the theory is the relations plus the notion of nuclearity so we need a different theory now i think alex sdrs allows both kinds so within a speaker turn you can have things like explanation and cause and all that but then across speaker turns you can have things like correction or i did not see any examples of agree or disagree in sdrs but i do not think so there are not i mean there are 30 relations so we should look at what they are but the relation names are less important i mean people have argued for ages about exactly what relations are in any of these theories well that is less important than what the theory is trying to achieve and it just seems to me that if you have got to have 2 separate apparatuses for the speaker internal argument structure and the group argument structure that that is more apparatus than you should actually need from doing this kind of analysis because in both cases you are building up an argument publicly in this group meeting and then you know adding something or changing the existing argument in some way by relating a contribution to what is already on the table yes but so i mean the relation names it is not the names that are important it is their semantics that are important right and the semantics of the relations are based on the semantics of the utterances that they are relating as alex is explained yesterday and so it is important that these relations have a semantics that is tied to the semantics of the utterances right yeah i just think well i mean all of these things when they encounter a new genre of actual data then they end up modifying building new kinds of relations with new kinds of semantics and adding them that is the entire history of this possibly but few people have tried to you know do the kind of truth conditional semantics that you know asher and lascarides have done right so the theory can handle the kinds of things that happen in meetings as she explained the actual computational mechanisms they have currently for building up the trees can not now i do not remember was which speaker you know in the in the example she showed yesterday was it the case that the representation of the utterances and the relations included which speaker was no just a turn exchange i mean it had the at the leaves they had the well she did not show it but they would have the semantics of the utterances and then she had relation names and handles actually she used some a bunch of features to do the automatic person right i just did not remember seeing in the representation which speaker contributed what because she did represent discourse structures because these were the verb mobile scheduling dialogs so the utterances in the representation were uttered by 2 different parties it was not you know in the example she showed us there were there was there was some that was just you know one party it was a monolog but then there was a snippet of dialog as well right so there you have got somebody who is doing theoretical development that would claim that you do not need 2 separate analytical frameworks for these 2 types of activities that weighting is about for this specific set of relations that they have defined there might be some higher level structure that you want to pull out of that i mean because you were saying you are just going to you want to look at this you know very local stuff but from that one would hope you would pull out something high you know just we need to see how well we could work out of the artranos no i think working from the bottom up sounds great but i am just i mean you can imagine someone saying well how did they decide to make it blue and maybe there was a long blow by blower and you do not you kind of want to abstract away from he said this she said that he said you know to anyway i do not think especially you know if you are not steeped in these theories that it makes a huge difference to you whether we use rst or something like sdrs but it might make much more you know i mean yeah i just prefer it to be one thing rather than 2 completely separate things and then the added explanation of why it is not possible to have the same thing applied to both halves well i think we should i think we should try to find a theory that allows multiple speakers and relations to span over more than one speak you know because it is true that i mean if you look at things like damsel and other attempts to come up with annotation schemes for dialog acts there was this notion of forward looking and backward looking anyway they are relational i mean in some way you know they are all relational but you needed but there is a different set of tags in there from the kinds of relations you see within rst for example so if you look at amanda stent is thesis and the analysis that she did sort of within a turn she used something like rst and then a cross turn she used damsel right yeah which probably would not give us the kind of of information that we need about relations across turns just because we really are critically interested in the argument that the group is building up so that we can find the decisions that they made and any counter proposals to proposals that have been made and things i mean it really is more in the spirit of the rst ish relations than in the damsel kinds of relationships what we want out of the the relationships between speakers here they are jointly building up an argument structure and that is the critical thing that we are missing in order to be able to understand what is happening in these meetings so we would not want to do the same thing as what she did in this case just because damsel does not give you the right sort of representation between speakers well it gives you things like accept and reject and 0 does it okay i have not seen it on that this kind of data so maybe in fact rst does not give you anything like that no right okay but what i mean is you want to make sure you get the argumentative relations however you do it whichever relations you have to build right you know not i am not saying i am not saying that there is a particular set of tags or relations that have been defined out there that are the right ones i am just saying we need the correct ones for the structure that we are trying to get out of this because we need certain information out of this analysis so i think it is much more i think it would make more sense to go about this problem figuring out what kinds of relationships we want and then finding and you know like setting down what the constraints are on but which he has been looking at the data so that is he has been looking at the data okay good and then and then there are lots of different theories of argument i mean you can also think about the sort of joint problem solving kinds of yeah approaches so you know clark and you know lock bomb and you know that kind of thing that were designed for dialog right i mean and can be expanded to more than 2 persons actually the lock bomb stuff is for groups yeah yeah yeah so yeah for interactiveity this is a very weak definition yeah if we need more than my 2 then we might use some more finer definition for interactivity well what you will need in the end is something operational so something that you can use to actually segment meetings into the areas that you think are important for this analysis yeah so it is not quite clear to me what the different kinds of things are so it would be i think handy if you could handy if you could take a meeting at some and say you know this is an interactive area and this is a monologic area because we could come up with what kinds of measurements you mean i mean we have not done past work here on segmenting meetings in that way but we have done work on characterizing different kinds of meetings in terms of how highly interactive they are but it is a different definition right in terms of how much all of the people involved in the meeting take equal part in it as opposed to the highest status people in the meeting just driving it and the other people following whatever the high status people want but those kinds of measures we could use to do this kind of segmentation if it is important to the analysis yeah okay but we need clear definitions of what the different things are how many different kinds of activities are there is it just interactive and non interactive or are there are there more grades of shading than that and what are the things that you expect in each what is the back channeling behavior what is the speaker turn what kind of dialog acts do you expect i think that would help you pin down what the definitions are okay okay so for dialog yeah there would be some kind of interaction structure so in previous discourse studies especially in discourse analysis exchange it is used to describe this kind of local structure and in conversation analysis so at just a patient is pairs are used to describe this kind of structure but in these 2 strands of work they just come up with a list maybe not exhaustive so i have not been thinking about if we could come up with some kind of more principle scheme taking some of the information in the interaction into account like is the small part of the interaction is exchanging some information here information is a little vague which could be some kind of knowledge what the speaker knows or or some kind of thought what the speaker is thinking so this 2 kinds of information might be some kind of difference so this is a little bit like the bels exchange of information for the knowledge exchange of opinion some kind of so it is very related to thought of thinking yeah do you think it matters whether something is knowledge or opinion knowledge or opinion in the way it contributes to the group process in some case it differs because when we exchange knowledge we might not challenge the speaker but when the speaker is talking about what he feels what he thinks is proposing something than other speakers might challenge him or her yeah i sort of have always taken the view that well people are not very good at this kind of rationality you know and the difference between information and opinion is not that prominent in the way people do their reasoning so that might be some difference in interaction patterns might be because for information if somebody is seeking some information then he asks some question and the one knows will give the answer but for the opinion or thought stuff then the speaker might just present some idea and others will commend or challenge the idea well you can not define them that way you can not define the difference between information and opinion that way because that is basically defining as what happened about it and i think what tends to happen is that if people challenge the facts you often discover that they were not actually solid facts anyway i think the boundary between these is kind of fluid so i guess the question is is it important to what is it important to so do you need the difference between information and opinion to do this analysis so i am still looking for a so called principle scheme for the interaction so what have you been looking at yeah i was just going to say it would be useful to know what so you have got books that high on your desk which ones are they yeah so yeah for the discuss analysis they are using exchange but they have no classification just to come up but who is they that is the question okay you mean who singleton code that right and also in geneva school i do not think they list some kind of exchange but the exchange structure and and yeah exchange is 1st developed in systemic functional grammar along the hardy and martin is work so actually they have something in the textbook about the exchange they call it negotiation so they have some flowchats of the detailed analysis of exchange structure okay so i am sort of getting that you think there is a lack in the theory to do with exchange structure at the moment but i am not quite clear yet what you think the lack is so i do not think there is any kind of principle classification of exchanges well are you thinking it has been a while since i have read this literature but i mean my understanding of exchanges is a is that the initiating move introduces a goal to the discourse and then the remaining moves will contribute to either the meeting of that discourse goal or the decision not to bother with that discourse goal right and so i would be surprised if there was not work describing things in those terms i mean i know so clarence colthard is basically about teaching yeah rf is kind of structure yeah yeah yeah so you would not expect that to necessarily apply in wider context in a way like you might expect to have to extend the theory to do that but i think there are other people who have looked at exchange structure and described it in these terms in a way that should really suit your needs because from my previous experience we see analysis analysis of information seeking dialogs of telephones so it is about the tourism information so from my analysis of that kind of dialogs i could say that there is only there are only 4 basic exchanges depending on the information a direction of information flow and also the speaker role because in the information signal the rules of the 2 participants are different one is mainly from the agency a tourism agency that is the main information provider while the customer is the information seeker mainly and also the direction of information flows for some kind of information like the price of a specific route the spots they are going to visit and also the kinds of vehicle transportation so this is a lot like george horton is work on saying what are the different parameters for these exchanges what are the things that characterize the set of exchanges that can happen yeah yeah yeah but there are also some variations which i call complex exchanges so that is just a list not like the basic exchange exchanges yeah so is your complaint less about less about knowing what makes an exchange and more about knowing how you parameterize exchanges for this genre so what is the range okay so you are worried about what is the range of exchange types for this genre and that is what you think is like that yeah yeah i am looking for the metric yeah yeah yeah that is yeah parameter or principle yeah something like yeah yeah and also yeah and also in because in the meetings the speakers might report something they have done in the past and talk about what they are currently doing and also planning some future actions so this kind of information might also be helpful so have you tried i mean i know of one previous attempt to establish what exchange is they are needed for some kinds industry meetings and our experience in that project was that you can not really identify exchange structures in these meetings because they do not really follow that pattern that exchanges are clear in well they are clearer in when you have got fewer people so there is fewer people trying to pull the discourse goals in different directions and they are clearer when your goals are quite limited so when you have just got information exchange and there is a clear task that the people are trying to do and they are focused on that task as is normal in dialog elicitation like map task or like your information seeking dialogs which was not dialog elicitation but it was very constrained in genre so the answer to the question might be nobody knows what the catalog of exchanges is for real meetings because or for these decision making meetings because you can not identify exchanges right so have you gone through the meetings and looked for initiations and responses with without a prior catalog just knowing that an initiation introduces a new discourse goal and a response meets or negates that goal in some way so have you tried marking these exchanges not marking just from my reading of the is 108 for a meeting series yeah just to read in this part yeah well i mean i think that i do not know it is probably good to hear you know more of your presentation because i think there are lots of different candidates and you certainly can not go through and mark up every meeting i mean markup meetings with every scheme we think of so i think we have to kind of get an overview of what the candidates are and if there is like 2 or 3 that we think are good contenders then we try to see what works better right but a reading of this could tell you that there is no case where you are going to be looking for initiation and responses that is going to work because you could go through and see that actually the pattern of addressing different discourse goals is too dispersed to be able to do that kind of markup but yeah i mean obviously you want to have a front runner scheme in mind before you there are a lot more options so let us see what else you have got okay so this is for the dialog part for malak so that is it 0 so that was it on dialog okay okay so no further information is here okay so it is very unclear to me why you think exchange of information is more principled did you i mean what did you have specific schemes in mind there specific people is work that you were because so for the information and action the action here is not that like but some kind of physical action that the participants is going to or has done in the past is going to do the future do you mean an action in terms of what the group is requiring them to do or do you mean an action in like picking something up so action in the technical meeting sense or just action in the physical sense physical sense yeah right okay yeah so this kind of might be some kind of semantics of the utterance or exchange so this is just one parameter but can you give us an example of a scheme that is based on exchange of information i mean here the scheme is for exchange scheme so yeah for exchange of information so they are my busy speaker and here here so who speaks to whom and so this is the direction of information flow and also so speaker row might be 1st parameters for the information exchanges so for action that might be some kind of proposed or accept or rejections or accept or reject kind of exchange structure 0 so that is not a physical action right if i pick this up which is physical action you can not reject that what do you mean no no that is the so we are going to do something and so could you do something this kind of action related happens i mean right that is right right that is an action in the sort of technical sense of meetings okay technically so you are actioned to do something which is part of the permanent record okay yeah yeah yeah okay yeah okay okay so i mean i think that there are a lot of things that we have to look at before i mean before coming with the scheme you mean yeah yeah i mean and well there are very few schemes for marking up exchanges on actual data right because people have not considered it very important well i always thought they get tired after marking up the dialog moves so they never bother with any higher level dialog structure so we did it in the map task but i do not actually know of other cases where people bothered marking anything higher level than the map than the move structure right well we are marking up a lot of higher level things in our stan might does some 0 yeah sorry but it is not been very popular no probably because it makes some okay so have you looked at nate blaylock is work i might not familiar with these so he has things like you know adopt so basically it is to you know collaborative problem solving you know more than one person involved and trying to characterize things as you know if i i can not remember the i can not remember the names of all the actions is it to interrupt the nc coconut project or something no so the coconut project is a project that i did with rich thomas and a barbara diageenio have you looked at that yeah look not very much to do no well there is a lot of detail in the paper but you have not looked at it in very much detail no okay so i think you should okay because it was all these things that are done for joint problem solving and joint action i think are very relevant to look at because these people are solving a problem together right so while these schemes may not have they have some of the essential components that you need to characterize joint problem solving so while you may think that they are missing something then we could certainly augment them but i think that they have to be looked at and so nate blaylock tried to okay so you know from gross and kraus and seidner came all this stuff about shared plans and how you really can not understand you know dialog unless you understand shared plans but the actual theoretical or formalization of it was incredibly complex yes and so lockbaum simplified it quite and you know seidner and rich and collagen have simplified it quite a bit but nate blaylock tried to you know define you know what happens after somebody suggests something well you can adopt it you can i can not remember you know okay but he has he has a taxonomy i know this guy yeah yeah right james salern student yeah i know yeah yeah yeah exactly right which we tried to apply to tutoring but it actually does not work work very well for tutoring because tutoring is just a totally different ball game but it you know we spent a fair amount of time looking at it and it did strike me that for joint problem solving there were some good ideas and there is you know a set of tags and what they mean and you know that sort of thing and then you can also look at you know clark on joint action now he is not going to give you a set of tags because that is not the kind of guy he is but the overarching idea you know i mean i think accounts for a lot of the ways in which you know these other people are thinking but i mean i think that we basically have to merge some of this stuff that has been done you know on meetings with some of the stuff that has been done on collaborative problem solving because there is elements of each that seem really important well the 1st step is find out how much of the material does the collaborative problem solving cover so how much of a genre clash is there here because it really seems to me that you know if you look at and some of this stuff on exchange is like ancient and you know they did not have the ability to mark up lots of data and count things you know so i mean all the conversational analysis stuff i mean those guys never count anything you know i mean so they see a pattern and they think it is kind of cute and then they scour the world for 3rd turn repairs they do not actually say how often do those happen i mean i know manny shagloff really well and they are not counters they are purposely not counters they define themselves as non counters but there are people in the field that are moving that direction like martin and also what is important here is the theoretical clarity about what an initiation end of response is so what is it that ties different acts together at a higher level yeah i mean it does seem like you know there is something that we want from that and then there is something that we want from collaborative problem solving and if you take the exchange stuff it is not going to quite do everything you want and if you look at the collaborative problem solving stuff maybe it will not do everything that you want but i think more important than looking at whether parts of the meeting are highly interactive or not so interactive is looking at what sort of task the people are trying to do at the moment so you know are they doing are they as in the kickoff meeting just exchanging information to get started with the project or are they actually addressing some specific design criteria at the moment to try to decide which shape the thing is because that is going to affect the kinds of relations that you get between the utterances and so that is much more like the the literature from mcgrath about the different kinds of activities groups get into so i think that is another thing that has to be pulled into this because you do require different kinds of relations or you expect different kinds of relations from some overarching set in those different kinds of activities so main actions i guess might be some high level problem solving on decision making concepts not like the exchange which i would like to be very low level utterance to utterance relations so what is the high level version again meeting action meeting actions might be some right okay the meeting actions is just going to be a segmentation it is not going to present any microstructure okay so i was just going to say from here to here they are doing some here to here they are doing something that is all that that scheme is going to do somebody is writing on the whiteboard i mean the meeting actions they presented at the review seemed really low level to me like somebody stood up that is not no those well that is individual actions and the meeting actions are meant to become more semantic and more theoretical then the kind ediop have traditionally used okay so we need to still preserve some sense of the activity but rather than say it as somebody is at the white board it will be you know that somebody is explaining something that the others are i mean i realize it is important to recognize that you know wayching got up went to the whiteboard i realized that is important you know yeah i was kind of surprised that those were meeting actions so it is a different scheme yeah yeah so yeah the main actions that you are talking about i think might be a built up from the exchange at the last exchange might be a more kind of important information well you might need the meeting action segmentation in order to know where what area you are circumventing so it is just exchange structure in this area that you are looking at and then that is a solid boundary between the next bit or something i do not know yeah but well time is getting on so you know what practical things can we do to help this work move forward so show what else you are thinking about yeah for the monarch i have not looked into the details about these kind of relations for the monarch part just a list of these relations and so next next next i need to review the literature and come up with schemes for the real data annotation and of that because this is not in the hapcobs so we are not going to get all the 1000000s anted with such kind of information so we might just try a few 1000000s and have a reliability study if it is reliable enough then we must just add a small part of the meetings and using some learning techniques to bootstrap the automatic analyze well marking up exchange structure if if it is there and if if you have got a working scheme for it is relatively cheap it is a lot cheaper than doing download and code in the 1st place yeah if it is personalize the data yeah right well one presumes it is otherwise it is going to be radically expensive to do any kind of markup so you know it is something that we can look at but you need to what we need it is more important than knowing you know that we need an exchange structure marked up on x amount of data at this point is knowing where you are headed with this i think so what is the good that the project gets out of this in the end so you know what is the headline results of the work on the project i think is what we are trying to get a sense of here so you get your exchange structure marked up and say we just put in the money get exchange structure marked up on that entire hub or you know figure out how to bootstrap it so we get this how do you sell that as a project goal because people are going to say exchange structure so what what do you what do you what do you what how do you plug that in the rest of the stuff right yeah okay so this might not be the final goal that could be fit into the browser but it might be some kind of information which might benefit meeting actions or some more high level decision making or problem solving because this is the building blocks i guess yeah see because i think that just to make this more efficient you should be thinking about the high level goal in order to focus the work that you do doing rather than thinking well you know there is these theories you know how can we apply these theories to it and then seeing what comes out of that that might be useful for the meeting browser actually trying to say yeah we have a goal of being able to do this with the meeting browser what kinds of dialog structure would we need to be able to do that i mean i do not know maybe you would want to disagree with that but this more goal directed might actually make it easier to bring these things to publication in the long run right well i think we should approach it both ways i mean you know the meaning browser people have a lot of constraints and so what they end up with saying is most critical for them right now might not be the most interesting thing in the longer term right it is not necessarily what they say but it is something at least identify something that we would hope to get out of it you know right i mean people have said over and over again that they would like to understand how a decision has evolved right and i think that well i do not know if all of this low level stuff would allow you to track that kind of thing but it might at least be the building blocks of allowing you to track that kind of thing yeah so after this if we might do some from here we might be some higher level yeah problem solving decision making process to track that kind of information yeah so i mean i think it is clear that something at this level needs to be there i mean and to ask questions like who disagreed or were there any counter proposals or whatever even those kind of questions you need some kind of markup like this it just helps you know which theory you should be concentrating on so you know that immediately discounts exchange structures that are mostly for information seeking alone because the notion of exchange is too restricted there and in favor of these more argumentative things so i do think you should keep in mind what the final goal is so that you can plug this into something at the end and show some benefit of having the markup well that brings up something interesting because i think probably what we should do is try to get you know some of these tasks sort of put on you know sort of as part of the official agenda being able to answer questions like were there any counter proposals to the banana shape or whatever or to some specific you know recommendation and things like that i mean you know those you know strike me as clear things that you know people who go to decision making meetings want to know if they have missed one or arriving at or anything like that so it seems like we should probably get some of these things on the official agenda and say look you know would not it be nice if you could have answers to these questions yeah i mean operationally it is not going to make much difference whether it is hub or not right because we have got the bank of people we have to get the hub stuff through but you know he is not ready with an annotation scheme that can be used at the moment anyway so no and it is not that expensive i mean it is something that we can afford off budget and i think we would not want to necessarily do this on icsi data i think the icsi data is too complex and there are not enough real decisions made you know that you could clearly identify it i mean there are some but and not that i have read more than about 2 meetings but i think you know well so one thing that is going to be interesting is the i think to me is the what can you do because presumably the dfki folk or somebody is going to mark up a lot of the data with sort of tags about ontology right so we know you know somebody will have gone in by hand and sort of marked up something like shallow semantic y kinds of things right yeah that is not how by there but yeah deeper semanticy things that but you know not really in a formal language but mostly just you know tags right and then the question is well how does you know just the way people typically do this is you know let us take these semantic features into or these features into account and see how well we do with the learning and then let us throw these features out and see how well we do because we know we are going to have them because one or 2 sites are really interested in this ontology stuff right i mean tno is and dfki is right so there will be some kind of domain semantics and there will also be presumably some kind of task hierarchy modeling going on in i mean because in the whole browser working group final sort of presentation on that 2nd day was about you know knowing the task right and very you know details about which task is this you know which sub task is this little chunk of the meeting addressing and furthering in various ways right which seems like it would be really hard in an open domain well i do not remember that about the you remember when steve came in steve whitaker came in at the end and said you know they would spent the morning talking and they came up with this idea 0 but that was about what task the person doing the browsing was going to do they did not say anything about what sort of information they would want from the meeting itself it was about they just mentioned that they want decision part issues action items i think they are more talking about user requirement for the meeting than subtext they want to know no they said people are talking about tasks you know i mean they really they made it clear that they were that was going to be happily that was the browser task you know they have identified these 7 different tasks people might want to do with browsers you know like join a meeting late and catch up on what is happened in the meeting so far be at home waiting for the washing repair guy no those are the scenarios those are the users scenarios the task what he came back and said that at the end i mean it actually kind of scared me because it was all based around tasks okay well and the domain tasks okay they are now they now have to decide how many buttons to put on this thing i mean it was all about the domain task okay and it is really and given that it is really scary to think about what you could possibly do in an open domain where you do not where you do not know what the tasks are and i do not think you are going to be able to figure them out very reliably but i thought you wanted to avoid doing stuff that was stuck in this particular domain if possible i do but the whole project is you know focused on ami hub data as their 1st priority well yeah but that does not mean you have to be using the techniques that you can be using open domain no no no no no but i mean if the if they had this sort of epiphany at the end of this meeting where they decided well why are people browsing this well it is all to see how you know how the participants were furthering their tasks and you do not think that can be done in open domain at all because we do not want to get into working with ontologies i mean one of the strengths of this grouping was that you know they all work with the closed domain ontologies and we will do the open domain side of things i do not know well right and then yes and then there was this confusion about the last spider phone conference that the message is telling people to call came out after the meeting happened at least to some of us 0 right yeah so we do not know what happened at that one yeah so i mean we will get that back on track but i think that we do not you know there will be some meeting browser demo that you know i think it is an interesting question of okay well if you have knowledge here is what you can do if you do not have that knowledge you have to do different things and you know you can not necessarily yeah so you are advising which that he should be using like any ontologies in propositional content that dfki and teno bank in the cvs repository or not that is what i am not clear on when he does these learning experiments he should do him 2 ways because he will have this opportunity with or without and then we can see like well without you know these tags so you know alex was showing yesterday well without knowing the head word of the utterance at the leaf you know you get these kinds of results and when you add that in you get these other kinds of results and you know i think and feature combinations yeah so i think if we have those features it is you know it is a rare opportunity that you can you know actually try you can try one head to head against the other and then of course to make sure that you really i do not know when they will bank these by the way because that is it is not hub so it is one of these things where everybody is supposed to bank as any things as possible right so you know well we will not i mean we will not use until they are ready obviously so we might do the sort of open domain i mean although it will be a closed domain i mean we can not use any domain information necessarily i mean there probably are a bunch of issues like if you have got 100 hours of meetings and they all talk about a lot of the same things yeah it is not really an open domain test it is not really open domain and somehow that sneaks into all the statistics anyway because you do not have such a wide vocabulary well eventually the hub is supposed to be 40% non scenario right but that data is definitely going to be coming in last sure so we might find that when we move from what we thought was open domain but on the hub you know our results go down when we truly are getting meeting yeah so i mean that will be interesting we have got fivisco meetings actually which are still an elicitation task but they are not our domain so that should help some okay well that is good and they have been declared hub so they will accrue all of the normal annotations of course yeah you will never get the t 0 and dfki stuff on it because they are only building stuff with the one domain so okay so we want to decide what waiting should do next and maybe think about meeting and i do not know yeah just figure out how we are going to help them to try to make this more efficient so before we do that can i just ask you a question about chunking in the beginning you said something about you were doing this chunking and you are doing it on the ixie data but this was did you say it was not picked up in ami i was confused about i mean i was supposed to chunk part of the amm meetings enough to see if the chunk of channel sweetboard works well enough or not if not then we have to use some kind of simsifized running or some bootstrapping to do chunk and you have not done it yet no no because i was a bit worried about this kind of this construct that that is fine so that is still on the agenda to do at some point yeah but we should make sure we finish that off because otherwise that works just wasted right yeah exactly we are going to use chunks for something useful yeah chunks are going to be made identities if nothing else right or building blocks i mean so i mean we have to think about how chunking interacts with the automatic segmentation you know because barbara was showing automatic segmentation at which level the low level dialog act yeah and i mean it seems to me that chunking should have some kind of so if your dialog act segmenter says there is a segment here but your chunker said no this is a coherent chunk yeah that is a problem yeah you know i mean maybe they could work together to do a better job then certainly if you have got chunks it ought to help you know where your dialog act foundries better not be right yeah yeah it may be even more mechanical than that based on the examples we are seeing from dfki because they are their hand marking stuff and the dialog act boundaries are supposed to be intentional but they seem to always be marking them that after verbs you know with claws right it is very it looks quite mechanical so we are querying them about this because that is not the way they are supposed to be doing it right okay good so it it sounds like well well hold on but when do we need chunking by and you know so i do not like the idea that you know you start something and then 18 months later you pick up to do the last 10% of the work to finish it off or whatever so i like things kind of tidy right so how much work is there left and when do we actually need it by would be useful to discuss well you wanted to do the chunking on the ami data right yeah because the ami data is is there there is i mean there is the hubs of a is transcribed i think in final form we can check the wikis but you know the data is rolling in so we got stuck at some point on evaluation of chunking right the problem was you did not have icsi data that was hand labeled for chunks and we decided labeling the icsi data for chunks would be probably difficult and maybe not very useful and so it barbara still says somebody is done this for chunks okay i will have to look up okay so if there was hand labeled data that you could evaluate against so train on some and evaluate on some would not that because you have trained on switchboard right where there are chunks so if you had some icsi data that you could maybe use i do not know as a what is the there is a training set and then there is the development set and then there is a test set maybe you would use some chunk ixie stuff as development or something do some last tweaks and then evaluate but you know i mean i can see why wei ching did not want to spend his time chunking ixie which means we have to get some portion of the data hand chunked but that should be that hard to do right how much data do you need how much i do not read lips what was that is that much and switchboard switchboard is all chunked yeah yeah the data how much data you need was not a problem with the switchboard because he had lots so you did not have to think about how much so we need to know how much data you need hand chunked and then how do you hand chunk stuff right so you were working on instructions for doing hand chunking no you had pen tree bank parses and you moved chunks from that but then there were 2 possible options for doing this on the ami data you could either essentially go the same route as they did with the switchboard you know do the pen tree bank and then create the chunks off that or you could chunk directly and it was not clue to us which one of those 2 methods was the right way to do this direct chunk because we are not going to build any tree out of the emi data right well so you have to produce this part of this exercise is anybody going to try to parse the emma data you would have thought so but i do not know so diff kai because they are working on some hpsg 0 yes yeah i would forgotten about that yeah it might be maybe yeah but that is not very similar well i do not know very much well i do not know very much about grammar is that similar enough that the same method could apply that we derive chunks off the hpsg output i guess if they are different then we might just sort of write narscript to do the translation i think you might lose a lot in you know the step so basically your chunker is trained on something that was derived off penn treebank and it might be different enough that that actually causes it not to match you know if you are suddenly evaluating against chunks that we are done using hpsg i do not know about grammar but you know to look at some of the details to see well they are using the erg right which probably has maybe finer which i believe has finer grain distinctions and different labels and the mapping can be pretty difficult for this right or is it easy to map between finer grain distinctions it is probably easy to dumb down the hpsg parse to because the penn tree bank parses are pretty minimal because they did not want to commit to a specific grammatical theory so i mean it probably is worth looking at because they are doing this with dan flickinger right which is the same parser as the redwoods tree bank was made with that alex talked about yesterday so you can look at redwoods parses and see how different they are from tree bank parses and you might want to talk to alex to find out if somebody is already done this turn redwood stuff into tree bank stuff so that they can compare you know against results that are reported on tree bank the names would suggest that this was the idea right well that was the idea i believe but it was supposed to be another tree bank right so that sort of assumes that we think it is better to do this by deriving chunks from some parsed format than just hand chunk in the stuff well the good thing about that is that then you know that when you go through and parse you will be getting the same chunks as when you do this process automatically because the problem when you go through and hand do it is then suddenly you introduce your automatic method and your accuracy can go way down sometimes it does not you know sort of for part of speech tagging you know whether you do it by hand or machine it is pretty much the same accuracy right so let us try it this way and then that puts off the question of when do we have to do this to when we get hpsg stuff right so can you ask dfki whether they can give us a couple of me meetings run through their hpsg process i do not even know what time scale they had in mind i mean is this something they were doing it at the moment but i do not know whether this was very preliminary or do you remember they talked about it at the meeting that we went to in 2020 i was thinking enska day why can not i remember the right name of this place yes twente which you were there gene yeah yeah you are expecting me to remember stuff are not you no no no no i know you missed a meeting and 0 yeah and so there may have been 0 i do not even know which one i missed brussels amsterdam you missed the one in tno tno okay see i was not there so i do not know where we were was yeah yeah difficult this work in mobile project we see hpsg person yeah on which projects are i did not get that verbal mobile right yes yes they did so ask them right well that is that is what alex talked about yesterday right the pars i mean that was not something she did but that is the data she is using right so we have all that stuff here you know the parsed verb mobile material so you can look at that and see what the parsed trees look like you do not need to wait for dfki and if it turns out that you think 0 there is no way i can map these into pan treebank style bracketing then we have to think of something else okay okay but i think they had that in mind so it is worth also talking to her about that okay good so it is talk to alex see if it looks like it maps by looking at the verb mobile if it does ask dfki either for data or a timescale and you are going to need to know how much data you need in order to do that so do you have a technique for estimating how much data you need not sure i do not know how you do that so who knows how you do that everybody always says there is no data like more data okay so just ask the fki how much data and when right you know i mean the typical thing is to do leave one out cross validation with you know yeah i know 89 dialogs okay so i train on 88 or i train on 1999 okay but we do not have any particular time in mind when we need a chunker that works on this stuff well the question is what is chunking going to help i mean i think chunking could help automatic dialog act segmentation i think chunking could help as you mentioned named entity recognition like if we did a chunked version of the ami data and stuck it in the cvs under the automatic contributions then people would end up using it because they do that right so the question is at what stage should we put that in well we would want to make sure we are pretty happy that it is reliable yeah so post evaluation so we can put in some idea of what the evaluation is like yeah i mean otherwise you get these very bizarre results because the data is not really very good so but nobody has expressed like massive interest in chunking so i think nobody is waiting for it nobody is relying on this summer but we could i think we should talk to barbara about how to do some joint experiments or if she is going to be sending other people maybe liz or whoever comes over next time and it would actually be great if it was liz because i think she is the one who did the dialog act segmentation so when do you think that would be when did she something she just said that hopefully by the next meeting she would be bringing someone else or someone else would be coming instead of her but that is a pretty tight time scale then right because it would be for the full we would want to know the evaluation is that the time scale you are thinking of not to have it done but just to talk about how it could interact with automatic dialog act segmentation and i guess we should talk to yoshi about it in relationship to named entities who is doing named entities besides yoshi vessel i think okay so we could talk to vessel about that as well i think you should still ask barbara again you know can you have another think about who might have the chunking stuff because she has said several times there must be somebody and she can help with that and i think she is just forgotten so and then we can evaluate on the ixie you know pretty easily so one thing that would be very helpful for me because i have not read i think we are in a situation where jean you know has read a lot of the thing lot of literature and i have read a lot of literature and we there is some overlap but there is also quite a big space where we do not know the literature from the other side and since you are going to be reading it i mean it would be probably be useful if you made some kind of you know report that described you know okay this is what these people you know this is you know a synthesis okay here is the basic idea behind exchange structure and you know here is a couple of you know here is the 2 most relevant works that i have read or something like that you know to just yeah even if it is just you know on a wiki the references and then why they are relevant right i mean you do not want to make this really formal or even just like a paragraph of you know the basic idea just because you know obviously i am not going to go off and read all this kind of stuff because i do not have time and you probably do not have time to read all this collaborative problem but that would help us be a lot more specific than just you know listing bullet points on slides because then we would know sort of which literature you are thinking of putting in and we could read the most important well and anyway every time you write a paper you have got to go back and do this related work thing anyway so it just makes it a lot easier to get that common ground i think i think it is a real advantage that we have read different things and come at it from a different side because i think the previous approaches have not really you know they have not addressed this problem yeah you know the good features from you know both sides and tried to put them together yeah i am going to have to go very soon because i have got this hearing to go to but i can send you so the practical actions you know what are the things we are going to go away and do well i think we both want to send well i do not know maybe you have already done this but you know i will send you or we can you know chat offline for a minute about the list of things that i think you really need to take a look at and gene probably maybe you should do the same okay so then later yeah okay and if sabrina has any ideas we would be happy to hear from sabrina as well right now i am trying to do the meeting action segmentation because after talking to jean just we have different ideas whether meeting action will be about the same size as topic segments so it would be good to right now i am doing this looking at the data and trying to see if i can do the mating action segmentation 1st on the skin and then do topic segmentation again to see if the boundary actually coincide with each other if that do then maybe we can talk again yeah so you can do this exercise and then come and talk to me sometime next week i am not in monday but the other day i will be around all right yeah i have just started and so far i think it is i think it is still about the same level of segmentation we are talking here but maybe the boundary will not be exactly the same yeah this is interesting but i do not think we can afford to look at this right now so why do not you bring it by next week and what i really want to do concentrate just now is finding out how we are going to help waging with making progress on this so we have already got you know he is going to construct a list of the references he thinks is important and i and you as well are going to contribute to that and we can put that on a wiki or we can do it some other way i mean you know i am wiki mad so if we share it it should go on a wiki because i can not find anything in my email but you know you do not have to do that and then how do we you are actually so what you are doing is taking all of the most important references for finding argument structure that might eventually be of some use to a meeting browser and trying to look at actual data and apply and see like which one helps you figure out what you are actually going to do with this particular data right so it seems to me that we should take like the frontrunner scheme and the data you should look at not frontrunner scheme but frontrunner theory and see if you can speak to the data you know using that kind of theory at some point that might be an interesting exercise right well i think we need to like you mentioned earlier factor in what this might be used for in the end and so even if it will not be used maybe if it would need some higher level analysis before it could be directly used for that purpose we should still keep that in mind and so i think going through the presentations that were given at the work package 6 meeting where i think it was it was not was it anita or was it i forgot in her name the other woman from 20 you know they talked about they had done a lot of questionnaires and they got some user requirements out of that and then steve presented the results of following people around and looking at how they used their notes from meetings and so that would give us some ideas about you know what people want you know out of meeting records but then we should also add things that we think are interesting it is a good starting point but just to keep some of these goals in mind because one scheme might let you answer this set of questions but would not help you at all with another set of questions exactly yeah so you want to go through basically the user requirements and maybe think about some user requirements if you want to make a case for these are the kinds of things that we would want and look you are going to need this kind of dialog or argument structure for it so should we expect that all the slides from like the work package 6 meeting or the joint work package 5 work will be up on one of the work package websites and if not how do we make that happen i will think about that okay some of this was presented in at least the one spider phone or the 2nd spider phone telecon i went to for work package 6 so if they are not on the web we can try to look at that you know at the materials that were presented in that telcon without necessarily listening to the whole telcon yeah yeah so we are just trying to get records i mean i am sure they are not on the web yet and steve and his i have forgotten the name was it simon yeah yeah they i think they may have written this up in a report but i would hope that is on the work package 6 website as well okay so we are going to go through the requirements like that and did you think that was reasonable to after that to try and actually you know take some theory that seems fairly reasonable and think about particular data and come and talk once he is read a couple more i mean he is just completely missed out the collaborative problem solving world which i really think we need to take into account i mean you may take it into account and say no you are wrong and here is why but i am going to need some convincing yep so then try a frontrunner scheme on well a frontrunner theory or 2 and see this one gives you this and this one and can not you know are they fundamentally incompatible or can we but try it informally yeah yep when do we get together again because i really do have to go okay i promise these guys i would be there i think this is going to take about 3 weeks because he has got plenty of reading to do huh so maybe we get together again reading lists before that though so yeah i mean i will send them the references immediately so we will get together in about 3 weeks is that enough time for you do you think is that enough time for you to read and think about i would try my best if it is too tight you know i think we can say 4 are you going to be on holiday i might yeah because julia and the boy is here so it is a little tiring because my parents have left yeah so i might take 2 weeks off in may okay so we should get together probably in about 4 weeks yes so when are you going to bring this beautiful baby in to see everybody i hope so very see good good that is great right i am going to head off yep well good luck because solving your problem floods are more important than dialog structure i am afraid you have to tick and tick it does something weird when you do that i do not know so i am going to leave it here for now 0 she said you can take them like this they have often recorded yeah i thought she would photocopied him before okay we will check with her okay okay okay thank you thanks a lot too thanks i always like dialog structure yeah i have done a lot of dialog structure so no i have done a lot of dialog structure so where is my mind so where is my point we were trying to call up here to tell you that i was going to be a few months late because i was going to phone with ivana trying to organize the biva for monday and so i said call the instrument in a meeting room and i want to have a phone number and david said yes but then he said by them i better have electricity to the change okay okay can i take it now yeah you can not check the next you do not check to see how good the pens are at before you well i mean you know some are a lot but i am not going to know okay okay i checked my note and okay box right i have checked my note and okay box right it is not too old here yeah we already told them we have already input that into me okay