A Weighted Congruence Measure

Document Sample
A Weighted Congruence Measure Powered By Docstoc
					                                     A Weighted Congruence Measure

                           Irwin Kwan, Adrian Schr¨ ter, and Daniela Damian
              Software Engineering Global interAction Lab, Department of Computer Science
                                     University of Victoria, Canada

                        Abstract                                interactions more precisely. Hence, our resulting congru-
                                                                ence measurement is a more precise measure of congru-
   Socio-technical congruence is an intuitive way to com-       ence than a measurement that uses dichotomized relation-
pare required coordination effort within a software devel-      ship strengths.
opment project with the actual ongoing coordination. The           This reﬁnement is necessary because the current way of
current model of congruence is limited because it builds on     modelling congruence is based on some implicit assump-
top of some simplifying assumptions. These assumptions,         tions. Two of those assumptions are that every dependency
such as placing equal importance of coordination needs, of-     is equally important, and that any coordination is good
ten fail to reﬂect the actual nature of a project. We propose   enough to satisfy a coordination need. Stepping away from
a model that derives actual coordination needs from ﬁne         those assumptions is important in order to more accurately
grained task interdependencies and task assignments. This       model coordination, and to apply congruence to a larger set
enables us to compare those needs with the real ongoing         of projects.
coordination other than just dichotomized measurements.            In the remainder of this work, we start with describing
                                                                the related work (Section 2), followed by the description of
                                                                our measurement (Section 3). After discussing the beneﬁts
1. Introduction                                                 of this new measurement in Section 4, we end with a short
                                                                conclusion (Section 5).
    Building good software requires coordination among de-
velopers and within teams. People who work on related           2. Related Work
software components should also coordinate their efforts
with each other. Socio-technical congruence is a measure-          In this section we discuss related work and current draw-
ment that computes the alignment between the social co-         backs in the concept of socio-technical congruence.
ordination and with the technical dependencies as imposed
by the project tasks [1]. The result is a ratio of how well     2.1. State of the Art
the social connections among people match the technical
dependencies among people.                                          Socio-technical congruence indicates the ﬁt between an
    The measurement of socio-technical congruence allows        organization’s coordination requirements and an organiza-
observers to study project with a “ﬁne-grained” perspective     tion’s social interactions. The coordination requirements
of dependent tasks. A manager can use this measurement          indicate who should talk to whom based on their task as-
to view the alignment of his teams’ coordination with the       signments and the task interdependencies. The idea can be
technical dependencies. Unfortunately, the measure of con-      informally described as, “If I work on a task, and my friend
gruence presented by Cataldo, et al [1] uses dichotomized       is working on a task that’s dependent on my task, then I
values for its relationships between people, resulting in a     should be coordinating with my friend”. An organization
coarse interaction model.                                       is considered congruent if the social interactions match the
    We propose an improvement to the original congruence        coordination requirements.
calculation. The improvement enables us to model the con-           Limited evidence shows that higher congruence leads to
nections more ﬁne grained by applying weights to each re-       faster completion of modiﬁcation requests in a software de-
lationship. Weights provides a measure of strength between      velopment project [1]. Hence, an interest in congruence
people and tasks, interdependent tasks, and between peo-        gaps arose to determine how to increase congruence. Pre-
ple. Thus, we can reﬂect organizational social and technical    vious research on congruence gaps speciﬁcally looked into
the effect of gaps on distributed development. Ehrlich et                        P1       0.7         0.4          P2
al. [2] found that the presence of gaps increased the number
of code changes.                                                                 0.2       0.6                     0.4
    Others like Valetto et al. propose remedial actions when                                            0.5
socio-technical congruence gaps are diagnosed [5]. Exam-
                                                                                        T1 0.3                     0.7
ples of actions include closing a gap by augmenting coor-                                               0.1
dination, eliminating the gap by refactoring software, and
eliminating the gap by rearranging schedules.                                    P3                                P4

                                                                       Figure 1. An assignment/task-dependency
2.2. Limitations of Congruence Measures                                network. 70% of person P1’s time is assigned
                                                                       to task T2, whereas T1 depends 60% on task
    The socio-technical congruence calculation as intro-               T2.
duced by Cataldo et al. [1] bears some limitations. As men-
tioned, two assumptions in the current congruence model
are that every dependency is equally important, and that any         3.1. Weighted Coordination Requirements
coordination is sufﬁcient to satisfy a coordination need. We
believe these assumptions are not always true.                          Our measurement applies weighted edges between 0 and
    An ongoing debate in the socio-technical congruence              1 to the task assignment, task-dependency, and actual coor-
community is whether gaps are “bad”, and to what degree              dination matrices. Figure 1 shows an example of weights
a gap is detrimental. Although limited studies show that             assigned to edges in a task assignment network. These
high congruence correlates with higher performance, Mar-             weights are incorporated into the matrices required to cal-
czak, et al. [3] illustrates that a project was able to deliver      culate coordination requirements.
requirements on time despite the existence of gaps. For in-
                                                                     Weighted Task Assignment matrix is a m × n matrix
stance, there may be communication structures in the orga-
                                                                         where m is the number of selected people and n de-
nization that compensates for a lack of congruence. Ehrlich,
                                                                         notes the number of selected tasks. Each entry in the
et al. [2] suggests that brokers may be able to mitigate the
                                                                         matrix deﬁnes the strength of the connection between
effect of gaps; this is corroborated by a study by Marczak,
                                                                         a person i and a task j.
et al. [3]. If this is in fact the case, then the connections from
individuals to the broker may be considered more important               The weighted task assignment matrix may show how
than other connections. Unfortunately, the current represen-             many hours one person is supposed to spend on a task.
tation of congruence does not give a priority of which gaps              To break those values down to be between 0 and 1,
are most severe.                                                         they should be seen in the relation to the maximum
    There are different suggestions on how to improve the                work hours that person can spend.
notion of socio-technical congruence. Valetto suggests that              One can also measure the expertise a person has about
edge weights can be incorporated in his socio-technical                  a task. This would yield the amount of knowledge a
software networks [6]. He suggests that number of changes                person can share about a task.
made to the same artifact, or the number of dependencies of
a source ﬁle onto another are possible candidates for edge           Weighted Task Dependency matrix is a n × n matrix
weighing. Schr¨ ter et al. [4] suggests annotating gaps in a
                    o                                                    where n denotes the number of selected tasks. Each
time interval with respective software quality measures, al-             entry in the matrix describes the strength of the rela-
lowing one to determine how problematic a gap can be with                tion between two tasks.
respect to software quality.                                             The weighted task dependency matrix may show the
                                                                         ratio of dependencies a task has with another over the
                                                                         number of total dependencies that task has on others.
3. A Weighted Congruence Measurement                                     For example, a task strongly coupled with another has
                                                                         a high task dependency.

   In this section we introduce our measurement to weigh                To calculate the coordination requirements matrix, we
dependencies between tasks and between people and tasks.             follow the calculation proposed by Cataldo, et al. [1]:
This, combined with a weighted actual coordination, will              Task Assignment × Task Dependency × (Task Assignment)T
result in a model that more accurately describes coordina-
           0.32                 0.35                       0.5                       0.7
      P1                               P2            P1                                    P2            P1                                      P2
                                            0.32                                                0.9            0.14
   0.56      0.14                                                                   0.3               0.56
                                0.32                                                                                                      0.08

   0.18                        0.12                0.2
             0.18                           0.32                 0.2                            0.6                                        0.12
      P3                               P4            P3                                    P4
           0.32                 0.24                       0.4                      0.4                  P3                                      P4
 (a) Coordination requirement derived from the           (b) The coordination that happened.             (c) The coordination that is lacking.
 given task assignment and task dependencies.

   Figure 2. Compare coordination requirements with actual coordination to ﬁnd lacking coordination.

   This calculation will yield the coordination requirement                  Communication A social network such as a communica-
matrix, which is a m × m people by people matrix (Figure                        tion network can be weighed according to the amount
2(a)). The coordination requirement matrix tells you how                        of ongoing communication. Lets consider Alice and
strong the relations between people are supposed to be.                         Bob that work on depended tasks:

3.2. Weighted Congruence                                                              1. If Alice talks to Bob about his interdependent
                                                                                         tasks, then for each task discussed the weight is
   To derive the ﬁnal congruence coefﬁcient, the coordina-                               increased by a · 1/n, where a is the weight of the
tion requirements matrix needs to be compared to the actual                              communication and n is the number of Alice’s
observed coordination. The comparison of the coordina-                                   tasks that depend on Bob’s tasks.
tion requirements with the actual coordination follows three                          2. In case Alice discusses a task dependency very
steps.                                                                                   frequently with Bob, then, given a three point
                                                                                         scale “occasionally/frequently/very frequently”,
 1. Create a people by people coordination matrix using
                                                                                         a is set to 1. For another task that is only oc-
    our method (Figure 2(a)).
                                                                                         casionally discussed, a would be 0.¯
 2. Subtract the actual coordination matrix (Figure 2(b))
    from the coordination requirement matrix. This step                            In the end, if only two of Alice’s tasks depend on Bob’s
    differs from existing methods, which compute congru-                           tasks, the weight of Alice actual coordinating with Bob
    ence with a ratio.                                                             would be 1.0 · 1/2 + 0.¯ · 1/2 = 0.6¯
                                                                                                            3              6.

 3. Set values that are less than zero to zero (Figure 2(c)).                Distance We can also measure the relationship between
                                                                                  Alice and Bob by looking at different types of distance.
   The resulting matrix illustrates areas that lack sufﬁcient                     We can measure
coordination to satisfy the coordination requirements. This
matrix is a lack-of-coordination matrix (Figure 2(c)) that                             • how far they are apart in terms of physical dis-
identiﬁes each gap along with its weight. The larger the                                 tance, like meters or kilometres,
value, the more severe the lack of coordination is.
   To convert this lack-of-coordination matrix to a single                             • the time difference between their time zones, and
socio-technical congruence measurement, sum the edge val-                              • the number of organizational units that part them.
ues and divide them by the sum of the edge values in the
coordination requirements matrix. Then, subtract the result                        These values are normalized to ﬁt within 0 and 1. All
from 1 to get the congruence measurement.                                          three of those measures result in a symmetric matrix,
                                                                                   meaning Alice will always have the same distance to
3.3. Weighing Actual Coordination                                                  Bob and vice versa.

   Different actual coordination measurements may be used                       After we weigh all entries within the task assignment,
for the congruence calculation. We present a few examples                    task dependency, and actual coordination matrix, we can
of techniques that can be used to weigh actual coordination                  compare the coordination requirements with the actual co-
matrices:                                                                    ordination using the method described in Section 3.1.
3.4. Illustration of Weighted Congruence                        gaps to ﬁll ﬁrst if at all.
                                                                   By allowing better locality and priority, we believe that
   In Figure 1 we see two overlaid networks. One connect-       our weighted model of congruence provides a better way to
ing tasks with each other, the other connecting people with     study congruence and to diagnose issues.
tasks. In the task dependency network shows for example
that task T1 depends on task T2 and the strength is denoted     5. Conclusions and Future Work
with 0.6. Whereas the allocation network in case of P1 and
T1 that person P1 is allocated to work on task T1 with de-         We extend the socio-technical congruence measurement
pendency of strength 0.2. This dependency strength can for      by proposing a model that involves weighted edges. By
instance denote the percentage of time the person is sup-       weighing each edge with a value between zero and one, we
posed to dedicate to that task per day.                         provide a method to weigh both gaps to decide which is
   Figure 2(a) depicts the coordination requirements be-        more critical, as well as ongoing coordination that should
tween people. Looking at P1 and P3 shows that P3 needs to       be intensiﬁed. This concept builds can identify properties
coordinate quite a lot with P1 but not the other way around.    of organization, such as locality to closer investigate the
After subtracting the actual coordination (Figure 2(b)) from    coordination network, and priority to see critical areas for
the coordination requirements yields a sparse network il-       improvement.
lustrated in Figure 2(c). Only in four cases is coordina-          We are currently exploring additional schemes for
tion lacking. The sum of the edge weights in Figure 2(a)        weighting edges, and intend to conduct studies to determine
is 3.37. Thus, the resulting congruence for this example is     their applications. Our future plans include determining
1 − [(0.56 + 0.14 + 0.08 + 0.12)/3.37] = 0.74, whereas the      techniques that allow us to weigh the relationships between
traditional coefﬁcient would have yielded a value of 0.9,       people and tasks, tasks and other tasks, and people and peo-
since there is only one coordination requirement not ful-       ple. We then intend to empirically investigate the weighing
ﬁlled (between P1 and P3 ).                                     techniques that we suggest by applying them to various case
                                                                studies. By weighing people and tasks relationship, task in-
4. The Beneﬁt of Weighted Congruence                            terdependencies, and people interrelationships, we hope to
                                                                more easily diagnose and resolve coordination issues.
   Using a weighted congruence model allows us to deal
with situations that are not handled with regular congru-       References
ence. We can investigate relationships between people and
the tasks in ﬁner-grained detail, including situations where    [1] M. Cataldo, P. A. Wagstrom, J. D. Herbsleb, and K. M. Carley.
people are partially allocated to tasks. The measure can            Identiﬁcation of coordination requirements: Implications for
also provide a ranking of missing coordination: It suggests         the design of collaboration and awareness tools. In CSCW
which lack-of-coordination gaps are most important to ﬁll.          ’06: Computer-supported Cooperative Work, November 4–8,
Using the weighted congruence measure allows a person di-           2006.
                                                                [2] K. Ehrlich, M. Helander, G. Valetto, S. Davies, and
agnosing the organization to utilize locality and priority.
                                                                    C. Williams. An analysis of congruence gaps and their ef-
                                                                    fect on distributed software development. In MSR’07: Mining
Locality allows us to identify which area of a network has          Software Repositories, May 20–26, 2007.
an issue to resolve. Not only does this give us an overall                                                             o
                                                                [3] S. Marczak, D. Damian, U. Stege, and A. Schr¨ ter. In-
measure of congruence, but now we can identify potential            formation brokers in requirement-dependent social networks.
coordination problems in the network. Although the origi-           In 16th Intl Requirements Engineering Conference (RE08),
                                                                    Barcelona, Spain, Sept. 8–12, 2008.
nal model does give us some limited locality in the sense of
                                                                [4] A. Schr¨ ter, I. Kwan, L. D. Panjer, and D. Damian. Chat to
identifying congruence gaps, it is far more coarse grained.         succeed. In RSSE ’08: Proceedings of the 2008 international
For instance, using our weighted model, we can identify             workshop on Recommendation systems for software engineer-
people working on two interdependent tasks who do not dis-          ing, pages 43–44, New York, NY, USA, 2008. ACM.
cuss both tasks.                                                [5] G. Valetto, S. Chulani, and C. Williams. Balancing the value
                                                                    and risk of socio-technical congruence. In Socio-Technical
                                                                    Congruence Workshop (in conj. ICSE 2008), May 2008.
Priority ranks people pairs where more coordination
                                                                [6] G. Valetto, M. Helander, K. Ehrlich, S. Chulani, M. Weg-
might be needed by the computed lack of congruence.                 man, and C. Williams. Using software repositories to investi-
Ranking is a considerable improvement over Cataldo’s                gate socio-technical congruence in development projects. In
model since it allows us to only extract an unordered list of       MSR’07: Mining Software Repositories, May 20–26, 2007.
gaps. Because ﬁlling congruence gaps can be risky [5, 2], a
ranking can provides us with a better assessment of which