1

# Mastery Learning-Like Teaching with Achievements

Tobias Wrigstad  Uppsala UniversitySweden<EMAIL_ADDRESS>and  Elias
Castegren  Uppsala UniversitySweden<EMAIL_ADDRESS>
###### Abstract.

This paper describes the design of a second-year, 20 ECTS credit course on
imperative and object-oriented programming. The key design rhetoric is
encouraging students to assume responsibility for their own learning, and, to
this end, elements from mastery learning and other teaching strategies are
used in concert to construct an achievement-based system where students are
put in charge of how and when they are examined. In addition to describing the
elements of the course design, the paper reports on experiences from teaching
in this format, and how the format has evolved over time.

††copyright: none††conference: ACM SIGPLAN Conference on Programming
Languages; January 01–03, 2017; New York, NY, USA††journalyear: 2017††isbn:
978-x-xxxx-xxxx-x/YY/MM††doi: 10.1145/nnnnnnn.nnnnnnn††copyright: none

##  1\. Introduction

Four years ago at Uppsala University, we changed the 2nd year 20 ECTS credit
course111An ECTS credit is a standard EU credit unit. 20 ECTS credits
corresponds to $\approx$533 hours of work. that introduces imperative and
object-oriented programming, with focus on coding and tools. With the new
course design, we hoped to change the way students approach learning and also
force students to assume more responsibility for their studies. This paper
summarises the outcomes of this undertaking: it explains the key course design
elements, how the course design has changed from its initial implementation to
its current format, lessons learned from the process, and a biased,
qualitative self-evaluation.

First and foremost, it is important to understand the setting in which this
takes place: Sweden, a country where university education is “free” in the
sense that there are no tuition fees and students are given a basic allowance
and state-sponsored loans for studying. The course in question, _Programming
Methodology for Imperative and Object-Oriented Programming_ , has 120–140
students yearly and is a mandatory 2nd-year course on both the traditional
computer science bachelor programme and the five-year engineering programme in
information technology. Before this course, students have been taught
functional programming (lately with Haskell), basic data structures, program
construction principles, and basic computer architecture including a little
assembly language. Each batch of students can be divided into 3 roughly equi-
sized groups — those who are already programmers (programming
extracurricularly), those who learned imperative programming in upper-
secondary school, and those who have only studied programming at the
university. The course runs for a full semester at 66% speed ($\approx$ 530
working hours for each student); students take one other course in parallel
first half of the semester, and another in the second half. These courses vary
with degree program but are usually math-oriented.

In our experience of programming courses (not just limited to Uppsala or
Sweden), student assessment usually comes in the shape of a number of
implementation-based assignments. These assignments are often cleverly
designed so that concepts just taught in lectures can be successfully employed
to solve the problems at hand. A student is thought to have mastered the key
concepts if she is able to provide a “working implementations” of the
assignments that pass some key tests.

We find this practice problematic for several reasons:

  1. a)

it rewards students’ ability to pattern match recent lecture content onto a
problem which has little bearing on “real-world” programming, and rewards the
ability to solve programs the way the teacher intended rather than using
independent thinking;

  2. b)

as students commonly collaborate, there is no real check that a student has
worked on all parts of a program and thus actively reflected on all goals of
the assignment;

  3. c)

because the problem of implementing a program that performs $X$ is so much
more concrete than grasping abstract topics like “encapsulation”, the former
tends to overshadow the latter. Many predominantly qualitative aspects of
software, like _e.g.,_ maintainability, are hard to understand without living
with code for a very long time and suffering from own mistakes/choices.

We set out to develop a course that helps students reflect on skills and
actively think about what they are doing, and where we could be reasonably
sure that all students were actively engaging not only with the practice of
programming, but also with more general concepts that go beyond the “random”
set of concepts that surface in the parts of a program a particular student
happens to write. At the same time, we wanted to cater equally to the above-
mentioned groups of students, allow experienced programmers to quickly dive
into more complicated matters, and leverage the fact that they are mostly
self-serving in a way that would help us direct more resources to those with
less programming experience. Also, in particular, we wanted to make the
students actively make decisions about their learning, as well as what to
learn when, rather than just sit down and “be taught”. In the process, we
developed a system we came to call “Achievement Unlocked”, which is similar to
mastery learning (Bloom, 1968; Guskey, 2010), although we had not heard about
that at the time. Our main source of inspiration was constructive alignment
(Biggs, 1996). In this paper we report on the use of Achievement Unlocked in
this one particular course. In future papers, we hope to explore the key ideas
more deeply, connect them to existing theories like mastery learning, and
discuss their application to teaching advanced software design.

##### Outline

Section 2 describes the course structure, the achievement system, and how
students demonstrate mastery, plus the interplay with _e.g.,_ lectures,
assignments, our version of a closed-book exam, and how to give a final grade
to each student. Section 3 reports on our self-evaluation, based to a large
extent on feedback from students and teaching assistants. Finally, Section 4
concludes.

##  2\. Course Structure

As this course fits in two larger degree programmes, it must serve many
masters, and has thus been subject to “feature creep” in its formal course
goals over the years. In addition to learning how to program in imperative and
object-oriented languages, students are expected to learn about programming
methodologies and tools, as well as basic software engineering, in particular
agile software development processes. Additionally, soft skills such as oral
presentation and report writing should be taught, as well as professional
ethics (to a limited extent). An overview of the achievements, a sample
achievement description and the course outline with syllabus, timings and
deadlines can be found in Appendix.

###  2.1. The Achievement System

The course is centered around a relatively large set of _achievements_ ,
around 70, which are derived from the formal course goals and checked against
the ACM/IEEE Curriculum for Computer Science (2013). Each achievement belongs
to a certain grade level—3, 4 or 5—and to pass with a certain grade, a student
must pass 100% of the achievements belonging to that grade, and those below.
Some achievements only exist on the basic grade level (3) whereas others exist
at increasing depth across all grade level. An example of the former is
“writing documentation”222Not because there is not enough depth in the
subject, but because of prioritising other subjects.. An example of the latter
is profiling, where running a profiler, producing profiling output and using
it to explain a program’s performance is at grade level 3, using profiling
output to improve and demonstrate the performance improvement is at level 4,
and level 5 includes reducing memory footprint.

Achievements are skills that students must demonstrate “mastery of” at least
once during the course in order to pass the course. Importantly, achievements
are not tied to any specific assignments or dates. Instead, students are
required to _map the achievements_ to activities during the course (for
example, programs they write as part of mandatory assignments) and may
_attempt to demonstrate_ mastery when they so choose. Section 2.4 discusses
demonstration in further detail.

Because any achievement can be demonstrated at any time, students are free to
attempt to demonstrate mastery when they feel they have “grokked” something,
not at an externally-controlled moment in time. Some students demonstrate
piecemeal interleaved with assignment work, others first complete an
assignment and then make fewer but more comprehensive demonstrations. Students
are strongly encouraged to make demonstrations that tell a coherent story. A
student that realises the connection between “identity and equality” and
“aliasing” will be able to demonstrate both of these achievements together,
and in less time than it would take to demonstrate them both separately.

###  2.2. Influencing Student Behaviour Through Resource Limitation

While there are abundant opportunities to demonstrate, the number of lab
sessions with demonstration slots are limited, as are the number of
achievements that can be demonstrated in an attempt and even successfully
demonstrated in a session. This is an integral part of the course design.
Making the demonstration attempts a limited resource increases their value,
and discourages “waste” in the sense of attempting to pass by brute force
(because students invariably learn something from failed demonstration
attempts). The last two times the course was given, there have been around 30
slots for presenting, and a maximal number of 4 achievements per
demonstration.333About 60 of the 70 achievements are demonstrated in lab
sessions. Thus, a student could theoretically successfully demonstrate enough
achievements to pass the course twice with the highest grade, or fail 50% of
all attempts and still pass with highest grade, within the allowed number of
demonstration attempts. This encourages _clustering of achievements_ to form
larger presentations. Students unable to form synthesis are at a disadvantage
as they will not be able to cluster achievements as effectively. This is on
purpose as the ability to form synthesis is something that we wish to have
reflected in a student’s grade. Students say that the emphasis on clustering
forces them to work with the subject continuously throughout the course (from
course evaluation and interviews).

To discourage students abandoning good practices after successfully
demonstrating mastery, we randomly select achievements from a student’s
collection of passed achievements for re-examination. The re-check is intended
to be superficial, and mostly serve as a means of jogging students’ memory and
keeping knowledge fresh. A successful demonstration cannot be “lost” if a re-
check fails, but the student will be re-checked again until the re-check
succeeds. This way, re-checks can pile up, effectively blocking students from
making progress with other demonstrations.

###  2.3. Achivement Choice & Specification

The actual selection of achievements seem not to be crucial to the course
design and we have tweaked both the number, distribution and content of the
achievements over the last years. Our course focuses on imperative and object-
oriented programming, and emphasises methodology and tools, which is reflected
in the achievements. For example, we have achievements related to using
version control, using the Unix shell, build management and source code
editing.

As part of the goal of forcing students to work with achievements is to force
them to make their own plans and map achievements to activities in the course,
we have experimented with presenting the achievements in list form, where
achievements belonging together were grouped, but without highlighting this or
giving names to the groups. This seems to overwhelm some students who panic at
the sheer volume, and who quickly filter out achievements connected to higher
grades simply to make the list more manageable. Our current design is to give
each group a name and present the achievements in this group together. It
seems that the grouping of assignments in named groups is more important than
the actual groups, likely because it makes the set of achievements less
daunting for the students.

As the course material is currently only available in Swedish, we provide a
list of the groups of achievements used in 2016, and a rough outline of the
achievements in each group in the Appendix, together with an example
achievement.

To simplify the course structure, we try to fit everything inside the
achievement system, even though that sometimes goes against the idea of
achievements. Thus, we represent each completed assignment as an achievement,
as well as the final project. This does not seem to be confusing to students,
but possibly comes at a cost of making the achievements idea less pure. At the
end of the course, there are often students who have demonstrated all
achievements except one or more complete assignments. To us, this highlights
the importance of decoupling skill demonstration from _finished_ artefact
implementation. By giving specific assignments, we ensure that students write
substantial amounts of code in the course,444Throughout the course, the
average student writes a total of 4–5.000 lines of code, not including
introduction labs, extensions of assignments to address a certain achievement,
or the course project. and that some parts of the code survive long enough
that the pain of bad choices, bad naming, etc. can be learned from.

###  2.4. Demonstrating Mastery

With few exceptions, all demonstrations take the form of oral examination in
front of a computer terminal with one examiner and two students working in a
pair. There are several reasons for pairing students together: the students
outnumber the examiners, reducing stress and taking pressure off the
situation; if a student gets stuck, the partner can take over or help out; it
is easier to create a meaningful discussion with more than two people—for
example, the examiner can ask the other student whether she agrees or
disagrees with a statement that the first student made, or if she can propose
a different way of doing something than what the first student just explained.
Examining two students at a time also cuts the total number of demonstrations
in half, saving examiner time.

Checks are individual, meaning that both students must demonstrate mastery. A
mostly passive student will not pass a check even with a brilliant partner. It
is clearly stated that the responsibility for making sure that both students
actively participate is on the students, and not on the examiner. This
encourages students to prepare the demonstration and discuss who says what,
which in our experience makes presentations run faster, have higher quality,
and increase student reflection.

The choice of oral examination serves several purposes. Explaining
interactively is time-efficient, both for the student and examiner, and
feedback is given immediately when the demonstration is still fresh in the
student’s mind. Technical communication is an important soft-skill that is
trained automatically in this setting, and students are both encouraged and
helped to improve their ability to communicate about software development.
Nevertheless, some students complain, saying that they are not interaction-
oriented and much prefer to explain themselves in writing. We explain to them
that communication and interaction are key skills, and do not provide any
alternative means for them to demonstrate.

####  2.4.1. Demonstration Process

Demonstrations take place in designated computer labs and (for students with
laptops) lecture halls, during 4 hour lab sessions, which are scheduled twice
weekly throughout the semester. Students request to demonstrate through a web
page where they choose which achievements they wish to demonstrate. An
achievement for re-check is added automatically by the system, using the
history of passed demonstrations. Re-checks are disabled early in the course,
until students have passed a sufficiently large number of assignments. On
average 7–8 TAs serve 100 students in each session. (About 3 senior TAs and
10–12 junior TAs are attached to the course.)

After requesting to demonstrate, students wait (ideally while continuing to
work). A feed of pending requests to demonstrate is shown on a screen to the
examiners, who claim demonstrations before dispatching to where the students
are located. We follow no special rules for claiming demonstrations: sometimes
it makes sense to have the same examiner claim a student’s second attempt,
other times it makes sense to hand over to someone else. To increase
throughput, it sometimes makes sense to have one examiner claim several
demonstrations in the same room. As the achievements a students wishes to
demonstrate is visible in the feed, examiners can avoid achievements they feel
uncertain about, request help from a more senior examiner, etc.

Students are notified that their request has been picked up and by whom. At
all times, students can see how many requests are waiting and how many are in
front of their request, if any. Students are not allowed to have more than one
pending request or make requests before lab sessions start.

When the examiner arrives, the students must _pitch their demonstration_ ,
which means:

  1. (1)

Stating the achievements they wish to demonstrate

  2. (2)

Explaining why/how it makes sense to demonstrate them together

  3. (3)

Explaining what evidence they will use in the demonstration

If the examiner is not satisfied with 1 or 2, she will explain why and should
reject the request (in practice, this has not been strictly enforced as we
want to avoid discouraging students from _trying_ to pitch a demonstration).
By evidence in 3, we mean artefacts developed in the course that will form the
basis of the demonstration. For example, “we profiled the binary search tree
that we developed in Assignment 2”. The key idea with evidence is that
students must relate the achievements to the programming assignments (and
programming project) of the course. In the course’s initial instalment, we
naively allowed any code to be used in demonstrations prompting students to
search for code rather than write their own, or spending lots of time trying
to write a minimal program that could be used to demonstrate a maximal number
of achievements. While there are merits to both these approaches, we are
seeking to teach how to write imperative and object-oriented code, which is
why we added the requirement of evidence in the form of assignments from the
course. Examiners will often ask questions to force students to go “off-
script”, _e.g.,_ ask what would happen if $X$ was changed for $Y$, or change
some lines of code, or introduce a bug etc.

If the examiner finds the answer to 1–3 satisfactory, she will hand over the
running of the demonstration to the students. Different examiners approach
this task differently. Some will interrupt and ask questions. Some delay all
questions to the end. What approach students prefer seems highly subjective,
with a slight preference for more interactive examiners.

Only the achievements explicitly stated at the beginning of the demonstration
can be passed in the demonstration. If a student in the process of
demonstrating happens to “meet the requirements” for some other achievement,
this will be ignored. Further we will not even tell the student. Our
philosophy is that _you can only know what you know you know_ , meaning that
accidentally demonstrating procedural abstraction not knowing that that is
what you are doing counts as nothing.

Each achievement will be marked as either _pass_ , _fail_ or _fail with push-
back_. A student that fails to demonstrate an achievement is free to try again
at her leisure. The limited attempts plus the waiting time discourages
students from brute forcing their way to a pass by making continuous attempts,
accumulating feedback from the previous examiners to present to the next. A
fail with push-back indicates that the student has misunderstood something
deeply, prompting us to block any re-attempt for the rest of the lab session
to encourage spending some more time understanding that achievement. There are
no special limits on the number of attempts to demonstrate one particular
achievement. Failures with or without push-back are not counted towards any
grade.

An examiner’s decision is final, and the examiner does not have to motivate
failing a presentation. However, we ask examiners to explain and motivate both
passing and failing grades on demonstrations to reinforce learning, build
confidence, and to help students transition from a failed demonstration to a
passing one.

####  2.4.2. Demonstrations & Plagiarism

Because the focus of a demonstration is the skill in relation to the evidence,
we feel that plagiarism is not really a problem. We encourage students to
discuss solutions, look at code written by fellow students, and collaborate.
Because demonstration commonly involves examiners pointing at code, and asking
questions about it, or altering the code as part of the discussion, students
that lack a deep understanding of the code have a hard time passing.
Ultimately, the student is demonstrating some abilities, not whether he or she
wrote a particular piece of code or not.

###  2.5. Lectures

The course uses traditional lectures. Many lectures have a corresponding
screen cast which covers similar content in a different way, often with a
focus on code, most of the time showing an editor and a terminal and not
slides. Several lectures are given through “live coding” where the lecturer
switches between drawing on a whiteboard and projecting an editor and terminal
and incrementally constructing a program from scratch. Each lecture is mapped
to a set of achievements, to help students (predominantly those without much
prior programming experience) make plans for what to demonstrate. Following
constructive alignment, mapping lectures to achievements has been very helpful
for lecture design—what should be covered in this lecture, what can be cut in
the event of running out of time, etc.

####  2.5.1. Discussion

Initially, students fear demonstrating mastery because they are not used to
the process, and find it difficult to understand beforehand. To this end, we
encourage students to attempt presentations early on, not with the goal of
passing, but with the goal of understanding and practising how to give a
demonstration. This concept is eye-opening to many, and is easily explained in
terms of sacrificing an abundant resource (lots of demonstration slots
remaining) in exchange for knowledge that will be useful throughout the entire
course. Usually, once the initial resistance is broken down, students have no
problem requesting demonstrations.

One goal of reducing expected learning outcomes to much smaller achievements
was to reduce the expertise and amount of teacher training required to be an
examiner for demonstrations (especially judging quality more than pass/fail).
The teaching staff on this course is predominantly made up of junior TAs
(bachelor or master students) and senior TAs (PhD students). Because each
achievement demonstration is pass/fail (and push-back), we feel that it is
easy to deliver reasonably consistent quality both on judgement and feedback.
In combination with each student going through at least 40–70 demonstrations,
unfairness due to examiner inconsistencies is not a problem in practice.

The focus on oral presentations put orally-skilled students at an advantage,
but this is mitigated by the evidence requirement and the probing when
examiners change code or introduce bugs. We experience considerable
improvement in presentation quality from many students who initially are
afraid, block, or otherwise are uneasy when forced to talk in-front of others.
We try to motivate the use of oral examination by relating it to technical job
interviews, and, according to the course evaluation, by the end of the course
most students are happy with doing oral presentations.

###  2.6. Working with Phases and Sprints

We divide the course into three phases, one devoted to imperative programming
(currently in C), one to object-oriented programming (currently in Java), and
one devoted to a programming project. The first two phases are further
subdivided into two sprints each, subdivision of the third phase is controlled
by the students themselves. Each sprint is two weeks and has one assignment.
At the end of each sprint is the _soft deadline_ for the assignment. The soft
deadline marks the time when an assignment should be finished for the student
to be perfectly on-time. At the end of the following sprint is the _hard
deadline_ , which is the final deadline for handing in the assignment. Failure
to meet a hard deadline means that the student is derailing and is
subsequently called to a face-to-face meeting with the goal of adjusting the
plan for finishing the course. Again, there is no punishment or points
deducted, but students work hard to meet soft deadlines and very hard not to
break hard deadlines. In fact, the idea of using deadlines as a tool to help
with planning was suggested by students after the first instalment of the
course (which had no deadlines).

As the course progresses, the number of assignments to choose from in each
sprint increases, reflecting our expectancy of student’s increasing maturity
and ability to branch out.

To reduce the administrative burden, we keep track of assignments as
achievements, meaning “Successful demonstration of the Assignment of Phase 1,
Sprint 1” is an achivement. For each sprint, students are asked to plan for a
number of achievements to demonstrate, in a way similar to story points (see
_e.g.,_ Cohn (2004)). We plot progress as remaining number of achievements to
desired grade and visualise this as a burndown chart. We have experimented
with several ways of visualising student progress, most of which led to
_increased_ stress. Burndown charts tell a rich story, allow plotting trends,
exploring possibility of getting several grades at the same time, and show
periods of high/low activity etc. Figure 1 shows an example of burndown charts
generated by students using a provided on-line spreadsheet template. At the
end of each sprint, students meet a TA in groups of $\approx$10 to discuss
their progress, compare burndown charts and trade insights about assignments,
achievements, demonstrations, etc. These meetings serve as a forcing function
for updating the burndown chart and also force students to openly discuss
their (lack of) progress and make plans. While some students are stressed by
always chasing the ideal burndown, they are also very happy about always
knowing what are the possible attainable grades and adjust their desired grade
up and down depending on a wide variety of reasons.

Figure 1. Left: a burndown charts for a student in 2016. The blue, straight
line is the ideal burndown. The red line is actual burndown. The vertical
labels are sprints and phases. Right: all students’ achievements visualised as
a burnup chart. It is clear that students’ progress differ. (Colours have no
meaning.)

The division of the course into discrete units of time is important to make
progression through the course visible, especially given that students are not
required to make demonstrations at any specific time. Judging from experiments
that we have made with more/fewer/longer/shorter sprints, this is a quite
flexible part of the design and finding a good balance between a sense of
urgency and student stress is key.

###  2.7. The Coding Exam

The only guaranteed individual marking on the course is called the coding
exam, in which students use lab computers to solve relatively simple
programming problems without access to the Internet in limited time. Cutting
students off from the Internet breaks the work-flow of many students who are
used to getting unstuck by searching and visiting web sites like Stack
Overflow, or discussing with a partner. We see that students must be
encouraged explicitly and repeatedly to practise solving problems alone
and—importantly—off-line, regardless of their programming skills.

While passing the coding exam is mandatory, it is strictly a pass/fail
examination and does not affect the final grade—its only purpose is to present
an obstacle that students will never be able to get past without actually
learning how to program (to some limited degree). In our experience, many
students have such low confidence in their programming ability that they will
avoid touching the keyboard unless forced to. These students pair up with
other more confident students and let them sit in the driver position
throughout the course, and will learn many concepts shallowly as is inevitable
without any actual coding. We are completely open with the students that the
coding exam is an artificial obstacle that is there to scare the students that
would otherwise never leave the passenger seat to take place in front of the
keyboard.

Because the coding exam is done on lab computers, students have access to the
editors and IDEs that they use in the rest of the course, and important tools
like gdb and valgrind. We also provide test cases that can be run easily from
the command prompt, allowing students to test their solutions. Ability to
compile and run test is a double-edged sword, however, as some students go
into panic mode when faced with hard error messages, and get stuck on
preliminaries such as forgetting what are the arguments to main in C rather
than focus on the parts that the exam wanted to test.

To some extent, the code exam can be corrected automatically by just running a
set of tests on the hand-ins. Students passing the code exam gets no other
feedback than a passing grade. Hand-ins for which the tests fail will be
manually inspected and the code handed back to the student annotated with a
given a list of errors and bad smells in the code, which corresponds to
numbers in a lengthy code exam report posted on the course web page.555Example
(in Swedish, with lots of code examples):
http://wrigstad.com/ioopm/20161221.php. Because of automatic testing, results
sans feedback can usually be given within 24 hours of the code exam. Only
functional correctness determines pass/fail. This is partly to reduce stress
and help students to focus, but also because code quality comes up in almost
all other parts in the course.

###  2.8. Final Grade

A student’s final grade is decided solely by the number of achievements
successfully demonstrated. The achievements are stratified after grade levels
to match qualitative assessment that could perhaps have been made using more
comprehensive grading of handed-in assignments, and in that sense “quantifying
quality.” We do not allow, for example, substituting one achievement on level
5 for another on level 4, etc. The achievements match the learning outcomes
decided by (in our case) higher powers, and when read in that light, it does
not make sense to substitute, for example, procedural abstraction for
knowledge of garbage collection.666A system identifying achievements that make
sense to allow substitution can of course be constructed, but would add more
complexity for questionable gain. Students initially baulk at the concept of
getting 100% (of anything) to get a corresponding grade, possibly because of
how other courses allow “points trading.”

##  3\. (Self-)Evaluation

In this section, we report on feedback from students (based on the 2016 non-
mandatory course evaluation which asked 53 questions and was filled out by 80
students), teaching assistants as well as our own reflections. We concentrate
on feedback on the achievement system and not on the course on a whole.
Feedback from teaching assistants has not been collected through any
systematic means.

###  3.1. Feedback from Students

In the 2016 course instalment, we felt that we had finally understood how to
explain the achievement system to students: by introducing it as a set of
checkboxes that they had to tick off at some point that made sense to them
during the course. Students commented very positively on the achievement
system in the course evaluation, and suggestions for change were generally in
the form of (perceived) improvements such as “merge these achievements”,
“provide more suggestions for clustering of achievements” or “clarify
achievement X to make it more clear”. Almost 3/4 of the students report that
they grouped achievements together for demonstration based on how the
achievements fit together. Almost 2/3 of students think that the setup with
constant small checks helped them avoid procrastinating, though a little less
than 1/3 disagree. Notably, 45% of students think that their stress-level
increased due to the large number of checks.

90% of the students felt that it was always clear to them what grades were
attainable (66% strongly, 24% less strong). 75% of the students felt that the
achievement system has helped them absorb the contents of the course. Students
generally perceive being more responsible for driving their own studies
(_e.g._ , clustering achievements, planning etc.) as something good.

1/3 of the students think that the oral examination has _greatly_ influenced
their ability to explain, motivate and communicate. Another 51% agree but to a
lesser extent. There is a strong preference for immediate oral feedback over
delayed feedback in written form.

40% of the students do not think that they would have gotten more out of the
course by doing _fewer_ oral checks, though 26% think that they would have
gotten a little more out of it. 18% think they would have gotten a lot more
out of it. Notably, oral checks get the most complaints due to the time
students spend queuing, waiting to demonstrate. Invariably in the last years,
illness or other unfortunate circumstances have caused a shortage of examiners
a few times each semester, causing waiting times to inflate and students to
complain. In 2016, the average waiting time was 53 minutes between asking to
demonstrate and having the grade entered into the system (so this includes
demonstration time). To our mind, this is relatively short, but there are also
many demonstrations. Some students say that “waiting time is not a problem,
since there is always something to work on while waiting.” Some students spend
the time waiting in the queue _waiting_ , which seems wasteful. In future
work, we wish to understand this behaviour — is it because a looming
demonstration is stressful, because a pending demonstration is an alibi to
take a break, because of fear of swapping out a rehearsed demonstration, etc.

###  3.2. Feedback from Teaching Assistants

An effect of allowing students to demonstrate any achievement at any time is
that teaching assistants need to be prepared to assess a mastery demonstration
about any part of the course material at any time777In practice though, there
are some natural constraints on which achievements are actually demonstrated.
For example, achievements concerning object-oriented programming are typically
not demonstrated until later in the course, and students often tend to
demonstrate achievements that their peers have successfully demonstrated. .
This is very different from other courses where students typically all work
with the same assignment at the same time. Senior TAs and more experienced
junior TAs see this as something positive with the course, as the work becomes
more stimulating and less monotonic. Junior TAs are sometimes instead stressed
by this, especially early on in the course, because they feel like they have
to be able to answer any question about the course. To mitigate this we
encourage TAs to ask each other for help, and to sit in and listen to each
other’s mastery assessments in order to learn from each other. Throughout the
course we also have semi-regular lunch meetings together with the TAs to
discuss what works well and not, as well as trading “assessment strategies”
for different achievements. TAs that have taken the course themselves are
typically more comfortable with the assessment situation as they have
experienced it from the other side.

Many TAs are stressed when there is high pressure on the demonstration request
list and long waiting times in the lab rooms. In these situations, TAs
sometimes have a hard time balancing the quality and quantity of assessments;
giving more detailed feedback means that other students have to wait even
longer for their chance to demonstrate.

###  3.3. Our Own Reflections

The main negative aspect of switching to the achievement-based system is that
a lot of time that was previously spent casually chatting with students during
lab sessions is now spent taking demonstrations. This leaves less time for
interested students to discuss with teaching assistants, and less time to form
stronger connections with students. It is also likely that students spend less
time programming, and more time reflecting, rehearsing, and demonstrating.

In our opinion, students learn _more_ and are more aware of what they learn.
Students are able to discuss concepts like abstraction, modularity,
inheritance, etc. at a much higher level and relate these concepts to their
own code. To a greater extent than before, students also form _opinions_ about
concepts and constructs, rather than just accepting them as decreed. To an
overwhelming extent, students now think of themselves as the active agents in
their learning, in contrast to previous years when students were expecting to
_be taught_. To a larger extent than before, students are aware of their own
strengths and weaknesses, and ability to plan and execute plans. The ability
to reflect on their own learning was also observed by Matthias Hauswirth, a
professor from Lugano visited the course for a day in 2016 and carried out
interviews with several students.888Matthias Hauswirth, personal
communication.

##  4\. Summary & Conclusion

Four years ago, we radically changed our course on imperative and object-
oriented programming to a design where students are forced to take
responsibility for their own learning. By reifying the formal course goals as
a large number of small achievements to “unlock”, we give the students a
quantifiable measurement of progress throughout the course. By separating the
achievements from the mandatory assignments, the students themselves get to
make the connection between the course concepts and their code.

In our experience, the students are able to explain and discuss course topics
at a higher level than before, and after the third iteration of the course, we
are at a point where a majority of students are at least as satisfied with the
course as they were before the change. The course requires a large staff of
teaching assistants to handle the oral examinations, but because the course
produces students who are well-versed in the course contents, there is never a
shortage of junior teaching assistants to employ. Scaling the course to
accommodate a growing number of students in the future requires employing more
teaching assistants to keep the waiting time in the lab sessions sufficiently
low.

Because the course has changed in many ways (_e.g.,_ doubled in size) it is
hard to say whether the course is more or less labour-intensive than before
switching to achievements. It is budgeted the same way as all other courses at
the department, and fits in that budget. A cost-neutral difference is that we
seem to go through _more_ TAs each working _fewer_ hours than before. Our
guess is that this is because TA work is now more intense as it is more
oriented towards examination than before.

###  4.1. Lessons Learned

We end with some of the lessons we have learned while teaching this course.

Over the years, we have reduced the number of achievements slightly, mostly to
make the list more manageable for students. Achievements that are hard to
group with other achievements were the first to go. Related to this point,
changes to the achievement system is a pain when dealing with old students,
who did not pass the course (to some degree or at all) in a previous year. It
is hard to translate the progress from one year to the other when the set of
achievements are different. This problem can be alleviated by supporting
partial grading. In our course we give 25% of all credits for completing 50%
of the mandatory assignments (excluding the project) and $\approx$50% of the
achievements needed for a passing grade (again, excluding achievements tied to
the project). This way, students who fall far behind for some reason can aim
to finish _e.g.,_ the assignments for the imperative phase and finish the
object-oriented phase from the beginning the following year.

Due to the large number of demonstrations, computer support is crucial, both
for handling the queue of demonstration requests and to keep track of the
progress of each student. The large volume also means that errors will happen,
meaning that the backend needs to support manual input of results. The fact
that everything is recorded digitally is also a great source for data-mining.
We can extract which achievements are currently the most popular, which
achievements most students fail, statistics on waiting times etc. We can also
report on the progress of the entire class to help students understand how
their progress relates to the average progress of the class.

###  4.2. Final Remark

Achievement Unlocked is an interesting combination: it has elements of mastery
learning (Bloom, 1968), flipped class-room (Bergmann and Sams, 2012) (in the
sense of students doing lots of work at home and then come to discuss with
teachers and TAs in lab sessions), gamification (with respect to resource
management and unlocking of grades), and its design was heavily influenced by
our belief in constructive alignment (Biggs, 1996). As course designs go, it
is complex, but purposely so—with the intent of forcing students to actively
work with their learning process and reflect on study techniques, but
reasonably scalable to the extent affordable junior assistants are available.

## References

  * (1)
  * Bergmann and Sams (2012) Jonathan Bergmann and Aaron Sams. 2012.  Flip your classroom: Reach every student in every class every day.  International Society for Technology in Education.
  * Biggs (1996) John Biggs. 1996\.  Enhancing teaching through constructive alignment.  Higher education 32, 3 (1996), 347–364.
  * Bloom (1968) Benjamin S Bloom. 1968\.  Learning for Mastery. Instruction and Curriculum. Regional Education Laboratory for the Carolinas and Virginia, Topical Papers and Reprints, Number 1.  Evaluation comment 1, 2 (1968), n2.
  * Cohn (2004) Mike Cohn. 2004\.  User stories applied: For agile software development.  Addison-Wesley Professional.
  * Guskey (2010) Thomas R Guskey. 2010\.  Lessons of mastery learning.  Educational Leadership 68, 2 (2010), 52\.
  * Joint Task Force on Computing Curricula and Society (2013) Association for Computing Machinery (ACM) Joint Task Force on Computing Curricula and IEEE Computer Society. 2013.  Computer Science Curricula 2013: Curriculum Guidelines for Undergraduate Degree Programs in Computer Science.  ACM, New York, NY, USA.  999133\.

##  Appendix A Achievements at a Glance

The list below gives a high-level overview of what the achievements in the
course are and how they are grouped together. Remark that the grouping serves
no other purpose than to help students abstract and thus reduce the perceived
volume of skills to learn and demonstrate mastery of.

Abstraction:

Achievements in this group concern procedural and object-oriented abstraction,
the importance of naming, and information hiding.

Code Review:

Informal code reviews, responding to code reviews, refactoring.

Communication:

Essay writing, giving a structured presentation, working as a teaching
assistant. The last achievement is on the highest grade-level and forces
students to work as TAs during half a lab session where they work through the
help list, and the students they are helping give them feedback. Part of the
reason for having the helped students producing feedback is so that the helper
must think actively about what the goals are of helping—not solving their
problem, but helping them coming up with the solution themselves.

Documentation:

The single achievement in this group deals with writing interface
documentation for other programmers to use.

Encapsulation:

Aliasing, name-based encapsulation, nested and inner classes, interplay of
strong encapsulation and testing.

Generics:

Dealing with generics both in Java and C, and designing with/for parametric
polymorphism.

Imperative Programming:

Recursive vs. iterative solutions, tail recursion elimination.

Inheritance:

Object-oriented inheritance, overriding, overloading, subtype polymorphism,
separation of concerns.

Memory Management:

Allocation on stack vs. heap, manual memory management, automatic memory
management, manual vs. automatic memory management.

Methodology:

Defensive programming, exception handling, failure management and fault
tolerace.

Modularisation:

Module boundaries and interfaces, coupling and cohesion, separation of
concerns.

Object, identity & structure:

Identity vs. equality, value semantics, concrete vs. abstract classes.

Planning:

The single achievement in this group deals exclusively with planning and
following-up, which is covered in Section 2.6.

Pointers:

Pointers and arrays in C, pointer-based linked structures, pointer semantics,
indirection and pointers to pointers.

Pragmatics:

Compilers, interpreters, JITing, linking, and binding.

Profiling & Optimisation:

Profiling for performance and memory usage, optimising performance and memory
usage guided by profiling results.

Testing:

Unit testing, test quality, static anlysis, fuzzing.

Tools:

Debuggers, documentation tools, editors, build tools, working the terminal.

Assignments:

This group collects the achievements mapping directly to assignments (_e.g.,_
Assignment in Phase 1, Sprint 1, etc.).

Project:

This group collects the achievements that are part of the programming project
at the end of the course. These are not demonstrated in the usual way, but we
unify them with the other assignments for simplicity. Project achievements
include using and evaluating the usefulness of pair programming, regression
testing, working with pull requests and code reviews, coding standards, etc.

###  A.1. Example Achievement: Implementing Generic Data Structures with Void
Pointers

Grade level: 3
Demonstrated: in lab sessions
_Use void pointers in C programs in relevant ways to implement genericity, for
example, a collection capable of storing any kind of data._
Multiple copies of the same code with minor differences is undesirable.
Consider for example two separate implementations of lists of integers and
lists of floats where the second was created from the first. If a bug is found
in one list, two sets of bug fixes must be carried out in equivalent ways to
make sure the lists’ behaviour stays identical.
C supports pointers to data of unknown type (void pointers, void *) which can
be used to build a general list (for example). Constructing a list where
elements are void pointers allows the list to hold arbitrary data.
Void pointers is a common C idiom that is also used in many places in the
standard C library to identify a memory location with unknown content. The
compiler does not know what is at the location, nor the size of the object at
the location.
Demonstrate your understanding of the void pointer idiom, and how the
compiler’s lack of knowledge of the type of the object at an address
influences programming. Start your demonstration by explaining the concept of
genericity, how and when it is useful, and what you think about C’s support
for genericity.

##  Appendix B Course Time-Line

With few exceptions, each week has two 2-hour lectures and three scheduled
4-hour sessions for demonstration (and getting help).

C Bootstrap (2 weeks):

(Broad but not deep): loops, recursion, basic I/O, functions, arrays,
pointers, strings, modularisation and separate compilation, typedefs and basic
data types, structs, unions, function pointers. What is good code?
All C assignments are made available at the end of the bootstrap. All Java
assignments are made available in C Sprint 1 or early in C Sprint 2. Each
sprint has one assignment and the soft deadline for the assignment is at the
end of that sprint. Hard deadline at the end of following sprint.
There are five mandatory labs in the bootstrap. Soft deadline on the same day
as the lab is scheduled. Hard deadline on the following lab.

C Sprint 1 (3 weeks):

(Students build a small interactive program.) Recaping from the bootstrap.
Imperative vs. functional programming. Stack, heap and manual memory
management. Pointers. Top-down and bottom-up design of programs. Layer design.
Scripting and automation. Basic testing. What is a good programmer?

C Sprint 2 (3 weeks):

(Students rebuild parts of the program in more dynamic ways, eg. replacing
arrays with trees, carve out reusable libraries and swap with each other.)
Linked structures and iterators. Macros and the C preprocessor. Defensive
programming. Modules, coupling and cohesion. Readable code. Bit manipulation.
Profiling and optimisation.

Java Bootstrap (1 week):

Objects and classes. Basic Java. What parts of C carry over to Java. Automatic
memory management. Execution environments (VMs, JITs, etc.)

Java Sprint 1 (3 weeks):

OOAD. Constructors. Initialisation. References. Inheritance. Overloading and
overriding. Abstract classes. Interfaces.
Students choose between a classic OO simulation (cashiers and waiting lines in
a store), or a very simple micro blogging application with a text-only
interface that uses the network. (Lots of code is provided for the latter.)

Java Sprint 2 (3 weeks):

Parametric polymorphism. Testing. Static and dynamic binding. Profiling.
Refactoring. Java Bytecode. Garbage collection.
Students choose between implementing a simple text-oriented MUD or a symbolic
calculator. Both assignments focus on object structures, interfaces,
inheritance and polymorphism.

Project (4 weeks):

Basic Scrum.
The project implements a basic automatic memory management system for simple C
programs (a conservative Bartlett-style collector).


# Analyzing and Mitigating the Impacts of GMD and EMP Events on the Electrical
Grid with PowerModelsGMD.jl

Adam Mate 1*  —, Arthur K. Barnes 1*  —, Russell W. Bent 1*  —, and Eduardo
Cotilla-Sanchez 2*  — 1 The authors are with the Advanced Network Science
Initiative at Los Alamos National Laboratory (LANL ANSI), Los Alamos, NM 87544
USA. Email:{amate, abarnes<EMAIL_ADDRESS>The author is with the School of
Electrical Engineering and Computer Science, Oregon State University,
Corvallis, OR 97331 USA<EMAIL_ADDRESS>Approved
for public release; distribution is unlimited.

###### Abstract

Geomagnetic disturbances and E3 high-altitude electromagnetic pulse events
pose a substantial threat to the electrical grid by adversely impacting and
damaging high-voltage transmission networks and equipment. To evaluate the
risks and mitigate the potential effects of these hazards, this work proposes
PowerModelsGMD.jl (abbr. PMsGMD). PMsGMD is an open-source Julia package that
solves for quasi-dc line flow and ac power flow problems in a system subjected
to geomagnetically induced currents. Unlike commercially available software
solutions, it is extensible and applicable to a variety of problems. The
flexibility of this framework is demonstrated by applying it to the problem of
identifying mitigation strategies via line switching for a time-extended
transformer heating problem.

An overview of PMsGMD is presented in this paper: introduction to its design,
validation of its implementation, demonstration of its performance and
effectiveness, and a description of how it may be applied to aid system-
operation decisions.

###### Index Terms:

power system analysis, geomagnetic disturbance, electromagnetic pulse, optimal
power flow, Julia, open-source.

##  I Introduction

High-impact low-frequency (abbr. HILF) events are the greatest threat to the
continuous and reliable operation of our energy infrastructures. Such events
have the potential to cause unpredictable system-wide disruptions and long-
term damage in the electrical grid, a key component of the critical
infrastructures. Geomagnetic disturbances (abbr. GMDs) and high-altitude
electromagnetic pulse (abbr. HEMP) events are among these extreme hazards [1,
2].

GMDs are caused by intense solar activity: charged and magnetized particles
are blown away from the Sun during severe space weather, which then interact
with and disrupt the Earth’s magnetic field causing rapid changes in its
configuration. These disturbances are mainly driven by large solar flares and
associated coronal mass ejections during solar maximums, and by co-rotating
interaction regions (high-speed solar winds) during solar minimums [3, 4, 5].

HEMP events are caused by nuclear explosions detonated high up in the
atmosphere. These are series of electromagnetic waveforms, covering times from
nanoseconds to hundreds of seconds, that propagate to the Earth’s surface.
Three main waveforms are generated during a detonation, among which the E3
late-time waveform produces electric fields with comparable time scales and
area coverage as those of geomagnetic storms. However, these events are likely
to have a higher peak field level resulting in greater impact and more severe
damage than naturally-occurring solar flares [6, 4].

Both GMD and E3 HEMP events pose a risk to the electrical grid by generating
geomagnetically induced currents (abbr. GICs), quasi-dc currents that appear
in the conductive infrastructure and flow into the high-voltage network
through the neutrals of power transformers [5, 6, 7, 4]. GICs may adversely
impact transmission networks and equipment as they have the potential to
induce harmonics by causing half-cycle saturation in transformers. Harmonics
may lead to the misoperation of protective devices, causing tripping of over-
current relays. Premature aging, lasting damage, or complete failure of large
high-voltage transformers due to overheating and thermal degradation is also a
great threat. Increased reactive power consumption, caused by the circulating
GICs in the system, may lead to the loss of reactive power support and to
voltage collapses. In the worst case, widespread infrastructure damage and
tripping of transmission lines may result in cascading failures and extended
power disruptions [8, 9, 10, 11, 12].

Understanding the danger that power systems face is critically important. This
paper presents an overview of PowerModelsGMD.jl111https://github.com/lanl-
ansi/PowerModelsGMD.jl (abbr. PMsGMD), a free and open-source package for
power system simulation, which was specifically designed to evaluate the risks
and mitigate the impacts of the above mentioned hazards. The PMsGMD framework
is implemented in Julia [13], a high-level just-in-time compiled programming
language designed specifically for scientific computing. It is an extension to
the PowerModels.jl (abbr. PMs) platform [14], a Julia/JuMP package for solving
and evaluating steady-state power network optimization problems. JuMP [15] is
a package for mathematical programming in Julia that specifies problems with
algebraic constraints.

The key contributions of this paper include:

  * •

An extensible, open-source software for modeling GICs in power systems.

  * •

The first time-extended model of GIC that includes detrimental effects on
transformers and that is appropriate for use in a mitigation optimization
setting.

  * •

A detailed discussion on modeling requirements for GIC calculations.

The remainder of this paper is organized as follows: A review of the PMsGMD
package and background of key problem formulations are provided in Section II;
it gives context to this work and explains design goals. Section III presents
the implemented steady-state formulations, and Section IV describes the
specifications of the time-extended GIC mitigation problem. Section V and VI
validates the implementations and provides a practical example of how this
framework can be used both in power system planning and operations
environments. Section VII finishes with a few concluding remarks.

##  II Problem Formulations

###  II-A Context of PMsGMD

The Julia programming language [13] is a high-level, high-performance,
flexible dynamic language, which is appropriate for technical computing, with
performance comparable to traditional statically-typed languages. Julia has
many free and open-source packages available to provide specific toolkits and
capabilities for diverse applications.

PMs [14] is a package for power system simulation: it provides a flexible
platform for implementing and solving a wide range of steady-state network
optimization and analysis problems; among others, it includes implementations
of power flow (abbr. PF), optimal power flow (abbr. OPF), and optimal
transmission switching (abbr. OTS) problem specifications. It relies on JuMP
[15], which provides an ideal modeling layer for the wide range of
optimization problems that arise in power systems research. The use of at
least one solver is required; e.g., Ipopt [16] a fast and scalable solver for
non-convex non-linear optimization. Commercial solvers such as Gurobi [17] and
CPLEX [18] are supported as well.

In recent years PMs has emerged and filled a gap between ad-hoc research code
and commercial tools such as PSS®E and PowerWorld® Simulator. It is similar to
the open-source MATPOWER package [19] that is widely used in the academic
environment to conduct research; however, dependence on the MATLAB®
environment, licensing challenges for parallel calculations in a computing
cluster, and difficulty in creating extensions present great limitations.
There have been efforts to improve GIC modeling and include related analysis
in power system simulation [20, 21, 22, 23, 24]. A number of free and
commercially available software solutions exist – MATGMD [25], PowerWorld®’s
GIC add-on [26], and PSS®E’s GIC module [27] – however, these all are focused
on modeling and analysis with often unverifiable and non-customizable
capabilities. The PMsGMD package presented here builds on the PMs platform and
provides an accessible and easy-to-handle framework to both analyze and
mitigate the impacts of GMD and E3 HEMP events on electrical grids.

PMsGMD solves for quasi-dc line flow and ac power flow on a network subjected
to GICs. It solves for mitigation strategies by treating the transformer
overheating problem as an optimal transmission switching problem. Due to its
open-source nature, it is easy to verify and customize its operation in order
to best fit the application circumstances. Due to its speed and reliability,
it is suitable to be a key component of toolkits (such as [28]) that monitor
GMD manifestations in real-time, that predict GICs on the electrical grid,
that assess risk, that enhance reliability by providing aid to system-
operators, and that recommend modifications in the network configuration.
Consequently, PMsGMD is equally useful for research and industry application.

###  II-B Input File Format

PMsGMD uses several extensions to the PMs data format [14] to provide input
for its problem formulations. For generality, it uses a separate dc network
defined by gmd_bus and gmd_branch tables. To correctly calculate the increased
reactive power consumption of each transformer, the branch_gmd table adds all
winding configuration related data; furthermore, the branch_thermal table adds
thermal data necessary to determine the temperature changes in transformers.
The bus_gmd table includes the latitude and longitude of buses in the ac
network for use in distributionally robust optimization [29] or for
convenience in plotting the system.

The description of B4GIC [30], an included four-bus test case is presented
below to demonstrate the use of the PMsGMD data format and introduce each
input field. Some fields and descriptions are abbreviated due to space
constraints; transformer is abbreviated as xfmr.

1) GMD Bus Data Table: Table I.

_parent_ : | index of corresponding ac network bus
---|---
_status_ : | binary value that defines the status of bus
_g_gnd_ : | admittance to ground [S]
_name_ : | a descriptive name for the bus

TABLE I: mpc.gmd_bus parent | status | g_gnd | name
---|---|---|---
1 | 1 | 5 | ‘dc_sub1’
2 | 1 | 5 | ‘dc_sub2’
1 | 1 | 0 | ‘dc_bus1’
2 | 1 | 0 | ‘dc_bus2’
3 | 1 | 0 | ‘dc_bus3’
4 | 1 | 0 | ‘dc_bus4’

2) GMD Branch Data Table: Table II.

_f_bus_ : | ‘from’ bus in the gmd bus table
---|---
_t_bus_ : | ‘to’ bus in the gmd bus table
_index_ : | index of corresponding ac network branch
_status_ : | binary value that defines the status of branch
_br_r_ : | branch resistance [$\Omega$]
_br_v_ : | induced quasi-dc voltage [V]
_len_km_ : | length of branch [km]
_name_ : | a descriptive name for the branch

3) Branch GMD Data Table: Table III.

_hi_bus_ : | index of high-side ac network bus
---|---
_lo_bus_ : | index of low-side ac network bus
_gmd_br_hi_ : | index of gmd branch corresponding to
| high-side winding (two-winding xfmrs)
_gmd_br_lo_ : | index of gmd branch corresponding to
| low-side winding (two-winding xfmrs)
_gmd_k_ : | scaling factor to calculate reactive
| power consumption as a function
| of effective winding current [p.u.]

TABLE II: mpc.gmd_branch f_bus | t_bus | parent | status | br_r | br_v | len_km | name
---|---|---|---|---|---|---|---
3 | 1 | 1 | 1 | 0.1 | 0 | 0 | ‘dc_xf1_hi’
3 | 4 | 2 | 1 | 1.001 | 170.788 | 170.788 | ‘dc_br1’
4 | 2 | 3 | 1 | 0.1 | 0 | 0 | ‘dc_xf2_hi’
TABLE III: mpc.branch_gmd hi_bus | lo_bus | gmd_br_hi | gmd_br_lo | gmd_k | gmd_br_se | gmd_br_co | baseMVA | dispatch | type | config
---|---|---|---|---|---|---|---|---|---|---
1 | 3 | 1 | -1 | 1.793 | -1 | -1 | 100 | 1 | ‘xfmr’ | ‘gwye-delta’
1 | 2 | -1 | -1 | -1 | -1 | -1 | -1 | 1 | ‘line’ | ‘none’
2 | 4 | 3 | -1 | 1.793 | -1 | -1 | 100 | 1 | ‘xfmr’ | ‘gwye-delta’
TABLE IV: mpc.branch_thermal xfmr | temp_amb | hs_inst_lim | hs_avg_lim | hs_rated | to_time_c | to_rated | to_init | to_inited | hs_coeff
---|---|---|---|---|---|---|---|---|---
1 | 25 | 280 | 240 | 150 | 71 | 75 | 0 | 1 | 0.63
0 | -1 | -1 | -1 | -1 | -1 | -1 | -1 | -1 | -1
1 | 25 | 280 | 240 | 150 | 71 | 75 | 0 | 1 | 0.63

_gmd_br_se_ : | index of gmd branch corresponding to
---|---
| series winding (auto-xfmrs)
_gmd_br_co_ : | index of gmd branch corresponding to
| common winding (auto-xfmrs)
_baseMVA_ : | [MVA] base of xfmr
_type_ : | type of branch – ‘xfmr’ / ‘transformer’
| or ‘line’ or ‘series_cap’
_config_ : | winding configuration of transformer

4) Branch Thermal Data Table: Table IV.

_xfmr_ : | binary value that defines if branch is an
---|---
| ’xfmr’ or not
_temp_amb_ : | ambient temperature of xfmr [$\degree\mathrm{C}$]
_hs_inst_lim_ : | 1-hour hot-spot temp. limit of xfmr [$\degree\mathrm{C}$]
_hs_avg_lim_ : | 8-hour hot-spot temp. limit of xfmr [$\degree\mathrm{C}$]
_hs_rated_ : | hot-spot temperature-rise of xfmr
| at rated power [$\degree\mathrm{C}$]
_to_time_c_ : | top-oil temperature-rise time-constant
| of xfmr [min]
_to_rated_ : | top-oil temperature-rise of xfmr at
| rated power [$\degree\mathrm{C}$]
_to_init_ : | initial top-oil temperature of xfmr [$\degree\mathrm{C}$]
_to_inited_ : | binary value that defines the initial top-
| oil temperature of xfmr: 1 with to_init
| value, 0 with steady-state value
_hs_coeff_ : | relationship of hot-spot temperature rise
| to Ieff [$\degree\mathrm{C/A}$]

5) Bus GMD Data Table: Table V.

lat: | latitude coordinate of ac network bus and
---|---
| corresponding dc network bus
lon: | longitude coordinate of ac network bus and
| corresponding dc network bus

TABLE V: mpc.bus_gmd lat | lon
---|---
40 | -89
40 | -87
40 | -89
40 | -87

The related fields of Branch GMD Data (Table III) and Branch Thermal Data
(Table IV) tables are set to ($-1$) when a system component is not a
transformer. PMsGMD is able to model both fixed (spatially uniform) and
realistic (non-uniform) electric fields as it does not model line coupling.

###  II-C Modeling GIC Impact

The goal of modeling is to accurately simulate GICs and determine the exact
threat at any particular place and time in a power system. It is a complex
task as the level of detail required to model a system during a GIC is more
than what is needed in a traditional positive-sequence simulation [22]. GICs
are dependent on system characteristics (geographical location of substations,
resistance of system components, characteristics of power transformers),
geomagnetic source fields (amplitude, frequency content, spatial
characteristics), and the Earth conductivity structure (modeling method,
substation grounding resistance, and influence on geo-electric fields); all of
these need to be considered in the modeling process. The greatest challenge is
the availability of data; verifiable information on the threat and detailed
description of the system is needed for accurate results.

First, the equivalent dc network must be created from the ac power system [5].
As the impedances of each phase of the system are identical – i.e., the phase
conductors (transmission lines and transformer windings in each phase) provide
identical parallel paths for GIC flows – the calculations of GICs are
performed for a single phase only, and the same result applies to each phase.
Each resistive branch is replaced by its corresponding admittance value and
voltage sources. Transmission lines with series capacitive compensation are
omitted in the dc model as series capacitors block the flow of GIC.
Transformers are modeled with their winding resistance to the substation
neutral and, in the case of auto-transformers, both series and common windings
are represented explicitly.

PMsGMD currently supports “gwye-gwye”, “gwye-delta”, “delta-delta”, and “gwye-
gwye-auto” transformer winding configurations. When unknown, this can be
estimated:
$\bullet$ GSU transformers: delta-gwye with delta on low side
$\bullet$ load transformers: delta-gywe with delta on high side
$\bullet$ gwye-gywe-auto-transformers: connect portions of the transformer
grid where voltage ratio $\leq$ 3
$\bullet$ gwye-gwye transformers: connect portions of the transformer grid
where voltage ratio $>$ 3.

In practice, the actual geo-electric field varies with geographical locations.
Using a common assumption that the north and east components of the geo-
electric field are constant in the geographical area of the transmission line
[31, 32, 22], the induced voltage $\mathcal{V}$ is calculated as

$\allowdisplaybreaks\small\mathcal{V}=\mathcal{E}_{N}L_{N}+\mathcal{E}_{E}L_{E}=|\mathcal{E}|(\sin(\phi)L_{N}+\cos(\phi)L_{E})$
(1)

where $L_{N},L_{E},\mathcal{E}_{N},\mathcal{E}_{E}$ and $\phi$ are as
described in the Nomenclature and in Appendix I of [22]. Given their short
length, $\mathcal{V}=0$ is assumed for transformers.222This formulation
assumes that $\phi$ is measured in the counter-clockwise direction from the
positive x axis. In geography, it is convention to measure $\phi$ in the
clockwise direction from the positive y axis, consequently conversion is
necessary sometimes.

The substation grounding resistance is often approximated using nonlinear
regression based on substation size, maximum kV level, and number of incoming
lines as predictor variables.

###  II-D Transformer Modeling

The formulation presented in [33] contains two equivalent power system models:
one for computing the ac power flows and one for computing the GICs; for some
formulations, coupling constraints are added to link GICs to the ac power flow
model in the form of reactive power losses. The main difference between them
occurs in transformer modeling. AC power flow models typically model
transformers as a single edge with a voltage transformation (phase shift and
tap change). GIC models, on the other hand, require models of transformers
that include details of the series and common windings, as well as other
transformer components [34].

Effective GIC in the ac network is computed using the following set of
equations:
for $\forall e\in E^{a},t\in T$

$\allowdisplaybreaks\small\widetilde{I}_{e}^{t}=\begin{cases}\mathinner{\\!\left\lvert
I^{t}_{e^{H}}\right\rvert}&\text{if $e\in E^{\Delta}$}\vspace{.1cm}\\\
\mathinner{\\!\left\lvert\frac{\alpha_{e}I^{t}_{e^{H}}+I^{t}_{e^{L}}}{\alpha_{e}}\right\rvert}&\text{if
$e\in E^{y}$}\vspace{.1cm}\\\
\mathinner{\\!\left\lvert\frac{\alpha_{e}I^{t}_{e^{S}}+I^{t}_{e^{C}}}{\alpha_{e}+1}\right\rvert}&\text{if
$e\in E^{\infty}$}\vspace{.1cm}\\\ 0&\text{otherwise}\end{cases}$ (2)

where Case 1 models a gywe-delta generator step-up (GSU) transformer (dc
equivalent circuit for this case is illustrated in Fig. 1a), Case 2 models a
gywe-gywe transformer (dc equivalent circuit for this case is illustrated in
Fig. 1b), and Case 3 models a gywe-gwye-auto-transformer (dc equivalent
circuit for this case is illustrated in Fig. 1c).

(a) gywe-delta

(b) gywe-gywe

(c) gywe-gywe-auto

Figure 1: DC equivalent circuit of different transformer-windings

As [33] notes, most test networks in the literature neglect GSU transformers.
GSUs are used to connect the output terminals of generators to the
transmission network. The GSUs and the neutral leg ground points they provide
are critical when modeling GICs and methods to mitigate the impact of GICs.
Consequently, in network models that lack GSU transformers, the model is
assumed to include GSU transformers using the method discussed in [33]. For
the work presented in this paper, it is assumed that GSU transformers are
delta-gywe transformers.

Here, for the first time, the model of [33] is adapted to a multi-period
dispatch to model the GIC impacts on power systems. The contribution is a
model of temperature change over time as a function of changing GIC and ac
flows. Instead of the equivalent Norton representation, the simpler Thevenin
representation is used as PMs [14] supports series voltage sources. The
continuous-time formulation of the transformer hot-spot thermal model is a 1st
order linear dynamic model which is defined in terms of hot-spot temperature
rise over top-oil temperature.

Given the active power $p_{e,ij}^{t}$ [p.u.] and reactive power $q_{e,ij}^{t}$
[p.u.] flowing into the input terminal of the transformer at time $t$ [min],
the apparent power is defined by

$\allowdisplaybreaks\small
s_{e,ij}^{t}=\sqrt{(p_{e,ij}^{t})^{2}+(q_{e,ij}^{t})^{2}}$ (3)

Fractional loading of the transformer is defined by

$\allowdisplaybreaks\small k_{e}^{t}=\frac{s_{e,ij}^{t}}{\overline{s}_{e}}$
(4)

Steady-state top-oil temperature rise $\delta$ over ambient is then

$\allowdisplaybreaks\small\delta_{eu}^{t}=\delta_{e}^{r}k^{2}$ (5)

where $\delta_{e}^{r}$ is the top-oil temperature rise in
[$\degree\mathrm{C}$] at rated apparent power. For a typical transformer,
$\delta_{e}^{r}=75[{\degree}\mathrm{C}]$ [35].

From [36], the top-oil temperature rise over ambient is

$\allowdisplaybreaks\small\delta_{e}(t)=\delta_{eu}(1-e^{t/\tau_{e}})$ (6)

where $\tau_{e}$ is the time constant of the top-oil temperature [min]. A
typical value is 71 [mins] [35]. By using a bilinear transform, the hot-spot
model is reformulated as a difference equation that is suitable for
optimization and simulation [37]:

$\delta_{e}^{t}=\frac{1}{1+\zeta_{e}}(\delta_{eu}^{t}+\delta_{eu}^{t-1})-\frac{1-\zeta_{e}}{1+\zeta_{e}}\delta_{e}^{t-1}$
(7)

where $\Delta$ is the sampling period, $\widetilde{I}_{e}^{t}$ is the
transformer effective GIC, $t$ is the sample index and
$\zeta_{e}=2\tau_{e}/\Delta$ is the discretized time constant.

The time constants of the hot-spot temperature rise over the top-oil
temperature are typically negligible compared to the top-oil temperature
dynamics. The hot-spot temperature rise is linear with respect to the
effective GIC ($R_{e}=0.63\enskip[\degree\mathrm{C}/A]$):

$\allowdisplaybreaks\small\eta_{e}^{t}=R_{e}\widetilde{I}_{e}^{t}$ (8)

The absolute value of the transformer hot-spot:
for $\forall e\in E^{\Delta}\cup E^{y}\cup E^{\infty},t\in T$

$\rho_{e}^{t}+\delta_{e}^{t}+\eta_{e}^{t}$ (9)

where $\rho_{e}^{t}$ is the ambient temperature. It is assumed that
$\rho_{e}^{t}=25\enskip[\degree\mathrm{C}]$.

##  III Steady-State Formulations

Before presenting the time-extended formulation, first the common industry and
academic steady-state formulations implemented in PMsGMD are discussed.

1) GIC DC: quasi-dc power flow.
Solves steady-state dc currents on lines resulting from induced dc voltages on
lines.

2) GIC $\xrightarrow{}$ AC – OPF: sequential quasi-dc power flow and ac
optimal power flow.
Solves for quasi-dc voltages and currents, and uses the calculated quasi-dc
currents through transformer windings as inputs to an AC-OPF in order to
calculate the increase in transformer reactive power consumption.

3) GIC + AC – OPF: ac optimal power flow coupled with a quasi-dc power flow.
Solves for quasi-dc voltages and currents, and solves the AC-OPF concurrently.
The dc network couples to the ac network by means of reactive power loss in
transformers. This formulation does not model increases in transformer
reactive power consumption caused by changes in the ac terminal voltages.
Additionally, it may report higher reactive power consumption than reality as
it relaxes the “effective” transformer quasi-dc winding current magnitude.

4) GIC + AC – MLS: ac minimum-load-shed coupled with a quasi-dc power flow.
Solves the minimum-load shedding problem for a network subjected to GIC with
fixed topology. It uses load shedding to protect the system from GIC-induced
voltage collapse and transformer over-heating.

5) GIC + AC – OTS: ac optimal transmission switching with load shed coupled
with a quasi-dc power flow.
Solves the minimum-load shedding problem for a network subjected to GIC, where
lines and transformers can be opened or closed. It uses transmission-switching
to protect the system from GIC-induced voltage collapse and transformer over-
heating.

##  IV Time-series Extension of the
AC-OTS Problem Specification

Below the problem specification for the AC-OTS use case of PMsGMD is
described. This problem uses the top-oil temperature dynamics described in
Section II to link time-periods. Line switching configurations are shared
across all time-periods. In this paper the dc power flow approximation of the
full ac non-convex power flow constraints is used to ease computational
burden. However, switching to the full set of ac power flow constraints or
relaxations is seamless given the PMs framework.

####  IV-1 Objective function

$\allowdisplaybreaks\small\min\smashoperator[]{\sum_{g\in G,t\in
T}^{}}\kappa_{g}^{0}+\kappa_{g}^{1}p_{g}^{t}+\kappa_{g}^{2}(p_{g}^{t})^{2}$
(10)

####  IV-2 DC power flow approximation equations

for $\forall i\in{N^{a}},t\in{T}$

$\allowdisplaybreaks\small\smashoperator[]{\sum_{e\in{E_{i}^{+}}}^{}}p_{e,ij}^{t}-\smashoperator[]{\sum_{e\in{E_{i}^{-}}}^{}}p_{e,ij}^{t}=\sum_{g\in
G_{i}}p_{g}^{t}-p_{i}^{t}-g_{i}$ (11)

for $\forall e\in{E^{a}},t\in{T}$

$\displaystyle p_{e,ij}^{t}=z_{e}b_{e}(\theta_{i}^{t}-\theta_{j}^{t})$ (12a)
$\displaystyle p_{e,ji}^{t}=-p_{e,ij}^{t}$ (12b)

####  IV-3 Operational limit constraints

for $\forall e\in{E^{a}},t\in{T}$

$\displaystyle p_{e,ij}^{t}\leq z_{e}\overline{s}_{e}$ (13a) $\displaystyle
p_{e,ji}^{t}\leq z_{e}\overline{s}_{e}$ (13b)
$\displaystyle|\theta_{i}^{t}-\theta_{j}^{t}|\leq
z_{e}\overline{\theta}+(1-z_{e})\theta^{M}$ (13c)

for $\forall g\in G,t\in{T}$

$\allowdisplaybreaks\small\underline{p}_{g}\leq p_{g}^{t}\leq\overline{p}_{g}$
(14)

####  IV-4 GIC effects on transformers

for $\forall i\in{N^{d}}$

$\allowdisplaybreaks\small\smashoperator[]{\sum_{e\in
E^{+}_{i}}^{}}I_{e}^{t}-\smashoperator[]{\sum_{e\in
E^{-}_{i}}^{}}I_{e}^{t}=a_{i}V_{i}^{d}$ (15)

for $\forall e\in\mathcal{E}^{d},t\in{T}$

$\allowdisplaybreaks\small
I_{e}^{t}=z_{\overrightarrow{e}}a_{e}(V_{i}^{t}-V_{j}^{t}+\mathcal{V}_{e}^{t})$
(16)

for $\forall e\in{E^{a}},t\in{T}$

$\allowdisplaybreaks\small\widetilde{I}_{e}^{t}\geq+/-\begin{cases}{I^{t}_{e^{H}}}&\text{if
$e\in E^{\Delta}$}\vspace{.1cm}\\\
{\frac{\alpha_{e}I^{t}_{e^{H}}+I^{t}_{e^{L}}}{\alpha_{e}}}&\text{if $e\in
E^{y}$}\vspace{.1cm}\\\
{\frac{\alpha_{e}I^{t}_{e^{S}}+I^{t}_{e^{C}}}{\alpha_{e}+1}}&\text{if $e\in
E^{\infty}$}\vspace{.1cm}\\\ 0&\text{otherwise}\end{cases}$ (17)

for $\forall e\in\mathcal{E}^{\tau}$

$\allowdisplaybreaks\small
0\leq{\widetilde{I}_{e}^{d}}\leq\overline{I}_{e}^{d}$ (18)

$\allowdisplaybreaks\small\mathrm{Eq.}\;\;\eqref{ac_temp},\eqref{eq:xfmr-hs-
rise}$ (19)

for $\forall e\in E^{\Delta}\cup E^{y}\cup E^{\infty},t\in T$

$\allowdisplaybreaks\small\rho_{e}^{t}+\delta_{e}^{t}+\eta_{e}^{t}\leq\mathcal{T}_{e}$
(20)

for $\forall e\in{\mathcal{E}^{a}},g\in G$

$\allowdisplaybreaks\small z_{e}\in\\{0,1\\}$ (21)

####  IV-5 Supporting constraints

for $\forall e\in E^{a},t\in T$

$\allowdisplaybreaks\small\widetilde{I}_{e}^{t}\leq z_{e}\overline{I}_{e}^{d}$
(22)

In this specification, the objective function (10) minimizes total generator
dispatch costs and load shedding costs. Constraints (11)-(12) describe the dc
power flow physics and engineering constraints associated with a power system.
Constraint (11) models power balance at each node (Kirchoff’s Law).
Constraints (12a) and (12b) model the dc approximation of ac power flow on
each transmission line with on-off variables $z_{e}$ (Ohm’s Law). The flow on
any line is forced to zero if the line is switched off.

Constraints (13)-(14) describe the operational limits of the power system.
Constraints (13a) and (13b) model the thermal limits of lines in both
directions. Constraint (13c) applies bounds on the phase angle difference
between two buses. Constraint (14) models the capacity of power generation.

The dc circuit and the effects associated with the GMD are formulated in
constraints (15)-(21). Here an edge ($e\in\mathcal{E}^{d}$) in the dc circuit
is linked to an edge in the ac circuit ($\overrightarrow{e}$). Constraints
(15) and (16) formulate the GIC flow on each dc line by applying Kirchhoff’s
current law using the Thevenin representation.

The GIC on a line is determined by the induced current source and the quasi-dc
voltage difference between two buses [22]. GIC flow is forced to 0 by
$z_{\overrightarrow{e}}$ when $\overrightarrow{e}$ is switched off. Constraint
(17) computes the effective (non-negative) GIC on each ac edge by relaxing the
absolute values in (2) with two inequalities. The effective GIC is calculated
at the time scale of the ac circuit. Constraint (18) models the maximum
allowed value of GIC flowing through a transformer for safe operation.
Constraint (19) computes the ac and GIC hot-spot heating temperature at every
time step using Equations (7) and (8). Constraint (20) forces transformer
temperatures below a thermal limit. Finally, constraint (22) explicitly forces
the effective GIC to 0 when an edge is inactive.

Code Block 1 (on Page V) shows how this specification is implemented in PMsGMD
in Julia. Some function names are abbreviated due to space constaints.

##  V Case Study

To demonstrate the application and performance of PMsGMD, in addition validate
the implementation of the AC-OTS “ac optimal transmission switching with load
shed coupled with a quasi-dc power flow” problem specification (presented in
Section IV), a GMD case study is carried out. The hypothetical network
designed by Horton et al. [31] for GIC modeling software validation is used.
This 21-bus extra high voltage power system model – depicted in Figs. 2 and 3
– consists of 345 [kV] and 500 [kV] lines and transformers, and is centered
over the State of Tennessee, USA. The model includes single transmission lines
as well as some that occupy the same transmission corridor; the substations
feature both conventional and auto-transformers, furthermore, series and
neutral connected GIC blocking devices are also included.

Figure 2: One-line diagram of the used case study system

Figure 3: Map of the used case study system and electric field vector

The Horton case study system provides all parameters necessary for
calculations of GICs, including bus-substation connectivity and substation
geographical locations. In order to simulate a realistic system operation
scenario, a number of lines are disconnected (as indicated in the above
figures) assuming planned system-maintenance and unforeseen outages due to a
GMD event; consequently, the size of the network is reduced to 11 buses. Table
VI summarizes these initial assumptions for each branch.

TABLE VI: Branch Statuses in the Case Study System i | j | Ckt. | Type | $z_{e}^{nom}$ | $z_{e}$ | $p_{e,ij}$ | $I_{e}$
---|---|---|---|---|---|---|---
1 | 2 | 1 | xf | 1 | 1 | 8.3 | 0.0
2 | 3 | 1 | line | 1 | 1 | 8.3 | 0.0
3 | 4 | 1 | xf | 1 | 1 | 2.1 | -56.6
3 | 4 | 2 | xf | 1 | 1 | 2.1 | -56.6
3 | 4 | 3 | xf | 1 | 1 | 2.1 | 127.8
3 | 4 | 4 | xf | 1 | 1 | 2.1 | 127.8
4 | 5 | 1 | line | 1 | 1 | -6.7 | 368.8
4 | 5 | 2 | line | 1 | 0 | 0.0 | 0.0
4 | 6 | 1 | line | 1 | 0 | 0.0 | 0.0
5 | 6 | 1 | line | 1 | 1 | -18.7 | 368.8
5 | 20 | 1 | xf | 0 | 0 | 0.0 | 0.0
5 | 20 | 2 | xf | 0 | 0 | 0.0 | 0.0
5 | 21 | 1 | series_cap | 0 | 0 | 0.0 | 0.0
6 | 7 | 1 | xf | 1 | 1 | -8.3 | 21.5
6 | 8 | 1 | xf | 1 | 1 | -8.3 | 21.5
6 | 11 | 1 | line | 1 | 1 | -5.0 | 325.9
11 | 12 | 1 | line | 1 | 1 | -5.0 | 325.9
12 | 13 | 1 | xf | 1 | 1 | -5.0 | 325.9
12 | 14 | 1 | xf | 0 | 0 | 0.0 | 0.0
15 | 4 | 1 | line | 0 | 0 | 0.0 | 0.0
15 | 6 | 1 | line | 0 | 0 | 0.0 | 0.0
15 | 6 | 2 | line | 0 | 0 | 0.0 | 0.0
16 | 15 | 1 | xf | 0 | 0 | 0.0 | 0.0
16 | 15 | 2 | xf | 0 | 0 | 0.0 | 0.0
16 | 17 | 1 | line | 0 | 0 | 0.0 | 0.0
16 | 20 | 1 | line | 0 | 0 | 0.0 | 0.0
17 | 2 | 1 | line | 0 | 0 | 0.0 | 0.0
17 | 18 | 1 | xf | 0 | 0 | 0.0 | 0.0
17 | 19 | 1 | xf | 0 | 0 | 0.0 | 0.0
17 | 20 | 1 | line | 0 | 0 | 0.0 | 0.0
21 | 11 | 1 | line | 0 | 0 | 0.0 | 0.0

The system is subjected to a time-varying uniform East-West electric field (as
indicated in Fig. 3), which varies linearly from 0 $\mathrm{V/km}$ to 3.2
$\mathrm{V/km}$ over the first 3 hours and then back down linearly to 0
$\mathrm{V/km}$ over the remaining 3 hours of the simulation, illustrated in
Fig. 4.

Figure 4: Time variation of the electric field magnitude

##  VI Evaluation of Results

The simulation was performed on a computer with a quad-core Inter® CoreTM
i5-6500 3.2GHz CPU, and 8GB RAM. The used software was Julia v1.0 [13], using
Gurobi v8.1 [17] for MacOS as nonlinear solver.

Parts of the results are presented in Table VI: the columns $z_{e}^{nom}$ and
$z_{e}$ list the nominal branch statuses along with the branch statuses
produced by the optimizer, respectively. The mitigation strategy determination
takes 21 [sec]. The optimizer chooses to relieve GIC flowing through the
transformers at substations 1, 4 and 6; it accomplishes this by opening the
longest predominantly East-West directional line 4-6-1, along with one of the
lines in the 4-5 transmission corridor.

Fig. 5 illustrates the temperature of transformer 12-13-1, located at
substation 8 of Fig. 3; the designed mitigation strategy kept its temperature
below the instantaneous limit of $280\degree\mathrm{C}$. This admittedly high
instantaneous limit is selected with a bias to supplying load power at the
cost of significant loss-of-life to transformer insulation; nevertheless, this
limit can be adjusted as needed.

Figure 5: Hot-spot temperature for transformer 12-13-1

##  VII Conclusion

As the threat of GMD and E3 HEMP events continues to pose a substantial risk
to our energy infrastructures, accurate simulation and effective mitigation
becomes increasingly important. This paper presented an extensible open-source
modeling framework for both analyzing and optimally mitigating the impacts of
such hazards.

By design, PMsGMD is suitable to monitor disturbance-manifestations in real-
time and predict GICs on the electrical grid. In addition, by extending PMsGMD
with the problem of mitigating transformer heating by switching lines given a
time-varying electric field, along with techniques to improve the scalability
of the problem, a GIC mitigation capability is enabled; providing mitigation-
strategies in a reasonable amount of time demonstrates the utility of this
modular framework.

The performed case study validates implementation and demonstrates
performance; further simulations on larger size networks are future tasks.
Other extensions to the presented GIC power flow problem are possible under
the PMsGMD framework, such as optimal GIC blocker placement, harmonic load
flow, and scenario-based problem formulations.

Acknowledgements This work was supported by the U.S. Department of Energy –
Laboratory Directed Research and Development Program at Los Alamos National
Laboratory under the _“Impacts of Extreme Space Weather Events on Power Grid
Infrastructure: Physics-Based Modelling of Geomagnetically-Induced Currents
(GICs) During Carrington-Class Geomagnetic Storms”_ project.

## References

  * [1] NERC and U.S. Department of Energy.  High-Impact, Low-Frequency Event Risk to the North American Bulk Power System.  Technical report, North American Electric Reliability Corporation, Jun. 2010.
  * [2] DHS Cybersecurity & Infrastructure Security Agency.  Homeland Security Presidential Directive 7.  Technical report, U.S. Department of Homeland Security, Dec. 2003.
  * [3] V. D. Albertson, J. M. Thorson, R. E. Clayton, and S. C. Tripathy.  Solar-Induced-Currents in Power Systems: Cause and Effects.  IEEE Transactions on Power Apparatus and Systems, PAS-92(2):471–477, Mar. 1973.
  * [4] A. Mate.  A Generation Prioritization Method for Power System Restoration with Renewable Resources.  PhD thesis, Oregon State University, Jun. 2020.
  * [5] D. H. Boteler and R. J. Pirjola.  Modeling Geomagnetically Induced Currents.  Space Weather, 15(1):258–276, Jan. 2017.
  * [6] T. R. Hutchins and T. J. Overbye.  Power System Dynamic Performance During the Late-Time (E3) High-Altitude Electromagnetic Pulse.  In Proceedings of the 2016 Power Systems Computation Conference, pages 1–6, Jun. 2016.
  * [7] R. Pirjola.  Geomagnetically Induced Currents During Magnetic Storms.  IEEE Transactions on Plasma Science, 28(6):1867–1873, Dec. 2000\.
  * [8] T. R. Hutchins and T. J. Overbye.  The Effect of Geomagnetic Disturbances on the Electric Grid and Appropriate Mitigation Strategies.  In Proceedings to the 2011 North American Power Symposium, pages 1–5, Aug. 2011.
  * [9] J. Berge and R. K. Varma.  Modeling and Mitigation of Geomagnetically Induced Currents on a Realistic Power System Network.  In Proceedings to the 2011 IEEE Electrical Power and Energy Conference, pages 485–490, Oct 2011.
  * [10] T. J. Overbye, K. S. Shetye, T. R. Hutchins, Q. Qiu, and J. D. Weber.  Power Grid Sensitivity Analysis of Geomagnetically Induced Currents.  IEEE Transactions on Power Systems, 28(4):4821–4828, Nov. 2013\.
  * [11] NERC.  Effects of Geomagnetic Disturbances on the Bulk Power System System.  Technical report, North American Electric Reliability Corporation, Feb. 2012.
  * [12] K. S. Shetye and T. J. Overbye.  Modeling and Analysis of GMD Effects on Power Systems: An Overview of the Impact on Large-Scale Power Systems.  IEEE Electrification Magazine, 3(4):13–21, Dec. 2015.
  * [13] J. Bezanson, S. Karpinski, V. B. Shah, and A. Edelman.  Julia: A Fast Dynamic Language for Technical Computing.  https://julialang.org, 2012.
  * [14] C. Coffrin, R. Bent, K. Sundar, Y. Ng, and M. Lubin.  PowerModels.jl: An Open-Source Framework for Exploring Power Flow Formulations.  In Proceedings of the 2018 Power Systems Computation Conference, pages 1–8, June 2018.
  * [15] I. Dunning, J. Huchette, and M. Lubin.  JuMP: A Modeling Language for Mathematical Optimization.  SIAM Review, 59(2):295–320, 2017.
  * [16] A. Wachter and L. T. Biegler.  On the Implementation of a Primal-Dual Interior Point Filter Line Search Algorithm for Large-Scale Nonlinear Programming.  Mathematical Programming, 106(1):25–57, Mar. 2006.
  * [17] Gurobi Optimization LLC.  Gurobi Optimizer Reference Manual.  http://www.gurobi.com, 2019.
  * [18] IBM Corp.  CPLEX User’s Manual.  https://www.ibm.com, 2019.
  * [19] R. D. Zimmerman, C. E. Murillo-Sánchez, and R. J. Thomas.  MATPOWER: Steady-State Operations, Planning and Analysis Tools for Power Systems Research and Education.  IEEE Transactions on Power Systems, 26(1):12–19, Feb. 2011.
  * [20] T. R. Hutchins and T. J. Overbye.  Power Flow Studies in the Presence of Geomagnetically Induced Currents.  In Proceedings of the 2012 IEEE Power and Energy Conference at Illinois, pages 1–4, Feb. 2012.
  * [21] T. J. Overbye, T. R. Hutchins, K. S. Shetye, J. D. Weber, and S. Dahman.  Integration of Geomagnetic Disturbance Modeling Into the Power Flow: A Methodology for Large-Scale System Studies.  In Proceedings of the 2012 North American Power Symposium, pages 1–7, Sep. 2012.
  * [22] NERC Application Guide.  Computing Geomagnetically-Induced Current in the Bulk-Power System.  Technical report, North American Electric Reliability Corporation, Dec. 2013.
  * [23] M. Kazerooni, H. Zhu, T. J. Overbye, and D. A. Wojtczak.  Transmission System Geomagnetically Induced Current Model Validation.  IEEE Transactions on Power Systems, 32(3):2183–2192, May. 2017\.
  * [24] C. Klauber, G. P. Juvekar, K. Davis, T. Overbye, and K. Shetye.  The Potential for a GIC-inclusive State Estimator.  In Proceedings of the 2018 North American Power Symposium, pages 1–6, Sep. 2018.
  * [25] G. P. Juvekar and K. Davis.  MATGMD: A Tool for Enabling GMD Studies in MATLAB.  In Proceedings of the 2019 IEEE Texas Power and Energy Conference, pages 1–6, Feb. 2019.
  * [26] PowerWorld Corporation.  PowerWorld® Simulatorl.  https://www.powerworld.com, 2019.
  * [27] Siemens.  PSS®E.  https://new.siemens.com, 2019.
  * [28] A. Schultz, S. Murphy, E. Cotilla-Sanchez, N. Imamura, A. Mate, and A. Barnes.  Fusing Magnetotelluric, Magnetic Observatory and Power Grid Sensor Data with Real-Time Power Flow Analysis: Providing Actionable Information to Mitigate Risk to the Power Grid from Geomagnetic Disturbances/EMP.  In Technical Presentation at the American Geophysical Union 2019 Fall Meeting, Dec. 2019.
  * [29] M. Lu, S. D. Eksioglu, S. J. Mason, R. Bent, and H. Nagarajan.  Distributionally Robust Optimization for a Resilient Transmission Grid During Geomagnetic Disturbances.  https://arxiv.org/abs/1906.04139, 2019.
  * [30] S. Dahman.  Geomagnetically Induced Current (GIC) Modeling.  In Proceedings of the 2016 PowerWorld Client Conference, Feb. 2016\.
  * [31] R. Horton, D. Boteler, T. J. Overbye, R. Pirjola, and R. C. Dugan.  A Test Case for the Calculation of Geomagnetically Induced Currents.  IEEE Transactions on Power Delivery, 27(4):2368–2373, Oct. 2012\.
  * [32] H. Zhu and T. J. Overbye.  Blocking Device Placement for Mitigating the Effects of Geomagnetically Induced Currents.  IEEE Transactions on Power Systems, 30(4):2081–2089, Jul. 2015\.
  * [33] M. Lu, H. Nagarajan, E. Yamangil, R. Bent, S. Backhaus, and A. Barnes.  Optimal Transmission Line Switching Under Geomagnetic Disturbances.  IEEE Transactions on Power Systems, 33(3):2539–2550, May 2018.
  * [34] K. Zheng, D. Boteler, R. J. Pirjola, L.-G. Liu, R. Becker, L. Marti, S. Boutilier, and S. Guillon.  Effects of System Characteristics on Geomagnetically Induced Currents.  IEEE Transactions on Power Delivery, 29(2):890–898, Apr. 2014.
  * [35] D. Susa, M. Lehtonen, and H. Nordman.  Dynamic Thermal Modelling of Power Transformers.  IEEE Transactions on Power Delivery, 20(1):197–204, Jan. 2005.
  * [36] X. Mao, D. J. Tylavsky, and G. A. McCulla.  Dynamic Thermal Modelling of Power Transformers.  IEE Proceedings on Generation, Transmission and Distribution, 153(4):414–422, Jul. 2006.
  * [37] R. Hurton.  Magnetohydrodynamic Electromagnetic Pulse Assessment of the Continental U.S. Electric Grid.  Technical report, Electric Power Research Institute, Feb. 2017.